First steps with client/server on Linux - bulk import; error message in dba
Tuesday, 10 May 2022 · 9 messages · last reply 18 Jul 2022
FS
Florian Schmitt10 May 2022 16:29 #
Hi all,
i'm currently evaluating BaseX running on a headless Linux Server. It took me quite a while to get it running (the docs seem to be focused on Windows environments)... Is there a way to contribute to the documentation?
First question: After importing some of documents nicely organized in collections, i've found that the REST client will display all documents on the same "level" instead of showing just the first level of collections. This isn't nice if the db contains a great number of documents.... When accessing the same db using WebDAV (Oxygen), one can nicelly navigate using the collection structure. With REST, adding the collection names to the URL will restrict the output to the db content of those collections. But without such a restriction, it would be great if the REST web interface would just return the first level of collections without any XML resource entries.
Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections? I've found that Oxygen "translates" subdirectories into collections when using the "import directories" feature. Is there somehing similar using the command line client?
And another issue: The "Logs" page of the DBA webapp shows the following error warning:
"Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact..."
Is this a critical error? How to debug this? So far, i'found the wrapper-basex log, but it doesn't contain anything useful. Since i'm using the Yajsw Wrapper, there's no debug output or debug option (?).
Yours sincerely.
Florian
FS
Florian Schmitt11 May 2022 10:37 #
Quoted text · Florian Schmitt · 11 lines
Am 10.05.2022 um 16:29 schrieb Florian Schmitt: > ... > And another issue: The "Logs" page of the DBA webapp shows the > following error warning: > > "Unexpected error: Improper use? Potential bug? Your feedback is > welcome: Contact..." > > Is this a critical error? How to debug this? So far, i'found the > wrapper-basex log, but it doesn't contain anything useful. Since i'm > using the Yajsw Wrapper, there's no debug output or debug option (?).
After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry:
[POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact:
basex-talk at mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace:
java.lang.NullPointerException at
org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at
org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at
org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at
org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at
org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R...
That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
HR
I agree with Florian that navigation of db contents is important and support is desirable - perhaps in analogy to
file:children and file:descendants. Kind regards, Hans-Jürgen PS: The same applies to the contents of archives.
Am Dienstag, 10. Mai 2022, 16:28:52 MESZ hat Florian Schmitt <ml-basex at florian-schmitt.net> Folgendes geschrieben:
Hi all,
i'm currently evaluating BaseX running on a headless Linux Server. It took me quite a while to get it running (the docs seem to be focused on Windows environments)... Is there a way to contribute to the documentation?
First question: After importing some of documents nicely organized in collections, i've found that the REST client will display all documents on the same "level" instead of showing just the first level of collections. This isn't nice if the db contains a great number of documents.... When accessing the same db using WebDAV (Oxygen), one can nicelly navigate using the collection structure. With REST, adding the collection names to the URL will restrict the output to the db content of those collections. But without such a restriction, it would be great if the REST web interface would just return the first level of collections without any XML resource entries.
Second question: What's the "best practice" for bulk imports while retaining the directory structure of the imported files using collections? I've found that Oxygen "translates" subdirectories into collections when using the "import directories" feature. Is there somehing similar using the command line client?
And another issue: The "Logs" page of the DBA webapp shows the following error warning:
"Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact..."
Is this a critical error? How to debug this? So far, i'found the wrapper-basex log, but it doesn't contain anything useful. Since i'm using the Yajsw Wrapper, there's no debug output or debug option (?).
Yours sincerely.
FlorianCG
Thanks for the observation, Florian.
Quoted text · Florian Schmitt · 1 line
> After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry:
I hope the latest snapshot resolves the Yajsw/.logs bug [1].
Quoted text · Florian Schmitt · 1 line
> [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at
I’ve added Vincent Lizzi to the conversation, who has gratefully added the documentation on YAJSW: Maybe it would be sensible to move the wrapper-basex.log file out of the .logs directory? I don’t expect any complications if it’s stored in data/.
I’ll additionally check if we can prevent
admin:logs from crashing if the log input does not conform to the usual pattern.
Best,
Christian
[1] https://github.com/BaseXdb/basex/issues/2105Quoted text · Florian Schmitt · 2 lines
> [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R... > That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
CG
Hi Florian,
Quoted text · Florian Schmitt · 4 lines
> i'm currently evaluating BaseX running on a headless Linux Server. It > took me quite a while to get it running (the docs seem to be focused > on Windows environments)... Is there a way to contribute to the > documentation?
Thanks for the offer. You are invited to register for a docs.basex.org Wiki acount. If you prefer, you can give us some specific hints on how to revise existing texts in the documentation.
Quoted text · Florian Schmitt · 5 lines
> First question: After importing some of documents nicely organized in > collections, i've found that the REST client will display all > documents on the same "level" instead of showing just the first level > of collections. This isn't nice if the db contains a great number of > documents....
I agree. The REST interface was included many years ago, but in the meantime there are various ways – e.g. via
db:dir() – to restrict the database view to specific directories. I have added an issue for that [1]. As it will be a breaking change, we’ll probably tackle it with BaseX 10 or later.Quoted text · Florian Schmitt · 3 lines
> Second question: What's the "best practice" for bulk imports while > retaining the directory structure of the imported files using > collections?
If you create a new database, you can use the CREATE command or
db:create and specify a directory as initial input. The existing file hierarchy will be mirrored in the resulting database.
Hope this helps,
Christian
[1] https://github.com/BaseXdb/basex/issues/2106LV
Hi Christian and Florian,
I recall that the DBA web interface didn't like the YAJSW log files, although that didn't seem surprising because the YAJSW log files are in a different format than BaseX log files. It's been a while since I tried this so I don't have the details now. YAJSW can be configured to save its log files in a different location.
Kind regards,
Vincent
_____________________________________________
Vincent M. Lizzi
Head of Information Standards | Taylor & Francis Group
vincent.lizzi at taylorandfrancis.com<mailto:vincent.lizzi at taylorandfrancis.com>
Information Classification: General
Quoted text · Christian Grün · 20 lines
From: Christian Grün <christian.gruen at gmail.com> Sent: Thursday, May 12, 2022 8:03 AM To: Florian Schmitt <ml-basex at florian-schmitt.net>; Lizzi, Vincent <Vincent.Lizzi at taylorandfrancis.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] First steps with client/server on Linux - bulk import; error message in dba Thanks for the observation, Florian. > After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry: I hope the latest snapshot resolves the Yajsw/.logs bug [1]. > [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at I've added Vincent Lizzi to the conversation, who has gratefully added the documentation on YAJSW: Maybe it would be sensible to move the wrapper-basex.log file out of the .logs directory? I don't expect any complications if it's stored in data/. I'll additionally check if we can prevent admin:logs from crashing if the log input does not conform to the usual pattern. Best, Christian [1] https://github.com/BaseXdb/basex/issues/2105<https://github.com/BaseXdb/basex/issues/2105> > [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de<mailto:basex-talk at mailman.uni-konstanz.de> Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R... > That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
CG
Christian Grün12 May 2022 19:08 #
Thanks, Vincent.
I have adapted the path in the documention.
Quoted text · Vincent · 64 lines
On Thu, May 12, 2022 at 3:24 PM Lizzi, Vincent <Vincent.Lizzi at taylorandfrancis.com> wrote: > > Hi Christian and Florian, > > > > I recall that the DBA web interface didn’t like the YAJSW log files, although that didn’t seem surprising because the YAJSW log files are in a different format than BaseX log files. It’s been a while since I tried this so I don’t have the details now. YAJSW can be configured to save its log files in a different location. > > > > Kind regards, > > Vincent > > > > > > _____________________________________________ > > Vincent M. Lizzi > > Head of Information Standards | Taylor & Francis Group > > vincent.lizzi at taylorandfrancis.com > > > > > Information Classification: General > > From: Christian Grün <christian.gruen at gmail.com> > Sent: Thursday, May 12, 2022 8:03 AM > To: Florian Schmitt <ml-basex at florian-schmitt.net>; Lizzi, Vincent <Vincent.Lizzi at taylorandfrancis.com> > Cc: BaseX <basex-talk at mailman.uni-konstanz.de> > Subject: Re: [basex-talk] First steps with client/server on Linux - bulk import; error message in dba > > > > Thanks for the observation, Florian. > > > After modifying permissions for the $basexhome/data/.logs directory, i now have a logfile showing the following entry: > > I hope the latest snapshot resolves the Yajsw/.logs bug [1]. > > > [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at > > I’ve added Vincent Lizzi to the conversation, who has gratefully added > the documentation on YAJSW: Maybe it would be sensible to move the > wrapper-basex.log file out of the .logs directory? I don’t expect any > complications if it’s stored in data/. > > I’ll additionally check if we can prevent admin:logs from crashing if > the log input does not conform to the usual pattern. > > Best, > Christian > > [1] https://github.com/BaseXdb/basex/issues/2105 > > > [POST] /dba/log?name=wrapper-basex&page=1; Unexpected error: Improper use? Potential bug? Your feedback is welcome: Contact: basex-talk at mailman.uni-konstanz.de Version: BaseX 9.7.1 Java: Private Build, 11.0.15 OS: Linux, amd64 Stack Trace: java.lang.NullPointerException at org.basex.query.func.admin.AdminLogs$1.next(AdminLogs.java:84) at org.basex.query.QueryContext.next(QueryContext.java:358) at org.basex.query.func.fn.FnReverse.iter(FnReverse.java:61) at org.basex.query.expr.gflwor.For$1.next(For.java:112) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:144) at org.basex.query.expr.gflwor.Where$1.next(Where.java:41) at org.basex.query.expr.gflwor.GFLWOR.value(GFLWOR.java:82) at org.basex.query.expr.gflwor.Let$LetEval.next(Let.java:146) at org.basex.query.expr.gflwor.GFLWOR$1.next(GFLWOR.java:68) at org.basex.query.scope.MainModule$1.next(MainModule.java:67) at org.basex.http.restxq.RestXqResponse.serialize(RestXqResponse.java:87) at org.basex.http.web.WebResponse.create(WebResponse.java:58) at org.basex.http.restxq.R... > > > That NPE is thrown if i try to access the wrapper-basex log using the DBA web interface. Accessing the BaseX logs works fine.
FS
Hi Christian,
Quoted text · Christian Grün · 7 lines
>> Second question: What's the "best practice" for bulk imports while >> retaining the directory structure of the imported files using >> collections? > > If you create a new database, you can use the CREATE command or > db:create and specify a directory as initial input. The existing file > hierarchy will be mirrored in the resulting database.
it seems i'm not quite used with the lightweight concept of databases in BaseX :-) - since it's very easy to create and use multiple databases, filling the db initially is a good and reasonable solution.
Regarding the server installation: i'm not sure what went wrong yesterday - anyway, i've ended up with another basex subdirectory in the home directory of my baseX user, containing an empty webapp directory - thus there was no DBA web interface and no logs to consult... but now (esp. after i've found the extremely valuable information about config files:
https://docs.basex.org/wiki/Configuration) everything works smooth and nice.
Thanks again for your help!
Florian
CG
Hi Florian,
Quoted text · Christian Grün · 6 lines
> I agree. The REST interface was included many years ago, but in the > meantime there are various ways – e.g. via db:dir() – to restrict the > database view to specific directories. I have added an issue for that > [1]. As it will be a breaking change, we’ll probably tackle it with > BaseX 10 or later. > [1] https://github.com/BaseXdb/basex/issues/2106
The REST output was revised: Resources in subdirectories have been replaced with a summarizing <dir> element. A new BaseX 10 snapshot is available [1].
Best,
Christian
[1] https://files.basex.org/maven/org/basex/basex/10.0-SNAPSHOT/basex-10.0-20220718.145526-8.jar
Written for BaseX 9.7. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery