Shouldn't CHOP = false make xml:space="preserve" the default behavior?
Sunday, 6 August 2017 · 6 messages · last reply 11 Aug 2017
OM
Ottid Mes6 Aug 2017 13:10 #
(Sorry for the noise, but because my previous mail was shown under trimmed content in my Sent Mail, I will resend it without format just in case it was not shown properly to others as well)
From the documentation about the CHOP option I assumed that since xml:space="preserve" sets CHOP = false for that part of the document, that if I set CHOP = false in my configuration file, that the behavior you get when you use xml:space="preserve" would be applied to the whole database (I created the database after setting the option). However the only way I have ever been able to get this behavior, has been to set xml:space="preserve" at the root element. Am I missing something, or is this a bug? How could I get this behavior by default in my databases?
I thought this would not warrent a thorough example given the clear conditions which cause the above situation, but I was asked for it anyway, so here it goes:
A little bit of context, although it should not matter, I have had this issue for years (at least 4) under Arch Linux. At the time I just assumed I did something wrong and went with xml:space="preserve" workaround everywhere.
Quoted text · 1 line
> uname -a
Linux phoenix 4.9.38 #1-NixOS SMP Sat Jul 15 10:17:55 UTC 2017 x86_64 GNU/Linux
The default situation:
No config file (.basex)
Quoted text · 1 line
> basex
BaseX 8.6.4 [Standalone]
Quoted text · 1 line
> create db chop-test
Database 'chop-test' created in 123.01 ms.
Quoted text · 1 line
> open chop-test
Database 'chop-test' was opened in 1.19 ms.
Quoted text · 1 line
> replace /a <root><a><b>foo</b></a> bar</root>
0 resource(s) replaced in 103.68 ms.
Quoted text · 1 line
> xquery /root
<root>
<a>
<b>foo</b>
</a>bar</root>
Query executed in 106.05 ms.I never use the REPL other than to create and drop databases, so I was a bit suprised that this did not work:
Quoted text · 1 line
> replace /a <root xml:space="preserve"><a><b>foo</b></a> bar</root>
"a.xml" (Line 1): Open quote is expected for attribute "xml:space" associated with an element type "root".
While this does:
Quoted text · 1 line
> replace /a <root xml:space='preserve'><a><b>foo</b></a> bar</root>
1 resource(s) replaced in 4.06 ms.
Quoted text · 1 line
> xquery /root
<root xml:space="preserve"><a><b>foo</b></a> bar</root>
Query executed in 0.95 ms.
Quoted text · 1 line
> quit
Have fun.
The resource with xml:space="preserve" is the behavior I want to have within my database, because all my documents are mixed content.
On the wiki (http://docs.basex.org/wiki/Options#CHOP) this is also mentioned:
It explicitly states that in my use case I should set CHOP to false: "The flag should be turned off if a document contains mixed content."
It also states that setting the xml:space="preserve" attribute is the same as having CHOP = false:
"If the xml:space="preserve" attribute is attached to an element, chopping will be turned off for all descendant text nodes."
So lets do that:
Let us first confirm that the config file is correctly read:
Quoted text · 2 lines
> echo 'FOO = 0' > /some/path/.basex > BASEX_JVM='-Dorg.basex.path=/some/path' basex
/some/path/.basex: Unknown option 'FOO'.
/some/path/.basex: writing new configuration file.
Now we set the option CHOP = false in our config:
Quoted text · 1 line
> echo 'CHOP = false' >> /some/path/.basex
So lets see what this changes in the basex REPL:
Quoted text · 1 line
> BASEX_JVM='-Dorg.basex.path=/some/path' basex
BaseX 8.6.4 [Standalone]
Quoted text · 1 line
> drop db chop-test
Database 'chop-test' was dropped.
Quoted text · 1 line
> create db chop-test
Database 'chop-test' created in 106.42 ms.
Quoted text · 1 line
> open chop-test
Database 'chop-test' was opened in 0.05 ms.
Quoted text · 1 line
> replace /a <root><a><b>foo</b></a> bar</root>
0 resource(s) replaced in 39.24 ms.
Quoted text · 1 line
> xquery /root
<root>
<a>
<b>foo</b>
</a> bar</root>
Query executed in 97.09 ms.Quoted text · 1 line
> quit
Have fun.
This is not what I expect, it should have been:
Quoted text · 1 line
> xquery /root
<root><a><b>foo</b></a> bar</root>And hence my question: Shouldn't CHOP = false make xml:space="preserve" the default behavior?
I even tried this:
Quoted text · 1 line
> BASEX_JVM='-Dorg.basex.path=/some/path' basex
BaseX 8.6.4 [Standalone]
Try 'help' to get more information.
Quoted text · 1 line
> open chop-test
Database 'chop-test' was opened in 90.65 ms.
Quoted text · 3 lines
> set chop off CHOP: false > replace /a <root><a><b>foo</b></a> bar</root>
1 resource(s) replaced in 45.58 ms.
Quoted text · 1 line
> xquery /root
<root>
<a>
<b>foo</b>
</a> bar</root>
Query executed in 96.78 ms.Quoted text · 1 line
> quit
See you.
Am I making some mistake in the above?
Is the wiki simply outdated and should this be configured differently? Is having having mixed content in basex so rare that this bug has gone unnoticed for years?
CG
Christian Grün7 Aug 2017 10:57 #
Dear Ottid,
Thanks for providing us with the helpful example, which helped me to understand the problem.
Quoted text · Ottid Mes · 3 lines
>> replace /a <root xml:space="preserve"><a><b>foo</b></a> bar</root> > "a.xml" (Line 1): Open quote is expected for attribute "xml:space" > associated with an element type "root".
Just a side note: Command-line parsing is restrictive when it comes to replacing XML. The reason is that is possible to send multiple commands in a single line, as shown in the following example:
create db db; replace /a <a/>; xquery .
Quoted text · Ottid Mes · 2 lines
>> xquery /root > <root><a><b>foo</b></a> bar</root>
You may be surprised to hear that whitespaces in your document were actually chopped, and that the whitespaces are added by the serializer, because the "indent" serialization parameter is by default set to "yes".
It was surprised to see that no one else pointed at this so far, and that was not mentioned in our documentation, so I have just added some explanatory lines [1,2].
Some more technical background:
If you call BaseX the "info storage" command, you will see which XML nodes are stored in the document:
Quoted text · Ottid Mes · 1 line
> set chop on;create db db <a> </a>; info storage
CHOP: true
Database 'db' created in 11.0 ms.
PRE DIS SIZ ATS ID NS KIND CONTENT
-----------------------------------------
0 1 2 1 0 0 DOC db.xml
1 1 1 1 1 0 ELEM aIf
Quoted text · Ottid Mes · 1 line
> set chop off;create db db <a> </a>; info storage
CHOP: false
Database 'db' created in 20.12 ms.
PRE DIS SIZ ATS ID NS KIND CONTENT
-----------------------------------------
0 1 3 1 0 0 DOC db.xml
1 1 2 1 1 0 ELEM a
2 1 1 1 2 0 TEXT
Serialization indentation was a chosen as default because it goes hand in hand with the CHOP option. It even works fine if CHOP is disabled if a document has whitespaces included (in which case no whitespaces will be added by the serialized). But it may definitely cause undesirable output if a document contains no superfluous whitespaces, such as in your case.
Hope this helps,
Christian
[1] http://docs.basex.org/wiki/Options#CHOP
[2] http://docs.basex.org/wiki/Full-Text#Mixed_Content
AB
Andy Bunce10 Aug 2017 18:53 #
Quoted text · Christian Grün · 1 line
>But it may definitely cause undesirable output if a document contains no
superfluous whitespaces,
One situation where the default serialization indentation can be problematic is WebDAV.
Can the WebDAV serialization be set independently of the default, in web.xml?
/Andy
Quoted text · Christian Grün · 64 lines
On 7 August 2017 at 09:57, Christian Grün <christian.gruen at gmail.com> wrote: > Dear Ottid, > > Thanks for providing us with the helpful example, which helped me to > understand the problem. > > >> replace /a <root xml:space="preserve"><a><b>foo</b></a> bar</root> > > "a.xml" (Line 1): Open quote is expected for attribute "xml:space" > > associated with an element type "root". > > Just a side note: Command-line parsing is restrictive when it comes to > replacing XML. The reason is that is possible to send multiple > commands in a single line, as shown in the following example: > > create db db; replace /a <a/>; xquery . > > > >> xquery /root > > <root><a><b>foo</b></a> bar</root> > > You may be surprised to hear that whitespaces in your document were > actually chopped, and that the whitespaces are added by the > serializer, because the "indent" serialization parameter is by default > set to "yes". > > It was surprised to see that no one else pointed at this so far, and > that was not mentioned in our documentation, so I have just added some > explanatory lines [1,2]. > > Some more technical background: > > If you call BaseX the "info storage" command, you will see which XML > nodes are stored in the document: > > > set chop on;create db db <a> </a>; info storage > CHOP: true > Database 'db' created in 11.0 ms. > PRE DIS SIZ ATS ID NS KIND CONTENT > ----------------------------------------- > 0 1 2 1 0 0 DOC db.xml > 1 1 1 1 1 0 ELEM aIf > > > set chop off;create db db <a> </a>; info storage > CHOP: false > Database 'db' created in 20.12 ms. > PRE DIS SIZ ATS ID NS KIND CONTENT > ----------------------------------------- > 0 1 3 1 0 0 DOC db.xml > 1 1 2 1 1 0 ELEM a > 2 1 1 1 2 0 TEXT > > Serialization indentation was a chosen as default because it goes hand > in hand with the CHOP option. It even works fine if CHOP is disabled > if a document has whitespaces included (in which case no whitespaces > will be added by the serialized). But it may definitely cause > undesirable output if a document contains no superfluous whitespaces, > such as in your case. > > Hope this helps, > Christian > > [1] http://docs.basex.org/wiki/Options#CHOP > [2] http://docs.basex.org/wiki/Full-Text#Mixed_Content >
CG
Christian Grün10 Aug 2017 19:40 #
Hi Andy,
Quoted text · Andy Bunce · 2 lines
> Can the WebDAV serialization be set independently of the default, in > web.xml?
The defaults for whitespace chopping and serialization can only be assigned globally for all features of BaseX. Did you try to set both 'org.basex.chop' and 'org.basex.serializer' in web.xml / does it introduce other unwanted side effects?
Cheers,
Christian
Quoted text · Andy Bunce · 68 lines
> /Andy > > On 7 August 2017 at 09:57, Christian Grün <christian.gruen at gmail.com> wrote: >> >> Dear Ottid, >> >> Thanks for providing us with the helpful example, which helped me to >> understand the problem. >> >> >> replace /a <root xml:space="preserve"><a><b>foo</b></a> bar</root> >> > "a.xml" (Line 1): Open quote is expected for attribute "xml:space" >> > associated with an element type "root". >> >> Just a side note: Command-line parsing is restrictive when it comes to >> replacing XML. The reason is that is possible to send multiple >> commands in a single line, as shown in the following example: >> >> create db db; replace /a <a/>; xquery . >> >> >> >> xquery /root >> > <root><a><b>foo</b></a> bar</root> >> >> You may be surprised to hear that whitespaces in your document were >> actually chopped, and that the whitespaces are added by the >> serializer, because the "indent" serialization parameter is by default >> set to "yes". >> >> It was surprised to see that no one else pointed at this so far, and >> that was not mentioned in our documentation, so I have just added some >> explanatory lines [1,2]. >> >> Some more technical background: >> >> If you call BaseX the "info storage" command, you will see which XML >> nodes are stored in the document: >> >> > set chop on;create db db <a> </a>; info storage >> CHOP: true >> Database 'db' created in 11.0 ms. >> PRE DIS SIZ ATS ID NS KIND CONTENT >> ----------------------------------------- >> 0 1 2 1 0 0 DOC db.xml >> 1 1 1 1 1 0 ELEM aIf >> >> > set chop off;create db db <a> </a>; info storage >> CHOP: false >> Database 'db' created in 20.12 ms. >> PRE DIS SIZ ATS ID NS KIND CONTENT >> ----------------------------------------- >> 0 1 3 1 0 0 DOC db.xml >> 1 1 2 1 1 0 ELEM a >> 2 1 1 1 2 0 TEXT >> >> Serialization indentation was a chosen as default because it goes hand >> in hand with the CHOP option. It even works fine if CHOP is disabled >> if a document has whitespaces included (in which case no whitespaces >> will be added by the serialized). But it may definitely cause >> undesirable output if a document contains no superfluous whitespaces, >> such as in your case. >> >> Hope this helps, >> Christian >> >> [1] http://docs.basex.org/wiki/Options#CHOP >> [2] http://docs.basex.org/wiki/Full-Text#Mixed_Content > >
AB
Andy Bunce10 Aug 2017 23:28 #
It seems globally setting
indent=no gets applied to WebDAV (and everywhere else where serialization is not explicit specified). This would be my preference for WebDAV, as it means documents can be round-tripped without any changes being introduced. The only side effect from this setting I have seen is view-source on generated html source is harder to read, but this is not a real issue.
I have not tried setting them in web.xml yet. I wondered if you would expect it to work :-).
I will try...
Cheers
/AndyQuoted text · Christian Grün · 86 lines
On 10 August 2017 at 18:40, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Andy, > > > Can the WebDAV serialization be set independently of the default, in > > web.xml? > > The defaults for whitespace chopping and serialization can only be > assigned globally for all features of BaseX. Did you try to set both > 'org.basex.chop' and 'org.basex.serializer' in web.xml / does it > introduce other unwanted side effects? > > Cheers, > Christian > > > > /Andy > > > > On 7 August 2017 at 09:57, Christian Grün <christian.gruen at gmail.com> > wrote: > >> > >> Dear Ottid, > >> > >> Thanks for providing us with the helpful example, which helped me to > >> understand the problem. > >> > >> >> replace /a <root xml:space="preserve"><a><b>foo</b></a> bar</root> > >> > "a.xml" (Line 1): Open quote is expected for attribute "xml:space" > >> > associated with an element type "root". > >> > >> Just a side note: Command-line parsing is restrictive when it comes to > >> replacing XML. The reason is that is possible to send multiple > >> commands in a single line, as shown in the following example: > >> > >> create db db; replace /a <a/>; xquery . > >> > >> > >> >> xquery /root > >> > <root><a><b>foo</b></a> bar</root> > >> > >> You may be surprised to hear that whitespaces in your document were > >> actually chopped, and that the whitespaces are added by the > >> serializer, because the "indent" serialization parameter is by default > >> set to "yes". > >> > >> It was surprised to see that no one else pointed at this so far, and > >> that was not mentioned in our documentation, so I have just added some > >> explanatory lines [1,2]. > >> > >> Some more technical background: > >> > >> If you call BaseX the "info storage" command, you will see which XML > >> nodes are stored in the document: > >> > >> > set chop on;create db db <a> </a>; info storage > >> CHOP: true > >> Database 'db' created in 11.0 ms. > >> PRE DIS SIZ ATS ID NS KIND CONTENT > >> ----------------------------------------- > >> 0 1 2 1 0 0 DOC db.xml > >> 1 1 1 1 1 0 ELEM aIf > >> > >> > set chop off;create db db <a> </a>; info storage > >> CHOP: false > >> Database 'db' created in 20.12 ms. > >> PRE DIS SIZ ATS ID NS KIND CONTENT > >> ----------------------------------------- > >> 0 1 3 1 0 0 DOC db.xml > >> 1 1 2 1 1 0 ELEM a > >> 2 1 1 1 2 0 TEXT > >> > >> Serialization indentation was a chosen as default because it goes hand > >> in hand with the CHOP option. It even works fine if CHOP is disabled > >> if a document has whitespaces included (in which case no whitespaces > >> will be added by the serialized). But it may definitely cause > >> undesirable output if a document contains no superfluous whitespaces, > >> such as in your case. > >> > >> Hope this helps, > >> Christian > >> > >> [1] http://docs.basex.org/wiki/Options#CHOP > >> [2] http://docs.basex.org/wiki/Full-Text#Mixed_Content > > > > >
CG
Christian Grün11 Aug 2017 00:44 #
I agree that it might be reasonable to introduce different defaults for WebDAV communication. Problems could arise if documents are opened with WebDAV that have been stored via REST or another API… But we could give it a try.
Quoted text · Andy Bunce · 103 lines
On Thu, Aug 10, 2017 at 11:28 PM, Andy Bunce <bunce.andy at gmail.com> wrote: > It seems globally setting `indent=no` gets applied to WebDAV (and everywhere > else where serialization is not explicit specified). This would be my > preference for WebDAV, as it means documents can be round-tripped without > any changes being introduced. The only side effect from this setting I have > seen is view-source on generated html source is harder to read, but this is > not a real issue. > > I have not tried setting them in web.xml yet. I wondered if you would expect > it to work :-). > I will try... > > Cheers > /Andy > > On 10 August 2017 at 18:40, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Andy, >> >> > Can the WebDAV serialization be set independently of the default, in >> > web.xml? >> >> The defaults for whitespace chopping and serialization can only be >> assigned globally for all features of BaseX. Did you try to set both >> 'org.basex.chop' and 'org.basex.serializer' in web.xml / does it >> introduce other unwanted side effects? >> >> Cheers, >> Christian >> >> >> > /Andy >> > >> > On 7 August 2017 at 09:57, Christian Grün <christian.gruen at gmail.com> >> > wrote: >> >> >> >> Dear Ottid, >> >> >> >> Thanks for providing us with the helpful example, which helped me to >> >> understand the problem. >> >> >> >> >> replace /a <root xml:space="preserve"><a><b>foo</b></a> bar</root> >> >> > "a.xml" (Line 1): Open quote is expected for attribute "xml:space" >> >> > associated with an element type "root". >> >> >> >> Just a side note: Command-line parsing is restrictive when it comes to >> >> replacing XML. The reason is that is possible to send multiple >> >> commands in a single line, as shown in the following example: >> >> >> >> create db db; replace /a <a/>; xquery . >> >> >> >> >> >> >> xquery /root >> >> > <root><a><b>foo</b></a> bar</root> >> >> >> >> You may be surprised to hear that whitespaces in your document were >> >> actually chopped, and that the whitespaces are added by the >> >> serializer, because the "indent" serialization parameter is by default >> >> set to "yes". >> >> >> >> It was surprised to see that no one else pointed at this so far, and >> >> that was not mentioned in our documentation, so I have just added some >> >> explanatory lines [1,2]. >> >> >> >> Some more technical background: >> >> >> >> If you call BaseX the "info storage" command, you will see which XML >> >> nodes are stored in the document: >> >> >> >> > set chop on;create db db <a> </a>; info storage >> >> CHOP: true >> >> Database 'db' created in 11.0 ms. >> >> PRE DIS SIZ ATS ID NS KIND CONTENT >> >> ----------------------------------------- >> >> 0 1 2 1 0 0 DOC db.xml >> >> 1 1 1 1 1 0 ELEM aIf >> >> >> >> > set chop off;create db db <a> </a>; info storage >> >> CHOP: false >> >> Database 'db' created in 20.12 ms. >> >> PRE DIS SIZ ATS ID NS KIND CONTENT >> >> ----------------------------------------- >> >> 0 1 3 1 0 0 DOC db.xml >> >> 1 1 2 1 1 0 ELEM a >> >> 2 1 1 1 2 0 TEXT >> >> >> >> Serialization indentation was a chosen as default because it goes hand >> >> in hand with the CHOP option. It even works fine if CHOP is disabled >> >> if a document has whitespaces included (in which case no whitespaces >> >> will be added by the serialized). But it may definitely cause >> >> undesirable output if a document contains no superfluous whitespaces, >> >> such as in your case. >> >> >> >> Hope this helps, >> >> Christian >> >> >> >> [1] http://docs.basex.org/wiki/Options#CHOP >> >> [2] http://docs.basex.org/wiki/Full-Text#Mixed_Content >> > >> > > >
Written for BaseX 8.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery