csv:parse in the age of XQuery 3.1

Joe Wicentowski <joewiz at gmail.com> schrieb am 6:14 Donnerstag, 8.September 2016:

Marc van Grootel <marc.van.grootel at gmail.com> schrieb am 15:55 Donnerstag, 8.September 2016:

From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Hans-Juergen Rennau Sent: Thursday, September 08, 2016 10:02 AM To: Marc van Grootel <marc.van.grootel at gmail.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] csv:parse in the age of XQuery 3.1 What concerns me, I definitely want the CSV as XML. But the performance problems have certainly nothing to do with XML versus CSV (I often deal with > 300 MB XML, which is parsed very fast!) - it is the parsing operation itself which, if I'm not mistaken, is handled by XQuery code and which must be shifted into the Java implementation. Kind regards, Hans-Jürgen Marc van Grootel <marc.van.grootel at gmail.com<mailto:marc.van.grootel at gmail.com>> schrieb am 15:55 Donnerstag, 8.September 2016: I'm currently dealing with CSV a lot as well. I tend to use the format=map approach but not nearly as large as 22 MB CSV yet. I'm wondering if, or how much more efficient it is to deal with this type of data as arrays and map data structures versus XML. For most processing I can leave serializing to XML to the very end. And if too large I would probably also chunk it before storing the end result. Intuitively I would think that dealing with CSV as maps/arrays should be much faster and less memory intensive. --Marc

"Lizzi, Vincent" <Vincent.Lizzi at taylorandfrancis.com> schrieb am 18:53 Donnerstag, 8.September 2016:

From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Hans-Juergen Rennau Sent: Thursday, September 08, 2016 10:02 AM To: Marc van Grootel <marc.van.grootel at gmail.com> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] csv:parse in the age of XQuery 3.1 What concerns me, I definitely want the CSV as XML. But the performance problems have certainly nothing to do with XML versus CSV (I often deal with > 300 MB XML, which is parsed very fast!) - it is the parsing operation itself which, if I'm not mistaken, is handled by XQuery code and which must be shifted into the Java implementation. Kind regards, Hans-Jürgen Marc van Grootel <marc.van.grootel at gmail.com> schrieb am 15:55 Donnerstag, 8.September 2016: I'm currently dealing with CSV a lot as well. I tend to use the format=map approach but not nearly as large as 22 MB CSV yet. I'm wondering if, or how much more efficient it is to deal with this type of data as arrays and map data structures versus XML. For most processing I can leave serializing to XML to the very end. And if too large I would probably also chunk it before storing the end result. Intuitively I would think that dealing with CSV as maps/arrays should be much faster and less memory intensive. --Marc

On Thu, Sep 8, 2016 at 6:13 AM, Joe Wicentowski <joewiz at gmail.com> wrote: > Dear BaseX developers, > > I noticed in example 3 under > http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with > option { 'format': 'map' } returns a map of maps, with hardcoded row > numbers: > > map { > 1: map { > "City": "Newton", > "Name": "John" > }, > 2: map { > "City": "Oldtown", > "Name": "Jack" > } > } > > Using maps, which are unordered, to represent something ordered like > rows in a CSV, hardcoded row numbers are necessary for reassembling > the map in document order. I assume this was a necessary approach > when the module was developed in the map-only world of XQuery 3.0. > Now that 3.1 supports arrays, might an array of maps be a closer fit > for CSV parsing? > > array { > map { > "City": "Newton", > "Name": "John" > }, > map { > "City": "Oldtown", > "Name": "Jack" > } > } > > I'm also curious, do you know of any efforts to create an EXPath spec > for CSV? Putting spec and CSV in the same sentence is dangerous, > since CSV is a notoriously under-specified format: "The CSV file > format is not standardized" (see > https://en.wikipedia.org/wiki/Comma-separated_values). But perhaps > there is a common enough need for CSV parsing that such a spec would > benefit the community? I thought I'd start by asking here, since > BaseX's seems to be the most developed (or only?) CSV module in > XQuery. > > Then there's the question of how to approach implementations of such a > spec. While XQuery is probably capable of parsing and serializing > small enough CSV, CSVs do get large and naive processing with XQuery > would tend to run into memory issues (as I found with xqjson). This > means implementations would tend to write in a lower-level language. > eXist, for example, uses Jackson for fn:parse-json(). I see Jackson > has a CSV extension too: > https://github.com/FasterXML/jackson-dataformat-csv. Any thoughts on > the suitability of XQuery for the task? > > Joe

Joe Wicentowski <joewiz at gmail.com> schrieb am 21:27 Sonntag, 11.September 2016:

On Sun, Sep 11, 2016 at 3:39 PM -0400, "Hans-Juergen Rennau" <hrennau at yahoo.de> wrote:

Joe Wicentowski <joewiz at gmail.com> schrieb am 21:27 Sonntag, 11.September 2016:

Joe Wicentowski <joewiz at gmail.com> schrieb am 21:44 Sonntag, 11.September 2016:

On Sun, Sep 11, 2016 at 3:39 PM -0400, "Hans-Juergen Rennau" <hrennau at yahoo.de> wrote:

Joe Wicentowski <joewiz at gmail.com> schrieb am 21:27 Sonntag, 11.September 2016:

On Sun, Sep 11, 2016 at 9:44 PM, Joe Wicentowski <joewiz at gmail.com> wrote: > Hans-Jürgen, > > I figured as much. I wonder if we can come up with an xsd-compliant regex > for this purpose? It may not give us a full-featured CSV parser, but would > handle reasonably uniform cases. > > Joe > > Sent from my iPhone > > > > > On Sun, Sep 11, 2016 at 3:39 PM -0400, "Hans-Juergen Rennau" > <hrennau at yahoo.de> wrote: > >> Joe, concerning your regex, I would complain, too! Already the first two >> characters >> (? >> render the expression invalid: >> (1) An unescaped ? is an occurrence indicator, making the preceding entity >> optional >> (2) An unescaped ( is used for grouping, it does not repesent anything >> => there is no entity preceding the ? which the ? could make optional => >> error >> >> Please keep in mind that the regex flavor supported by XPath is the regex >> flavor defined by the XSD spec. There are a few constructs used in Perl & Co >> which are not defined in XPath regex. >> >> What concerns the CSV implementation, I came to realize my error: the >> BaseX implementation *is* Java code, not XQuery code - the xqm module just >> contains the function signature, marked "external". >> >> Cheers, >> Hans >> >> >> Joe Wicentowski <joewiz at gmail.com> schrieb am 21:27 Sonntag, 11.September >> 2016: >> >> >> Thanks for your replies and interest, Hans-Jürgen, Marc, Vincent, and >> Christian. >> >> The other day, short of a comprehensive solution, I went in search of >> a regex that would handle quoted values that contain commas that >> shouldn't serve as delimiters. I found one that worked in eXist but >> not in BaseX. >> >> Source for the regex: http://stackoverflow.com/a/13259681/659732 >> >> The query: >> >> ``` >> xquery version "3.1"; >> >> let $csv := 'Author,Title,ISBN,Binding,Year Published >> Jeannette Walls,The Glass Castle,074324754X,Paperback,2006 >> James Surowiecki,The Wisdom of Crowds,9780385503860,Paperback,2005 >> Lawrence Lessig,The Future of Ideas,9780375505782,Paperback,2002 >> "Larry Bossidy, Ram Charan, Charles >> Burck",Execution,9780609610572,Hardcover,2002 >> Kurt Vonnegut,Slaughterhouse-Five,9780791059258,Paperback,1999' >> let $lines := tokenize($csv, '\n') >> let $header-row := fn:head($lines) >> let $body-rows := fn:tail($lines) >> let $headers := fn:tokenize($header-row, ",") ! fn:replace(., " ", "") >> for $row in $body-rows >> let $cells := fn:analyze-string($row, >> '(?:\s*(?:\"([^\"]*)\"|([^,]+))\s*,?|(?<=,)(),?)+?')//fn:group >> return >> element Book { >> for $cell at $count in $cells >> return element {$headers[$count]} {$cell/string()} >> } >> It produces the desired results: >> >> <Book> >> <Author>Jeannette Walls</Author> >> <Title>The Glass Castle</Title> >> <ISBN>074324754X</ISBN> >> <Binding>Paperback</Binding> >> <YearPublished>2006</YearPublished> >> </Book> >> <Book> >> <Author>James Surowiecki</Author> >> <Title>The Wisdom of Crowds</Title> >> <ISBN>9780385503860</ISBN> >> <Binding>Paperback</Binding> >> <YearPublished>2005</YearPublished> >> </Book> >> <Book> >> <Author>Lawrence Lessig</Author> >> <Title>The Future of Ideas</Title> >> <ISBN>9780375505782</ISBN> >> <Binding>Paperback</Binding> >> <YearPublished>2002</YearPublished> >> </Book> >> <Book> >> <Author>Larry Bossidy, Ram Charan, Charles Burck</Author> >> <Title>Execution</Title> >> <ISBN>9780609610572</ISBN> >> <Binding>Hardcover</Binding> >> <YearPublished>2002</YearPublished> >> </Book> >> <Book> >> <Author>Kurt Vonnegut</Author> >> <Title>Slaughterhouse-Five</Title> >> <ISBN>9780791059258</ISBN> >> <Binding>Paperback</Binding> >> <YearPublished>1999</YearPublished> >> </Book> >> >> Unfortunately BaseX complains about the regex, with the following error: >> >> Stopped at /Users/joe/file, 9/32: [FORX0002] Invalid regular >> expression: (?:\s(?:\"([^\"])\"|([^,]+))\s*,?|(?<=,)(),?)+?. >> >> Without a column location, I'm unable to tell where the problem is. >> Is there something used in this expression that BaseX doesn't support? >> >> On the topic of the potential memory pitfalls of a pure XQuery >> solution for our hypothetical EXPath library, I think the primary >> problem is that the entire CSV has to be loaded into memory. I wonder >> if implementations could use the new `fn:unparsed-text-lines()` >> function from XQuery 3.0 to stream the CSV through XQuery without >> requiring the entire thing to be in memory? Or are we basically >> setting ourselves up for the EXPath solution being a wrapper around an >> external library written in a lower level language? >> >> Joe >> >> On Sun, Sep 11, 2016 at 4:53 AM, Christian Grün >> <christian.gruen at gmail.com> wrote: >> > Hi Joe, >> > >> > Thanks for your mail. You are completely right, using an array would >> > be the natural choice with csv:parse. It’s mostly due to backward >> > compatibility that we didn’t update the function. >> > >> > @All: I’m pretty sure that all of us would like having an EXPath spec >> > for parsing CSV data. We still need one volunteer to make it happen ;) >> > Anyone out there? >> > >> > Cheers >> > Christian >> > >> > >> > On Thu, Sep 8, 2016 at 6:13 AM, Joe Wicentowski <joewiz at gmail.com> >> > wrote: >> >> Dear BaseX developers, >> >> >> >> I noticed in example 3 under >> >> http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with >> >> option { 'format': 'map' } returns a map of maps, with hardcoded row >> >> numbers: >> >> >> >> map { >> >> 1: map { >> >> "City": "Newton", >> >> "Name": "John" >> >> }, >> >> 2: map { >> >> "City": "Oldtown", >> >> "Name": "Jack" >> >> } >> >> } >> >> >> >> Using maps, which are unordered, to represent something ordered like >> >> rows in a CSV, hardcoded row numbers are necessary for reassembling >> >> the map in document order. I assume this was a necessary approach >> >> when the module was developed in the map-only world of XQuery 3.0. >> >> Now that 3.1 supports arrays, might an array of maps be a closer fit >> >> for CSV parsing? >> >> >> >> array { >> >> map { >> >> "City": "Newton", >> >> "Name": "John" >> >> }, >> >> map { >> >> "City": "Oldtown", >> >> "Name": "Jack" >> >> } >> >> } >> >> >> >> I'm also curious, do you know of any efforts to create an EXPath spec >> >> for CSV? Putting spec and CSV in the same sentence is dangerous, >> >> since CSV is a notoriously under-specified format: "The CSV file >> >> format is not standardized" (see >> >> https://en.wikipedia.org/wiki/Comma-separated_values). But perhaps >> >> there is a common enough need for CSV parsing that such a spec would >> >> benefit the community? I thought I'd start by asking here, since >> >> BaseX's seems to be the most developed (or only?) CSV module in >> >> XQuery. >> >> >> >> Then there's the question of how to approach implementations of such a >> >> spec. While XQuery is probably capable of parsing and serializing >> >> small enough CSV, CSVs do get large and naive processing with XQuery >> >> would tend to run into memory issues (as I found with xqjson). This >> >> means implementations would tend to write in a lower-level language. >> >> eXist, for example, uses Jackson for fn:parse-json(). I see Jackson >> >> has a CSV extension too: >> >> https://github.com/FasterXML/jackson-dataformat-csv. Any thoughts on >> >> the suitability of XQuery for the task? >> >> >> >> Joe >> >> >

On Sun, Sep 11, 2016 at 10:23 PM, Hans-Juergen Rennau <hrennau at yahoo.de> wrote: > Joe, just in case it is of interest to you: the TopicTools framework, > downloadable at > > https://github.com/hrennau/topictools > > contains an XQuery-implemented, full-featured csv parser (module > _csvParser.xqm, 212 lines). Writing XQuery tools using the framework, the > parser is automatically added to your application code and you can declare > command-line parameters of your tool to have (among many others) a csv-based > data type. Having declared the parameter to have such a type (e.g. csvURI or > csvFOX), you can forget csv, as your application code sees nothing else but > XML. More offline, if this is interesting to you. > > Cheers, > Hans > > PS - illustrating ... > > Tool invocations: > basex -b "request=csve?cox=joe.csv<header=true" /mytool/mytool.xq > basex -b "request=csve?cox=/foo//(*foobar*.csv except > betternot*.csv)<header=true" /mytool/mytool.xq > > Parameter declaration (within a module of the mytool app): > <param name="cox" type="csvFOX?" sep="WS" pgroup="input"/> > > Application code (within a module of the mytool app): > let $docs := tt:getParams($request, 'cox') > (: here you go - $docs is bound to the XML documents obtained by parsing > all files matching the FOXpath expression > supplied by the caller into an XML representation; ...) > > > > > Joe Wicentowski <joewiz at gmail.com> schrieb am 21:44 Sonntag, 11.September > 2016: > > > Hans-Jürgen, > > I figured as much. I wonder if we can come up with an xsd-compliant regex > for this purpose? It may not give us a full-featured CSV parser, but would > handle reasonably uniform cases. > > Joe > > Sent from my iPhone > > > > > On Sun, Sep 11, 2016 at 3:39 PM -0400, "Hans-Juergen Rennau" > <hrennau at yahoo.de> wrote: > > Joe, concerning your regex, I would complain, too! Already the first two > characters > (? > render the expression invalid: > (1) An unescaped ? is an occurrence indicator, making the preceding entity > optional > (2) An unescaped ( is used for grouping, it does not repesent anything > => there is no entity preceding the ? which the ? could make optional => > error > > Please keep in mind that the regex flavor supported by XPath is the regex > flavor defined by the XSD spec. There are a few constructs used in Perl & Co > which are not defined in XPath regex. > > What concerns the CSV implementation, I came to realize my error: the BaseX > implementation *is* Java code, not XQuery code - the xqm module just > contains the function signature, marked "external". > > Cheers, > Hans > > > Joe Wicentowski <joewiz at gmail.com> schrieb am 21:27 Sonntag, 11.September > 2016: > > > Thanks for your replies and interest, Hans-Jürgen, Marc, Vincent, and > Christian. > > The other day, short of a comprehensive solution, I went in search of > a regex that would handle quoted values that contain commas that > shouldn't serve as delimiters. I found one that worked in eXist but > not in BaseX. > > Source for the regex: http://stackoverflow.com/a/13259681/659732 > > The query: > > ``` > xquery version "3.1"; > > let $csv := 'Author,Title,ISBN,Binding,Year Published > Jeannette Walls,The Glass Castle,074324754X,Paperback,2006 > James Surowiecki,The Wisdom of Crowds,9780385503860,Paperback,2005 > Lawrence Lessig,The Future of Ideas,9780375505782,Paperback,2002 > "Larry Bossidy, Ram Charan, Charles > Burck",Execution,9780609610572,Hardcover,2002 > Kurt Vonnegut,Slaughterhouse-Five,9780791059258,Paperback,1999' > let $lines := tokenize($csv, '\n') > let $header-row := fn:head($lines) > let $body-rows := fn:tail($lines) > let $headers := fn:tokenize($header-row, ",") ! fn:replace(., " ", "") > for $row in $body-rows > let $cells := fn:analyze-string($row, > '(?:\s*(?:\"([^\"]*)\"|([^,]+))\s*,?|(?<=,)(),?)+?')//fn:group > return > element Book { > for $cell at $count in $cells > return element {$headers[$count]} {$cell/string()} > } > It produces the desired results: > > <Book> > <Author>Jeannette Walls</Author> > <Title>The Glass Castle</Title> > <ISBN>074324754X</ISBN> > <Binding>Paperback</Binding> > <YearPublished>2006</YearPublished> > </Book> > <Book> > <Author>James Surowiecki</Author> > <Title>The Wisdom of Crowds</Title> > <ISBN>9780385503860</ISBN> > <Binding>Paperback</Binding> > <YearPublished>2005</YearPublished> > </Book> > <Book> > <Author>Lawrence Lessig</Author> > <Title>The Future of Ideas</Title> > <ISBN>9780375505782</ISBN> > <Binding>Paperback</Binding> > <YearPublished>2002</YearPublished> > </Book> > <Book> > <Author>Larry Bossidy, Ram Charan, Charles Burck</Author> > <Title>Execution</Title> > <ISBN>9780609610572</ISBN> > <Binding>Hardcover</Binding> > <YearPublished>2002</YearPublished> > </Book> > <Book> > <Author>Kurt Vonnegut</Author> > <Title>Slaughterhouse-Five</Title> > <ISBN>9780791059258</ISBN> > <Binding>Paperback</Binding> > <YearPublished>1999</YearPublished> > </Book> > > Unfortunately BaseX complains about the regex, with the following error: > > Stopped at /Users/joe/file, 9/32: [FORX0002] Invalid regular > expression: (?:\s(?:\"([^\"])\"|([^,]+))\s*,?|(?<=,)(),?)+?. > > Without a column location, I'm unable to tell where the problem is. > Is there something used in this expression that BaseX doesn't support? > > On the topic of the potential memory pitfalls of a pure XQuery > solution for our hypothetical EXPath library, I think the primary > problem is that the entire CSV has to be loaded into memory. I wonder > if implementations could use the new `fn:unparsed-text-lines()` > function from XQuery 3.0 to stream the CSV through XQuery without > requiring the entire thing to be in memory? Or are we basically > setting ourselves up for the EXPath solution being a wrapper around an > external library written in a lower level language? > > Joe > > On Sun, Sep 11, 2016 at 4:53 AM, Christian Grün > <christian.gruen at gmail.com> wrote: >> Hi Joe, >> >> Thanks for your mail. You are completely right, using an array would >> be the natural choice with csv:parse. It’s mostly due to backward >> compatibility that we didn’t update the function. >> >> @All: I’m pretty sure that all of us would like having an EXPath spec >> for parsing CSV data. We still need one volunteer to make it happen ;) >> Anyone out there? >> >> Cheers >> Christian >> >> >> On Thu, Sep 8, 2016 at 6:13 AM, Joe Wicentowski <joewiz at gmail.com> wrote: >>> Dear BaseX developers, >>> >>> I noticed in example 3 under >>> http://docs.basex.org/wiki/CSV_Module#Examples that csv:parse() with >>> option { 'format': 'map' } returns a map of maps, with hardcoded row >>> numbers: >>> >>> map { >>> 1: map { >>> "City": "Newton", >>> "Name": "John" >>> }, >>> 2: map { >>> "City": "Oldtown", >>> "Name": "Jack" >>> } >>> } >>> >>> Using maps, which are unordered, to represent something ordered like >>> rows in a CSV, hardcoded row numbers are necessary for reassembling >>> the map in document order. I assume this was a necessary approach >>> when the module was developed in the map-only world of XQuery 3.0. >>> Now that 3.1 supports arrays, might an array of maps be a closer fit >>> for CSV parsing? >>> >>> array { >>> map { >>> "City": "Newton", >>> "Name": "John" >>> }, >>> map { >>> "City": "Oldtown", >>> "Name": "Jack" >>> } >>> } >>> >>> I'm also curious, do you know of any efforts to create an EXPath spec >>> for CSV? Putting spec and CSV in the same sentence is dangerous, >>> since CSV is a notoriously under-specified format: "The CSV file >>> format is not standardized" (see >>> https://en.wikipedia.org/wiki/Comma-separated_values). But perhaps >>> there is a common enough need for CSV parsing that such a spec would >>> benefit the community? I thought I'd start by asking here, since >>> BaseX's seems to be the most developed (or only?) CSV module in >>> XQuery. >>> >>> Then there's the question of how to approach implementations of such a >>> spec. While XQuery is probably capable of parsing and serializing >>> small enough CSV, CSVs do get large and naive processing with XQuery >>> would tend to run into memory issues (as I found with xqjson). This >>> means implementations would tend to write in a lower-level language. >>> eXist, for example, uses Jackson for fn:parse-json(). I see Jackson >>> has a CSV extension too: >>> https://github.com/FasterXML/jackson-dataformat-csv. Any thoughts on >>> the suitability of XQuery for the task? >>> >>> Joe > > > >

Hans-Jürgen, wrote: ! Already the first > two characters > (?render the expression invalid:(1) An unescaped ? is an > occurrence indicator, making the preceding entity optional(2) An > unescaped ( is used for grouping, it does not repesent anything > => there is no entity preceding the ? which the ? could make optional > => error

Liam R. E. Quin <liam at w3.org> schrieb am 5:54 Montag, 12.September 2016: Hans-Jürgen, wrote: ! Already the first > two characters > (?render the expression invalid:(1) An unescaped ? is an > occurrence indicator, making the preceding entity optional(2) An > unescaped ( is used for grouping, it does not repesent anything > => there is no entity preceding the ? which the ? could make optional > => error

On Mon, Sep 12, 2016 at 1:15 PM, Hans-Juergen Rennau <hrennau at yahoo.de> wrote: > Cordial thanks, Liam - I was not aware of that! > > @Joe: Rule of life: when one is especially sure to be right, one is surely > wrong, and so was I, and right were you(r first two characters). > > > Liam R. E. Quin <liam at w3.org> schrieb am 5:54 Montag, 12.September 2016: > > > Hans-Jürgen, wrote: > > ! Already the first >> two characters >> (?render the expression invalid:(1) An unescaped ? is an >> occurrence indicator, making the preceding entity optional(2) An >> unescaped ( is used for grouping, it does not repesent anything >> => there is no entity preceding the ? which the ? could make optional >> => error > > > Actually (?: .... ) is a non-capturing group, defined in XPath 3.0 and > XQuery 3.0, based on the same syntax in other languages. > > This extension, like a number of others, is useful because the > expression syntax defined by XSD doesn't make use of capturing groups > (there's no \1 or $1 or whatever), and so it doesn't need non-capturing > groups, but in XPath and XQuery they are used. > > See e.g. https://www.w3.org/TR/xpath-functions-30/#regex-syntax > > Liam > > > -- > Liam R. E. Quin <liam at w3.org> > The World Wide Web Consortium (W3C) > > >

> Christian: I tried removing the quote escaping but still get an error. > Here's a small test to reproduce: > > fn:analyze-string($row, '(?:\s*(?:"([^"]*)"|([^,]+))\s*,?|(?<=,)(),?)+?')

On Mon, Sep 12, 2016 at 7:29 AM, Christian Grün <christian.gruen at gmail.com> wrote: > I didn’t check the regex in general, but one reason I think why it > fails is the escaped quote. For example, the following query is > illegal in XQuery 3.1… > > matches('a"b', 'a\"b') > > …where as the following one is ok: > > matches('a"b', 'a"b') > > > > On Mon, Sep 12, 2016 at 1:15 PM, Hans-Juergen Rennau <hrennau at yahoo.de> wrote: >> Cordial thanks, Liam - I was not aware of that! >> >> @Joe: Rule of life: when one is especially sure to be right, one is surely >> wrong, and so was I, and right were you(r first two characters). >> >> >> Liam R. E. Quin <liam at w3.org> schrieb am 5:54 Montag, 12.September 2016: >> >> >> Hans-Jürgen, wrote: >> >> ! Already the first >>> two characters >>> (?render the expression invalid:(1) An unescaped ? is an >>> occurrence indicator, making the preceding entity optional(2) An >>> unescaped ( is used for grouping, it does not repesent anything >>> => there is no entity preceding the ? which the ? could make optional >>> => error >> >> >> Actually (?: .... ) is a non-capturing group, defined in XPath 3.0 and >> XQuery 3.0, based on the same syntax in other languages. >> >> This extension, like a number of others, is useful because the >> expression syntax defined by XSD doesn't make use of capturing groups >> (there's no \1 or $1 or whatever), and so it doesn't need non-capturing >> groups, but in XPath and XQuery they are used. >> >> See e.g. https://www.w3.org/TR/xpath-functions-30/#regex-syntax >> >> Liam >> >> >> -- >> Liam R. E. Quin <liam at w3.org> >> The World Wide Web Consortium (W3C) >> >> >>

On Mon, Sep 12, 2016 at 10:11 AM, Christian Grün <christian.gruen at gmail.com> wrote: >> Christian: I tried removing the quote escaping but still get an error. >> Here's a small test to reproduce: >> >> fn:analyze-string($row, '(?:\s*(?:"([^"]*)"|([^,]+))\s*,?|(?<=,)(),?)+?') > > I assume it’s the lookbehind assertion that is not allowed in XQuery > (but I should definitely spend more time on it to give you a better > answer..).