slow GUI-editor when using package-manager eXPath

On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > > > New tools, new challenges. > > I’ve rearranged my source-code in packages/libraries conform the > eXPath-specs in BaseX now. > > But since than the GUI-editor is becoming very slow, even unworkably slow. > > What is causing this and can you give me any tips how speed up the > GUI-editor? > > > > I’ve got 7 libraries now. The one with the largest interface counts 11 > entries in file:expath-pkg.xml, all public entries, the smallest library > counts one entry. > > > > TIA, > > Rob

Van: Christian Grün [mailto:christian.gruen at gmail.com] Verzonden: donderdag 10 maart 2016 10:53 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath Hi Rob, you could try to disable automatic parsing of XQuery files via the Preference Dialog (see screenshot). Does this help? Apart from that, feel free to directly send me your sources, and I can have a second look. Cheers, Christian On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > > > New tools, new challenges. > > I’ve rearranged my source-code in packages/libraries conform the > eXPath-specs in BaseX now. > > But since than the GUI-editor is becoming very slow, even unworkably slow. > > What is causing this and can you give me any tips how speed up the > GUI-editor? > > > > I’ve got 7 libraries now. The one with the largest interface counts 11 > entries in file:expath-pkg.xml, all public entries, the smallest > library counts one entry. > > > > TIA, > > Rob

On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > New tools, new challenges. > > I’ve rearranged my source-code in packages/libraries conform the > eXPath-specs in BaseX now. > > But since than the GUI-editor is becoming very slow, even unworkably slow. > > What is causing this and can you give me any tips how speed up the > GUI-editor? > > > > I’ve got 7 libraries now. The one with the largest interface counts 11 > entries in file:expath-pkg.xml, all public entries, the smallest library > counts one entry. > > > > TIA, > > Rob

Van: Christian Grün [mailto:christian.gruen at gmail.com] Verzonden: vrijdag 1 april 2016 14:06 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath Hi Rob, I managed to play around with BaseX on an older machine (Celeron, 1.3 GHz, 224 MB RAM), and I was surprised to see it worked pretty well. Maybe it was due to Windows XP, though, and the fact that it was a fresh installation. I had a closer look at the files you sent to me, and the most noticeable delays I noticed happened when editing an XQuery file with numerous module imports. The reason for that is that all the imported modules need to be parsed with each key click. Spontaneously, I got two ideas how this could be improved: * XQuery parsing could be delayed for some milliseconds. This way, it would only be parsed if a user has (presumably) finished his input. * An obsolete query parsing thread could be stopped before a new parsing thread is started. * The least thrilling solution: A new option in the Preferences dialog to disable realtime parsing, and only parse if a file is saved. I’ll see what can be done. Cheers, Christian On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > New tools, new challenges. > > I’ve rearranged my source-code in packages/libraries conform the > eXPath-specs in BaseX now. > > But since than the GUI-editor is becoming very slow, even unworkably slow. > > What is causing this and can you give me any tips how speed up the > GUI-editor? > > > > I’ve got 7 libraries now. The one with the largest interface counts 11 > entries in file:expath-pkg.xml, all public entries, the smallest > library counts one entry. > > > > TIA, > > Rob

On Fri, Apr 1, 2016 at 3:11 PM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > Thanx for having a closer look at it. > > I have already abandoned the ex-path package mechanism, it just was to slow to work with on my computer. > Now I can type away once again. Once in a while my computer has to take a breath, probably updating the project files, and I'll have to wait for a second, but this doesn't bother me. > The problem with the ex-path mechanism was that I had to wait with every keystroke. > It looks to me that the timing-mechanism when using of ex-path is different than without using ex-path. > > So if you can do something about the timing, just like you proposed below, I could again use ex-path. > > In the meanwhile I've been thinking about reorganizing my modules. > Maybe it would be good practice if each package is accessed by just one interface module which addresses the other modules through a relative path. I haven't tested this > > Let me know if you have figured something out, I'll be happy to test it for you. > > Take care, > Rob > > -----Oorspronkelijk bericht----- > Van: Christian Grün [mailto:christian.gruen at gmail.com] > Verzonden: vrijdag 1 april 2016 14:06 > Aan: Rob Stapper > CC: BaseX > Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath > > Hi Rob, > > I managed to play around with BaseX on an older machine (Celeron, 1.3 GHz, 224 MB RAM), and I was surprised to see it worked pretty well. > Maybe it was due to Windows XP, though, and the fact that it was a fresh installation. > > I had a closer look at the files you sent to me, and the most noticeable delays I noticed happened when editing an XQuery file with numerous module imports. The reason for that is that all the imported modules need to be parsed with each key click. > > Spontaneously, I got two ideas how this could be improved: > > * XQuery parsing could be delayed for some milliseconds. This way, it would only be parsed if a user has (presumably) finished his input. > > * An obsolete query parsing thread could be stopped before a new parsing thread is started. > > * The least thrilling solution: A new option in the Preferences dialog to disable realtime parsing, and only parse if a file is saved. > > I’ll see what can be done. > > Cheers, > Christian > > > > On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: >> Hi Christian, >> >> New tools, new challenges. >> >> I’ve rearranged my source-code in packages/libraries conform the >> eXPath-specs in BaseX now. >> >> But since than the GUI-editor is becoming very slow, even unworkably slow. >> >> What is causing this and can you give me any tips how speed up the >> GUI-editor? >> >> >> >> I’ve got 7 libraries now. The one with the largest interface counts 11 >> entries in file:expath-pkg.xml, all public entries, the smallest >> library counts one entry. >> >> >> >> TIA, >> >> Rob >

Van: Christian Grün [mailto:christian.gruen at gmail.com] Verzonden: vrijdag 1 april 2016 15:47 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath Hi Rob, I think I’ve found a solution that does justice to both older and newer systems: query parsing will now be generally delayed for some ms), and only one query parsing process will be active at the same time (before, every key click triggered an extra thread). Even if you have switched to another mechanism now, could you give it a try [1]? Looking forward to your feedback, Christian [1] http://files.basex.org/releases/latest On Fri, Apr 1, 2016 at 3:11 PM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > Thanx for having a closer look at it. > > I have already abandoned the ex-path package mechanism, it just was to slow to work with on my computer. > Now I can type away once again. Once in a while my computer has to take a breath, probably updating the project files, and I'll have to wait for a second, but this doesn't bother me. > The problem with the ex-path mechanism was that I had to wait with every keystroke. > It looks to me that the timing-mechanism when using of ex-path is different than without using ex-path. > > So if you can do something about the timing, just like you proposed below, I could again use ex-path. > > In the meanwhile I've been thinking about reorganizing my modules. > Maybe it would be good practice if each package is accessed by just > one interface module which addresses the other modules through a > relative path. I haven't tested this > > Let me know if you have figured something out, I'll be happy to test it for you. > > Take care, > Rob > > -----Oorspronkelijk bericht----- > Van: Christian Grün [mailto:christian.gruen at gmail.com] > Verzonden: vrijdag 1 april 2016 14:06 > Aan: Rob Stapper > CC: BaseX > Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager > eXPath > > Hi Rob, > > I managed to play around with BaseX on an older machine (Celeron, 1.3 GHz, 224 MB RAM), and I was surprised to see it worked pretty well. > Maybe it was due to Windows XP, though, and the fact that it was a fresh installation. > > I had a closer look at the files you sent to me, and the most noticeable delays I noticed happened when editing an XQuery file with numerous module imports. The reason for that is that all the imported modules need to be parsed with each key click. > > Spontaneously, I got two ideas how this could be improved: > > * XQuery parsing could be delayed for some milliseconds. This way, it would only be parsed if a user has (presumably) finished his input. > > * An obsolete query parsing thread could be stopped before a new parsing thread is started. > > * The least thrilling solution: A new option in the Preferences dialog to disable realtime parsing, and only parse if a file is saved. > > I’ll see what can be done. > > Cheers, > Christian > > > > On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: >> Hi Christian, >> >> New tools, new challenges. >> >> I’ve rearranged my source-code in packages/libraries conform the >> eXPath-specs in BaseX now. >> >> But since than the GUI-editor is becoming very slow, even unworkably slow. >> >> What is causing this and can you give me any tips how speed up the >> GUI-editor? >> >> >> >> I’ve got 7 libraries now. The one with the largest interface counts >> 11 entries in file:expath-pkg.xml, all public entries, the smallest >> library counts one entry. >> >> >> >> TIA, >> >> Rob >

Van: Christian Grün [mailto:christian.gruen at gmail.com] Verzonden: vrijdag 1 april 2016 15:47 Aan: Rob Stapper CC: BaseX Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager eXPath Hi Rob, I think I’ve found a solution that does justice to both older and newer systems: query parsing will now be generally delayed for some ms), and only one query parsing process will be active at the same time (before, every key click triggered an extra thread). Even if you have switched to another mechanism now, could you give it a try [1]? Looking forward to your feedback, Christian [1] http://files.basex.org/releases/latest On Fri, Apr 1, 2016 at 3:11 PM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: > Hi Christian, > > Thanx for having a closer look at it. > > I have already abandoned the ex-path package mechanism, it just was to slow to work with on my computer. > Now I can type away once again. Once in a while my computer has to take a breath, probably updating the project files, and I'll have to wait for a second, but this doesn't bother me. > The problem with the ex-path mechanism was that I had to wait with every keystroke. > It looks to me that the timing-mechanism when using of ex-path is different than without using ex-path. > > So if you can do something about the timing, just like you proposed below, I could again use ex-path. > > In the meanwhile I've been thinking about reorganizing my modules. > Maybe it would be good practice if each package is accessed by just > one interface module which addresses the other modules through a > relative path. I haven't tested this > > Let me know if you have figured something out, I'll be happy to test it for you. > > Take care, > Rob > > -----Oorspronkelijk bericht----- > Van: Christian Grün [mailto:christian.gruen at gmail.com] > Verzonden: vrijdag 1 april 2016 14:06 > Aan: Rob Stapper > CC: BaseX > Onderwerp: Re: [basex-talk] slow GUI-editor when using package-manager > eXPath > > Hi Rob, > > I managed to play around with BaseX on an older machine (Celeron, 1.3 GHz, 224 MB RAM), and I was surprised to see it worked pretty well. > Maybe it was due to Windows XP, though, and the fact that it was a fresh installation. > > I had a closer look at the files you sent to me, and the most noticeable delays I noticed happened when editing an XQuery file with numerous module imports. The reason for that is that all the imported modules need to be parsed with each key click. > > Spontaneously, I got two ideas how this could be improved: > > * XQuery parsing could be delayed for some milliseconds. This way, it would only be parsed if a user has (presumably) finished his input. > > * An obsolete query parsing thread could be stopped before a new parsing thread is started. > > * The least thrilling solution: A new option in the Preferences dialog to disable realtime parsing, and only parse if a file is saved. > > I’ll see what can be done. > > Cheers, > Christian > > > > On Thu, Mar 10, 2016 at 9:45 AM, Rob Stapper <r.stapper at lijbrandt.nl> wrote: >> Hi Christian, >> >> New tools, new challenges. >> >> I’ve rearranged my source-code in packages/libraries conform the >> eXPath-specs in BaseX now. >> >> But since than the GUI-editor is becoming very slow, even unworkably slow. >> >> What is causing this and can you give me any tips how speed up the >> GUI-editor? >> >> >> >> I’ve got 7 libraries now. The one with the largest interface counts >> 11 entries in file:expath-pkg.xml, all public entries, the smallest >> library counts one entry. >> >> >> >> TIA, >> >> Rob >

On Fri, Apr 1, 2016 at 5:24 PM, Graydon Saunders <graydonish at gmail.com> wrote: > Hello - > > I've got a problem I'm not sure how to best approach. > > I've got triplets of names -- class.operation.specifier -- that I need to > match against much longer sequences of names. (Which are in attributes in an > XML hierarchy; each sequence of names derives from a path to a leaf > element.) > > If there is a match (as there usually is not) one of the names in the > sequence of names will match to the class, a subsequent name to the > operation, and a name subsequent to that match to the specifier. (All > simple string values.) > > The naive n^2 version is much too slow for the amount of data involved. > > Is there an efficient way to do this kind of matching? > > Thanks! > Graydon

On Fri, Apr 1, 2016 at 12:04 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Graydon, > > Do you think there’d be a chance for us to get a minimized, > self-contained example, which demonstrates the n^2 solution? > > Thanks in advance, > Christian > > > > On Fri, Apr 1, 2016 at 5:24 PM, Graydon Saunders <graydonish at gmail.com> > wrote: > > Hello - > > > > I've got a problem I'm not sure how to best approach. > > > > I've got triplets of names -- class.operation.specifier -- that I need to > > match against much longer sequences of names. (Which are in attributes > in an > > XML hierarchy; each sequence of names derives from a path to a leaf > > element.) > > > > If there is a match (as there usually is not) one of the names in the > > sequence of names will match to the class, a subsequent name to the > > operation, and a name subsequent to that match to the specifier. (All > > simple string values.) > > > > The naive n^2 version is much too slow for the amount of data involved. > > > > Is there an efficient way to do this kind of matching? > > > > Thanks! > > Graydon >

> I can't give you a real example because it's the client's health care data,

> let $found := //*[@name eq $match(1)][./descendant::*[@name eq > $match(2)][./descendant::*[@name eq $match(3)]]]

> > This works, but it's going over the entire database for every three part > class-operation-specifier compound name. I can't shake the feeling that > there's a more efficient way to do this, but I can't see what it might be. > > Thanks! > Graydon > > On Fri, Apr 1, 2016 at 12:04 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Graydon, >> >> Do you think there’d be a chance for us to get a minimized, >> self-contained example, which demonstrates the n^2 solution? >> >> Thanks in advance, >> Christian >> >> >> >> On Fri, Apr 1, 2016 at 5:24 PM, Graydon Saunders <graydonish at gmail.com> >> wrote: >> > Hello - >> > >> > I've got a problem I'm not sure how to best approach. >> > >> > I've got triplets of names -- class.operation.specifier -- that I need >> > to >> > match against much longer sequences of names. (Which are in attributes >> > in an >> > XML hierarchy; each sequence of names derives from a path to a leaf >> > element.) >> > >> > If there is a match (as there usually is not) one of the names in the >> > sequence of names will match to the class, a subsequent name to the >> > operation, and a name subsequent to that match to the specifier. (All >> > simple string values.) >> > >> > The naive n^2 version is much too slow for the amount of data involved. >> > >> > Is there an efficient way to do this kind of matching? >> > >> > Thanks! >> > Graydon > >

On Mon, Apr 4, 2016 at 1:27 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Graydon, > > > I can't give you a real example because it's the client's health care > data, > > No problem, your example looks fine. > > > let $found := //*[@name eq $match(1)][./descendant::*[@name eq > > $match(2)][./descendant::*[@name eq $match(3)]]] > > Right. You could try to rewrite this for index access: > > 1. You’ll have to mark the generated arrays as string arrays: > > let $composedNames as array(xs:string) := > for $x in $composed//composed > return array { tokenize($x/string(),'\.') } > > 2. You need to replace "eq" with "=", and you can simplify the > predicates a little: > > let $found := //*[@name = $match(1)] > [descendant::*/@name = $match(2)] > [descendant::*/@name = $match(3)] > > You indicated that you’ll have thousands of paths. How do they look > like? Could you add some more examples (besides > "class.operation.specifier")? Are some parts of the paths more > specific than others? E.g... > > A.A.A > A.A.B > A.A.C > A.B.D > A.B.E > A.B.F > ... > > In this case, it could make sense to only look for the last path > segment via the index. You could also try to group your results by the > first segment, then do the search on the second segment, etc. See my > attached query as example (I’m sure it needs to be revised to work > properly, because I have only run it with your simple example file). > > Does this help? > Christian > > > > > > > > This works, but it's going over the entire database for every three part > > class-operation-specifier compound name. I can't shake the feeling that > > there's a more efficient way to do this, but I can't see what it might > be. > > > > Thanks! > > Graydon > > > > On Fri, Apr 1, 2016 at 12:04 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> Hi Graydon, > >> > >> Do you think there’d be a chance for us to get a minimized, > >> self-contained example, which demonstrates the n^2 solution? > >> > >> Thanks in advance, > >> Christian > >> > >> > >> > >> On Fri, Apr 1, 2016 at 5:24 PM, Graydon Saunders <graydonish at gmail.com> > >> wrote: > >> > Hello - > >> > > >> > I've got a problem I'm not sure how to best approach. > >> > > >> > I've got triplets of names -- class.operation.specifier -- that I need > >> > to > >> > match against much longer sequences of names. (Which are in attributes > >> > in an > >> > XML hierarchy; each sequence of names derives from a path to a leaf > >> > element.) > >> > > >> > If there is a match (as there usually is not) one of the names in the > >> > sequence of names will match to the class, a subsequent name to the > >> > operation, and a name subsequent to that match to the specifier. (All > >> > simple string values.) > >> > > >> > The naive n^2 version is much too slow for the amount of data > involved. > >> > > >> > Is there an efficient way to do this kind of matching? > >> > > >> > Thanks! > >> > Graydon > > > > >