Memory management

Am 19.01.2017 18:53 schrieb "meumapple" <meumapple at gmail.com>: > I have 223 XML files and I need to perform some actions on each of them, > so that the output should be the modified texts (223 new files). > > What I have noticed is that if I apply my script recursively to each file > and then save the results in new files (file:write()), everything works > fast. On the other hand, If I do not write out the results in new files > (but I expect the results from all the files in the GUI), the GUI freezes > and then runs out of memory. > > My question is: is there a way to get the script working well without > using file:write()? Should I always apply this script strategy when I work > with many files? > > Thanks, > Joseph

Il giorno 19 gen 2017, alle ore 19:52, Christian Grün <christian.gruen at gmail.com> ha scritto:

Am 19.01.2017 18:53 schrieb "meumapple" <meumapple at gmail.com>: > I have 223 XML files and I need to perform some actions on each of them, so that the output should be the modified texts (223 new files). > > What I have noticed is that if I apply my script recursively to each file and then save the results in new files (file:write()), everything works fast. On the other hand, If I do not write out the results in new files (but I expect the results from all the files in the GUI), the GUI freezes and then runs out of memory. > > My question is: is there a way to get the script working well without using file:write()? Should I always apply this script strategy when I work with many files? > > Thanks, > Joseph

On Fri, Jan 20, 2017 at 5:06 PM, meumapple <meumapple at gmail.com> wrote: > Hi Cristian, > > I have actually written a Library module. I have probably understood the > problem, in that a few user-defined functions which I called in a nested way > contained, each of them, a for-loop. I tried to not add for-loops in the > definition of functions but just have one main for-loop in the body of the > query and now I can easily get my results in the GUI. However, another > problem arose: when I try to launch the command "basex myquery.xq" from the > command line, I start getting the output of the query correctly, but at a > certain point it gets stuck and java runs out of memory, even though the > same query performs in less than a minute in the GUI. Any idea? > > Thanks! > Joseph > > Il giorno 19 gen 2017, alle ore 19:52, Christian Grün > <christian.gruen at gmail.com> ha scritto: > > Hi Joseph, > > Could you please post your script? > > Thanks in advance, > Christian > > > > Am 19.01.2017 18:53 schrieb "meumapple" <meumapple at gmail.com>: >> >> I have 223 XML files and I need to perform some actions on each of them, >> so that the output should be the modified texts (223 new files). >> >> What I have noticed is that if I apply my script recursively to each file >> and then save the results in new files (file:write()), everything works >> fast. On the other hand, If I do not write out the results in new files >> (but I expect the results from all the files in the GUI), the GUI freezes >> and then runs out of memory. >> >> My question is: is there a way to get the script working well without >> using file:write()? Should I always apply this script strategy when I work >> with many files? >> >> Thanks, >> Joseph

Il giorno 20 gen 2017, alle ore 17:33, Christian Grün <christian.gruen at gmail.com> ha scritto:

> On Fri, Jan 20, 2017 at 5:06 PM, meumapple <meumapple at gmail.com> wrote: > Hi Cristian, > > I have actually written a Library module. I have probably understood the > problem, in that a few user-defined functions which I called in a nested way > contained, each of them, a for-loop. I tried to not add for-loops in the > definition of functions but just have one main for-loop in the body of the > query and now I can easily get my results in the GUI. However, another > problem arose: when I try to launch the command "basex myquery.xq" from the > command line, I start getting the output of the query correctly, but at a > certain point it gets stuck and java runs out of memory, even though the > same query performs in less than a minute in the GUI. Any idea? > > Thanks! > Joseph > > Il giorno 19 gen 2017, alle ore 19:52, Christian Grün > <christian.gruen at gmail.com> ha scritto: > > Hi Joseph, > > Could you please post your script? > > Thanks in advance, > Christian > > > > Am 19.01.2017 18:53 schrieb "meumapple" <meumapple at gmail.com>: >> >> I have 223 XML files and I need to perform some actions on each of them, >> so that the output should be the modified texts (223 new files). >> >> What I have noticed is that if I apply my script recursively to each file >> and then save the results in new files (file:write()), everything works >> fast. On the other hand, If I do not write out the results in new files >> (but I expect the results from all the files in the GUI), the GUI freezes >> and then runs out of memory. >> >> My question is: is there a way to get the script working well without >> using file:write()? Should I always apply this script strategy when I work >> with many files? >> >> Thanks, >> Joseph

On Tue, Jan 24, 2017 at 10:44 PM, meumapple <meumapple at gmail.com> wrote: > Hi Christian , > > I have been able to find the source of the problem: the RAM. I was wrongly assuming that basex/basexgui in the bin folder had by default more RAM available than the GUI available from the BaseX.jar file. I simply changed the value in the basex/basexgui files and everything worked. Why is 512m the default value (isn't it too low)? Is it possible to run the basexgui/basex files with more RAM using an argument at the command line or is it necessary to modify the files internally? Thanks. > > Joseph > > Il giorno 20 gen 2017, alle ore 17:33, Christian Grün <christian.gruen at gmail.com> ha scritto: > > Hi Joseph, there is a vast number of reasons why XQuery code can cause > out of memory errors, just as in every other language, so I need to > look at your code to give you more hints. Minimized examples are > preferred as usual. – Cheers, Christian > > >> On Fri, Jan 20, 2017 at 5:06 PM, meumapple <meumapple at gmail.com> wrote: >> Hi Cristian, >> >> I have actually written a Library module. I have probably understood the >> problem, in that a few user-defined functions which I called in a nested way >> contained, each of them, a for-loop. I tried to not add for-loops in the >> definition of functions but just have one main for-loop in the body of the >> query and now I can easily get my results in the GUI. However, another >> problem arose: when I try to launch the command "basex myquery.xq" from the >> command line, I start getting the output of the query correctly, but at a >> certain point it gets stuck and java runs out of memory, even though the >> same query performs in less than a minute in the GUI. Any idea? >> >> Thanks! >> Joseph >> >> Il giorno 19 gen 2017, alle ore 19:52, Christian Grün >> <christian.gruen at gmail.com> ha scritto: >> >> Hi Joseph, >> >> Could you please post your script? >> >> Thanks in advance, >> Christian >> >> >> >> Am 19.01.2017 18:53 schrieb "meumapple" <meumapple at gmail.com>: >>> >>> I have 223 XML files and I need to perform some actions on each of them, >>> so that the output should be the modified texts (223 new files). >>> >>> What I have noticed is that if I apply my script recursively to each file >>> and then save the results in new files (file:write()), everything works >>> fast. On the other hand, If I do not write out the results in new files >>> (but I expect the results from all the files in the GUI), the GUI freezes >>> and then runs out of memory. >>> >>> My question is: is there a way to get the script working well without >>> using file:write()? Should I always apply this script strategy when I work >>> with many files? >>> >>> Thanks, >>> Joseph