Main Page » Mailing List » February 2015

Repo and development workflow

Tuesday, 17 February 2015 · 9 messages · last reply 19 Feb 2015
MG
Marc van Grootel17 Feb 2015 21:30 #
Hi, As part of the basex-gradle-starter scripts [1] I am now looking at how to deal with the repo. Until now I have always developed by putting modules under webapp. It allows to just edit a file and query away. I like to have the same workflow during development and take advantage of having modules not depend on a fixed import path. When working with the repo modules have to be installed first. Just copying them under the repo doesn't make basex aware of them. Of course I could arrange it so that I run a build task that does this via repo:install but that involves the extra step of copying files from the install location into the repo instead of just registering the files that are already in the repo location. I would like to have, for example, a 'src' directory with the repo module directory structure. Is there a way to say to basex that my repo lives in 'src' and that it auto registers what it finds there. Say a command like repo:register('dir') or maybe even an auto-register option. Once a module is already installed editing and saving is enough for basex to pick up the change. No issue there. Of course for production I probably want to compile the code to jar or xar in order to install it and share it. But during development I would like to be able to just have the code be picked up so that I can edit, save and query. I am not sure if what I'm asking is a feature or if I just have to do things differently. BTW one more thing I noticed. When I sometimes get into a situation where individual empty directories are also recognized as packages (but I cannot delete them from the GUI package manager). With repo:list() I then see something like this.
<package name="org." type="Internal"/>
<package name="org.basex." type="Internal"/>
<package name="org.xokomola." type="Internal"/>
<package name="org.xokomola.xq.Hello" type="Internal"/>
[1] https://github.com/theapsgroup/basex-gradle-starter --Marc
CG
Christian Grün19 Feb 2015 11:24 #
Hi Marc,
Quoted text · Marc van Grootel · 4 lines
> I like to have the same workflow during development and take advantage > of having modules not depend on a fixed import path. When working with > the repo modules have to be installed first. Just copying them under > the repo doesn't make basex aware of them.
That's true (at least for jar and xar files; it actually works for simple XQuery modules). We could add a REPO UPDATE command and a repo:update() XQuery function, which rebuild the internal repository cache; do you think this would already help? Another, more flexible, alternative would be to remember the timestamps of already parsed modules and rebuild the repository whenever the timestamp has changed or the addressed module does not exist.
Quoted text · Marc van Grootel · 4 lines
> BTW one more thing I noticed. When I sometimes get into a situation > where individual empty directories are also recognized as packages > (but I cannot delete them from the GUI package manager). With > repo:list() I then see something like this.
Feel free to provide me with a snapshot of your repo directory whenever you can reproduce it. Best, Christian
MG
Marc van Grootel19 Feb 2015 11:35 #
Hi Christian, Yes, I figure just doing a quick REPO UPDATE where it goes and checks the repo dir for modules and registers them if necessary would be perfect. I figure when I change the repo path before running this it would pick up the modules from the new directory? Thus, so to speak, being able to "mount" another repo dir without file copying. Remembering timestamps etc. would make this a little trickier though as the new path may contain different modules possibly even older then the previous repo dir. I will keep an eye on this package listing issue (I noticed it on OSX) and try to come up with some repro steps. --Marc
Quoted text · Christian Grün · 31 lines
On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Marc, > >> I like to have the same workflow during development and take advantage >> of having modules not depend on a fixed import path. When working with >> the repo modules have to be installed first. Just copying them under >> the repo doesn't make basex aware of them. > > That's true (at least for jar and xar files; it actually works for > simple XQuery modules). > > We could add a REPO UPDATE command and a repo:update() XQuery > function, which rebuild the internal repository cache; do you think > this would already help? > > Another, more flexible, alternative would be to remember the > timestamps of already parsed modules and rebuild the repository > whenever the timestamp has changed or the addressed module does not > exist. > >> BTW one more thing I noticed. When I sometimes get into a situation >> where individual empty directories are also recognized as packages >> (but I cannot delete them from the GUI package manager). With >> repo:list() I then see something like this. > > Feel free to provide me with a snapshot of your repo directory > whenever you can reproduce it. > > Best, > Christian
-- --Marc
CG
Christian Grün19 Feb 2015 11:44 #
Hi Marc,
Quoted text · Marc van Grootel · 2 lines
> I figure when I change the repo path before running this it > would pick up the modules from the new directory?
This is probably not that easy, because REPOPATH is a global/static option, which is supposed not to change over the runtime of a BaseX instance. How/where do you currently change the repo path at runtime? Christian
Quoted text · Marc van Grootel · 47 lines
Thus, so to speak, > being able to "mount" another repo dir without file copying. > Remembering timestamps etc. would make this a little trickier though > as the new path may contain different modules possibly even older then > the previous repo dir. > > I will keep an eye on this package listing issue (I noticed it on OSX) > and try to come up with some repro steps. > > --Marc > > On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün > <christian.gruen at gmail.com> wrote: >> Hi Marc, >> >>> I like to have the same workflow during development and take advantage >>> of having modules not depend on a fixed import path. When working with >>> the repo modules have to be installed first. Just copying them under >>> the repo doesn't make basex aware of them. >> >> That's true (at least for jar and xar files; it actually works for >> simple XQuery modules). >> >> We could add a REPO UPDATE command and a repo:update() XQuery >> function, which rebuild the internal repository cache; do you think >> this would already help? >> >> Another, more flexible, alternative would be to remember the >> timestamps of already parsed modules and rebuild the repository >> whenever the timestamp has changed or the addressed module does not >> exist. >> >>> BTW one more thing I noticed. When I sometimes get into a situation >>> where individual empty directories are also recognized as packages >>> (but I cannot delete them from the GUI package manager). With >>> repo:list() I then see something like this. >> >> Feel free to provide me with a snapshot of your repo directory >> whenever you can reproduce it. >> >> Best, >> Christian > > > > -- > --Marc
MG
Marc van Grootel19 Feb 2015 11:53 #
In gradle.properties I define system properties and then in a Gradle build script I invoke a JavaExec task with the system properties. task http(type: JavaExec) { classpath configurations.basex main = "org.basex.BaseXHTTP" systemProperties System.getProperties() } --Marc
Quoted text · Christian Grün · 63 lines
On Thu, Feb 19, 2015 at 11:44 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Marc, > >> I figure when I change the repo path before running this it >> would pick up the modules from the new directory? > > This is probably not that easy, because REPOPATH is a global/static > option, which is supposed not to change over the runtime of a BaseX > instance. > > How/where do you currently change the repo path at runtime? > > Christian > > > Thus, so to speak, >> being able to "mount" another repo dir without file copying. >> Remembering timestamps etc. would make this a little trickier though >> as the new path may contain different modules possibly even older then >> the previous repo dir. >> >> I will keep an eye on this package listing issue (I noticed it on OSX) >> and try to come up with some repro steps. >> >> --Marc >> >> On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün >> <christian.gruen at gmail.com> wrote: >>> Hi Marc, >>> >>>> I like to have the same workflow during development and take advantage >>>> of having modules not depend on a fixed import path. When working with >>>> the repo modules have to be installed first. Just copying them under >>>> the repo doesn't make basex aware of them. >>> >>> That's true (at least for jar and xar files; it actually works for >>> simple XQuery modules). >>> >>> We could add a REPO UPDATE command and a repo:update() XQuery >>> function, which rebuild the internal repository cache; do you think >>> this would already help? >>> >>> Another, more flexible, alternative would be to remember the >>> timestamps of already parsed modules and rebuild the repository >>> whenever the timestamp has changed or the addressed module does not >>> exist. >>> >>>> BTW one more thing I noticed. When I sometimes get into a situation >>>> where individual empty directories are also recognized as packages >>>> (but I cannot delete them from the GUI package manager). With >>>> repo:list() I then see something like this. >>> >>> Feel free to provide me with a snapshot of your repo directory >>> whenever you can reproduce it. >>> >>> Best, >>> Christian >> >> >> >> -- >> --Marc
-- --Marc
CG
Christian Grün19 Feb 2015 12:04 #
Quoted text · Marc van Grootel · 2 lines
> In gradle.properties I define system properties and then in a Gradle > build script I invoke a JavaExec task with the system properties.
So, if I get it right, the repo path will be set at startup time. But will you change it while BaseXHTTP is running?
Quoted text · Marc van Grootel · 78 lines
> > > task http(type: JavaExec) { > classpath configurations.basex > main = "org.basex.BaseXHTTP" > systemProperties System.getProperties() > } > > --Marc > > On Thu, Feb 19, 2015 at 11:44 AM, Christian Grün > <christian.gruen at gmail.com> wrote: >> Hi Marc, >> >>> I figure when I change the repo path before running this it >>> would pick up the modules from the new directory? >> >> This is probably not that easy, because REPOPATH is a global/static >> option, which is supposed not to change over the runtime of a BaseX >> instance. >> >> How/where do you currently change the repo path at runtime? >> >> Christian >> >> >> Thus, so to speak, >>> being able to "mount" another repo dir without file copying. >>> Remembering timestamps etc. would make this a little trickier though >>> as the new path may contain different modules possibly even older then >>> the previous repo dir. >>> >>> I will keep an eye on this package listing issue (I noticed it on OSX) >>> and try to come up with some repro steps. >>> >>> --Marc >>> >>> On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün >>> <christian.gruen at gmail.com> wrote: >>>> Hi Marc, >>>> >>>>> I like to have the same workflow during development and take advantage >>>>> of having modules not depend on a fixed import path. When working with >>>>> the repo modules have to be installed first. Just copying them under >>>>> the repo doesn't make basex aware of them. >>>> >>>> That's true (at least for jar and xar files; it actually works for >>>> simple XQuery modules). >>>> >>>> We could add a REPO UPDATE command and a repo:update() XQuery >>>> function, which rebuild the internal repository cache; do you think >>>> this would already help? >>>> >>>> Another, more flexible, alternative would be to remember the >>>> timestamps of already parsed modules and rebuild the repository >>>> whenever the timestamp has changed or the addressed module does not >>>> exist. >>>> >>>>> BTW one more thing I noticed. When I sometimes get into a situation >>>>> where individual empty directories are also recognized as packages >>>>> (but I cannot delete them from the GUI package manager). With >>>>> repo:list() I then see something like this. >>>> >>>> Feel free to provide me with a snapshot of your repo directory >>>> whenever you can reproduce it. >>>> >>>> Best, >>>> Christian >>> >>> >>> >>> -- >>> --Marc > > > > -- > --Marc
MG
Marc van Grootel19 Feb 2015 12:23 #
Oh no I wouldn't. --Marc
Quoted text · Christian Grün · 87 lines
On Thu, Feb 19, 2015 at 12:04 PM, Christian Grün <christian.gruen at gmail.com> wrote: >> In gradle.properties I define system properties and then in a Gradle >> build script I invoke a JavaExec task with the system properties. > > So, if I get it right, the repo path will be set at startup time. But > will you change it while BaseXHTTP is running? > > >> >> >> task http(type: JavaExec) { >> classpath configurations.basex >> main = "org.basex.BaseXHTTP" >> systemProperties System.getProperties() >> } >> >> --Marc >> >> On Thu, Feb 19, 2015 at 11:44 AM, Christian Grün >> <christian.gruen at gmail.com> wrote: >>> Hi Marc, >>> >>>> I figure when I change the repo path before running this it >>>> would pick up the modules from the new directory? >>> >>> This is probably not that easy, because REPOPATH is a global/static >>> option, which is supposed not to change over the runtime of a BaseX >>> instance. >>> >>> How/where do you currently change the repo path at runtime? >>> >>> Christian >>> >>> >>> Thus, so to speak, >>>> being able to "mount" another repo dir without file copying. >>>> Remembering timestamps etc. would make this a little trickier though >>>> as the new path may contain different modules possibly even older then >>>> the previous repo dir. >>>> >>>> I will keep an eye on this package listing issue (I noticed it on OSX) >>>> and try to come up with some repro steps. >>>> >>>> --Marc >>>> >>>> On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün >>>> <christian.gruen at gmail.com> wrote: >>>>> Hi Marc, >>>>> >>>>>> I like to have the same workflow during development and take advantage >>>>>> of having modules not depend on a fixed import path. When working with >>>>>> the repo modules have to be installed first. Just copying them under >>>>>> the repo doesn't make basex aware of them. >>>>> >>>>> That's true (at least for jar and xar files; it actually works for >>>>> simple XQuery modules). >>>>> >>>>> We could add a REPO UPDATE command and a repo:update() XQuery >>>>> function, which rebuild the internal repository cache; do you think >>>>> this would already help? >>>>> >>>>> Another, more flexible, alternative would be to remember the >>>>> timestamps of already parsed modules and rebuild the repository >>>>> whenever the timestamp has changed or the addressed module does not >>>>> exist. >>>>> >>>>>> BTW one more thing I noticed. When I sometimes get into a situation >>>>>> where individual empty directories are also recognized as packages >>>>>> (but I cannot delete them from the GUI package manager). With >>>>>> repo:list() I then see something like this. >>>>> >>>>> Feel free to provide me with a snapshot of your repo directory >>>>> whenever you can reproduce it. >>>>> >>>>> Best, >>>>> Christian >>>> >>>> >>>> >>>> -- >>>> --Marc >> >> >> >> -- >> --Marc
-- --Marc
CG
Christian Grün19 Feb 2015 12:29 #
…Perfect, so we shouldn't encounter problems when refreshing the repository directory.
Quoted text · Marc van Grootel · 97 lines
On Thu, Feb 19, 2015 at 12:23 PM, Marc van Grootel <marc.van.grootel at gmail.com> wrote: > Oh no I wouldn't. > --Marc > > On Thu, Feb 19, 2015 at 12:04 PM, Christian Grün > <christian.gruen at gmail.com> wrote: >>> In gradle.properties I define system properties and then in a Gradle >>> build script I invoke a JavaExec task with the system properties. >> >> So, if I get it right, the repo path will be set at startup time. But >> will you change it while BaseXHTTP is running? >> >> >>> >>> >>> task http(type: JavaExec) { >>> classpath configurations.basex >>> main = "org.basex.BaseXHTTP" >>> systemProperties System.getProperties() >>> } >>> >>> --Marc >>> >>> On Thu, Feb 19, 2015 at 11:44 AM, Christian Grün >>> <christian.gruen at gmail.com> wrote: >>>> Hi Marc, >>>> >>>>> I figure when I change the repo path before running this it >>>>> would pick up the modules from the new directory? >>>> >>>> This is probably not that easy, because REPOPATH is a global/static >>>> option, which is supposed not to change over the runtime of a BaseX >>>> instance. >>>> >>>> How/where do you currently change the repo path at runtime? >>>> >>>> Christian >>>> >>>> >>>> Thus, so to speak, >>>>> being able to "mount" another repo dir without file copying. >>>>> Remembering timestamps etc. would make this a little trickier though >>>>> as the new path may contain different modules possibly even older then >>>>> the previous repo dir. >>>>> >>>>> I will keep an eye on this package listing issue (I noticed it on OSX) >>>>> and try to come up with some repro steps. >>>>> >>>>> --Marc >>>>> >>>>> On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün >>>>> <christian.gruen at gmail.com> wrote: >>>>>> Hi Marc, >>>>>> >>>>>>> I like to have the same workflow during development and take advantage >>>>>>> of having modules not depend on a fixed import path. When working with >>>>>>> the repo modules have to be installed first. Just copying them under >>>>>>> the repo doesn't make basex aware of them. >>>>>> >>>>>> That's true (at least for jar and xar files; it actually works for >>>>>> simple XQuery modules). >>>>>> >>>>>> We could add a REPO UPDATE command and a repo:update() XQuery >>>>>> function, which rebuild the internal repository cache; do you think >>>>>> this would already help? >>>>>> >>>>>> Another, more flexible, alternative would be to remember the >>>>>> timestamps of already parsed modules and rebuild the repository >>>>>> whenever the timestamp has changed or the addressed module does not >>>>>> exist. >>>>>> >>>>>>> BTW one more thing I noticed. When I sometimes get into a situation >>>>>>> where individual empty directories are also recognized as packages >>>>>>> (but I cannot delete them from the GUI package manager). With >>>>>>> repo:list() I then see something like this. >>>>>> >>>>>> Feel free to provide me with a snapshot of your repo directory >>>>>> whenever you can reproduce it. >>>>>> >>>>>> Best, >>>>>> Christian >>>>> >>>>> >>>>> >>>>> -- >>>>> --Marc >>> >>> >>> >>> -- >>> --Marc > > > > -- > --Marc
CG
Christian Grün19 Feb 2015 14:49 #
Hi Marc, Instead of creating a new command, I decided to update the REPO LIST command, which now invalidates the internal repo structure before listing all all packages. The command creates some textual output, but you can simply ignore it. Could you try the latest snapshot [1] and see if calling REPO LIST or repo:list() meets your requirements? Best, Christian [1] http://files.basex.org/releases/latest/
Quoted text · Christian Grün · 103 lines
On Thu, Feb 19, 2015 at 12:29 PM, Christian Grün <christian.gruen at gmail.com> wrote: > …Perfect, so we shouldn't encounter problems when refreshing the > repository directory. > > > On Thu, Feb 19, 2015 at 12:23 PM, Marc van Grootel > <marc.van.grootel at gmail.com> wrote: >> Oh no I wouldn't. >> --Marc >> >> On Thu, Feb 19, 2015 at 12:04 PM, Christian Grün >> <christian.gruen at gmail.com> wrote: >>>> In gradle.properties I define system properties and then in a Gradle >>>> build script I invoke a JavaExec task with the system properties. >>> >>> So, if I get it right, the repo path will be set at startup time. But >>> will you change it while BaseXHTTP is running? >>> >>> >>>> >>>> >>>> task http(type: JavaExec) { >>>> classpath configurations.basex >>>> main = "org.basex.BaseXHTTP" >>>> systemProperties System.getProperties() >>>> } >>>> >>>> --Marc >>>> >>>> On Thu, Feb 19, 2015 at 11:44 AM, Christian Grün >>>> <christian.gruen at gmail.com> wrote: >>>>> Hi Marc, >>>>> >>>>>> I figure when I change the repo path before running this it >>>>>> would pick up the modules from the new directory? >>>>> >>>>> This is probably not that easy, because REPOPATH is a global/static >>>>> option, which is supposed not to change over the runtime of a BaseX >>>>> instance. >>>>> >>>>> How/where do you currently change the repo path at runtime? >>>>> >>>>> Christian >>>>> >>>>> >>>>> Thus, so to speak, >>>>>> being able to "mount" another repo dir without file copying. >>>>>> Remembering timestamps etc. would make this a little trickier though >>>>>> as the new path may contain different modules possibly even older then >>>>>> the previous repo dir. >>>>>> >>>>>> I will keep an eye on this package listing issue (I noticed it on OSX) >>>>>> and try to come up with some repro steps. >>>>>> >>>>>> --Marc >>>>>> >>>>>> On Thu, Feb 19, 2015 at 11:24 AM, Christian Grün >>>>>> <christian.gruen at gmail.com> wrote: >>>>>>> Hi Marc, >>>>>>> >>>>>>>> I like to have the same workflow during development and take advantage >>>>>>>> of having modules not depend on a fixed import path. When working with >>>>>>>> the repo modules have to be installed first. Just copying them under >>>>>>>> the repo doesn't make basex aware of them. >>>>>>> >>>>>>> That's true (at least for jar and xar files; it actually works for >>>>>>> simple XQuery modules). >>>>>>> >>>>>>> We could add a REPO UPDATE command and a repo:update() XQuery >>>>>>> function, which rebuild the internal repository cache; do you think >>>>>>> this would already help? >>>>>>> >>>>>>> Another, more flexible, alternative would be to remember the >>>>>>> timestamps of already parsed modules and rebuild the repository >>>>>>> whenever the timestamp has changed or the addressed module does not >>>>>>> exist. >>>>>>> >>>>>>>> BTW one more thing I noticed. When I sometimes get into a situation >>>>>>>> where individual empty directories are also recognized as packages >>>>>>>> (but I cannot delete them from the GUI package manager). With >>>>>>>> repo:list() I then see something like this. >>>>>>> >>>>>>> Feel free to provide me with a snapshot of your repo directory >>>>>>> whenever you can reproduce it. >>>>>>> >>>>>>> Best, >>>>>>> Christian >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> --Marc >>>> >>>> >>>> >>>> -- >>>> --Marc >> >> >> >> -- >> --Marc

Written for BaseX 8.0. BaseX may have changed since: please check the current documentation.

« Trying to get started with IntelliJ and BaseXIssue with dtd validation »
⚡Generated with XQuery