Working with custom stores

> Using the store module, is it possible to read and merge multiple custom stores into main memory

> set a particular store as context for a query?

> Or, do I need to serialize the stores, e.g., as JSON and then create databases for them?

Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 18:50: > That's perfect--I knew I was missing something :) > > Are db:get-value and db:put-value side-effecting as well? > > Thanks, > Tim > > > -- > Tim A. Thompson (he, him) > Librarian for Applied Metadata Research > www.linkedin.com/in/timathompson > timothy.thompson at yale.edu > > > El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( > christian.gruen at gmail.com) escribió: > >> Hi Tim, >> >> > Using the store module, is it possible to read and merge multiple >> custom stores into main memory >> >> We could add an argument (or options) to file:read to allow entries to >> be added to an existing main-memory store. >> >> > set a particular store as context for a query? >> >> The recommended approach is to invoke store:read before any other >> operation in the query. Please note, however, that the store will be >> globally available. If a second query runs on the same server and >> opens another store, it will change the behavior of the first query. >> For such use cases, it’s better to use databases. >> >> > Or, do I need to serialize the stores, e.g., as JSON and then create >> databases for them? >> >> You could have a look at the new db:get-value and db:put-value >> functions [1]: These functions have particularly been designed to >> organize XQuery items in databases, making casts to JSON or XML >> obsolete. >> >> Hope this helps, >> Christian >> >> [1] https://docs.basex.org/wiki/Database_Module#db:get-value >> >

Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 19:16: > Thanks! I just tried to dump the contents of the store to a database value: > > db:put-value( > "entries", > array { > store:keys() ! store:get(.) (: sequence of arrays :) > }, > "x" > ) > > But when I query with db:get-value("entries", "x"), I get a "Not Expected" > error. Should this kind of operation be possible? > > Error: > Improper use? Potential bug? Your feedback is welcome: > Contact: basex-talk at mailman.uni-konstanz.de > Version: BaseX 10.3 beta > Java: Azul Systems, Inc., 11.0.16 > OS: Mac OS X, aarch64 > Stack Trace: > java.lang.RuntimeException: Not Expected. > at org.basex.util.Util.notExpected(Util.java:54) > at org.basex.query.value.type.AtomType.read(AtomType.java:983) > at org.basex.core.Store.read(Store.java:315) > at org.basex.query.func.db.DbGetValue.resource(DbGetValue.java:70) > at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:46) > at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:27) > at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) > at org.basex.query.scope.MainModule.iter(MainModule.java:48) > at org.basex.query.QueryContext.lambda$4(QueryContext.java:354) > at org.basex.query.QueryContext.run(QueryContext.java:763) > at org.basex.query.QueryContext.iter(QueryContext.java:354) > at org.basex.query.QueryContext.lambda$6(QueryContext.java:601) > at org.basex.query.QueryContext.run(QueryContext.java:763) > at org.basex.query.QueryContext.cache(QueryContext.java:598) > at org.basex.query.QueryProcessor.cache(QueryProcessor.java:118) > at org.basex.core.cmd.AQuery.run(AQuery.java:82) > at org.basex.core.Command.run(Command.java:233) > at org.basex.core.Command.execute(Command.java:93) > at org.basex.gui.GUI.execute(GUI.java:415) > at org.basex.gui.GUI.lambda$5(GUI.java:360) > at java.base/java.lang.Thread.run(Thread.java:829) > > > > -- > Tim A. Thompson (he, him) > Librarian for Applied Metadata Research > www.linkedin.com/in/timathompson > timothy.thompson at yale.edu > > > El mar, 15 nov 2022 a la(s) 13:07, Christian Grün ( > christian.gruen at gmail.com) escribió: > >> You'll be safe: Updates with db:put-value will be managed via the Pending >> Update List [1], and future queries that operate on the same database will >> be queued until the update is finished. >> >> [1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List >> >> >> >> Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 18:50: >> >>> That's perfect--I knew I was missing something :) >>> >>> Are db:get-value and db:put-value side-effecting as well? >>> >>> Thanks, >>> Tim >>> >>> >>> -- >>> Tim A. Thompson (he, him) >>> Librarian for Applied Metadata Research >>> www.linkedin.com/in/timathompson >>> timothy.thompson at yale.edu >>> >>> >>> El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( >>> christian.gruen at gmail.com) escribió: >>> >>>> Hi Tim, >>>> >>>> > Using the store module, is it possible to read and merge multiple >>>> custom stores into main memory >>>> >>>> We could add an argument (or options) to file:read to allow entries to >>>> be added to an existing main-memory store. >>>> >>>> > set a particular store as context for a query? >>>> >>>> The recommended approach is to invoke store:read before any other >>>> operation in the query. Please note, however, that the store will be >>>> globally available. If a second query runs on the same server and >>>> opens another store, it will change the behavior of the first query. >>>> For such use cases, it’s better to use databases. >>>> >>>> > Or, do I need to serialize the stores, e.g., as JSON and then create >>>> databases for them? >>>> >>>> You could have a look at the new db:get-value and db:put-value >>>> functions [1]: These functions have particularly been designed to >>>> organize XQuery items in databases, making casts to JSON or XML >>>> obsolete. >>>> >>>> Hope this helps, >>>> Christian >>>> >>>> [1] https://docs.basex.org/wiki/Database_Module#db:get-value >>>> >>>

El mar, 15 nov 2022 a la(s) 12:28, Christian Grün (christian.gruen at gmail.com) escribió: > Hi Tim, > > > Using the store module, is it possible to read and merge multiple custom > stores into main memory > > We could add an argument (or options) to file:read to allow entries to > be added to an existing main-memory store. > > > set a particular store as context for a query? > > The recommended approach is to invoke store:read before any other > operation in the query. Please note, however, that the store will be > globally available. If a second query runs on the same server and > opens another store, it will change the behavior of the first query. > For such use cases, it’s better to use databases. > > > Or, do I need to serialize the stores, e.g., as JSON and then create > databases for them? > > You could have a look at the new db:get-value and db:put-value > functions [1]: These functions have particularly been designed to > organize XQuery items in databases, making casts to JSON or XML > obsolete. > > Hope this helps, > Christian > > [1] https://docs.basex.org/wiki/Database_Module#db:get-value >

El mar, 15 nov 2022 a la(s) 13:07, Christian Grün (christian.gruen at gmail.com) escribió: > You'll be safe: Updates with db:put-value will be managed via the Pending > Update List [1], and future queries that operate on the same database will > be queued until the update is finished. > > [1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List > > > > Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 18:50: > >> That's perfect--I knew I was missing something :) >> >> Are db:get-value and db:put-value side-effecting as well? >> >> Thanks, >> Tim >> >> >> -- >> Tim A. Thompson (he, him) >> Librarian for Applied Metadata Research >> www.linkedin.com/in/timathompson >> timothy.thompson at yale.edu >> >> >> El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( >> christian.gruen at gmail.com) escribió: >> >>> Hi Tim, >>> >>> > Using the store module, is it possible to read and merge multiple >>> custom stores into main memory >>> >>> We could add an argument (or options) to file:read to allow entries to >>> be added to an existing main-memory store. >>> >>> > set a particular store as context for a query? >>> >>> The recommended approach is to invoke store:read before any other >>> operation in the query. Please note, however, that the store will be >>> globally available. If a second query runs on the same server and >>> opens another store, it will change the behavior of the first query. >>> For such use cases, it’s better to use databases. >>> >>> > Or, do I need to serialize the stores, e.g., as JSON and then create >>> databases for them? >>> >>> You could have a look at the new db:get-value and db:put-value >>> functions [1]: These functions have particularly been designed to >>> organize XQuery items in databases, making casts to JSON or XML >>> obsolete. >>> >>> Hope this helps, >>> Christian >>> >>> [1] https://docs.basex.org/wiki/Database_Module#db:get-value >>> >>

El mar, 15 nov 2022 a la(s) 13:21, Christian Grün (christian.gruen at gmail.com) escribió: > Wow. I’ll look into that soon. Do you think you can provide me with a > little test case (e.g., a BaseX command script) to reproduce this? > > > > > Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 19:16: > >> Thanks! I just tried to dump the contents of the store to a database >> value: >> >> db:put-value( >> "entries", >> array { >> store:keys() ! store:get(.) (: sequence of arrays :) >> }, >> "x" >> ) >> >> But when I query with db:get-value("entries", "x"), I get a "Not >> Expected" error. Should this kind of operation be possible? >> >> Error: >> Improper use? Potential bug? Your feedback is welcome: >> Contact: basex-talk at mailman.uni-konstanz.de >> Version: BaseX 10.3 beta >> Java: Azul Systems, Inc., 11.0.16 >> OS: Mac OS X, aarch64 >> Stack Trace: >> java.lang.RuntimeException: Not Expected. >> at org.basex.util.Util.notExpected(Util.java:54) >> at org.basex.query.value.type.AtomType.read(AtomType.java:983) >> at org.basex.core.Store.read(Store.java:315) >> at org.basex.query.func.db.DbGetValue.resource(DbGetValue.java:70) >> at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:46) >> at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:27) >> at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) >> at org.basex.query.scope.MainModule.iter(MainModule.java:48) >> at org.basex.query.QueryContext.lambda$4(QueryContext.java:354) >> at org.basex.query.QueryContext.run(QueryContext.java:763) >> at org.basex.query.QueryContext.iter(QueryContext.java:354) >> at org.basex.query.QueryContext.lambda$6(QueryContext.java:601) >> at org.basex.query.QueryContext.run(QueryContext.java:763) >> at org.basex.query.QueryContext.cache(QueryContext.java:598) >> at org.basex.query.QueryProcessor.cache(QueryProcessor.java:118) >> at org.basex.core.cmd.AQuery.run(AQuery.java:82) >> at org.basex.core.Command.run(Command.java:233) >> at org.basex.core.Command.execute(Command.java:93) >> at org.basex.gui.GUI.execute(GUI.java:415) >> at org.basex.gui.GUI.lambda$5(GUI.java:360) >> at java.base/java.lang.Thread.run(Thread.java:829) >> >> >> >> -- >> Tim A. Thompson (he, him) >> Librarian for Applied Metadata Research >> www.linkedin.com/in/timathompson >> timothy.thompson at yale.edu >> >> >> El mar, 15 nov 2022 a la(s) 13:07, Christian Grün ( >> christian.gruen at gmail.com) escribió: >> >>> You'll be safe: Updates with db:put-value will be managed via the >>> Pending Update List [1], and future queries that operate on the same >>> database will be queued until the update is finished. >>> >>> [1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List >>> >>> >>> >>> Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 18:50: >>> >>>> That's perfect--I knew I was missing something :) >>>> >>>> Are db:get-value and db:put-value side-effecting as well? >>>> >>>> Thanks, >>>> Tim >>>> >>>> >>>> -- >>>> Tim A. Thompson (he, him) >>>> Librarian for Applied Metadata Research >>>> www.linkedin.com/in/timathompson >>>> timothy.thompson at yale.edu >>>> >>>> >>>> El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( >>>> christian.gruen at gmail.com) escribió: >>>> >>>>> Hi Tim, >>>>> >>>>> > Using the store module, is it possible to read and merge multiple >>>>> custom stores into main memory >>>>> >>>>> We could add an argument (or options) to file:read to allow entries to >>>>> be added to an existing main-memory store. >>>>> >>>>> > set a particular store as context for a query? >>>>> >>>>> The recommended approach is to invoke store:read before any other >>>>> operation in the query. Please note, however, that the store will be >>>>> globally available. If a second query runs on the same server and >>>>> opens another store, it will change the behavior of the first query. >>>>> For such use cases, it’s better to use databases. >>>>> >>>>> > Or, do I need to serialize the stores, e.g., as JSON and then create >>>>> databases for them? >>>>> >>>>> You could have a look at the new db:get-value and db:put-value >>>>> functions [1]: These functions have particularly been designed to >>>>> organize XQuery items in databases, making casts to JSON or XML >>>>> obsolete. >>>>> >>>>> Hope this helps, >>>>> Christian >>>>> >>>>> [1] https://docs.basex.org/wiki/Database_Module#db:get-value >>>>> >>>>

On Tue, Nov 15, 2022 at 7:40 PM Tim Thompson <timathom at gmail.com> wrote: > Sure, attached are two command scripts: one that fails (when dumping store > contents) and one that passes (when adding a value directly). > > Thanks, > Tim > > > -- > Tim A. Thompson (he, him) > Librarian for Applied Metadata Research > www.linkedin.com/in/timathompson > timothy.thompson at yale.edu > > > El mar, 15 nov 2022 a la(s) 13:21, Christian Grün ( > christian.gruen at gmail.com) escribió: > >> Wow. I’ll look into that soon. Do you think you can provide me with a >> little test case (e.g., a BaseX command script) to reproduce this? >> >> >> >> >> Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 19:16: >> >>> Thanks! I just tried to dump the contents of the store to a database >>> value: >>> >>> db:put-value( >>> "entries", >>> array { >>> store:keys() ! store:get(.) (: sequence of arrays :) >>> }, >>> "x" >>> ) >>> >>> But when I query with db:get-value("entries", "x"), I get a "Not >>> Expected" error. Should this kind of operation be possible? >>> >>> Error: >>> Improper use? Potential bug? Your feedback is welcome: >>> Contact: basex-talk at mailman.uni-konstanz.de >>> Version: BaseX 10.3 beta >>> Java: Azul Systems, Inc., 11.0.16 >>> OS: Mac OS X, aarch64 >>> Stack Trace: >>> java.lang.RuntimeException: Not Expected. >>> at org.basex.util.Util.notExpected(Util.java:54) >>> at org.basex.query.value.type.AtomType.read(AtomType.java:983) >>> at org.basex.core.Store.read(Store.java:315) >>> at org.basex.query.func.db.DbGetValue.resource(DbGetValue.java:70) >>> at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:46) >>> at org.basex.query.func.db.DbGetValue.value(DbGetValue.java:27) >>> at org.basex.query.expr.ParseExpr.iter(ParseExpr.java:46) >>> at org.basex.query.scope.MainModule.iter(MainModule.java:48) >>> at org.basex.query.QueryContext.lambda$4(QueryContext.java:354) >>> at org.basex.query.QueryContext.run(QueryContext.java:763) >>> at org.basex.query.QueryContext.iter(QueryContext.java:354) >>> at org.basex.query.QueryContext.lambda$6(QueryContext.java:601) >>> at org.basex.query.QueryContext.run(QueryContext.java:763) >>> at org.basex.query.QueryContext.cache(QueryContext.java:598) >>> at org.basex.query.QueryProcessor.cache(QueryProcessor.java:118) >>> at org.basex.core.cmd.AQuery.run(AQuery.java:82) >>> at org.basex.core.Command.run(Command.java:233) >>> at org.basex.core.Command.execute(Command.java:93) >>> at org.basex.gui.GUI.execute(GUI.java:415) >>> at org.basex.gui.GUI.lambda$5(GUI.java:360) >>> at java.base/java.lang.Thread.run(Thread.java:829) >>> >>> >>> >>> -- >>> Tim A. Thompson (he, him) >>> Librarian for Applied Metadata Research >>> www.linkedin.com/in/timathompson >>> timothy.thompson at yale.edu >>> >>> >>> El mar, 15 nov 2022 a la(s) 13:07, Christian Grün ( >>> christian.gruen at gmail.com) escribió: >>> >>>> You'll be safe: Updates with db:put-value will be managed via the >>>> Pending Update List [1], and future queries that operate on the same >>>> database will be queued until the update is finished. >>>> >>>> [1] https://docs.basex.org/wiki/XQuery_Update#Pending_Update_List >>>> >>>> >>>> >>>> Tim Thompson <timathom at gmail.com> schrieb am Di., 15. Nov. 2022, 18:50: >>>> >>>>> That's perfect--I knew I was missing something :) >>>>> >>>>> Are db:get-value and db:put-value side-effecting as well? >>>>> >>>>> Thanks, >>>>> Tim >>>>> >>>>> >>>>> -- >>>>> Tim A. Thompson (he, him) >>>>> Librarian for Applied Metadata Research >>>>> www.linkedin.com/in/timathompson >>>>> timothy.thompson at yale.edu >>>>> >>>>> >>>>> El mar, 15 nov 2022 a la(s) 12:28, Christian Grün ( >>>>> christian.gruen at gmail.com) escribió: >>>>> >>>>>> Hi Tim, >>>>>> >>>>>> > Using the store module, is it possible to read and merge multiple >>>>>> custom stores into main memory >>>>>> >>>>>> We could add an argument (or options) to file:read to allow entries to >>>>>> be added to an existing main-memory store. >>>>>> >>>>>> > set a particular store as context for a query? >>>>>> >>>>>> The recommended approach is to invoke store:read before any other >>>>>> operation in the query. Please note, however, that the store will be >>>>>> globally available. If a second query runs on the same server and >>>>>> opens another store, it will change the behavior of the first query. >>>>>> For such use cases, it’s better to use databases. >>>>>> >>>>>> > Or, do I need to serialize the stores, e.g., as JSON and then >>>>>> create databases for them? >>>>>> >>>>>> You could have a look at the new db:get-value and db:put-value >>>>>> functions [1]: These functions have particularly been designed to >>>>>> organize XQuery items in databases, making casts to JSON or XML >>>>>> obsolete. >>>>>> >>>>>> Hope this helps, >>>>>> Christian >>>>>> >>>>>> [1] https://docs.basex.org/wiki/Database_Module#db:get-value >>>>>> >>>>>