Main Page » Mailing List » January 2016

Java client and binary files

Friday, 15 January 2016 · 10 messages · last reply 29 Jan 2016
EJ
E. Wray Johnson15 Jan 2016 21:11 #
When using the sample Java client to store/retrieve binary/raw files I have to execute separate open/close commands. It is my understanding that this locks the database. In a multi-user scenario, such locking is undesirable as binary/raw files can be quite large. And since there is no alteration of the indexes etc during such operations, is there a better way? Also, why isn't there a corresponding retrieve method on the Java client, similar to the store method? My code to store:
BaseXClient session = new BaseXClient(host, port, username, password);
session.execute("open " + database);
session.store(path, inputStream);
session.execute("close");
My code to retrieve:
BaseXClient session = new BaseXClient(host, port, username, password);
session.execute("open " + database);
session.execute("retrieve " + path, outputStream);
session.execute("close");
Thanks in advance!
CG
Christian Grün16 Jan 2016 18:18 #
Quoted text · E. Wray Johnson · 2 lines
> And since there is no alteration of > the indexes etc during such operations, is there a better way?
Hm. I can’t think of better ways (but there will be some, as usual ;). Suggestions from other users are welcome.
Quoted text · E. Wray Johnson · 2 lines
> isn't there a corresponding retrieve method on the Java client, similar to > the store method?
Just use RETRIEVE or db:retrieve() (with BaseX 8.4, binary resources will directly be returned in their binary representation [1]). Please note that you’ll have to specify an output stream for your client, because the result cannot be returned as string. [1] http://docs.basex.org/wiki/XQuery_Extensions#Serialization
ME
Mike Engledewin reply to E. Wray Johnson, 15 Jan 2016, 21:1116 Jan 2016 21:06 #
If like me your using BaseX 8.3.1 don't forget to add ...
declare option output:method "raw";
before the db:retrieve() otherwise you won't get back what you put in. I learned this the hard way trying to extract binary files that were stored in codepage IBM1047 and it took me a while to figure out why they came back translated;
Quoted text · E. Wray Johnson · 26 lines
On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson <wray.johnson at gmail.com> wrote: > When using the sample Java client to store/retrieve binary/raw files I > have to execute separate open/close commands. It is my understanding that > this locks the database. In a multi-user scenario, such locking is > undesirable as binary/raw files can be quite large. And since there is no > alteration of the indexes etc during such operations, is there a better > way? Also, why isn't there a corresponding retrieve method on the Java > client, similar to the store method? > > My code to store: > BaseXClient session = new BaseXClient(host, port, username, password); > session.execute("open " + database); > session.store(path, inputStream); > session.execute("close"); > > My code to retrieve: > BaseXClient session = new BaseXClient(host, port, username, password); > session.execute("open " + database); > session.execute("retrieve " + path, outputStream); > session.execute("close"); > > Thanks in advance! > > >
EJ
E. Wray Johnson28 Jan 2016 08:12 #
With 8.4...
declare option output:method "raw";
I get [SEPM0016] Value of 'method' must be one of (xml,xhtml,html,json,csv,text,adaptive,basex).
Quoted text · Mike Engledew · 40 lines
On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew <mike.engledew at gmail.com> wrote: > If like me your using BaseX 8.3.1 don't forget to add ... > > declare option output:method "raw"; > > before the db:retrieve() otherwise you won't get back what you put in. > I learned this the hard way trying to extract binary files that were > stored in codepage IBM1047 and it took me a while to figure out why they > came back translated; > > > On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson <wray.johnson at gmail.com> > wrote: > >> When using the sample Java client to store/retrieve binary/raw files I >> have to execute separate open/close commands. It is my understanding that >> this locks the database. In a multi-user scenario, such locking is >> undesirable as binary/raw files can be quite large. And since there is no >> alteration of the indexes etc during such operations, is there a better >> way? Also, why isn't there a corresponding retrieve method on the Java >> client, similar to the store method? >> >> My code to store: >> BaseXClient session = new BaseXClient(host, port, username, password); >> session.execute("open " + database); >> session.store(path, inputStream); >> session.execute("close"); >> >> My code to retrieve: >> BaseXClient session = new BaseXClient(host, port, username, password); >> session.execute("open " + database); >> session.execute("retrieve " + path, outputStream); >> session.execute("close"); >> >> Thanks in advance! >> >> >> >
CG
Christian Grün28 Jan 2016 09:08 #
See here: http://docs.basex.org/wiki/Serialization
Quoted text · E. Wray Johnson · 52 lines
Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" <wray.johnson at gmail.com>: > With 8.4... > > declare option output:method "raw"; > > I get > > [SEPM0016] Value of 'method' must be one of > (xml,xhtml,html,json,csv,text,adaptive,basex). > > On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew <mike.engledew at gmail.com> > wrote: > >> If like me your using BaseX 8.3.1 don't forget to add ... >> >> declare option output:method "raw"; >> >> before the db:retrieve() otherwise you won't get back what you put in. >> I learned this the hard way trying to extract binary files that were >> stored in codepage IBM1047 and it took me a while to figure out why they >> came back translated; >> >> >> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson <wray.johnson at gmail.com >> > wrote: >> >>> When using the sample Java client to store/retrieve binary/raw files I >>> have to execute separate open/close commands. It is my understanding that >>> this locks the database. In a multi-user scenario, such locking is >>> undesirable as binary/raw files can be quite large. And since there is no >>> alteration of the indexes etc during such operations, is there a better >>> way? Also, why isn't there a corresponding retrieve method on the Java >>> client, similar to the store method? >>> >>> My code to store: >>> BaseXClient session = new BaseXClient(host, port, username, password); >>> session.execute("open " + database); >>> session.store(path, inputStream); >>> session.execute("close"); >>> >>> My code to retrieve: >>> BaseXClient session = new BaseXClient(host, port, username, password); >>> session.execute("open " + database); >>> session.execute("retrieve " + path, outputStream); >>> session.execute("close"); >>> >>> Thanks in advance! >>> >>> >>> >> >
EJ
E. Wray Johnson28 Jan 2016 17:57 #
So the problem was my fault as I suspected. And I fixed it and I am now getting the correct data. I was attempting to retrieve a binary/raw document using a Java client without the open/close which locks a database by switching from using a retrieve command to db:retrieve XQuery. I added the following method to the client Query object. It works fine, but let me know if you see anything wrong with it. I particular, is the if(!ok()) needed? I suggest this as an enhancement. Also, is it possible to achieve, by enhancement, similar results using db:store in an XQuery from Java client and an InputStream?
public void execute(final OutputStream output) throws
           IOException {
            out.write(4);
            send(id);
            if (in.read() > 0) {
                receive(in, output);
            }
            info = receive();
            if (!ok()) {
                throw new IOException(info);
            }
        }
Quoted text · Christian Grün · 139 lines
On Thu, Jan 28, 2016 at 10:01 AM, Christian Grün <christian.gruen at gmail.com> wrote: > > I am getting the incorrect data downloaded as Mike Engledew said he used > the > > "raw" output method to fix. However, "raw" is not listed as one of the > > valid types for this option. > > So how should I ensure binary files get downloaded. accurately? > > Any SSCCE? > > > > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> See here: http://docs.basex.org/wiki/Serialization > >> > >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" > >> <wray.johnson at gmail.com>: > >>> > >>> With 8.4... > >>> > >>> declare option output:method "raw"; > >>> > >>> I get > >>> > >>> [SEPM0016] Value of 'method' must be one of > >>> (xml,xhtml,html,json,csv,text,adaptive,basex). > >>> > >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew < > mike.engledew at gmail.com> > >>> wrote: > >>>> > >>>> If like me your using BaseX 8.3.1 don't forget to add ... > >>>> > >>>> declare option output:method "raw"; > >>>> > >>>> before the db:retrieve() otherwise you won't get back what you put > in. > >>>> I learned this the hard way trying to extract binary files that were > >>>> stored in codepage IBM1047 and it took me a while to figure out why > they > >>>> came back translated; > >>>> > >>>> > >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson > >>>> <wray.johnson at gmail.com> wrote: > >>>>> > >>>>> When using the sample Java client to store/retrieve binary/raw files > I > >>>>> have to execute separate open/close commands. It is my > understanding that > >>>>> this locks the database. In a multi-user scenario, such locking is > >>>>> undesirable as binary/raw files can be quite large. And since there > is no > >>>>> alteration of the indexes etc during such operations, is there a > better way? > >>>>> Also, why isn't there a corresponding retrieve method on the Java > client, > >>>>> similar to the store method? > >>>>> > >>>>> My code to store: > >>>>> BaseXClient session = new BaseXClient(host, port, username, > password); > >>>>> session.execute("open " + database); > >>>>> session.store(path, inputStream); > >>>>> session.execute("close"); > >>>>> > >>>>> My code to retrieve: > >>>>> BaseXClient session = new BaseXClient(host, port, username, > password); > >>>>> session.execute("open " + database); > >>>>> session.execute("retrieve " + path, outputStream); > >>>>> session.execute("close"); > >>>>> > >>>>> Thanks in advance! > >>>>> > >>>>> > >>>> > >>> > > > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün <christian.gruen at gmail.com> wrote: > See here: http://docs.basex.org/wiki/Serialization > Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" <wray.johnson at gmail.com > >: > >> With 8.4... >> >> declare option output:method "raw"; >> >> I get >> >> [SEPM0016] Value of 'method' must be one of >> (xml,xhtml,html,json,csv,text,adaptive,basex). >> >> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew <mike.engledew at gmail.com> >> wrote: >> >>> If like me your using BaseX 8.3.1 don't forget to add ... >>> >>> declare option output:method "raw"; >>> >>> before the db:retrieve() otherwise you won't get back what you put in. >>> I learned this the hard way trying to extract binary files that were >>> stored in codepage IBM1047 and it took me a while to figure out why they >>> came back translated; >>> >>> >>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson < >>> wray.johnson at gmail.com> wrote: >>> >>>> When using the sample Java client to store/retrieve binary/raw files I >>>> have to execute separate open/close commands. It is my understanding that >>>> this locks the database. In a multi-user scenario, such locking is >>>> undesirable as binary/raw files can be quite large. And since there is no >>>> alteration of the indexes etc during such operations, is there a better >>>> way? Also, why isn't there a corresponding retrieve method on the Java >>>> client, similar to the store method? >>>> >>>> My code to store: >>>> BaseXClient session = new BaseXClient(host, port, username, password); >>>> session.execute("open " + database); >>>> session.store(path, inputStream); >>>> session.execute("close"); >>>> >>>> My code to retrieve: >>>> BaseXClient session = new BaseXClient(host, port, username, password); >>>> session.execute("open " + database); >>>> session.execute("retrieve " + path, outputStream); >>>> session.execute("close"); >>>> >>>> Thanks in advance! >>>> >>>> >>>> >>> >>
CG
Christian Grün29 Jan 2016 10:50 #
Please be more specific: Could you give me the exact reference to the class you have extended? If it’s [1], the following might work:
    public void execute(final OutputStream output) throws IOException {
      out.write(4);
      send(id);
      receive(in, output);
      if(!ok()) throw new IOException(receive());
    }
Did you try to use the BaseXClient.execute method as well?
    BaseXClient client = new BaseXClient("localhost", 1984, "admin", "admin");
    try {
      ByteArrayOutputStream os = new ByteArrayOutputStream();
      client.execute("xquery db:retrieve('db', 'file.bin)", os);
      final byte[] data = os.toByteArray();
      System.out.println("Size of binary data: " + data.length);
    } finally {
      client.close();
    }
[1] https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/org/basex/examples/api/BaseXClient.java
Quoted text · E. Wray Johnson · 169 lines
On Thu, Jan 28, 2016 at 4:57 PM, E. Wray Johnson <wray.johnson at gmail.com> wrote: > So the problem was my fault as I suspected. And I fixed it and I am now > getting the correct data. I was attempting to retrieve a binary/raw > document using a Java client without the open/close which locks a database > by switching from using a retrieve command to db:retrieve XQuery. I added > the following method to the client Query object. It works fine, but let me > know if you see anything wrong with it. I particular, is the if(!ok()) > needed? I suggest this as an enhancement. Also, is it possible to achieve, > by enhancement, similar results using db:store in an XQuery from Java client > and an InputStream? > > public void execute(final OutputStream output) throws > IOException { > out.write(4); > send(id); > if (in.read() > 0) { > receive(in, output); > } > info = receive(); > if (!ok()) { > throw new IOException(info); > } > } > > > On Thu, Jan 28, 2016 at 10:01 AM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> > I am getting the incorrect data downloaded as Mike Engledew said he used >> > the >> > "raw" output method to fix. However, "raw" is not listed as one of the >> > valid types for this option. >> > So how should I ensure binary files get downloaded. accurately? >> >> Any SSCCE? >> >> >> > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> See here: http://docs.basex.org/wiki/Serialization >> >> >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" >> >> <wray.johnson at gmail.com>: >> >>> >> >>> With 8.4... >> >>> >> >>> declare option output:method "raw"; >> >>> >> >>> I get >> >>> >> >>> [SEPM0016] Value of 'method' must be one of >> >>> (xml,xhtml,html,json,csv,text,adaptive,basex). >> >>> >> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew >> >>> <mike.engledew at gmail.com> >> >>> wrote: >> >>>> >> >>>> If like me your using BaseX 8.3.1 don't forget to add ... >> >>>> >> >>>> declare option output:method "raw"; >> >>>> >> >>>> before the db:retrieve() otherwise you won't get back what you put >> >>>> in. >> >>>> I learned this the hard way trying to extract binary files that were >> >>>> stored in codepage IBM1047 and it took me a while to figure out why >> >>>> they >> >>>> came back translated; >> >>>> >> >>>> >> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson >> >>>> <wray.johnson at gmail.com> wrote: >> >>>>> >> >>>>> When using the sample Java client to store/retrieve binary/raw files >> >>>>> I >> >>>>> have to execute separate open/close commands. It is my >> >>>>> understanding that >> >>>>> this locks the database. In a multi-user scenario, such locking is >> >>>>> undesirable as binary/raw files can be quite large. And since there >> >>>>> is no >> >>>>> alteration of the indexes etc during such operations, is there a >> >>>>> better way? >> >>>>> Also, why isn't there a corresponding retrieve method on the Java >> >>>>> client, >> >>>>> similar to the store method? >> >>>>> >> >>>>> My code to store: >> >>>>> BaseXClient session = new BaseXClient(host, port, username, >> >>>>> password); >> >>>>> session.execute("open " + database); >> >>>>> session.store(path, inputStream); >> >>>>> session.execute("close"); >> >>>>> >> >>>>> My code to retrieve: >> >>>>> BaseXClient session = new BaseXClient(host, port, username, >> >>>>> password); >> >>>>> session.execute("open " + database); >> >>>>> session.execute("retrieve " + path, outputStream); >> >>>>> session.execute("close"); >> >>>>> >> >>>>> Thanks in advance! >> >>>>> >> >>>>> >> >>>> >> >>> >> > > > > > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> See here: http://docs.basex.org/wiki/Serialization >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" >> <wray.johnson at gmail.com>: >>> >>> With 8.4... >>> >>> declare option output:method "raw"; >>> >>> I get >>> >>> [SEPM0016] Value of 'method' must be one of >>> (xml,xhtml,html,json,csv,text,adaptive,basex). >>> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew <mike.engledew at gmail.com> >>> wrote: >>>> >>>> If like me your using BaseX 8.3.1 don't forget to add ... >>>> >>>> declare option output:method "raw"; >>>> >>>> before the db:retrieve() otherwise you won't get back what you put in. >>>> I learned this the hard way trying to extract binary files that were >>>> stored in codepage IBM1047 and it took me a while to figure out why they >>>> came back translated; >>>> >>>> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson >>>> <wray.johnson at gmail.com> wrote: >>>>> >>>>> When using the sample Java client to store/retrieve binary/raw files I >>>>> have to execute separate open/close commands. It is my understanding that >>>>> this locks the database. In a multi-user scenario, such locking is >>>>> undesirable as binary/raw files can be quite large. And since there is no >>>>> alteration of the indexes etc during such operations, is there a better way? >>>>> Also, why isn't there a corresponding retrieve method on the Java client, >>>>> similar to the store method? >>>>> >>>>> My code to store: >>>>> BaseXClient session = new BaseXClient(host, port, username, password); >>>>> session.execute("open " + database); >>>>> session.store(path, inputStream); >>>>> session.execute("close"); >>>>> >>>>> My code to retrieve: >>>>> BaseXClient session = new BaseXClient(host, port, username, password); >>>>> session.execute("open " + database); >>>>> session.execute("retrieve " + path, outputStream); >>>>> session.execute("close"); >>>>> >>>>> Thanks in advance! >>>>> >>>>> >>>> >>> >
CG
Christian Grünin reply to E. Wray Johnson, 29 Jan 2016, 16:5529 Jan 2016 16:00 #
Quoted text · E. Wray Johnson · 2 lines
> However, what I really need is a way to do db:store and db:replace via > Xquery with an InputStream (e.g. without requiring the open/close commands).
That will be difficult, because data to be inserted may occur anywhere inside an XQuery expression..
Quoted text · E. Wray Johnson · 232 lines
> > > On Fri, Jan 29, 2016 at 4:50 AM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Please be more specific: Could you give me the exact reference to the >> class you have extended? >> >> If it’s [1], the following might work: >> >> public void execute(final OutputStream output) throws IOException { >> out.write(4); >> send(id); >> receive(in, output); >> if(!ok()) throw new IOException(receive()); >> } >> >> Did you try to use the BaseXClient.execute method as well? >> >> BaseXClient client = new BaseXClient("localhost", 1984, "admin", >> "admin"); >> try { >> ByteArrayOutputStream os = new ByteArrayOutputStream(); >> client.execute("xquery db:retrieve('db', 'file.bin)", os); >> final byte[] data = os.toByteArray(); >> System.out.println("Size of binary data: " + data.length); >> } finally { >> client.close(); >> } >> >> >> [1] >> https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/org/basex/examples/api/BaseXClient.java >> >> >> On Thu, Jan 28, 2016 at 4:57 PM, E. Wray Johnson <wray.johnson at gmail.com> >> wrote: >> > So the problem was my fault as I suspected. And I fixed it and I am now >> > getting the correct data. I was attempting to retrieve a binary/raw >> > document using a Java client without the open/close which locks a >> > database >> > by switching from using a retrieve command to db:retrieve XQuery. I >> > added >> > the following method to the client Query object. It works fine, but let >> > me >> > know if you see anything wrong with it. I particular, is the if(!ok()) >> > needed? I suggest this as an enhancement. Also, is it possible to >> > achieve, >> > by enhancement, similar results using db:store in an XQuery from Java >> > client >> > and an InputStream? >> > >> > public void execute(final OutputStream output) throws >> > IOException { >> > out.write(4); >> > send(id); >> > if (in.read() > 0) { >> > receive(in, output); >> > } >> > info = receive(); >> > if (!ok()) { >> > throw new IOException(info); >> > } >> > } >> > >> > >> > On Thu, Jan 28, 2016 at 10:01 AM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> > I am getting the incorrect data downloaded as Mike Engledew said he >> >> > used >> >> > the >> >> > "raw" output method to fix. However, "raw" is not listed as one of >> >> > the >> >> > valid types for this option. >> >> > So how should I ensure binary files get downloaded. accurately? >> >> >> >> Any SSCCE? >> >> >> >> >> >> > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün >> >> > <christian.gruen at gmail.com> >> >> > wrote: >> >> >> >> >> >> See here: http://docs.basex.org/wiki/Serialization >> >> >> >> >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" >> >> >> <wray.johnson at gmail.com>: >> >> >>> >> >> >>> With 8.4... >> >> >>> >> >> >>> declare option output:method "raw"; >> >> >>> >> >> >>> I get >> >> >>> >> >> >>> [SEPM0016] Value of 'method' must be one of >> >> >>> (xml,xhtml,html,json,csv,text,adaptive,basex). >> >> >>> >> >> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew >> >> >>> <mike.engledew at gmail.com> >> >> >>> wrote: >> >> >>>> >> >> >>>> If like me your using BaseX 8.3.1 don't forget to add ... >> >> >>>> >> >> >>>> declare option output:method "raw"; >> >> >>>> >> >> >>>> before the db:retrieve() otherwise you won't get back what you >> >> >>>> put >> >> >>>> in. >> >> >>>> I learned this the hard way trying to extract binary files that >> >> >>>> were >> >> >>>> stored in codepage IBM1047 and it took me a while to figure out >> >> >>>> why >> >> >>>> they >> >> >>>> came back translated; >> >> >>>> >> >> >>>> >> >> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson >> >> >>>> <wray.johnson at gmail.com> wrote: >> >> >>>>> >> >> >>>>> When using the sample Java client to store/retrieve binary/raw >> >> >>>>> files >> >> >>>>> I >> >> >>>>> have to execute separate open/close commands. It is my >> >> >>>>> understanding that >> >> >>>>> this locks the database. In a multi-user scenario, such locking >> >> >>>>> is >> >> >>>>> undesirable as binary/raw files can be quite large. And since >> >> >>>>> there >> >> >>>>> is no >> >> >>>>> alteration of the indexes etc during such operations, is there a >> >> >>>>> better way? >> >> >>>>> Also, why isn't there a corresponding retrieve method on the Java >> >> >>>>> client, >> >> >>>>> similar to the store method? >> >> >>>>> >> >> >>>>> My code to store: >> >> >>>>> BaseXClient session = new BaseXClient(host, port, username, >> >> >>>>> password); >> >> >>>>> session.execute("open " + database); >> >> >>>>> session.store(path, inputStream); >> >> >>>>> session.execute("close"); >> >> >>>>> >> >> >>>>> My code to retrieve: >> >> >>>>> BaseXClient session = new BaseXClient(host, port, username, >> >> >>>>> password); >> >> >>>>> session.execute("open " + database); >> >> >>>>> session.execute("retrieve " + path, outputStream); >> >> >>>>> session.execute("close"); >> >> >>>>> >> >> >>>>> Thanks in advance! >> >> >>>>> >> >> >>>>> >> >> >>>> >> >> >>> >> >> > >> > >> > >> > >> > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> See here: http://docs.basex.org/wiki/Serialization >> >> >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" >> >> <wray.johnson at gmail.com>: >> >>> >> >>> With 8.4... >> >>> >> >>> declare option output:method "raw"; >> >>> >> >>> I get >> >>> >> >>> [SEPM0016] Value of 'method' must be one of >> >>> (xml,xhtml,html,json,csv,text,adaptive,basex). >> >>> >> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew >> >>> <mike.engledew at gmail.com> >> >>> wrote: >> >>>> >> >>>> If like me your using BaseX 8.3.1 don't forget to add ... >> >>>> >> >>>> declare option output:method "raw"; >> >>>> >> >>>> before the db:retrieve() otherwise you won't get back what you put >> >>>> in. >> >>>> I learned this the hard way trying to extract binary files that were >> >>>> stored in codepage IBM1047 and it took me a while to figure out why >> >>>> they >> >>>> came back translated; >> >>>> >> >>>> >> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson >> >>>> <wray.johnson at gmail.com> wrote: >> >>>>> >> >>>>> When using the sample Java client to store/retrieve binary/raw files >> >>>>> I >> >>>>> have to execute separate open/close commands. It is my >> >>>>> understanding that >> >>>>> this locks the database. In a multi-user scenario, such locking is >> >>>>> undesirable as binary/raw files can be quite large. And since there >> >>>>> is no >> >>>>> alteration of the indexes etc during such operations, is there a >> >>>>> better way? >> >>>>> Also, why isn't there a corresponding retrieve method on the Java >> >>>>> client, >> >>>>> similar to the store method? >> >>>>> >> >>>>> My code to store: >> >>>>> BaseXClient session = new BaseXClient(host, port, username, >> >>>>> password); >> >>>>> session.execute("open " + database); >> >>>>> session.store(path, inputStream); >> >>>>> session.execute("close"); >> >>>>> >> >>>>> My code to retrieve: >> >>>>> BaseXClient session = new BaseXClient(host, port, username, >> >>>>> password); >> >>>>> session.execute("open " + database); >> >>>>> session.execute("retrieve " + path, outputStream); >> >>>>> session.execute("close"); >> >>>>> >> >>>>> Thanks in advance! >> >>>>> >> >>>>> >> >>>> >> >>> >> > > >
EJ
E. Wray Johnsonin reply to Christian Grün, 29 Jan 2016, 10:5029 Jan 2016 16:55 #
Yes the code I modified is indeed [1] java/org/basex/examples/api/BaseXClient.java <https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/org/basex/examples/api/BaseXClient.java> The following line was needed to handle the first byte of the query result. if (in.read() > 0) I do like your "client.execute("xquery db:retrieve" suggestion better as it does not require me to change the example client code. However, what I really need is a way to do db:store and db:replace via Xquery with an InputStream (e.g. without requiring the open/close commands).
Quoted text · Christian Grün · 226 lines
On Fri, Jan 29, 2016 at 4:50 AM, Christian Grün <christian.gruen at gmail.com> wrote: > Please be more specific: Could you give me the exact reference to the > class you have extended? > > If it’s [1], the following might work: > > public void execute(final OutputStream output) throws IOException { > out.write(4); > send(id); > receive(in, output); > if(!ok()) throw new IOException(receive()); > } > > Did you try to use the BaseXClient.execute method as well? > > BaseXClient client = new BaseXClient("localhost", 1984, "admin", > "admin"); > try { > ByteArrayOutputStream os = new ByteArrayOutputStream(); > client.execute("xquery db:retrieve('db', 'file.bin)", os); > final byte[] data = os.toByteArray(); > System.out.println("Size of binary data: " + data.length); > } finally { > client.close(); > } > > > [1] > https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/org/basex/examples/api/BaseXClient.java > > > On Thu, Jan 28, 2016 at 4:57 PM, E. Wray Johnson <wray.johnson at gmail.com> > wrote: > > So the problem was my fault as I suspected. And I fixed it and I am now > > getting the correct data. I was attempting to retrieve a binary/raw > > document using a Java client without the open/close which locks a > database > > by switching from using a retrieve command to db:retrieve XQuery. I > added > > the following method to the client Query object. It works fine, but let > me > > know if you see anything wrong with it. I particular, is the if(!ok()) > > needed? I suggest this as an enhancement. Also, is it possible to > achieve, > > by enhancement, similar results using db:store in an XQuery from Java > client > > and an InputStream? > > > > public void execute(final OutputStream output) throws > > IOException { > > out.write(4); > > send(id); > > if (in.read() > 0) { > > receive(in, output); > > } > > info = receive(); > > if (!ok()) { > > throw new IOException(info); > > } > > } > > > > > > On Thu, Jan 28, 2016 at 10:01 AM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> > I am getting the incorrect data downloaded as Mike Engledew said he > used > >> > the > >> > "raw" output method to fix. However, "raw" is not listed as one of > the > >> > valid types for this option. > >> > So how should I ensure binary files get downloaded. accurately? > >> > >> Any SSCCE? > >> > >> > >> > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün > >> > <christian.gruen at gmail.com> > >> > wrote: > >> >> > >> >> See here: http://docs.basex.org/wiki/Serialization > >> >> > >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" > >> >> <wray.johnson at gmail.com>: > >> >>> > >> >>> With 8.4... > >> >>> > >> >>> declare option output:method "raw"; > >> >>> > >> >>> I get > >> >>> > >> >>> [SEPM0016] Value of 'method' must be one of > >> >>> (xml,xhtml,html,json,csv,text,adaptive,basex). > >> >>> > >> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew > >> >>> <mike.engledew at gmail.com> > >> >>> wrote: > >> >>>> > >> >>>> If like me your using BaseX 8.3.1 don't forget to add ... > >> >>>> > >> >>>> declare option output:method "raw"; > >> >>>> > >> >>>> before the db:retrieve() otherwise you won't get back what you put > >> >>>> in. > >> >>>> I learned this the hard way trying to extract binary files that > were > >> >>>> stored in codepage IBM1047 and it took me a while to figure out why > >> >>>> they > >> >>>> came back translated; > >> >>>> > >> >>>> > >> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson > >> >>>> <wray.johnson at gmail.com> wrote: > >> >>>>> > >> >>>>> When using the sample Java client to store/retrieve binary/raw > files > >> >>>>> I > >> >>>>> have to execute separate open/close commands. It is my > >> >>>>> understanding that > >> >>>>> this locks the database. In a multi-user scenario, such locking > is > >> >>>>> undesirable as binary/raw files can be quite large. And since > there > >> >>>>> is no > >> >>>>> alteration of the indexes etc during such operations, is there a > >> >>>>> better way? > >> >>>>> Also, why isn't there a corresponding retrieve method on the Java > >> >>>>> client, > >> >>>>> similar to the store method? > >> >>>>> > >> >>>>> My code to store: > >> >>>>> BaseXClient session = new BaseXClient(host, port, username, > >> >>>>> password); > >> >>>>> session.execute("open " + database); > >> >>>>> session.store(path, inputStream); > >> >>>>> session.execute("close"); > >> >>>>> > >> >>>>> My code to retrieve: > >> >>>>> BaseXClient session = new BaseXClient(host, port, username, > >> >>>>> password); > >> >>>>> session.execute("open " + database); > >> >>>>> session.execute("retrieve " + path, outputStream); > >> >>>>> session.execute("close"); > >> >>>>> > >> >>>>> Thanks in advance! > >> >>>>> > >> >>>>> > >> >>>> > >> >>> > >> > > > > > > > > > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> See here: http://docs.basex.org/wiki/Serialization > >> > >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" > >> <wray.johnson at gmail.com>: > >>> > >>> With 8.4... > >>> > >>> declare option output:method "raw"; > >>> > >>> I get > >>> > >>> [SEPM0016] Value of 'method' must be one of > >>> (xml,xhtml,html,json,csv,text,adaptive,basex). > >>> > >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew < > mike.engledew at gmail.com> > >>> wrote: > >>>> > >>>> If like me your using BaseX 8.3.1 don't forget to add ... > >>>> > >>>> declare option output:method "raw"; > >>>> > >>>> before the db:retrieve() otherwise you won't get back what you put > in. > >>>> I learned this the hard way trying to extract binary files that were > >>>> stored in codepage IBM1047 and it took me a while to figure out why > they > >>>> came back translated; > >>>> > >>>> > >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson > >>>> <wray.johnson at gmail.com> wrote: > >>>>> > >>>>> When using the sample Java client to store/retrieve binary/raw files > I > >>>>> have to execute separate open/close commands. It is my > understanding that > >>>>> this locks the database. In a multi-user scenario, such locking is > >>>>> undesirable as binary/raw files can be quite large. And since there > is no > >>>>> alteration of the indexes etc during such operations, is there a > better way? > >>>>> Also, why isn't there a corresponding retrieve method on the Java > client, > >>>>> similar to the store method? > >>>>> > >>>>> My code to store: > >>>>> BaseXClient session = new BaseXClient(host, port, username, > password); > >>>>> session.execute("open " + database); > >>>>> session.store(path, inputStream); > >>>>> session.execute("close"); > >>>>> > >>>>> My code to retrieve: > >>>>> BaseXClient session = new BaseXClient(host, port, username, > password); > >>>>> session.execute("open " + database); > >>>>> session.execute("retrieve " + path, outputStream); > >>>>> session.execute("close"); > >>>>> > >>>>> Thanks in advance! > >>>>> > >>>>> > >>>> > >>> > > >
EJ
E. Wray Johnsonin reply to Christian Grün, 29 Jan 2016, 16:0029 Jan 2016 17:11 #
Yes, I figured that. I would have been nice if the ADD(09), REPLACE(0C), STORE(0D) protocol commands would have accepted a database name as a first parameter, like their corresponding Database Module functions. This would make it easier to work on multiple databases.
Quoted text · Christian Grün · 255 lines
On Fri, Jan 29, 2016 at 10:00 AM, Christian Grün <christian.gruen at gmail.com> wrote: > > However, what I really need is a way to do db:store and db:replace via > > Xquery with an InputStream (e.g. without requiring the open/close > commands). > > That will be difficult, because data to be inserted may occur anywhere > inside an XQuery expression.. > > > > > > > On Fri, Jan 29, 2016 at 4:50 AM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> Please be more specific: Could you give me the exact reference to the > >> class you have extended? > >> > >> If it’s [1], the following might work: > >> > >> public void execute(final OutputStream output) throws IOException { > >> out.write(4); > >> send(id); > >> receive(in, output); > >> if(!ok()) throw new IOException(receive()); > >> } > >> > >> Did you try to use the BaseXClient.execute method as well? > >> > >> BaseXClient client = new BaseXClient("localhost", 1984, "admin", > >> "admin"); > >> try { > >> ByteArrayOutputStream os = new ByteArrayOutputStream(); > >> client.execute("xquery db:retrieve('db', 'file.bin)", os); > >> final byte[] data = os.toByteArray(); > >> System.out.println("Size of binary data: " + data.length); > >> } finally { > >> client.close(); > >> } > >> > >> > >> [1] > >> > https://github.com/BaseXdb/basex/blob/master/basex-examples/src/main/java/org/basex/examples/api/BaseXClient.java > >> > >> > >> On Thu, Jan 28, 2016 at 4:57 PM, E. Wray Johnson < > wray.johnson at gmail.com> > >> wrote: > >> > So the problem was my fault as I suspected. And I fixed it and I am > now > >> > getting the correct data. I was attempting to retrieve a binary/raw > >> > document using a Java client without the open/close which locks a > >> > database > >> > by switching from using a retrieve command to db:retrieve XQuery. I > >> > added > >> > the following method to the client Query object. It works fine, but > let > >> > me > >> > know if you see anything wrong with it. I particular, is the > if(!ok()) > >> > needed? I suggest this as an enhancement. Also, is it possible to > >> > achieve, > >> > by enhancement, similar results using db:store in an XQuery from Java > >> > client > >> > and an InputStream? > >> > > >> > public void execute(final OutputStream output) throws > >> > IOException { > >> > out.write(4); > >> > send(id); > >> > if (in.read() > 0) { > >> > receive(in, output); > >> > } > >> > info = receive(); > >> > if (!ok()) { > >> > throw new IOException(info); > >> > } > >> > } > >> > > >> > > >> > On Thu, Jan 28, 2016 at 10:01 AM, Christian Grün > >> > <christian.gruen at gmail.com> > >> > wrote: > >> >> > >> >> > I am getting the incorrect data downloaded as Mike Engledew said he > >> >> > used > >> >> > the > >> >> > "raw" output method to fix. However, "raw" is not listed as one of > >> >> > the > >> >> > valid types for this option. > >> >> > So how should I ensure binary files get downloaded. accurately? > >> >> > >> >> Any SSCCE? > >> >> > >> >> > >> >> > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün > >> >> > <christian.gruen at gmail.com> > >> >> > wrote: > >> >> >> > >> >> >> See here: http://docs.basex.org/wiki/Serialization > >> >> >> > >> >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" > >> >> >> <wray.johnson at gmail.com>: > >> >> >>> > >> >> >>> With 8.4... > >> >> >>> > >> >> >>> declare option output:method "raw"; > >> >> >>> > >> >> >>> I get > >> >> >>> > >> >> >>> [SEPM0016] Value of 'method' must be one of > >> >> >>> (xml,xhtml,html,json,csv,text,adaptive,basex). > >> >> >>> > >> >> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew > >> >> >>> <mike.engledew at gmail.com> > >> >> >>> wrote: > >> >> >>>> > >> >> >>>> If like me your using BaseX 8.3.1 don't forget to add ... > >> >> >>>> > >> >> >>>> declare option output:method "raw"; > >> >> >>>> > >> >> >>>> before the db:retrieve() otherwise you won't get back what you > >> >> >>>> put > >> >> >>>> in. > >> >> >>>> I learned this the hard way trying to extract binary files that > >> >> >>>> were > >> >> >>>> stored in codepage IBM1047 and it took me a while to figure out > >> >> >>>> why > >> >> >>>> they > >> >> >>>> came back translated; > >> >> >>>> > >> >> >>>> > >> >> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson > >> >> >>>> <wray.johnson at gmail.com> wrote: > >> >> >>>>> > >> >> >>>>> When using the sample Java client to store/retrieve binary/raw > >> >> >>>>> files > >> >> >>>>> I > >> >> >>>>> have to execute separate open/close commands. It is my > >> >> >>>>> understanding that > >> >> >>>>> this locks the database. In a multi-user scenario, such > locking > >> >> >>>>> is > >> >> >>>>> undesirable as binary/raw files can be quite large. And since > >> >> >>>>> there > >> >> >>>>> is no > >> >> >>>>> alteration of the indexes etc during such operations, is there > a > >> >> >>>>> better way? > >> >> >>>>> Also, why isn't there a corresponding retrieve method on the > Java > >> >> >>>>> client, > >> >> >>>>> similar to the store method? > >> >> >>>>> > >> >> >>>>> My code to store: > >> >> >>>>> BaseXClient session = new BaseXClient(host, port, username, > >> >> >>>>> password); > >> >> >>>>> session.execute("open " + database); > >> >> >>>>> session.store(path, inputStream); > >> >> >>>>> session.execute("close"); > >> >> >>>>> > >> >> >>>>> My code to retrieve: > >> >> >>>>> BaseXClient session = new BaseXClient(host, port, username, > >> >> >>>>> password); > >> >> >>>>> session.execute("open " + database); > >> >> >>>>> session.execute("retrieve " + path, outputStream); > >> >> >>>>> session.execute("close"); > >> >> >>>>> > >> >> >>>>> Thanks in advance! > >> >> >>>>> > >> >> >>>>> > >> >> >>>> > >> >> >>> > >> >> > > >> > > >> > > >> > > >> > On Thu, Jan 28, 2016 at 3:08 AM, Christian Grün > >> > <christian.gruen at gmail.com> > >> > wrote: > >> >> > >> >> See here: http://docs.basex.org/wiki/Serialization > >> >> > >> >> Am 28.01.2016 7:13 vorm. schrieb "E. Wray Johnson" > >> >> <wray.johnson at gmail.com>: > >> >>> > >> >>> With 8.4... > >> >>> > >> >>> declare option output:method "raw"; > >> >>> > >> >>> I get > >> >>> > >> >>> [SEPM0016] Value of 'method' must be one of > >> >>> (xml,xhtml,html,json,csv,text,adaptive,basex). > >> >>> > >> >>> On Sat, Jan 16, 2016 at 2:06 PM, Mike Engledew > >> >>> <mike.engledew at gmail.com> > >> >>> wrote: > >> >>>> > >> >>>> If like me your using BaseX 8.3.1 don't forget to add ... > >> >>>> > >> >>>> declare option output:method "raw"; > >> >>>> > >> >>>> before the db:retrieve() otherwise you won't get back what you put > >> >>>> in. > >> >>>> I learned this the hard way trying to extract binary files that > were > >> >>>> stored in codepage IBM1047 and it took me a while to figure out why > >> >>>> they > >> >>>> came back translated; > >> >>>> > >> >>>> > >> >>>> On Fri, Jan 15, 2016 at 11:11 AM, E. Wray Johnson > >> >>>> <wray.johnson at gmail.com> wrote: > >> >>>>> > >> >>>>> When using the sample Java client to store/retrieve binary/raw > files > >> >>>>> I > >> >>>>> have to execute separate open/close commands. It is my > >> >>>>> understanding that > >> >>>>> this locks the database. In a multi-user scenario, such locking > is > >> >>>>> undesirable as binary/raw files can be quite large. And since > there > >> >>>>> is no > >> >>>>> alteration of the indexes etc during such operations, is there a > >> >>>>> better way? > >> >>>>> Also, why isn't there a corresponding retrieve method on the Java > >> >>>>> client, > >> >>>>> similar to the store method? > >> >>>>> > >> >>>>> My code to store: > >> >>>>> BaseXClient session = new BaseXClient(host, port, username, > >> >>>>> password); > >> >>>>> session.execute("open " + database); > >> >>>>> session.store(path, inputStream); > >> >>>>> session.execute("close"); > >> >>>>> > >> >>>>> My code to retrieve: > >> >>>>> BaseXClient session = new BaseXClient(host, port, username, > >> >>>>> password); > >> >>>>> session.execute("open " + database); > >> >>>>> session.execute("retrieve " + path, outputStream); > >> >>>>> session.execute("close"); > >> >>>>> > >> >>>>> Thanks in advance! > >> >>>>> > >> >>>>> > >> >>>> > >> >>> > >> > > > > > >

Written for BaseX 8.3. BaseX may have changed since: please check the current documentation.

« SecurityAny (hopefully representative) reference korpora of XML-ish documents? »
⚡Generated with XQuery