Gzip/RESTXQ
Tuesday, 21 February 2017 · 7 messages · last reply 23 Feb 2017
M
meumapple21 Feb 2017 19:12 #
Hi All,
I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful?
Best,
Joseph
CG
Christian Grün21 Feb 2017 19:53 #
Hi joseph,
Quoted text · meumapple · 1 line
> I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful?
Compression is always useful if you transfer data that can still be reduced in size. In other words, if you store JPG files, compression won’t help so much, but it will surely shrink JSON or WAV files.
Cheers,
Christian
KS
Kendall Shaw23 Feb 2017 02:22 #
There is some point at which the time it takes to compress is worse in some measure than serving uncompressed data.
What I did was store precompressed data and serve that, instead of enabling compression. So if that is an option you can use:
web:response-header(map {'media-type': 'application/wordstar'}, map {'Content-Encoding': 'gzip'}) , db:retrieve('test-binary', 'out.bin')
where out.bin contains gzipped data.
The web server might then serve the compressed data if Accept-Encoding contains deflate or gzip
KendallQuoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün · 1 line
On 2/21/17, 10:53 AM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of christian.gruen at gmail.com> wrote:
Hi joseph,
Quoted text · Christian Grün · 1 line
> I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful?
Compression is always useful if you transfer data that can still be reduced in size. In other words, if you store JPG files, compression won’t help so much, but it will surely shrink JSON or WAV files.
Cheers,
Christian
DK
Dirk Kirsten23 Feb 2017 08:51 #
Hello Kendall,
I have to go off-topic here for once, because it caught my eye and I couldn't believe it: wordstar? Really? And I thought working with XML is sometimes an ancient technology, but wordstar is... well, let's call it well-established ;) Or are you working for George R.R. Martin (see http://grrm.livejournal.com/197075.html), and if so how will Game Of Thrones end? ;)
I am glad to see that XQuery is really a technology used for all kind of data formats...
Cheers
Dirk
Quoted text · Kendall Shaw · 28 lines
On 02/23/2017 04:52 AM, Kendall Shaw wrote: > There is some point at which the time it takes to compress is worse in some measure than serving uncompressed data. > > What I did was store precompressed data and serve that, instead of enabling compression. So if that is an option you can use: > > web:response-header(map {'media-type': 'application/wordstar'}, map {'Content-Encoding': 'gzip'}) > , db:retrieve('test-binary', 'out.bin') > > where out.bin contains gzipped data. > > The web server might then serve the compressed data if Accept-Encoding contains deflate or gzip > > Kendall > > On 2/21/17, 10:53 AM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of christian.gruen at gmail.com> wrote: > > Hi joseph, > > > I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful? > > Compression is always useful if you transfer data that can still be > reduced in size. In other words, if you store JPG files, compression > won’t help so much, but it will surely shrink JSON or WAV files. > > Cheers, > Christian > >
--
Dirk Kirsten, BaseX GmbH, http://basexgmbh.de
|-- Firmensitz: Blarerstrasse 56, 78462 Konstanz
|-- Registergericht Freiburg, HRB: 708285, Geschäftsführer:
| Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22
M
Hi Kendall,
Thanks for that. Just a question: can the compressed data on your server be searched if a RESTXQ function is supposed to perform a simple query on them before serving? Thanks.
Best,
Joseph
Quoted text · Kendall Shaw · 1 line
Il giorno 23 feb 2017, alle ore 02:22, Kendall Shaw <kendall.shaw at workday.com> ha scritto:
There is some point at which the time it takes to compress is worse in some measure than serving uncompressed data.
What I did was store precompressed data and serve that, instead of enabling compression. So if that is an option you can use:
web:response-header(map {'media-type': 'application/wordstar'}, map {'Content-Encoding': 'gzip'}) , db:retrieve('test-binary', 'out.bin')
where out.bin contains gzipped data.
The web server might then serve the compressed data if Accept-Encoding contains deflate or gzip
KendallQuoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün · 1 line
On 2/21/17, 10:53 AM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of christian.gruen at gmail.com> wrote:
Hi joseph,
Quoted text · Kendall Shaw · 1 line
> I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful?
Compression is always useful if you transfer data that can still be reduced in size. In other words, if you store JPG files, compression won’t help so much, but it will surely shrink JSON or WAV files.
Cheers,
Christian
KS
Kendall Shaw23 Feb 2017 18:50 #
I was kidding. Yes wordstar was great. My mother used troff as a typist back then.
Kendall
Quoted text · meumapple · 1 line
On 2/23/17, 12:44 AM, "meumapple" <meumapple at gmail.com> wrote:
Hi Kendall,
Thanks for that. Just a question: can the compressed data on your server be searched if a RESTXQ function is supposed to perform a simple query on them before serving? Thanks.
Best,
Joseph
Quoted text · Kendall Shaw · 1 line
Il giorno 23 feb 2017, alle ore 02:22, Kendall Shaw <kendall.shaw at workday.com> ha scritto:
There is some point at which the time it takes to compress is worse in some measure than serving uncompressed data.
What I did was store precompressed data and serve that, instead of enabling compression. So if that is an option you can use:
web:response-header(map {'media-type': 'application/wordstar'}, map {'Content-Encoding': 'gzip'}) , db:retrieve('test-binary', 'out.bin')
where out.bin contains gzipped data.
The web server might then serve the compressed data if Accept-Encoding contains deflate or gzip
KendallQuoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün · 1 line
On 2/21/17, 10:53 AM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of christian.gruen at gmail.com> wrote:
Hi joseph,
Quoted text · meumapple · 1 line
> I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful?
Compression is always useful if you transfer data that can still be reduced in size. In other words, if you store JPG files, compression won’t help so much, but it will surely shrink JSON or WAV files.
Cheers,
Christian
KS
As far as I know you have to explicitly decompress and search through the content. For example:
convert:binary-to-string(db:retrieve(…))Kendall
Quoted text · meumapple · 1 line
On 2/23/17, 12:44 AM, "meumapple" <meumapple at gmail.com> wrote:
Hi Kendall,
Thanks for that. Just a question: can the compressed data on your server be searched if a RESTXQ function is supposed to perform a simple query on them before serving? Thanks.
Best,
Joseph
Quoted text · Kendall Shaw · 1 line
Il giorno 23 feb 2017, alle ore 02:22, Kendall Shaw <kendall.shaw at workday.com> ha scritto:
There is some point at which the time it takes to compress is worse in some measure than serving uncompressed data.
What I did was store precompressed data and serve that, instead of enabling compression. So if that is an option you can use:
web:response-header(map {'media-type': 'application/wordstar'}, map {'Content-Encoding': 'gzip'}) , db:retrieve('test-binary', 'out.bin')
where out.bin contains gzipped data.
The web server might then serve the compressed data if Accept-Encoding contains deflate or gzip
KendallQuoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün · 1 line
On 2/21/17, 10:53 AM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Christian Grün" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of christian.gruen at gmail.com> wrote:
Hi joseph,
Quoted text · meumapple · 1 line
> I am wondering whether for RESTXQ one should enable gzip encoding to improve data transmission: if the database is used for storage (where the data are in binary form), is the gzip option still useful?
Compression is always useful if you transfer data that can still be reduced in size. In other words, if you store JPG files, compression won’t help so much, but it will surely shrink JSON or WAV files.
Cheers,
Christian
Written for BaseX 8.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery