Improvements. On-fly in-memory temporary index
Monday, 6 May 2013 · 2 messages · last reply 7 May 2013
AS
Alexander Shpack6 May 2013 23:40 #
Hi Folks!
I had one task with csv parsing and transforming. Main file contains approximately 100k of nodes.
Transformation process should be in memory only, and I thought, that in-memory temporary index would be useful for this type of tasks.
Useful scenario
copy $clone := $csvContent
modify
(
tmp:optimize($clone),
...
)
return $clone
Main goal - increase performance with huge document fragments, modified and changed on-fly.
Is it possible?
CG
Christian Grün7 May 2013 17:11 #
Hi Alex,
if I get it right, you need a faster solution to parse and transform your CSV files? The transform expression (using copy...return) does indeed allow you to do main-memory-only transformations of your data. I’m not sure, though, how the temporary index you propose would look like.
Best,
Christian
Quoted text · Alexander Shpack · 26 lines
On Mon, May 6, 2013 at 11:40 PM, Alexander Shpack <shadowkin at gmail.com> wrote: > Hi Folks! > > I had one task with csv parsing and transforming. Main file contains > approximately 100k of nodes. > Transformation process should be in memory only, and I thought, that > in-memory temporary index would be useful for this type of tasks. > > Useful scenario > > copy $clone := $csvContent > modify > ( > tmp:optimize($clone), > ... > ) > return $clone > > Main goal - increase performance with huge document fragments, > modified and changed on-fly. > > Is it possible? > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Written for BaseX 7.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery