Improvements. On-fly in-memory temporary index
Hi Folks!
I had one task with csv parsing and transforming. Main file contains approximately 100k of nodes. Transformation process should be in memory only, and I thought, that in-memory temporary index would be useful for this type of tasks.
Useful scenario
copy $clone := $csvContent modify ( tmp:optimize($clone), ... ) return $clone
Main goal - increase performance with huge document fragments, modified and changed on-fly.
Is it possible?
Hi Alex,
if I get it right, you need a faster solution to parse and transform your CSV files? The transform expression (using copy...return) does indeed allow you to do main-memory-only transformations of your data. I’m not sure, though, how the temporary index you propose would look like.
Best, Christian
Quoted text · Alexander Shpack · 26 lines
On Mon, May 6, 2013 at 11:40 PM, Alexander Shpack <shadowkin at gmail.com> wrote: > Hi Folks! > > I had one task with csv parsing and transforming. Main file contains > approximately 100k of nodes. > Transformation process should be in memory only, and I thought, that > in-memory temporary index would be useful for this type of tasks. > > Useful scenario > > copy $clone := $csvContent > modify > ( > tmp:optimize($clone), > ... > ) > return $clone > > Main goal - increase performance with huge document fragments, > modified and changed on-fly. > > Is it possible? > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk
Written for BaseX 7.6. BaseX may have changed since: please check the current documentation.