Export database to SQL?

From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Klavs Prieditis Sent: Wednesday, March 02, 2016 11:21 AM To: Christian Grün <christian.gruen at gmail.com>; Dirk Kirsten <dk at basex.org> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Parallel execution Hi, Thanks Dirk and Christian! My apologies for using such a strong word to describe such a minor issue. It is not my intention to underestimate your work. My main concern is that `async:fork-join()` is too heavy. I mean, in essence, I am solving the same problem in these two examples.

If you solve the same problem in Java 8 using Streams API you get the following two solutions.

Considering these two Java 8 solutions, I guess what I actually want is the `async` keyword. So that the second Java example could be rewritten as follows.

Problem here is that you need to introduce a new keyword to the language. I am just trying to figure out if there could be a solution like this.

Thanks for your input! Currently I lack in-depth knowledge on XQuery, I'll try to get a spare time to study the XQuery specification so that I am able reason about the various consequences. 2016-03-02 11:36 GMT+02:00 Christian Grün <christian.gruen at gmail.com<mailto:christian.gruen at gmail.com>>: Hi Klāvs, Dirk already gave some good reasons why implicit parallelization is not the default in BaseX, although it may seems to be the most natural and desirable choice. Indeed it’s fairly easy to create FLWOR expressions which are slower when executed in parallel. The XQuery specification, by the way, does not say anything about parallelization, so it’s completely up to the implementation to choose the most promising evaluation strategy. You asked if it was not possible to introduce sth. like a `can-be-optimized-for-parallel-execution` flag, and you may be interested to hear that we had some first thoughts on an 'async' keyword, which could be used in FLWOR expressions [1]. In the GitHub issue, only the most basic cases have been considered so far, but if we manage to push this further, we would have a syntax that will be more easily accessible to most users. Ideally, some time in future, we could then apply this pattern automatically if it will be absolutely clear that parallelization leads to faster results. Everyone: If you notice that the async:fork-join function makes your code faster, don’t hesitate and send us your queries. Hope this helps, Christian [1] https://github.com/james-jw/xq-promise/issues/15 On Wed, Mar 2, 2016 at 10:05 AM, Dirk Kirsten <dk at basex.org<mailto:dk at basex.org>> wrote: > Labrīt Klāvs, > > well, I would the word "broken" is a strong word for something, which > still works correctly and returns the correct result. It might not be > perform optimal, but broken is something different. Also, given that the > feature is still not even one day old, you might have to be a bit more > patient with us in applying optimization techniques. > > The problem here has several dimensions: > - XQuery returns results in order. So in order to not break BaseX we > would have to cache the results of the fetch and return it in the > correct order > - This caching might be much more complex, if it is a more complicated > expression > - It might not always be desirable to turn on parallelisation. In this > case you call an external HTTP server 10 times, in this case there might > be a speedup. But using fetch:text() could also point to a local URI and > to parallelize I/O is not always beneficial. > > So given the many different use cases here I don't think an implicit > parallelization will work here to the satisfaction of everyone. > Also, I invite you to read this closely related question about Haskell > on SO and the linked research: > https://stackoverflow.com/questions/15005670/why-is-there-no-implicit-parallelism-in-haskell > > Cheers > Dirk > > On 03/02/2016 01:41 AM, Klāvs Priedītis wrote: >> Hello, BaseX team and community! >> >> Disclosure: I am not yet an expert on these issues and I have not >> fully studied the specs of XQuery. >> >> Now that Async Module is incorporated, I am wondering about some basic >> properties of xquery. >> >> Example: >> ```xquery >> for $i in 1 to 10 >> return fetch:text("http://example.com/magic-story/for-number/" || $i) >> ``` >> >> It feels to me that the xquery processor is broken if it cannot figure >> out on its own that the fetch can be done in parallel. >> Why does this not work out of the box? Can it work out of the box? >> Wouldn't it be able to do this in parallel if a flag >> `can-be-optimized-for-parallel-execution` was defined for the >> `fetch:text`? >> >> -- >> Veiksmi vēlot, >> Klāvs Priedītis > > -- > Dirk Kirsten, BaseX GmbH, http://basexgmbh.de > |-- Firmensitz: Blarerstrasse 56, 78462 Konstanz > |-- Registergericht Freiburg, HRB: 708285, Geschäftsführer: > | Dr. Christian Grün, Dr. Alexander Holupirek, Michael Seiferle > `-- Phone: 0049 7531 91 68 276, Fax: 0049 7531 20 05 22 > -- Veiksmi vēlot, Klāvs Priedītis

> Is there any way to export data from the database to other databases?

-----Original Message----- From: Christian Grün [mailto:christian.gruen at gmail.com] Sent: Wednesday, March 02, 2016 11:40 AM To: Bhander, Gurbakhash S. <Bhander.Gurbakhash at epa.gov> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: Export database to SQL? Hi Raj, > Is there any way to export data from the database to other databases? Sure! But please be more specific. Christian

On Wed, Mar 2, 2016 at 5:51 PM, Bhander, Gurbakhash S. <Bhander.Gurbakhash at epa.gov> wrote: > Hi Christian, > > I am using C# and trying to connect BaseX database to my grid (tables on the interface). > Q1: Is there any way I can connect basex database to my grid directly? > > If not, how can I export to SQL database from the basex because I will have to connect with my grid? My database is very big in size. I had exported to XML but I was unable to export data from XML to SQL using readXML and XDocument because of database size. I get out of memory error. > > Is there any way that I can use to export to SQL database? > > Thanks - Raj > > > > -----Original Message----- > From: Christian Grün [mailto:christian.gruen at gmail.com] > Sent: Wednesday, March 02, 2016 11:40 AM > To: Bhander, Gurbakhash S. <Bhander.Gurbakhash at epa.gov> > Cc: BaseX <basex-talk at mailman.uni-konstanz.de> > Subject: Re: Export database to SQL? > > Hi Raj, > >> Is there any way to export data from the database to other databases? > > Sure! But please be more specific. > > Christian

On Wed, 2016-03-02 at 16:24 +0000, Bhander, Gurbakhash S. wrote: > Hi, > > Is there any way to export data from the database to other databases?

-----Original Message----- From: Liam R. E. Quin [mailto:liam at w3.org] Sent: Wednesday, March 02, 2016 11:49 AM To: Bhander, Gurbakhash S. <Bhander.Gurbakhash at epa.gov> Cc: BaseX <basex-talk at mailman.uni-konstanz.de> Subject: Re: [basex-talk] Export database to SQL? On Wed, 2016-03-02 at 16:24 +0000, Bhander, Gurbakhash S. wrote: > Hi, > > Is there any way to export data from the database to other databases? There are several ways. But note that there's not a 1:1 mapping between relational data and XML, so you first have to choose how you will represent your information in the relational database. After that, if it's simple unstructured values (e.g. no mixed content) you might write out CSV as text using XQuery, or use the BaseX SQL Module [1] to connect directly to the other database if it supports JDBC. Liam [1] http://docs.basex.org/wiki/SQL_Module -- Liam R. E. Quin <liam at w3.org> The World Wide Web Consortium (W3C)

On Thu, 2016-03-03 at 14:25 +0000, Bhander, Gurbakhash S. wrote: > Hi Liam, > > Yes, C# supports JDBC and I am trying to connect it to the basex > database using following codes. But it doesn't work. I guess, my > database name is not correct.