Database housekeeping for newbies

> Hey all! > > > > I am considering BaseX to hold event information for a broadcast graphics > play out system. This system gets a lot of event data from the planning > system. Each segment can have up to 12 events, and a day in a typical > commercial tv-channel has about 120-160 segments. The commercial breaks not > included. > > > > Every time a full day of programming is made for a channel, an xml file is > produced containing a lot of (badly formatted) EVENT nodes, below a typical > “coming next” event: > > > > <EVENT IDdeko="109213416" DATE="2012-09-21" CHANNEL_ID="123" > VIDEOFORMAT="2"> > > <SCENARIO ID="29"> > > <METADATA pos="0" value="00:20"/> > > <METADATA pos="1" value="The Godfather pt III"/> > > <METADATA pos="2" value="American drama from 1990 with Al Pacino and > Andy Garcia"/> > > </SCENARIO> > > </EVENT> > > > > I have manually created a database from one of these export xml files and > manged to call a BaseXClient dll that I compiled from your examples to > retrieve data from the database. All is well with that. Now to my question, > since I’m not really familiar with databases in general… > > > > Is it possible to set up a “watch folder” for the incoming xml files that > will add them to a specified database or would I have to create an > application for this? > > > > The second question is about housekeeping, since we are handling about 20 > channels with this database, the number of events in the database will > increase quite fast. Is it possible to specify an xpath search that would > match all events with a date more than X days back in time and delete them > from the database, i.e delete data from the database without calling the > exact file the data was imported with? > > > > > > Best regards > > > > SamTheSwede > > Stockholm > > > > > > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >

---- Original Message ---- From: gaerber at axxepta.de To: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database housekeeping for newbies Date: Wed, 31 Oct 2012 15:00:09 +0100 >Hi, > >I don't know of any hot-folder mechanism in basex but you might be >interested in the webdav api: http://docs.basex.org/wiki/WebDAV > >Concerning the housekeeping, I'd create new databases containing a >defined time period (e.g. one week or so). You can query several >databases at once, so you always have the option to: > >- query over single or multiple time periods >- drop a complete time period > >Changing a lot of nodes (or deleting) inside a single database is >sometimes slow. But that might be due the structure of my data. > >Regards, > >Max > > >2012/10/31 Sam Eklund <sam.eklund at gluebox.se>: >> Hey all! >> >> >> >> I am considering BaseX to hold event information for a broadcast >graphics >> play out system. This system gets a lot of event data from the >planning >> system. Each segment can have up to 12 events, and a day in a >typical >> commercial tv-channel has about 120-160 segments. The commercial >breaks not >> included. >> >> >> >> Every time a full day of programming is made for a channel, an xml >file is >> produced containing a lot of (badly formatted) EVENT nodes, below a >typical >> coming next event: >> >> >> >> <EVENT IDdeko="109213416" DATE="2012-09-21" CHANNEL_ID="123" >> VIDEOFORMAT="2"> >> >> <SCENARIO ID="29"> >> >> <METADATA pos="0" value="00:20"/> >> >> <METADATA pos="1" value="The Godfather pt III"/> >> >> <METADATA pos="2" value="American drama from 1990 with Al >Pacino and >> Andy Garcia"/> >> >> </SCENARIO> >> >> </EVENT> >> >> >> >> I have manually created a database from one of these export xml >files and >> manged to call a BaseXClient dll that I compiled from your examples >to >> retrieve data from the database. All is well with that. Now to my >question, >> since Im not really familiar with databases in general >> >> >> >> Is it possible to set up a watch folder for the incoming xml >files that >> will add them to a specified database or would I have to create an >> application for this? >> >> >> >> The second question is about housekeeping, since we are handling >about 20 >> channels with this database, the number of events in the database >will >> increase quite fast. Is it possible to specify an xpath search that >would >> match all events with a date more than X days back in time and >delete them >> from the database, i.e delete data from the database without >calling the >> exact file the data was imported with? >> >> >> >> >> >> Best regards >> >> >> >> SamTheSwede >> >> Stockholm >> >> >> >> >> >> >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > > >-- > >Maximilian Gärber > >axxepta solutions GmbH >Postfach 51 02 38 >13362 Berlin > >Tel +49 (0)30 499 147 66 >Fax +49 (0)30 499 147 67 >Mail gaerber at axxepta.de >_______________________________________________ >BaseX-Talk mailing list >BaseX-Talk at mailman.uni-konstanz.de >https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

-----Original Message----- From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Maximilian Gärber Sent: den 31 oktober 2012 15:00 To: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database housekeeping for newbies Hi, I don't know of any hot-folder mechanism in basex but you might be interested in the webdav api: http://docs.basex.org/wiki/WebDAV Concerning the housekeeping, I'd create new databases containing a defined time period (e.g. one week or so). You can query several databases at once, so you always have the option to: - query over single or multiple time periods - drop a complete time period Changing a lot of nodes (or deleting) inside a single database is sometimes slow. But that might be due the structure of my data. Regards, Max 2012/10/31 Sam Eklund <sam.eklund at gluebox.se>: > Hey all! > > > > I am considering BaseX to hold event information for a broadcast > graphics play out system. This system gets a lot of event data from > the planning system. Each segment can have up to 12 events, and a day > in a typical commercial tv-channel has about 120-160 segments. The > commercial breaks not included. > > > > Every time a full day of programming is made for a channel, an xml > file is produced containing a lot of (badly formatted) EVENT nodes, > below a typical "coming next" event: > > > > <EVENT IDdeko="109213416" DATE="2012-09-21" CHANNEL_ID="123" > VIDEOFORMAT="2"> > > <SCENARIO ID="29"> > > <METADATA pos="0" value="00:20"/> > > <METADATA pos="1" value="The Godfather pt III"/> > > <METADATA pos="2" value="American drama from 1990 with Al Pacino > and Andy Garcia"/> > > </SCENARIO> > > </EVENT> > > > > I have manually created a database from one of these export xml files > and manged to call a BaseXClient dll that I compiled from your > examples to retrieve data from the database. All is well with that. > Now to my question, since I'm not really familiar with databases in > general. > > > > Is it possible to set up a "watch folder" for the incoming xml files > that will add them to a specified database or would I have to create > an application for this? > > > > The second question is about housekeeping, since we are handling about > 20 channels with this database, the number of events in the database > will increase quite fast. Is it possible to specify an xpath search > that would match all events with a date more than X days back in time > and delete them from the database, i.e delete data from the database > without calling the exact file the data was imported with? > > > > > > Best regards > > > > SamTheSwede > > Stockholm > > > > > > > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > -- Maximilian Gärber axxepta solutions GmbH Postfach 51 02 38 13362 Berlin Tel +49 (0)30 499 147 66 Fax +49 (0)30 499 147 67 Mail gaerber at axxepta.de BaseX-Talk mailing list BaseX-Talk at mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

> Thanks for the input Max! > > You say "changing a lot of nodes (or deleting) .... is sometimes slow". But is it possible to delete data from a given xpath or xquery? > > The structure of this data is quite simple, so I'd like to test this, but as a total beginner in this I do not understand the syntax, it seems the DELETE command wants a file path rather than an Xpath? > > All the best, > > Sam > > -----Original Message----- > From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Maximilian Gärber > Sent: den 31 oktober 2012 15:00 > To: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] Database housekeeping for newbies > > Hi, > > I don't know of any hot-folder mechanism in basex but you might be interested in the webdav api: http://docs.basex.org/wiki/WebDAV > > Concerning the housekeeping, I'd create new databases containing a defined time period (e.g. one week or so). You can query several databases at once, so you always have the option to: > > - query over single or multiple time periods > - drop a complete time period > > Changing a lot of nodes (or deleting) inside a single database is sometimes slow. But that might be due the structure of my data. > > Regards, > > Max > > > 2012/10/31 Sam Eklund <sam.eklund at gluebox.se>: >> Hey all! >> >> >> >> I am considering BaseX to hold event information for a broadcast >> graphics play out system. This system gets a lot of event data from >> the planning system. Each segment can have up to 12 events, and a day >> in a typical commercial tv-channel has about 120-160 segments. The >> commercial breaks not included. >> >> >> >> Every time a full day of programming is made for a channel, an xml >> file is produced containing a lot of (badly formatted) EVENT nodes, >> below a typical "coming next" event: >> >> >> >> <EVENT IDdeko="109213416" DATE="2012-09-21" CHANNEL_ID="123" >> VIDEOFORMAT="2"> >> >> <SCENARIO ID="29"> >> >> <METADATA pos="0" value="00:20"/> >> >> <METADATA pos="1" value="The Godfather pt III"/> >> >> <METADATA pos="2" value="American drama from 1990 with Al Pacino >> and Andy Garcia"/> >> >> </SCENARIO> >> >> </EVENT> >> >> >> >> I have manually created a database from one of these export xml files >> and manged to call a BaseXClient dll that I compiled from your >> examples to retrieve data from the database. All is well with that. >> Now to my question, since I'm not really familiar with databases in >> general. >> >> >> >> Is it possible to set up a "watch folder" for the incoming xml files >> that will add them to a specified database or would I have to create >> an application for this? >> >> >> >> The second question is about housekeeping, since we are handling about >> 20 channels with this database, the number of events in the database >> will increase quite fast. Is it possible to specify an xpath search >> that would match all events with a date more than X days back in time >> and delete them from the database, i.e delete data from the database >> without calling the exact file the data was imported with? >> >> >> >> >> >> Best regards >> >> >> >> SamTheSwede >> >> Stockholm >> >> >> >> >> >> >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > > > -- > > Maximilian Gärber > > axxepta solutions GmbH > Postfach 51 02 38 > 13362 Berlin > > Tel +49 (0)30 499 147 66 > Fax +49 (0)30 499 147 67 > Mail gaerber at axxepta.de > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk

-----Original Message----- From: basex-talk-bounces at mailman.uni-konstanz.de [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of Maximilian Gärber Sent: den 1 november 2012 18:28 To: basex-talk at mailman.uni-konstanz.de Subject: Re: [basex-talk] Database housekeeping for newbies Hi, I'm assuming you are using the C# client? If you are working with a client dll, then you're just sending xquery snippets to the server. For "delete", please refer to: http://docs.basex.org/wiki/Updates#delete Did you see the example in https://github.com/BaseXdb/basex-api/blob/master/src/main/c%23/QueryExample.cs ? Regards, Max 2012/11/1 Sam Eklund <sam.eklund at gluebox.se>: > Thanks for the input Max! > > You say "changing a lot of nodes (or deleting) .... is sometimes slow". But is it possible to delete data from a given xpath or xquery? > > The structure of this data is quite simple, so I'd like to test this, but as a total beginner in this I do not understand the syntax, it seems the DELETE command wants a file path rather than an Xpath? > > All the best, > > Sam > > -----Original Message----- > From: basex-talk-bounces at mailman.uni-konstanz.de > [mailto:basex-talk-bounces at mailman.uni-konstanz.de] On Behalf Of > Maximilian Gärber > Sent: den 31 oktober 2012 15:00 > To: basex-talk at mailman.uni-konstanz.de > Subject: Re: [basex-talk] Database housekeeping for newbies > > Hi, > > I don't know of any hot-folder mechanism in basex but you might be > interested in the webdav api: http://docs.basex.org/wiki/WebDAV > > Concerning the housekeeping, I'd create new databases containing a defined time period (e.g. one week or so). You can query several databases at once, so you always have the option to: > > - query over single or multiple time periods > - drop a complete time period > > Changing a lot of nodes (or deleting) inside a single database is sometimes slow. But that might be due the structure of my data. > > Regards, > > Max > > > 2012/10/31 Sam Eklund <sam.eklund at gluebox.se>: >> Hey all! >> >> >> >> I am considering BaseX to hold event information for a broadcast >> graphics play out system. This system gets a lot of event data from >> the planning system. Each segment can have up to 12 events, and a day >> in a typical commercial tv-channel has about 120-160 segments. The >> commercial breaks not included. >> >> >> >> Every time a full day of programming is made for a channel, an xml >> file is produced containing a lot of (badly formatted) EVENT nodes, >> below a typical "coming next" event: >> >> >> >> <EVENT IDdeko="109213416" DATE="2012-09-21" CHANNEL_ID="123" >> VIDEOFORMAT="2"> >> >> <SCENARIO ID="29"> >> >> <METADATA pos="0" value="00:20"/> >> >> <METADATA pos="1" value="The Godfather pt III"/> >> >> <METADATA pos="2" value="American drama from 1990 with Al Pacino >> and Andy Garcia"/> >> >> </SCENARIO> >> >> </EVENT> >> >> >> >> I have manually created a database from one of these export xml files >> and manged to call a BaseXClient dll that I compiled from your >> examples to retrieve data from the database. All is well with that. >> Now to my question, since I'm not really familiar with databases in >> general. >> >> >> >> Is it possible to set up a "watch folder" for the incoming xml files >> that will add them to a specified database or would I have to create >> an application for this? >> >> >> >> The second question is about housekeeping, since we are handling >> about >> 20 channels with this database, the number of events in the database >> will increase quite fast. Is it possible to specify an xpath search >> that would match all events with a date more than X days back in time >> and delete them from the database, i.e delete data from the database >> without calling the exact file the data was imported with? >> >> >> >> >> >> Best regards >> >> >> >> SamTheSwede >> >> Stockholm >> >> >> >> >> >> >> >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> > > > > -- > > Maximilian Gärber > > axxepta solutions GmbH > Postfach 51 02 38 > 13362 Berlin > > Tel +49 (0)30 499 147 66 > Fax +49 (0)30 499 147 67 > Mail gaerber at axxepta.de > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk -- Maximilian Gärber axxepta solutions GmbH Postfach 51 02 38 13362 Berlin Tel +49 (0)30 499 147 66 Fax +49 (0)30 499 147 67 Mail gaerber at axxepta.de BaseX-Talk mailing list BaseX-Talk at mailman.uni-konstanz.de https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk