Best practice for storing timestamp information in XML
Tuesday, 29 August 2017 · 5 messages · last reply 31 Aug 2017
DD
Dave Day29 Aug 2017 23:50 #
Greetings List,
I've got an 8 byte value that is the output of a Store Clock instruction on a mainframe. The value created by the instruction is a 64 bit hex value where bit 51 is equal to one microsecond. To get a value that is microseconds, the original clock value is shifted to the right 12 bits. The clock started at all 0's on January 1st, 1900.
I need to be able to store and do basic math(addition and subtraction) on the value in microseconds. And, these values will need to be displayed on a browser in HH:MM:SS.TTTTTT format.
If I create the decimal value for the integer on the mainframe when creating the XML record, I will have to go thru some conversion logic. There could be possibly millions of these records in one XML database.
It seems to me to make more sense to create the value as a hexBinary element, and then have some code running within BaseX do the conversion to create the correct time value.
Is this the way to do something like this?
Regards,
-- Dave Day
KS
Kendall Shaw30 Aug 2017 03:52 #
I assume these are durations, but I’m not sure if the clock start date is relevant. If you can assume it’s not going to span days, my first thought was something like:
string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT0.0001S"))
which produces:
00:00:00.0001
The duration is being expressed as an arbitrary number of seconds. “PT0.0001S” is 1 microsecond, I think.
The duration added to a time results in a time and the default format is what you asked for.
Another example:
string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT9999999.9999S"))
is:
Quoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Dave Day · 1 line
On 8/29/17, 12:51 PM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Dave Day" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of David.Day at duke-software.com> wrote:
Greetings List,
I've got an 8 byte value that is the output of a Store Clock instruction on a mainframe. The value created by the instruction is a 64 bit hex value where bit 51 is equal to one microsecond. To get a value that is microseconds, the original clock value is shifted to the right 12 bits. The clock started at all 0's on January 1st, 1900.
I need to be able to store and do basic math(addition and subtraction) on the value in microseconds. And, these values will need to be displayed on a browser in HH:MM:SS.TTTTTT format.
If I create the decimal value for the integer on the mainframe when creating the XML record, I will have to go thru some conversion logic. There could be possibly millions of these records in one XML database.
It seems to me to make more sense to create the value as a hexBinary element, and then have some code running within BaseX do the conversion to create the correct time value.
Is this the way to do something like this?
Regards,
-- Dave Day
KS
Kendall Shaw30 Aug 2017 04:08 #
Oops. I didn’t mean to click send.
string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT9999999.9999S")) is:
17:46:39.9999
So, if an element contained microseconds, you could do arithmetic with the values and use something like above to format the result.
let $ms := 99999999999
return xs:time('00:00:00.000') + xs:dayTimeDuration(concat('PT', $ms div 10000, 'S'))Kendall
Quoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Kendall Shaw · 1 line
On 8/29/17, 6:52 PM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Kendall Shaw" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of kendall.shaw at workday.com> wrote:
I assume these are durations, but I’m not sure if the clock start date is relevant. If you can assume it’s not going to span days, my first thought was something like:
string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT0.0001S")) which produces:
00:00:00.0001
The duration is being expressed as an arbitrary number of seconds. “PT0.0001S” is 1 microsecond, I think.
The duration added to a time results in a time and the default format is what you asked for.
Another example:
string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT9999999.9999S")) is:
Quoted text · basex-talk-bounces at mailman.uni-konstanz.de on behalf of Dave Day · 1 line
On 8/29/17, 12:51 PM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Dave Day" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of David.Day at duke-software.com> wrote:
Greetings List,
I've got an 8 byte value that is the output of a Store Clock instruction on a mainframe. The value created by the instruction is a 64 bit hex value where bit 51 is equal to one microsecond. To get a value that is microseconds, the original clock value is shifted to the right 12 bits. The clock started at all 0's on January 1st, 1900.
I need to be able to store and do basic math(addition and subtraction) on the value in microseconds. And, these values will need to be displayed on a browser in HH:MM:SS.TTTTTT format.
If I create the decimal value for the integer on the mainframe when creating the XML record, I will have to go thru some conversion logic. There could be possibly millions of these records in one XML database.
It seems to me to make more sense to create the value as a hexBinary element, and then have some code running within BaseX do the conversion to create the correct time value.
Is this the way to do something like this?
Regards,
-- Dave Day
ML
Hi Kendall,
just pay attention to the fact that one micro is 10^-6 of a unit. Thus 0.000001 of a second is one microsecond.
Regards,
Marco.
Quoted text · Kendall Shaw · 51 lines
On 30/08/2017 03:52, Kendall Shaw wrote: > I assume these are durations, but I’m not sure if the clock start date is relevant. If you can assume it’s not going to span days, my first thought was something like: > > string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT0.0001S")) > > which produces: > > 00:00:00.0001 > > The duration is being expressed as an arbitrary number of seconds. “PT0.0001S” is 1 microsecond, I think. > > The duration added to a time results in a time and the default format is what you asked for. > > Another example: > > string(xs:time("00:00:00.000") + xs:dayTimeDuration("PT9999999.9999S")) > > is: > > > > On 8/29/17, 12:51 PM, "basex-talk-bounces at mailman.uni-konstanz.de on behalf of Dave Day" <basex-talk-bounces at mailman.uni-konstanz.de on behalf of David.Day at duke-software.com> wrote: > > Greetings List, > > I've got an 8 byte value that is the output of a Store Clock > instruction on a mainframe. The value created by the instruction is a > 64 bit hex value where bit 51 is equal to one microsecond. To get a > value that is microseconds, the original clock value is shifted to the > right 12 bits. The clock started at all 0's on January 1st, 1900. > > I need to be able to store and do basic math(addition and > subtraction) on the value in microseconds. And, these values will need > to be displayed on a browser in HH:MM:SS.TTTTTT format. > > If I create the decimal value for the integer on the mainframe when > creating the XML record, I will have to go thru some conversion logic. > There could be possibly millions of these records in one XML database. > > It seems to me to make more sense to create the value as a > hexBinary element, and then have some code running within BaseX do the > conversion to create the correct time value. > > Is this the way to do something like this? > > Regards, > > -- Dave Day > > >
AH
Quoted text · Dave Day · 17 lines
> On 29. Aug 2017, at 21:51, Dave Day <David.Day at duke-software.com> wrote: > > Greetings List, > > I've got an 8 byte value that is the output of a Store Clock instruction on a mainframe. The value created by the instruction is a 64 bit hex value where bit 51 is equal to one microsecond. To get a value that is microseconds, the original clock value is shifted to the right 12 bits. The clock started at all 0's on January 1st, 1900. > > I need to be able to store and do basic math(addition and subtraction) on the value in microseconds. And, these values will need to be displayed on a browser in HH:MM:SS.TTTTTT format. > > If I create the decimal value for the integer on the mainframe when creating the XML record, I will have to go thru some conversion logic. There could be possibly millions of these records in one XML database. > > It seems to me to make more sense to create the value as a hexBinary element, and then have some code running within BaseX do the conversion to create the correct time value. > > Is this the way to do something like this? > > Regards, > > -- Dave Day
Hi Dave,
regarding performance a more dedicated approach may be needed. However, to do some first experiments with XML data that originates form the mainframe, BaseX has extensions to do bit shifting and conversions. Following examples can be copied into the XQuery editor of the latest BaseX version for testing.
tl;dr:
<mainframe>
<event start="00000000FF"/>
</mainframe>
/event/@start/string() => trace('t1: ')
=> convert:integer-from-base(16) => trace('t2: ')
=> bin:pack-integer(8, 'big-endian') => trace('t3: ')
=> bin:shift(-1) => trace('t4: ')
=> bin:unpack-integer(0, 8, 'big-endian') => trace('t5: ')
=> convert:integer-to-dateTime() => trace('t6: ')
=> fn:format-dateTime("[H01]:[m01]:[s01].[f000001]") => trace('t7: ')
t1: "00000000FF"
t2: 255
t3: xs:base64Binary("AAAAAAAAAP8=")
t4: xs:base64Binary("AAAAAAAAAH8=")
t5: 127
t6: xs:dateTime("1970-01-01T00:00:00.127Z")
t7: "00:00:00.127000"============================================================
long version:
let's assume you have an input xml such as:
<mainframe>
<event start="00000000FF"/>
</mainframe>- you access the value using XPath as xs:string
<mainframe>
<event start="0" start1="D2DE66B76C0FF200"/>
</mainframe>
/event/@start1/string()
D2DE66B76C0FF200- from (hex) xs:string to xs:integer one can use [1]:
convert:integer-from-base($str as xs:string, $base as xs:integer) as xs:integer
convert:integer-from-base('0000000000FF', 16)
255- bit operations work on xs:base64Binary
- from xs:integer to xs:base64Binary one can use [2]:
bin:pack-integer($in as xs:integer, $size as xs:integer, $octet-order as xs:string) as xs:base64Binary
bin:pack-integer(255, 8, 'big-endian')
xs:base64Binary("AAAAAAAAAP8=") - to perform bit shifting one can use [3]
bin:shift($in as xs:base64Binary?, $by as xs:integer) as xs:base64Binary?
bin:shift(xs:base64Binary("AAAAAAAAAP8="), -1)
xs:base64Binary("AAAAAAAAAH8=") - to go back from xs:base64Binary to xs:integer one can use [4]:
bin:unpack-integer($in as xs:base64Binary, $offset as xs:integer, $size as xs:integer, $octet-order as xs:string) as xs:integer
bin:unpack-integer(xs:base64Binary("AAAAAAAAAH8="), 0, 8, 'big-endian')
127- to interprete an xs:integer as xs:dateTime one can use [5]
convert:integer-to-dateTime($ms as xs:integer) as xs:dateTime
convert:integer-to-dateTime(127)
1970-01-01T00:00:00.127Z- to display an xs:dateTime according to a pattern such as HH:MM:SS.TTTTTT one can use [6]
fn:format-dateTime($value as xs:dateTime?, $picture as xs:string) as xs:string?
fn:format-dateTime(
xs:dateTime("1970-01-01T00:00:00.127Z"),
"[H01]:[m01]:[s01].[f000001]"
)
00:00:00.127000Cheers,
Alex
[1] http://docs.basex.org/wiki/Conversion_Module#convert:integer-from-base [2] http://docs.basex.org/wiki/Binary_Module#bin:pack-integer [3] http://docs.basex.org/wiki/Binary_Module#bin:shift
[4] http://docs.basex.org/wiki/Binary_Module#bin:unpack-integer [5] http://docs.basex.org/wiki/Conversion_Module#convert:integer-to-dateTime [6] https://www.w3.org/TR/xpath-functions-30/#func-format-dateTime
Written for BaseX 8.6. BaseX may have changed since: please check the current documentation.
⚡Generated with XQuery