Main Page » Mailing List » December 2019

Huge No of XML files.

Tuesday, 17 December 2019 · 24 messages · last reply 3 Mar 2020
SY
Sreenivasulu Yadavalli17 Dec 2019 07:18 #
Hi Team, We have a big problem for creating the collection bcoz of large no of xmls. Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. Please help and do needful. Regards, YSL
CG
Christian Grün17 Dec 2019 07:56 #
Hi YSL,
Quoted text · Sreenivasulu Yadavalli · 1 line
> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection.
Some questions back: 1. What is the average size of each of your document? 2. Will this sum up to a yearly amount of appr. 20 million files, or do you have an upper limit? 3. Which of our API(s) are you using to add the documents? 4. How do your queries for reporting data look like?
Quoted text · Sreenivasulu Yadavalli · 1 line
> Please help and do needful.
https://ell.stackexchange.com/a/17626 ;) Best, Christian
SY
Sreenivasulu Yadavalli17 Dec 2019 08:08 #
Hi Christian, Every day we are generating upto 50k xml files per account. At the time of collection generation i need to generate report on the selected account. Then immediately that collection got terminated. Not able to generate report. Even collection creation time is also taken more than 15 hours that particular account. Pls help me. Regards, YSL
Quoted text · Christian Grün · 24 lines
On Tue, Dec 17, 2019 at 12:27 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi YSL, > > > Every day we are moving collections around 55k to 60k no of xml files > large account. Its taking more than 18 hours. At that time we want access > the collection for generating report its on lock mode and scrip that > collection. > > Some questions back: > > 1. What is the average size of each of your document? > 2. Will this sum up to a yearly amount of appr. 20 million files, or > do you have an upper limit? > 3. Which of our API(s) are you using to add the documents? > 4. How do your queries for reporting data look like? > > > Please help and do needful. > > https://ell.stackexchange.com/a/17626 ;) > > Best, > Christian >
CG
Christian Grün17 Dec 2019 08:12 #
Quoted text · Sreenivasulu Yadavalli · 1 line
> Pls help me.
Trying. Could you try to help us first und answer all of the questions of my initial reply?
SY
Sreenivasulu Yadavalliin reply to Christian Grün, 17 Dec 2019, 07:5617 Dec 2019 08:30 #
Hi Christian, 1. What is the average size of each of your document? A: 5k to 7k 2. Will this sum up to a yearly amount of appr. 20 million files, or do you have an upper limit? A: Every day we are generating collections (Unbilled Mobile Transactionn data) Per account appr 50k - 55k files. 3. Which of our API(s) are you using to add the documents? A: import org.basex.server.ClientSession; 4. How do your queries for reporting data look like? A: Usage Summary Per Line:10 minutes
declare function local:getLabelName($serNo as xs:string) {  let $doc :=
collection('LabelCollection_1573624392934')   let $label :=
$doc//row[@ser_no = $serNo] let $lbCount:=count($doc//row[@ser_no =
$serNo])  return     if(data($label) != '' and $lbCount<=1) then
data($label)     else ('') }; declare function local:getRecords() {  let
$Rows :=
(collection("867509_Voice_OCTOBER-19_Unbilled_1")/SUBCUSTBRK[AccNo[@NO =
(6945045)]][DOB >='2019-10-01' and DOB <='2019-10-31'])   let $billNo :=
data($Rows/CONN/@NO)  let $topRecords := $billNo
   for $details in  $Rows[CONN[data(@NO)=$topRecords]]/Country/ROW
group by $billNo := data($details/../../CONN/@NO)
          return <R> {               element Acno
{distinct-values(data($details/../../AccNo/@NO))},               element
BNO {$billNo},               element SLBl
{distinct-values(local:getLabelName($billNo))},               element Stat
{}, for $serviceDetails in $details   group by $groupService :=
data($serviceDetails/@ServType)   return (element
{concat($groupService,'C') } {xs:decimal(sum(data($serviceDetails/@Calls)))
} ), element TCls {xs:decimal(sum(data($details/@Calls)))} }</R> }; (
      let $records := local:getRecords()[position() le 10000]
let $totalCalls := sum($records/TCls)  let $pstncls :=
xs:decimal(sum($records/PSTNC)) let $vpncls :=
xs:decimal(sum($records/VPNC)) let $tnbscls :=
xs:decimal(sum($records/TNBSC)) let $fmccls :=
xs:decimal(sum($records/FMCC)) return               ($records,
   if(count($records) > 0) then ( <R>{ <Acno>{ 'Total' }</Acno>,
<BNO></BNO>, <SLBl></SLBl>, <Stat></Stat>, <PSTNC>{ xs:decimal($pstncls)
}</PSTNC>,  <VPNC>{ xs:decimal($vpncls) }</VPNC>, <TNBSC>{
xs:decimal($tnbscls) }</TNBSC>,  <FMCC>{ xs:decimal($fmccls) }</FMCC>,
<TCls>{ xs:decimal($totalCalls) }</TCls> }</R>)                else ())
      )
Trend Report by Call type:4 minutes
declare function local:getDurationPer($ctDur as xs:double, $ctTotalDur as
xs:double) {    if($ctTotalDur=0)then()else(   let $ctPer :=
concat(xs:decimal(round-half-to-even((($ctDur div $ctTotalDur) * 100), 3)),
"%")   return $ctPer) };
declare function local:getHHMMSSFromNumber($num as xs:double) {  let $ss :=
xs:integer($num mod 60)  let $mm1 := xs:integer($num div 60)  let $mm :=
xs:integer($mm1 mod 60)  let $hh := xs:integer($mm1 div 60)  let $hhFinal
:= if(string-length(xs:string($hh)) >= 2) then $hh else (concat('0', $hh))
 let $mmFinal := if(string-length(xs:string($mm)) >= 2) then $mm else
(concat('0', $mm))  let $ssFinal := if(string-length(xs:string($ss)) >= 2)
then $ss else (concat('0', $ss))  return
concat($hhFinal,':',$mmFinal,':',$ssFinal) };
declare function local:getMMSSFromNumber($num as xs:double) {  let $ss :=
xs:integer($num mod 60)  let $mm := xs:integer($num div 60)  let $mmFinal
:= if(string-length(xs:string($mm)) >= 2) then $mm else (concat('0', $mm))
 let $ssFinal := if(string-length(xs:string($ss)) >= 2) then $ss else
(concat('0', $ss))  return concat($mmFinal,':',$ssFinal) };

let $subbrk :=
(collection("867509_Voice_OCTOBER-19_Unbilled_1")/SUBCUSTBRK[AccNo[@NO =
(6945045)]][DOB >= '2019-10-01' and DOB <= '2019-10-31']) let
$detailUsageTxn := $subbrk/DETAIL/TRANSACTION[@Usage = 'usage'] let
$DistCallTypes := distinct-values($detailUsageTxn/SUB_SECTION/@Type) let
$allmonths := distinct-values($subbrk/Month) let $rowcnt :=
count($detailUsageTxn/SUB_SECTION) let $months := $allmonths[position() le
10000] let $opttype := 'NumVal' let $durop := 'hh:mm:ss' return   (
if($DistCallTypes!='')then(for $month in $months   return     <R>
 <GRPBY>{ $month }</GRPBY>{                   if($opttype = 'NumVal') then
                    for $ct in $DistCallTypes                      let
$ctCnt := sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Calls)                     return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {xs:decimal($ctCnt)}                   else
      if($opttype = 'NumPerViewPoint') then                       for $ct
in $DistCallTypes                       let $ctCnt :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Calls)                       let
$ctTotalCnt := sum($detailUsageTxn/SUB_SECTION[data(@Type) =
data($ct)]/@Calls)                       let $ctPer :=
concat(xs:decimal(round-half-to-even((($ctCnt div $ctTotalCnt) * 100), 3)),
'%')                       return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {$ctPer}                     else
if($opttype = 'NumPerCallType') then                         for $ct in
$DistCallTypes                         let $ctCnt :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Calls)                         let
$ctTotalCnt := sum($detailUsageTxn/SUB_SECTION[data(../../../Month) =
$month]/@Calls)                         let $ctPer :=
concat(xs:decimal(round-half-to-even(($ctCnt div $ctTotalCnt) * 100, 3)),
'%')                         return element {
replace(concat('_',data($ct)),
'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)| |&apos;)', '_') } {$ctPer}
                      else                         if($opttype = 'DurVal')
then                           for $ct in $DistCallTypes
        let $ctDur := sum($detailUsageTxn/SUB_SECTION[data(@Type) =
data($ct) and data(../../../Month) = $month]/@Duration)
      return                             element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {if($durop = 'hh:mm:ss') then
local:getHHMMSSFromNumber($ctDur)
                            else (local:getMMSSFromNumber($ctDur))}
                else                           if($opttype =
'DurPerViewPoint') then                             for $ct in
$DistCallTypes                             let $ctDur :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Duration)                             let
$ctTotalDur := sum($detailUsageTxn/SUB_SECTION[data(@Type) =
data($ct)]/@Duration)                             return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {local:getDurationPer($ctDur, $ctTotalDur)}
            else                             if($opttype =
'DurPerCallType') then                               for $ct in
$DistCallTypes                               let $ctDur :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Duration)                               let
$ctTotalDur := sum($detailUsageTxn/SUB_SECTION[data(../../../Month) =
$month]/@Duration)                               return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {local:getDurationPer($ctDur, $ctTotalDur)}
              else                               if($opttype = 'CostVal')
then                                 for $ct in $DistCallTypes
                    let $ctSum :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Amt)                                 return
element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {xs:decimal(round-half-to-even($ctSum, 3))}
                else                                 if($opttype =
'CostPerViewPoint') then                                   for $ct in
$DistCallTypes                                   let $ctSum :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct) and
data(../../../Month) = $month]/@Amt)                                   let
$ctTotalSum := sum($detailUsageTxn/SUB_SECTION[data(@Type) =
data($ct)]/@Amt)                                   let $ctPer :=
concat(xs:decimal(round-half-to-even((($ctSum div $ctTotalSum) * 100), 3)),
'%')                                   return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {$ctPer}                                 else
                      if($opttype = 'CostPerCallType') then
                    for $ct in $DistCallTypes
      let $ctSum := sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct)
and data(../../../Month) = $month]/@Amt)
  let $ctTotalSum := sum($detailUsageTxn/SUB_SECTION[data(../../../Month) =
$month]/@Amt)                                     let $ctPer :=
concat(xs:decimal(round-half-to-even((($ctSum div $ctTotalSum) * 100), 3)),
'%')                                     return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {$ctPer}                                   else ()
          }   <TOTAL>{   if($opttype = 'NumVal') then
xs:decimal(sum($detailUsageTxn/SUB_SECTION[data(../../../Month) =
$month]/@Calls))   else    if($opttype = 'NumPerViewPoint') then      let
$ctCnt := sum($detailUsageTxn/SUB_SECTION[data(../../../Month) =
$month]/@Calls)      let $ctTotalCnt :=
sum($detailUsageTxn/SUB_SECTION/@Calls)      let $ctPer :=
concat(xs:decimal(round-half-to-even((($ctCnt div $ctTotalCnt) * 100), 3)),
'%')      return $ctPer    else      if($opttype = 'NumPerCallType') then
data('100%')      else        if($opttype = 'DurVal') then          let
$DurSum := sum($detailUsageTxn/SUB_SECTION[data(../../../Month) =
$month]/@Duration)          return            if($durop = 'hh:mm:ss') then
local:getHHMMSSFromNumber($DurSum)            else
(local:getMMSSFromNumber($DurSum))        else          if($opttype =
'DurPerViewPoint') then            let $ctDur :=
sum($detailUsageTxn/SUB_SECTION[data(../../../Month) = $month]/@Duration)
         let $ctTotalDur := sum($detailUsageTxn/SUB_SECTION/@Duration)
       return local:getDurationPer($ctDur, $ctTotalDur)          else
     if($opttype = 'DurPerCallType') then data('100%')            else
         if($opttype = 'CostVal') then
xs:decimal(round-half-to-even(sum($detailUsageTxn/SUB_SECTION[data(../../../Month)
= $month]/@Amt), 3))              else                if($opttype =
'CostPerViewPoint') then                  let $ctSum :=
sum($detailUsageTxn/SUB_SECTION[data(../../../Month) = $month]/@Amt)
           let $ctTotalSum := sum($detailUsageTxn/SUB_SECTION/@Amt)
         let $ctPer := concat(xs:decimal(round-half-to-even((($ctSum div
$ctTotalSum) * 100), 3)), '%')                  return $ctPer
 else                  if($opttype = 'CostPerCallType') then data('100%')
               else ()   }</TOTAL></R>)else()    , (   if($rowcnt > 0) then
  <R><GRPBY>Total</GRPBY>{                 if($opttype = 'NumVal') then
              for $ct in $DistCallTypes                   let $ctCnt :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct)]/@Calls)
        return element { replace(concat('_',data($ct)),
'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)| |&apos;)', '_') }
{xs:decimal($ctCnt)}                 else                   if($opttype =
'NumPerViewPoint') then                     for $ct in $DistCallTypes
              return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {data('100%')}                   else
if($opttype = 'NumPerCallType') then                       for $ct in
$DistCallTypes                       let $ctCnt :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct)]/@Calls)
            let $ctTotalCnt := sum($detailUsageTxn/SUB_SECTION/@Calls)
                  let $ctPer :=
concat(xs:decimal(round-half-to-even((($ctCnt div $ctTotalCnt) * 100), 3)),
'%')                       return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {$ctPer}                     else
if($opttype = 'DurVal') then                         for $ct in
$DistCallTypes                         let $DurSum :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct)]/@Duration)
                return                           element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {if($durop = 'hh:mm:ss') then
local:getHHMMSSFromNumber($DurSum)
                            else (local:getMMSSFromNumber($DurSum))}
                else                         if($opttype =
'DurPerViewPoint') then data('100%')                         else
                if($opttype = 'DurPerCallType') then
      for $ct in $DistCallTypes                             let $ctDur :=
sum($detailUsageTxn/SUB_SECTION[data(@Type) = data($ct)]/@Duration)
                    let $ctTotalDur :=
sum($detailUsageTxn/SUB_SECTION/@Duration)
return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {local:getDurationPer($ctDur, $ctTotalDur)}
            else                             if($opttype = 'CostVal') then
                              for $ct in $DistCallTypes
          let $ctSum := sum($detailUsageTxn/SUB_SECTION[data(@Type) =
data($ct)]/@Amt)                               return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {xs:decimal(round-half-to-even($ctSum, 3))}
              else                               if($opttype =
'CostPerViewPoint') then data('100%')                               else
                              if($opttype = 'CostPerCallType') then
                          for $ct in $DistCallTypes
          let $ctSum := sum($detailUsageTxn/SUB_SECTION[data(@Type) =
data($ct)]/@Amt)                                   let $ctTotalSum :=
sum($detailUsageTxn/SUB_SECTION/@Amt)                                   let
$ctPer := concat(xs:decimal(round-half-to-even((($ctSum div $ctTotalSum) *
100), 3)), '%')                                   return element {
replace(concat('_',data($ct)),'(\.|\[|\]|\\|\||\-|\^|\$|\?|\*|\+|\{|\}|\(|\)|
|&apos;)', '_') } {$ctPer}                                 else ()
      }<TOTAL>{                         if($opttype = 'NumVal') then
xs:decimal(sum($detailUsageTxn/SUB_SECTION/@Calls))
else                           if($opttype = 'NumPerViewPoint') then
data('100%')                           else
if($opttype = 'NumPerCallType') then data('100%')
  else                               if($opttype = 'DurVal') then
                      let $durSum :=
sum($detailUsageTxn/SUB_SECTION/@Duration)
return                                   if($durop = 'hh:mm:ss') then
local:getHHMMSSFromNumber($durSum)                                   else
(local:getMMSSFromNumber($durSum))                               else
                          if($opttype = 'DurPerViewPoint') then
data('100%')                                 else
        if($opttype = 'DurPerCallType') then data('100%')
                else                                     if($opttype =
'CostVal') then
xs:decimal(round-half-to-even(sum($detailUsageTxn/SUB_SECTION/@Amt), 3))
                                  else
  if($opttype = 'CostPerViewPoint') then data('100%')
                else                                         if($opttype =
'CostPerCallType') then data('100%')
  else ()                       }</TOTAL></R>    else ()      ))
Multilevel activity Summary report:-4-minitutes
declare function local:getMinsSecs($mins, $calls) {   let $mm :=
floor($mins div $calls)   let $pre :=
substring(substring-after(string(round-half-to-even(($mins div $calls) -
$mm, 5)), '.'), 1, 3)   let $precision := if($pre != '') then number($pre)
    else 0   let $ss := string(round($precision * 0.06))   return
concat(if($mm < 10) then concat('0', $mm)            else $mm, ':', if($ss
< 10) then concat('0', $ss)           else $ss) };
declare function local:getHHMMSSFromNumber($num as xs:integer) {  let $ss
:= xs:integer($num mod 60)  let $mm1 := xs:integer($num div 60)  let $mm :=
xs:integer($mm1 mod 60)  let $hh := xs:integer($mm1 div 60)  let $hhFinal
:= if(string-length(xs:string($hh)) >= 2) then $hh else (concat('0', $hh))
 let $mmFinal := if(string-length(xs:string($mm)) >= 2) then $mm else
(concat('0', $mm))  let $ssFinal := if(string-length(xs:string($ss)) >= 2)
then $ss else (concat('0', $ss))  return
concat($hhFinal,':',$mmFinal,':',$ssFinal) };
declare function local:getMinutesFromDutation($duration as xs:string) {
let $duration := tokenize($duration, ":")   let $hh := $duration[1]   let
$mm := $duration[2]   let $ss := $duration[3]   return
xs:decimal(round-half-to-even((number(data($hh)) * 60 + number(data($mm)) +
(number(data($ss)) div 60)), 3)) };
declare function local:getViewPontChargesData() {   for $viewPontLopp in
<root><viewpoint><name>Default</name><accountnos>6945045</accountnos></viewpoint></root>//viewpoint
            let $Rows := if(data($viewPontLopp/name)=
"Default")then((collection("867509_Voice_OCTOBER-19_Unbilled_1")/SUBCUSTBRK[AccNo[@NO
= (6945045)]][DOB >='2019-10-01' and DOB
<='2019-10-31']))else((collection("867509_Voice_OCTOBER-19_Unbilled_1")/SUBCUSTBRK[AccNo[@NO
= (6945045)]][DOB >='2019-10-01' and DOB <='2019-10-31']))         let
$accNos := $viewPontLopp/accountnos          let $viewPointName :=
data($viewPontLopp/name)          let $viewPointdata
:=$Rows[AccNo[@NO=tokenize($accNos, ",")]]/DETAIL/TRANSACTION/SUB_SECTION
let $nCalls := xs:decimal(sum(data($viewPointdata/@Calls)))          let
$totalAmt := round-half-to-even(sum(data($viewPointdata/@Amt)), 3)
 let $BNCount :=
count(data(distinct-values($viewPointdata/../../../CONN/@NO)))    let
$durSum :=  sum(data($viewPointdata/@Duration))     let $totDur :=
local:getHHMMSSFromNumber(xs:integer($durSum))          let $tMs :=
local:getMinutesFromDutation($totDur)          let $totalDurSec := $durSum
        return            <ROW>            <VPNM>{ $viewPointName }</VPNM>
           <CLS>{ $nCalls }</CLS>     <BNC>{$BNCount}</BNC>        <DUR>{
$totDur }</DUR>            <DUS>{$totalDurSec}</DUS>            <MIN>{ $tMs
}</MIN>            <AM>{if($tMs > 0 and $nCalls > 0) then   (let $avgMin :=
local:getMinsSecs($tMs, $nCalls)    return $avgMin) else
('00:00')}</AM><AMT>{ $totalAmt }</AMT><PC>{   if($totalAmt > 0 and $nCalls
Quoted text · Christian Grün · 1 line
> 0) then (let $avgPeRCall := round-half-to-even($totalAmt div $nCalls,
3)      return $avgPeRCall)   else (0.00) }</PC><PM>{        if($totalAmt >
0 and $tMs > 0) then          (let $avgPerMin :=
round-half-to-even($totalAmt div $tMs, 3)           return $avgPerMin)
   else (0.00)      }</PM><PN>{            if($totalAmt > 0 and $BNCount >
0) then              (let $avgPerNum := round-half-to-even($totalAmt div
$BNCount, 3)               return $avgPerNum)            else (0.000)
   }</PN></ROW>        };
let $viewPointCData := local:getViewPontChargesData()[position() le 10000]
let $TotCalls := xs:decimal(sum(data($viewPointCData/CLS))) let $TotAmt :=
sum(data($viewPointCData/AMT)) let $totDur := data($viewPointCData/DUR) let
$totalDur :=
local:getHHMMSSFromNumber(xs:integer(sum($viewPointCData/DUS))) let
$totalMins := local:getMinutesFromDutation($totalDur) let $BNcount :=
sum(data($viewPointCData/BNC)) let $TAPAmt := sum(data($viewPointCData/PN))
let $TotNoofAccs := count(data($viewPointCData/VPNM)) return   (
    if(count($viewPointCData) > 0) then (for $eachRow in $viewPointCData
 let $eachAmount := $eachRow/AMT  return    <R>{         $eachRow/node(),
      <PER>{    if($eachAmount > 0 and $TotAmt > 0) then      (let $perAll
:= round-half-to-even(($eachAmount div $TotAmt) * 100, 3)       return
$perAll)    else (0.00)  }</PER>       }</R>,
 <R><VPNM>Total/Average</VPNM><CLS>{ $TotCalls }</CLS><DUR>{ $totalDur
}</DUR><MIN>{  format-number($totalMins,'0.000') }</MIN><AM>{
 if($totalMins > 0 and $TotCalls > 0) then            (let $avgMinutes :=
local:getMinsSecs($totalMins, $TotCalls)             return $avgMinutes)
       else ('00:00')        }</AM><BNC>{$BNcount}</BNC><AMT>{
 format-number($TotAmt,'0.000') }</AMT><PC>{          if($TotAmt > 0 and
$TotCalls > 0) then            (let $avgCallAmt :=
round-half-to-even($TotAmt div $TotCalls, 3)             return
 format-number($avgCallAmt,'0.000'))          else (0.000)
 }</PC><PM>{               if($TotAmt > 0 and $totalMins > 0) then (let
$avgPerMin := round-half-to-even($TotAmt div $totalMins, 3)  return
 format-number($avgPerMin,'0.000'))               else (0.000)
}</PM><PN>{  if($TAPAmt > 0 and $TotNoofAccs > 0) then    (let $avgNumAmt
:= round-half-to-even($TAPAmt div $TotNoofAccs, 3)     return
 format-number($avgNumAmt,'0.000'))  else (0.000)}</PN><PER>100</PER></R>)
              else ()             )
Multilevel activity Summary report (Site level drill down report):-5minutes
declare function local:getMinsSecs($mins, $calls) {   let $mm :=
floor($mins div $calls)   let $pre :=
substring(substring-after(string(round-half-to-even(($mins div $calls) -
$mm, 5)), '.'), 1, 3)   let $precision := if($pre != '') then number($pre)
      else 0   let $ss := string(round($precision * 0.06))   return
concat(if($mm < 10) then concat('0', $mm)            else $mm, ':', if($ss
< 10) then concat('0', $ss)             else $ss) };
declare function local:getHHMMSSFromNumber($num as xs:integer) {  let $ss
:= xs:integer($num mod 60)  let $mm1 := xs:integer($num div 60)  let $mm :=
xs:integer($mm1 mod 60)  let $hh := xs:integer($mm1 div 60)  let $hhFinal
:= if(string-length(xs:string($hh)) >= 2) then $hh else (concat('0', $hh))
 let $mmFinal := if(string-length(xs:string($mm)) >= 2) then $mm else
(concat('0', $mm))  let $ssFinal := if(string-length(xs:string($ss)) >= 2)
then $ss else (concat('0', $ss))  return
concat($hhFinal,':',$mmFinal,':',$ssFinal) };
declare function local:getMinutesFromDutation($duration as xs:string) {
let $duration := tokenize($duration, ":")   let $hh := $duration[1]   let
$mm := $duration[2]   let $ss := $duration[3]   return
round-half-to-even((number(data($hh)) * 60 + number(data($mm)) +
(number(data($ss)) div 60)), 3) };
declare function local:getViewPontChargesData() {      let $Rows
:=(collection("867509_Voice_OCTOBER-19_Unbilled_1")/SUBCUSTBRK[AccNo[@NO =
(6945045)]][DOB >='2019-10-01' and DOB <='2019-10-31'])           let
$sites := "No Site"          let $eachSite := tokenize($sites, ",")
 for $site at $index in $eachSite          let $siteName := if($site ='No
Site')then('')else($site)     let $siteData := $Rows[Site[@NAME =
$siteName]]/DETAIL/TRANSACTION/SUB_SECTION     let $nCalls :=
xs:decimal(sum(data($siteData/@Calls)))          let $totalAmt :=
round-half-to-even(sum(data($siteData/@Amt)), 3)          let $BNCount :=
count(data(distinct-values($siteData/../../../CONN/@NO)))      let $durSum
:= xs:decimal(sum(data($siteData/@Duration)))    let $totDur :=
local:getHHMMSSFromNumber(xs:integer($durSum))          let $tMs :=
xs:decimal(local:getMinutesFromDutation($totDur))          let $totalDurSec
:= $durSum     return            if($nCalls > 0) then(<ROW><SNM>{ $site
}</SNM><CLS>{ $nCalls }</CLS><DUR>{ $totDur
}</DUR><DUS>{$totalDurSec}</DUS><MIN>{ $tMs }</MIN><AM>{        if($tMs > 0
and $nCalls > 0) then          (let $avgMin := local:getMinsSecs($tMs,
$nCalls)           return $avgMin)        else ('0:00')      }</AM>
 <AMT>{ $totalAmt }</AMT><PC>{   if($totalAmt > 0 and $nCalls > 0) then
(let $avgPeRCall := round-half-to-even($totalAmt div $nCalls, 3)
 return $avgPeRCall)   else (0.00) }</PC><PM>{          if($totalAmt > 0
and $tMs > 0) then            (let $avgPerMin :=
round-half-to-even($totalAmt div $tMs, 3)             return $avgPerMin)
       else (0.00)        }</PM><PN>{  if($totalAmt > 0 and $BNCount > 0)
then    (let $avgPerNum := round-half-to-even($totalAmt div $BNCount, 3)
  return $avgPerNum)  else (0.000)}</PN></ROW>)            else () };
let $viewPointCData := local:getViewPontChargesData() let $TotCalls :=
xs:decimal(sum(data($viewPointCData/CLS))) let $TotAmt :=
sum(data($viewPointCData/AMT)) let $totDur := data($viewPointCData/DUR) let
$totalDur :=
local:getHHMMSSFromNumber(xs:integer(sum($viewPointCData/DUS))) let
$totalMins := local:getMinutesFromDutation($totalDur) let $TAPAmt :=
sum(data($viewPointCData/PN)) let $TotNoofAccs :=
count(data($viewPointCData/SNM)) return   ( if(count($viewPointCData) > 0)
then   (for $eachRow in $viewPointCData    let $eachAmount := $eachRow/AMT
   return      <R>{           $eachRow/node(),           <PER>{
 if($eachAmount > 0 and $TotAmt > 0) then          (let $perAll :=
round-half-to-even(($eachAmount div $TotAmt) * 100, 3)           return
$perAll)        else (0.00)      }</PER>         }</R>,
 <R><SNM>Total/Average</SNM><CLS>{ $TotCalls }</CLS><DUR>{ $totalDur
}</DUR><MIN>{  format-number($totalMins,'0.000') }</MIN><AM>{
if($totalMins > 0 and $TotCalls > 0) then       (let $avgMinutes :=
local:getMinsSecs($totalMins, $TotCalls)        return $avgMinutes)
else ('00:00')   }</AM><AMT>{  format-number($TotAmt,'0.000') }</AMT><PC>{
           if($TotAmt > 0 and $TotCalls > 0) then(let $avgCallAmt :=
round-half-to-even($TotAmt div $TotCalls, 3) return
 format-number($avgCallAmt,'0.000'))            else (0.000)
 }</PC><PM>{     if($TotAmt > 0 and $totalMins > 0) then       (let
$avgPerMin := round-half-to-even($TotAmt div $totalMins, 3)        return
 format-number($avgPerMin,'0.000'))     else (0.000)   }</PM><PN>{
  if($TAPAmt > 0 and $TotNoofAccs > 0) then             (let $avgNumAmt :=
round-half-to-even($TAPAmt div $TotNoofAccs, 3)              return
 format-number($avgNumAmt,'0.000'))           else (0.000)
}</PN><PER>100</PER></R>) else ()             )
Multilevel activity Summary report (service number level drill down report):error-no-collection
declare function local:getLabelName($serNo as xs:string) {   let $doc :=
collection('LabelCollection_1573625140685') for $label in $doc//row[@ser_no
= $serNo] order by $label/@upDate descending return $label   };declare
function local:getHHMMSSFromNumber($num as xs:integer) {  let $ss :=
xs:integer($num mod 60)  let $mm1 := xs:integer($num div 60)  let $mm :=
xs:integer($mm1 mod 60)  let $hh := xs:integer($mm1 div 60)  let $hhFinal
:= if(string-length(xs:string($hh)) >= 2) then $hh else (concat('0', $hh))
 let $mmFinal := if(string-length(xs:string($mm)) >= 2) then $mm else
(concat('0', $mm))  let $ssFinal := if(string-length(xs:string($ss)) >= 2)
then $ss else (concat('0', $ss))  return
concat($hhFinal,':',$mmFinal,':',$ssFinal) }; declare function
local:getMinutesFromDutation($duration as xs:string) {   let $duration :=
tokenize($duration, ":")   let $hh := $duration[1]   let $mm :=
$duration[2]   let $ss := $duration[3]   return number(data($hh)) * 60 +
number(data($mm)) + floor(number(data($ss)) div 60) }; declare function
local:getViewPontChargesData() {           let $Rows :=
(collection("867509_Voice_OCTOBER-19_Unbilled_1")/SUBCUSTBRK[AccNo[@NO =
(6945045)]][Site[@NAME='']][DOB >='2019-10-01' and DOB <='2019-10-31'])
   let $serviceNumbers :=
string-join(distinct-values(data($Rows/CONN/@NO)),',')          let
$eachServiceNumber := tokenize($serviceNumbers, ",")         for $service
at $index in $eachServiceNumber          let $serviceNumber := $service
 let $serviceData := $Rows[CONN[@NO =
$serviceNumber]]/DETAIL/TRANSACTION/SUB_SECTION          let $nCalls :=
xs:decimal(sum(data($serviceData/@Calls)))          let $totalAmt :=
round-half-to-even(sum(data($serviceData/@Amt)), 3)          let $BNCount
:= count(data(distinct-values($serviceData/../../../CONN/@NO)))    let
$durSum := sum(data($serviceData/@Duration))      let $totDur :=
local:getHHMMSSFromNumber(xs:integer($durSum))          let $tMs :=
local:getMinutesFromDutation($totDur)          let $totalDurSec := $durSum
        return  <ROW><SNB>{ $serviceNumber }</SNB><SNL>{
distinct-values(local:getLabelName($serviceNumber)[position()=1])
}</SNL><CLS>{ $nCalls }</CLS><DUR>{ $totDur
}</DUR><DUS>{$totalDurSec}</DUS><MIN>{ $tMs }</MIN><AM>{
           if($tMs > 0 and $nCalls > 0) then                      (let
$avgMin := round-half-to-even($tMs div $nCalls, 2)
return $avgMin)                              else (0.00)
         }</AM><AMT>{ $totalAmt }</AMT><PC>{
if($totalAmt > 0 and $nCalls > 0) then                                 (let
$avgPeRCall := round-half-to-even($totalAmt div $nCalls, 2)
                 return $avgPeRCall)                               else
(0.00)                             }</PC><PM>{
   if($totalAmt > 0 and $tMs > 0) then
 (let $avgPerMin := round-half-to-even($totalAmt div $tMs, 3)
                    return $avgPerMin)                                else
(0.00)                              }</PM><PN>{
   if($totalAmt > 0 and $BNCount > 0) then
 (let $avgPerNum := round-half-to-even($totalAmt div $BNCount, 3)
                        return $avgPerNum)
 else (0.000)                              }</PN></ROW>       }; let
$viewPointCData := local:getViewPontChargesData() let $TotCalls :=
xs:decimal(sum(data($viewPointCData/CLS))) let $TotAmt :=
sum(data($viewPointCData/AMT)) let $totalDur :=
local:getHHMMSSFromNumber(xs:integer(sum($viewPointCData/DUS))) let
$totalMins := local:getMinutesFromDutation($totalDur) let $TAPAmt :=
sum(data($viewPointCData/PN)) let $TotNoofAccs :=
count(data($viewPointCData/SNB)) return   (     if(count($viewPointCData) >
0) then       (for $eachRow in $viewPointCData        let $eachAmount :=
$eachRow/AMT        return          <R>{     $eachRow/node(),     <PER>{
   if($eachAmount > 0 and $TotAmt > 0) then        (let $perAll :=
round-half-to-even(($eachAmount div $TotAmt) * 100, 2)         return
$perAll)      else (0.00)    }</PER>   }</R>,
 <R><SNB>Total/Average</SNB><CLS>{ $TotCalls }</CLS><DUR>{ $totalDur
}</DUR><MIN>{  format-number($totalMins,'0.000') }</MIN><AM>{
    if($totalMins > 0 and $TotCalls > 0) then                     (let
$avgMinutes := round-half-to-even($totalMins div $TotCalls, 2)
         return  format-number($avgMinutes,'0.000'))                   else
(0.000)                 }</AM><AMT>{  format-number($TotAmt,'0.000')
}</AMT><PC>{                  if($TotAmt > 0 and $TotCalls > 0) then
             (let $avgCallAmt := round-half-to-even($TotAmt div $TotCalls,
2)                     return  format-number($avgCallAmt,'0.000'))
         else (0.000)                          }</PC><PM>{
            if($TotAmt > 0 and $totalMins > 0) then
(let $avgPerMin := round-half-to-even($TotAmt div $totalMins, 3)
           return  format-number($avgPerMin,'0.000'))
      else (0.000)                           }</PM><PN>{
          if($TAPAmt > 0 and $TotNoofAccs > 0) then
(let $avgNumAmt := round-half-to-even($TAPAmt div $TotNoofAccs, 3)
             return  format-number($avgNumAmt,'0.000'))
        else (0.000)                           }</PN><PER>100</PER></R>)
  else ()   )
Regards, YSL
Quoted text · Christian Grün · 24 lines
On Tue, Dec 17, 2019 at 12:27 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi YSL, > > > Every day we are moving collections around 55k to 60k no of xml files > large account. Its taking more than 18 hours. At that time we want access > the collection for generating report its on lock mode and scrip that > collection. > > Some questions back: > > 1. What is the average size of each of your document? > 2. Will this sum up to a yearly amount of appr. 20 million files, or > do you have an upper limit? > 3. Which of our API(s) are you using to add the documents? > 4. How do your queries for reporting data look like? > > > Please help and do needful. > > https://ell.stackexchange.com/a/17626 ;) > > Best, > Christian >
CG
Christian Grün17 Dec 2019 08:48 #
Quoted text · Sreenivasulu Yadavalli · 1 line
> A: Every day we are generating collections (Unbilled Mobile Transactionn data) Per account appr 50k - 55k files.
So you are creating a new collection every day, and it will have around 60k documents at the end of the day? Will you discard older collections? And I assume you need to incrementally add the documents, or do you have all of them at hand before you run the first queries? In the latter case, you might need to partition your database and work with multiple instances (see [1,2] for details). In both cases, it might be recommendable to experience with the UPDINDEX option [3].
Quoted text · Sreenivasulu Yadavalli · 1 line
> 4. How do your queries for reporting data look like?
Feel invited to give us more information on the database that you accessing in your query. [1] http://docs.basex.org/wiki/Databases [2] http://docs.basex.org/wiki/Statistics [3] http://docs.basex.org/wiki/Options#UPDINDEX
LQ
Liam R. E. Quinin reply to Sreenivasulu Yadavalli, 17 Dec 2019, 07:1817 Dec 2019 10:06 #
Quoted text · Sreenivasulu Yadavalli · 5 lines
On Tue, 2019-12-17 at 11:48 +0530, Sreenivasulu Yadavalli wrote: > > Every day we are moving collections around 55k to 60k no of xml files > large > account.
Here, i just created a BaseX database with 80,000 XML files. It took under one minute on the Linux desktop system i use.
Quoted text · Sreenivasulu Yadavalli · 1 line
> Its taking more than 18 hours.
This make no sense. How much memory do you have on the computer? What exactly do you mean by moving collections around? Are you taking a database with 100 million documents and renaming 50,000 of them? What operations exactly are slow? Liam
-- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
LQ
Liam R. E. Quin18 Dec 2019 10:41 #
Quoted text · Sreenivasulu Yadavalli · 5 lines
On Wed, 2019-12-18 at 11:10 +0530, Sreenivasulu Yadavalli wrote: > > > What exactly do you mean by moving collections around?. > > A: moving the collections in the same system.
So, you use the Linux "mv" command to do this? Or what? What exactly do you mean by collections? I for one would find it easier if you would stop talking in riddles, as my telepathy skills are weak.
Quoted text · Liam R. E. Quin · 4 lines
> And every day we have to > update the existing collection with call data. So finding the > collection is > taking more time
How do you look for the collection? Isn't it a separate BaseX database?
Quoted text · Liam R. E. Quin · 7 lines
> > Are you taking a database with 100 million documents and renaming > 50,000 of them? > > What operations exactly are slow? > > A: finding the existing collection.
find / -name collection.db ? This is a little frustrating in that you are asking for people's help but not explaining the problem. Are you saying that fn:collection() is slow in BaseX? What arguments are you passing it exactly? What is the size, in gigabytes, of the database, on disk? How many documents are in it? Can you give step-by-step EXACT AND PRECISSE instructions so someone else could reproduce the problem you have having? Complete and exact instructions, with sample files if needed, so they can reproduce the problem on their own computer? A database with 80,000 files is easy to "find" here, and opens quickly, in a small fraction of a second. It doesn't take hours. Is something else running on your computer that makes it slow?? Note: please remember to copy the list in your replies, as the BaseX people are far more knowledgeable about BaseX than i am :) My goal as an analyst is to get you to explain the problem you are having clearly enough that you can get an answer :) Liam
-- Liam Quin, https://www.delightfulcomputing.com/ Available for XML/Document/Information Architecture/XSLT/ XSL/XQuery/Web/Text Processing/A11Y training, work & consulting. Barefoot Web-slave, antique illustrations: http://www.fromoldbooks.org
SY
Sreenivasulu Yadavalliin reply to Sreenivasulu Yadavalli, 17 Dec 2019, 07:1820 Dec 2019 10:58 #
Hi Christian, Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names.
We're able to get process using the memory file stream,
File ofile= new File("<<FILEPATH");
FileInputStream xmlFile = new FileInputStream(ofile);
clientSession.add(<<FILE_NAME>>, xmlFile);
We're not able to process these files using the command *ADD * clientSession.execute("ADD <<FILE_PATH>>"); Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. Regards, YSL
Quoted text · Sreenivasulu Yadavalli · 27 lines
On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < srinivasuluy75 at gmail.com> wrote: > Hi Team, > > > > > > We have a big problem for creating the collection bcoz of large no of xmls. > > > > Every day we are moving collections around 55k to 60k no of xml files > large account. Its taking more than 18 hours. At that time we want access > the collection for generating report its on lock mode and scrip that > collection. > > > > Please help and do needful. > > > > Regards, > > YSL >
SY
Sreenivasulu Yadavalli20 Dec 2019 11:02 #
Hi Christian, Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names.
We're able to get process using the memory file stream,
File ofile= new File("<<FILEPATH");
FileInputStream xmlFile = new FileInputStream(ofile);
clientSession.add(<<FILE_NAME>>, xmlFile);
We're not able to process these files using the command *ADD * clientSession.execute("ADD <<FILE_PATH>>"); Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. Regards, YSL
CG
Christian Grünin reply to Sreenivasulu Yadavalli, 20 Dec 2019, 10:5820 Dec 2019 12:44 #
Dear Sreenivasulu, Kindly provide me with a self-contained example first (at earliest, or whenever you have time to do so). See [1] for more instructions. Thanks in advance, Christian [1] http://www.sscce.org
Quoted text · Sreenivasulu Yadavalli · 45 lines
On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > > Hi Christian, > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. > > We're able to get process using the memory file stream, > File ofile= new File("<<FILEPATH"); > FileInputStream xmlFile = new FileInputStream(ofile); > clientSession.add(<<FILE_NAME>>, xmlFile); > > We're not able to process these files using the command ADD > clientSession.execute("ADD <<FILE_PATH>>"); > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. > > > Regards, > YSL > > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: >> >> Hi Team, >> >> >> >> >> >> We have a big problem for creating the collection bcoz of large no of xmls. >> >> >> >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. >> >> >> >> Please help and do needful. >> >> >> >> Regards, >> >> YSL
CG
Christian Grün29 Dec 2019 20:36 #
Dear Sreenivasulu (cc to the list), Thanks for the Java code; I could reproduce the behavior. A new stable snapshot will be available in a few minutes [1]; could you check for us if it resolves the problem? Thanks, Christian [1] http://files.basex.org/releases/latest/
Quoted text · Christian Grün · 45 lines
> On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > <srinivasuluy75 at gmail.com> wrote: > > > > Hi Christian, > > > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. > > > > We're able to get process using the memory file stream, > > File ofile= new File("<<FILEPATH"); > > FileInputStream xmlFile = new FileInputStream(ofile); > > clientSession.add(<<FILE_NAME>>, xmlFile); > > > > We're not able to process these files using the command ADD > > clientSession.execute("ADD <<FILE_PATH>>"); > > > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. > > > > > > Regards, > > YSL > > > > > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > >> > >> Hi Team, > >> > >> > >> > >> > >> > >> We have a big problem for creating the collection bcoz of large no of xmls. > >> > >> > >> > >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. > >> > >> > >> > >> Please help and do needful. > >> > >> > >> > >> Regards, > >> > >> YSL
SY
Sreenivasulu Yadavalli3 Jan 2020 08:32 #
Hi Christian, could you please provide \ suggest the solution for the shared concern. Regards, YSL
Quoted text · Christian Grün · 69 lines
On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: > Dear Sreenivasulu (cc to the list), > > Thanks for the Java code; I could reproduce the behavior. > > A new stable snapshot will be available in a few minutes [1]; could > you check for us if it resolves the problem? > > Thanks, > Christian > > [1] http://files.basex.org/releases/latest/ > > > > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > > <srinivasuluy75 at gmail.com> wrote: > > > > > > Hi Christian, > > > > > > Currently, as part of our business use case we are receiving some XML > file, which contains special characters like semicolon ": " in the file > names. > > > > > > We're able to get process using the memory file stream, > > > File ofile= new File("<<FILEPATH"); > > > FileInputStream xmlFile = new FileInputStream(ofile); > > > clientSession.add(<<FILE_NAME>>, xmlFile); > > > > > > We're not able to process these files using the command ADD > > > clientSession.execute("ADD <<FILE_PATH>>"); > > > > > > Kindly suggest us the way forward \ alternative approach to the > resolve the concern at earliest. > > > > > > > > > Regards, > > > YSL > > > > > > > > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < > srinivasuluy75 at gmail.com> wrote: > > >> > > >> Hi Team, > > >> > > >> > > >> > > >> > > >> > > >> We have a big problem for creating the collection bcoz of large no of > xmls. > > >> > > >> > > >> > > >> Every day we are moving collections around 55k to 60k no of xml files > large account. Its taking more than 18 hours. At that time we want access > the collection for generating report its on lock mode and scrip that > collection. > > >> > > >> > > >> > > >> Please help and do needful. > > >> > > >> > > >> > > >> Regards, > > >> > > >> YSL >
CG
Christian Grün3 Jan 2020 10:58 #
Quoted text · Sreenivasulu Yadavalli · 1 line
> could you please provide \ suggest the solution for the shared concern.
So what happened when you tried the provided snapshot as suggested?
Quoted text · Sreenivasulu Yadavalli · 71 lines
On Fri, Jan 3, 2020 at 8:32 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > > Hi Christian, > > > > Regards, > YSL > > > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: >> >> Dear Sreenivasulu (cc to the list), >> >> Thanks for the Java code; I could reproduce the behavior. >> >> A new stable snapshot will be available in a few minutes [1]; could >> you check for us if it resolves the problem? >> >> Thanks, >> Christian >> >> [1] http://files.basex.org/releases/latest/ >> >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli >> > <srinivasuluy75 at gmail.com> wrote: >> > > >> > > Hi Christian, >> > > >> > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. >> > > >> > > We're able to get process using the memory file stream, >> > > File ofile= new File("<<FILEPATH"); >> > > FileInputStream xmlFile = new FileInputStream(ofile); >> > > clientSession.add(<<FILE_NAME>>, xmlFile); >> > > >> > > We're not able to process these files using the command ADD >> > > clientSession.execute("ADD <<FILE_PATH>>"); >> > > >> > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. >> > > >> > > >> > > Regards, >> > > YSL >> > > >> > > >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: >> > >> >> > >> Hi Team, >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> We have a big problem for creating the collection bcoz of large no of xmls. >> > >> >> > >> >> > >> >> > >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. >> > >> >> > >> >> > >> >> > >> Please help and do needful. >> > >> >> > >> >> > >> >> > >> Regards, >> > >> >> > >> YSL
SY
Sreenivasulu Yadavalli8 Jan 2020 07:17 #
Christian, I haven't received the snapshot. could you please provide it?
Quoted text · Christian Grün · 89 lines
On Fri, Jan 3, 2020 at 3:28 PM Christian Grün <christian.gruen at gmail.com> wrote: > > could you please provide \ suggest the solution for the shared concern. > > So what happened when you tried the provided snapshot as suggested? > > > > On Fri, Jan 3, 2020 at 8:32 AM Sreenivasulu Yadavalli > <srinivasuluy75 at gmail.com> wrote: > > > > Hi Christian, > > > > > > > > Regards, > > YSL > > > > > > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün < > christian.gruen at gmail.com> wrote: > >> > >> Dear Sreenivasulu (cc to the list), > >> > >> Thanks for the Java code; I could reproduce the behavior. > >> > >> A new stable snapshot will be available in a few minutes [1]; could > >> you check for us if it resolves the problem? > >> > >> Thanks, > >> Christian > >> > >> [1] http://files.basex.org/releases/latest/ > >> > >> > >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > >> > <srinivasuluy75 at gmail.com> wrote: > >> > > > >> > > Hi Christian, > >> > > > >> > > Currently, as part of our business use case we are receiving some > XML file, which contains special characters like semicolon ": " in the file > names. > >> > > > >> > > We're able to get process using the memory file stream, > >> > > File ofile= new File("<<FILEPATH"); > >> > > FileInputStream xmlFile = new FileInputStream(ofile); > >> > > clientSession.add(<<FILE_NAME>>, xmlFile); > >> > > > >> > > We're not able to process these files using the command ADD > >> > > clientSession.execute("ADD <<FILE_PATH>>"); > >> > > > >> > > Kindly suggest us the way forward \ alternative approach to the > resolve the concern at earliest. > >> > > > >> > > > >> > > Regards, > >> > > YSL > >> > > > >> > > > >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < > srinivasuluy75 at gmail.com> wrote: > >> > >> > >> > >> Hi Team, > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> We have a big problem for creating the collection bcoz of large no > of xmls. > >> > >> > >> > >> > >> > >> > >> > >> Every day we are moving collections around 55k to 60k no of xml > files large account. Its taking more than 18 hours. At that time we want > access the collection for generating report its on lock mode and scrip that > collection. > >> > >> > >> > >> > >> > >> > >> > >> Please help and do needful. > >> > >> > >> > >> > >> > >> > >> > >> Regards, > >> > >> > >> > >> YSL >
AH
Alexander Holupirek8 Jan 2020 07:59 #
Please download the latest snapshot from http://files.basex.org/releases/latest/
Quoted text · Sreenivasulu Yadavalli · 82 lines
> On 8. Jan 2020, at 07:17, Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > > Christian, I haven't received the snapshot. could you please provide it? > > On Fri, Jan 3, 2020 at 3:28 PM Christian Grün <christian.gruen at gmail.com> wrote: > > could you please provide \ suggest the solution for the shared concern. > > So what happened when you tried the provided snapshot as suggested? > > > > On Fri, Jan 3, 2020 at 8:32 AM Sreenivasulu Yadavalli > <srinivasuluy75 at gmail.com> wrote: > > > > Hi Christian, > > > > > > > > Regards, > > YSL > > > > > > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: > >> > >> Dear Sreenivasulu (cc to the list), > >> > >> Thanks for the Java code; I could reproduce the behavior. > >> > >> A new stable snapshot will be available in a few minutes [1]; could > >> you check for us if it resolves the problem? > >> > >> Thanks, > >> Christian > >> > >> [1] http://files.basex.org/releases/latest/ > >> > >> > >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > >> > <srinivasuluy75 at gmail.com> wrote: > >> > > > >> > > Hi Christian, > >> > > > >> > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. > >> > > > >> > > We're able to get process using the memory file stream, > >> > > File ofile= new File("<<FILEPATH"); > >> > > FileInputStream xmlFile = new FileInputStream(ofile); > >> > > clientSession.add(<<FILE_NAME>>, xmlFile); > >> > > > >> > > We're not able to process these files using the command ADD > >> > > clientSession.execute("ADD <<FILE_PATH>>"); > >> > > > >> > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. > >> > > > >> > > > >> > > Regards, > >> > > YSL > >> > > > >> > > > >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > >> > >> > >> > >> Hi Team, > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> We have a big problem for creating the collection bcoz of large no of xmls. > >> > >> > >> > >> > >> > >> > >> > >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. > >> > >> > >> > >> > >> > >> > >> > >> Please help and do needful. > >> > >> > >> > >> > >> > >> > >> > >> Regards, > >> > >> > >> > >> YSL
SY
Sreenivasulu Yadavalliin reply to Christian Grün, 29 Dec 2019, 20:3610 Feb 2020 10:31 #
Dear Christian, With this latest version it is working fine,Also please provide guidance for below issue *we have one collection with 3,00,000(3 lakhs) xmls and the collection size is 4 gb,on this collection we are running xquery ,this querytaking more than 20 minutes.Here we need to run xquery all 3 lakhs xmls(for report purpose we need to get data from all xmls) .Note : Same query running very quick on huge size of collection(size more than 30gb) but in this case number of xmls in that collection isless(nearly 350).Please suggest any clue to tune the query.* Regards, YSL
Quoted text · Christian Grün · 69 lines
On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: > Dear Sreenivasulu (cc to the list), > > Thanks for the Java code; I could reproduce the behavior. > > A new stable snapshot will be available in a few minutes [1]; could > you check for us if it resolves the problem? > > Thanks, > Christian > > [1] http://files.basex.org/releases/latest/ > > > > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > > <srinivasuluy75 at gmail.com> wrote: > > > > > > Hi Christian, > > > > > > Currently, as part of our business use case we are receiving some XML > file, which contains special characters like semicolon ": " in the file > names. > > > > > > We're able to get process using the memory file stream, > > > File ofile= new File("<<FILEPATH"); > > > FileInputStream xmlFile = new FileInputStream(ofile); > > > clientSession.add(<<FILE_NAME>>, xmlFile); > > > > > > We're not able to process these files using the command ADD > > > clientSession.execute("ADD <<FILE_PATH>>"); > > > > > > Kindly suggest us the way forward \ alternative approach to the > resolve the concern at earliest. > > > > > > > > > Regards, > > > YSL > > > > > > > > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < > srinivasuluy75 at gmail.com> wrote: > > >> > > >> Hi Team, > > >> > > >> > > >> > > >> > > >> > > >> We have a big problem for creating the collection bcoz of large no of > xmls. > > >> > > >> > > >> > > >> Every day we are moving collections around 55k to 60k no of xml files > large account. Its taking more than 18 hours. At that time we want access > the collection for generating report its on lock mode and scrip that > collection. > > >> > > >> > > >> > > >> Please help and do needful. > > >> > > >> > > >> > > >> Regards, > > >> > > >> YSL >
CG
Christian Grün10 Feb 2020 11:29 #
Hi Sreenivasulu,
Quoted text · Sreenivasulu Yadavalli · 1 line
> Please suggest any clue to tune the query.
Which query? Best, Christian
Quoted text · Sreenivasulu Yadavalli · 66 lines
> > Regards, > YSL > > > > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: >> >> Dear Sreenivasulu (cc to the list), >> >> Thanks for the Java code; I could reproduce the behavior. >> >> A new stable snapshot will be available in a few minutes [1]; could >> you check for us if it resolves the problem? >> >> Thanks, >> Christian >> >> [1] http://files.basex.org/releases/latest/ >> >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli >> > <srinivasuluy75 at gmail.com> wrote: >> > > >> > > Hi Christian, >> > > >> > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. >> > > >> > > We're able to get process using the memory file stream, >> > > File ofile= new File("<<FILEPATH"); >> > > FileInputStream xmlFile = new FileInputStream(ofile); >> > > clientSession.add(<<FILE_NAME>>, xmlFile); >> > > >> > > We're not able to process these files using the command ADD >> > > clientSession.execute("ADD <<FILE_PATH>>"); >> > > >> > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. >> > > >> > > >> > > Regards, >> > > YSL >> > > >> > > >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: >> > >> >> > >> Hi Team, >> > >> >> > >> >> > >> >> > >> >> > >> >> > >> We have a big problem for creating the collection bcoz of large no of xmls. >> > >> >> > >> >> > >> >> > >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. >> > >> >> > >> >> > >> >> > >> Please help and do needful. >> > >> >> > >> >> > >> >> > >> Regards, >> > >> >> > >> YSL
SY
Sreenivasulu Yadavalli10 Feb 2020 11:49 #
Dear Christian, Please find the attachment for query Regards, YSL
Quoted text · Christian Grün · 89 lines
On Mon, Feb 10, 2020 at 3:59 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi Sreenivasulu, > > > Please suggest any clue to tune the query. > > Which query? > > Best, > Christian > > > > > > > Regards, > > YSL > > > > > > > > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün < > christian.gruen at gmail.com> wrote: > >> > >> Dear Sreenivasulu (cc to the list), > >> > >> Thanks for the Java code; I could reproduce the behavior. > >> > >> A new stable snapshot will be available in a few minutes [1]; could > >> you check for us if it resolves the problem? > >> > >> Thanks, > >> Christian > >> > >> [1] http://files.basex.org/releases/latest/ > >> > >> > >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > >> > <srinivasuluy75 at gmail.com> wrote: > >> > > > >> > > Hi Christian, > >> > > > >> > > Currently, as part of our business use case we are receiving some > XML file, which contains special characters like semicolon ": " in the file > names. > >> > > > >> > > We're able to get process using the memory file stream, > >> > > File ofile= new File("<<FILEPATH"); > >> > > FileInputStream xmlFile = new FileInputStream(ofile); > >> > > clientSession.add(<<FILE_NAME>>, xmlFile); > >> > > > >> > > We're not able to process these files using the command ADD > >> > > clientSession.execute("ADD <<FILE_PATH>>"); > >> > > > >> > > Kindly suggest us the way forward \ alternative approach to the > resolve the concern at earliest. > >> > > > >> > > > >> > > Regards, > >> > > YSL > >> > > > >> > > > >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < > srinivasuluy75 at gmail.com> wrote: > >> > >> > >> > >> Hi Team, > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> We have a big problem for creating the collection bcoz of large no > of xmls. > >> > >> > >> > >> > >> > >> > >> > >> Every day we are moving collections around 55k to 60k no of xml > files large account. Its taking more than 18 hours. At that time we want > access the collection for generating report its on lock mode and scrip that > collection. > >> > >> > >> > >> > >> > >> > >> > >> Please help and do needful. > >> > >> > >> > >> > >> > >> > >> > >> Regards, > >> > >> > >> > >> YSL >
CG
Christian Grün10 Feb 2020 20:01 #
Hi YSL, You could try to rewrite the number test to a string comparison: OLD: ....[AccNo[@NO = (12345)]] NEW: ....[AccNo/@NO = '12345'] If you run your query, are some expressions rewritten for index access? Just have a closer look at the Info View output and look for "applying... index" strings. Best, Christian
Quoted text · Sreenivasulu Yadavalli · 91 lines
On Mon, Feb 10, 2020 at 11:49 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > > Dear Christian, > Please find the attachment for query > > > > Regards, > YSL > > > On Mon, Feb 10, 2020 at 3:59 PM Christian Grün <christian.gruen at gmail.com> wrote: >> >> Hi Sreenivasulu, >> >> > Please suggest any clue to tune the query. >> >> Which query? >> >> Best, >> Christian >> >> >> >> > >> > Regards, >> > YSL >> > >> > >> > >> > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: >> >> >> >> Dear Sreenivasulu (cc to the list), >> >> >> >> Thanks for the Java code; I could reproduce the behavior. >> >> >> >> A new stable snapshot will be available in a few minutes [1]; could >> >> you check for us if it resolves the problem? >> >> >> >> Thanks, >> >> Christian >> >> >> >> [1] http://files.basex.org/releases/latest/ >> >> >> >> >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli >> >> > <srinivasuluy75 at gmail.com> wrote: >> >> > > >> >> > > Hi Christian, >> >> > > >> >> > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. >> >> > > >> >> > > We're able to get process using the memory file stream, >> >> > > File ofile= new File("<<FILEPATH"); >> >> > > FileInputStream xmlFile = new FileInputStream(ofile); >> >> > > clientSession.add(<<FILE_NAME>>, xmlFile); >> >> > > >> >> > > We're not able to process these files using the command ADD >> >> > > clientSession.execute("ADD <<FILE_PATH>>"); >> >> > > >> >> > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. >> >> > > >> >> > > >> >> > > Regards, >> >> > > YSL >> >> > > >> >> > > >> >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: >> >> > >> >> >> > >> Hi Team, >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> We have a big problem for creating the collection bcoz of large no of xmls. >> >> > >> >> >> > >> >> >> > >> >> >> > >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. >> >> > >> >> >> > >> >> >> > >> >> >> > >> Please help and do needful. >> >> > >> >> >> > >> >> >> > >> >> >> > >> Regards, >> >> > >> >> >> > >> YSL
SY
Sreenivasulu Yadavalli2 Mar 2020 07:43 #
Hi Christian, We're having a requirement to move Our application to AWS cloud. Please suggest on how to use Base-X DB on AWS cloud environment ? Regards, YSL
Quoted text · Christian Grün · 123 lines
On Tue, Feb 11, 2020 at 12:31 AM Christian Grün <christian.gruen at gmail.com> wrote: > Hi YSL, > > You could try to rewrite the number test to a string comparison: > > OLD: > ....[AccNo[@NO = (12345)]] > > NEW: > ....[AccNo/@NO = '12345'] > > If you run your query, are some expressions rewritten for index > access? Just have a closer look at the Info View output and look for > "applying... index" strings. > > Best, > Christian > > > > On Mon, Feb 10, 2020 at 11:49 AM Sreenivasulu Yadavalli > <srinivasuluy75 at gmail.com> wrote: > > > > Dear Christian, > > Please find the attachment for query > > > > > > > > Regards, > > YSL > > > > > > On Mon, Feb 10, 2020 at 3:59 PM Christian Grün < > christian.gruen at gmail.com> wrote: > >> > >> Hi Sreenivasulu, > >> > >> > Please suggest any clue to tune the query. > >> > >> Which query? > >> > >> Best, > >> Christian > >> > >> > >> > >> > > >> > Regards, > >> > YSL > >> > > >> > > >> > > >> > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün < > christian.gruen at gmail.com> wrote: > >> >> > >> >> Dear Sreenivasulu (cc to the list), > >> >> > >> >> Thanks for the Java code; I could reproduce the behavior. > >> >> > >> >> A new stable snapshot will be available in a few minutes [1]; could > >> >> you check for us if it resolves the problem? > >> >> > >> >> Thanks, > >> >> Christian > >> >> > >> >> [1] http://files.basex.org/releases/latest/ > >> >> > >> >> > >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > >> >> > <srinivasuluy75 at gmail.com> wrote: > >> >> > > > >> >> > > Hi Christian, > >> >> > > > >> >> > > Currently, as part of our business use case we are receiving > some XML file, which contains special characters like semicolon ": " in the > file names. > >> >> > > > >> >> > > We're able to get process using the memory file stream, > >> >> > > File ofile= new File("<<FILEPATH"); > >> >> > > FileInputStream xmlFile = new FileInputStream(ofile); > >> >> > > clientSession.add(<<FILE_NAME>>, xmlFile); > >> >> > > > >> >> > > We're not able to process these files using the command ADD > >> >> > > clientSession.execute("ADD <<FILE_PATH>>"); > >> >> > > > >> >> > > Kindly suggest us the way forward \ alternative approach to the > resolve the concern at earliest. > >> >> > > > >> >> > > > >> >> > > Regards, > >> >> > > YSL > >> >> > > > >> >> > > > >> >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < > srinivasuluy75 at gmail.com> wrote: > >> >> > >> > >> >> > >> Hi Team, > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> We have a big problem for creating the collection bcoz of large > no of xmls. > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> Every day we are moving collections around 55k to 60k no of xml > files large account. Its taking more than 18 hours. At that time we want > access the collection for generating report its on lock mode and scrip that > collection. > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> Please help and do needful. > >> >> > >> > >> >> > >> > >> >> > >> > >> >> > >> Regards, > >> >> > >> > >> >> > >> YSL >
CG
Christian Grün2 Mar 2020 14:15 #
Hi YSL, What about my last response: Did you check the query plans, could you observe any improvements? Best, Christian
Quoted text · Sreenivasulu Yadavalli · 124 lines
On Mon, Mar 2, 2020 at 7:44 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: > > Hi Christian, > > We're having a requirement to move Our application to AWS cloud. > > Please suggest on how to use Base-X DB on AWS cloud environment ? > > Regards, > YSL > > On Tue, Feb 11, 2020 at 12:31 AM Christian Grün <christian.gruen at gmail.com> wrote: >> >> Hi YSL, >> >> You could try to rewrite the number test to a string comparison: >> >> OLD: >> ....[AccNo[@NO = (12345)]] >> >> NEW: >> ....[AccNo/@NO = '12345'] >> >> If you run your query, are some expressions rewritten for index >> access? Just have a closer look at the Info View output and look for >> "applying... index" strings. >> >> Best, >> Christian >> >> >> >> On Mon, Feb 10, 2020 at 11:49 AM Sreenivasulu Yadavalli >> <srinivasuluy75 at gmail.com> wrote: >> > >> > Dear Christian, >> > Please find the attachment for query >> > >> > >> > >> > Regards, >> > YSL >> > >> > >> > On Mon, Feb 10, 2020 at 3:59 PM Christian Grün <christian.gruen at gmail.com> wrote: >> >> >> >> Hi Sreenivasulu, >> >> >> >> > Please suggest any clue to tune the query. >> >> >> >> Which query? >> >> >> >> Best, >> >> Christian >> >> >> >> >> >> >> >> > >> >> > Regards, >> >> > YSL >> >> > >> >> > >> >> > >> >> > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün <christian.gruen at gmail.com> wrote: >> >> >> >> >> >> Dear Sreenivasulu (cc to the list), >> >> >> >> >> >> Thanks for the Java code; I could reproduce the behavior. >> >> >> >> >> >> A new stable snapshot will be available in a few minutes [1]; could >> >> >> you check for us if it resolves the problem? >> >> >> >> >> >> Thanks, >> >> >> Christian >> >> >> >> >> >> [1] http://files.basex.org/releases/latest/ >> >> >> >> >> >> >> >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli >> >> >> > <srinivasuluy75 at gmail.com> wrote: >> >> >> > > >> >> >> > > Hi Christian, >> >> >> > > >> >> >> > > Currently, as part of our business use case we are receiving some XML file, which contains special characters like semicolon ": " in the file names. >> >> >> > > >> >> >> > > We're able to get process using the memory file stream, >> >> >> > > File ofile= new File("<<FILEPATH"); >> >> >> > > FileInputStream xmlFile = new FileInputStream(ofile); >> >> >> > > clientSession.add(<<FILE_NAME>>, xmlFile); >> >> >> > > >> >> >> > > We're not able to process these files using the command ADD >> >> >> > > clientSession.execute("ADD <<FILE_PATH>>"); >> >> >> > > >> >> >> > > Kindly suggest us the way forward \ alternative approach to the resolve the concern at earliest. >> >> >> > > >> >> >> > > >> >> >> > > Regards, >> >> >> > > YSL >> >> >> > > >> >> >> > > >> >> >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> wrote: >> >> >> > >> >> >> >> > >> Hi Team, >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> We have a big problem for creating the collection bcoz of large no of xmls. >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> Every day we are moving collections around 55k to 60k no of xml files large account. Its taking more than 18 hours. At that time we want access the collection for generating report its on lock mode and scrip that collection. >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> Please help and do needful. >> >> >> > >> >> >> >> > >> >> >> >> > >> >> >> >> > >> Regards, >> >> >> > >> >> >> >> > >> YSL
SY
Sreenivasulu Yadavalli3 Mar 2020 00:54 #
Hi Christian, We checked with the latest snapshot and we couldn't get an improved results. We're looking for the possible index creations.and check the execution time on old snapshot . Regards, YSL
Quoted text · Christian Grün · 150 lines
On Mon, Mar 2, 2020 at 6:45 PM Christian Grün <christian.gruen at gmail.com> wrote: > Hi YSL, > > What about my last response: Did you check the query plans, could you > observe any improvements? > > Best, > Christian > > > > > On Mon, Mar 2, 2020 at 7:44 AM Sreenivasulu Yadavalli > <srinivasuluy75 at gmail.com> wrote: > > > > Hi Christian, > > > > We're having a requirement to move Our application to AWS cloud. > > > > Please suggest on how to use Base-X DB on AWS cloud environment ? > > > > Regards, > > YSL > > > > On Tue, Feb 11, 2020 at 12:31 AM Christian Grün < > christian.gruen at gmail.com> wrote: > >> > >> Hi YSL, > >> > >> You could try to rewrite the number test to a string comparison: > >> > >> OLD: > >> ....[AccNo[@NO = (12345)]] > >> > >> NEW: > >> ....[AccNo/@NO = '12345'] > >> > >> If you run your query, are some expressions rewritten for index > >> access? Just have a closer look at the Info View output and look for > >> "applying... index" strings. > >> > >> Best, > >> Christian > >> > >> > >> > >> On Mon, Feb 10, 2020 at 11:49 AM Sreenivasulu Yadavalli > >> <srinivasuluy75 at gmail.com> wrote: > >> > > >> > Dear Christian, > >> > Please find the attachment for query > >> > > >> > > >> > > >> > Regards, > >> > YSL > >> > > >> > > >> > On Mon, Feb 10, 2020 at 3:59 PM Christian Grün < > christian.gruen at gmail.com> wrote: > >> >> > >> >> Hi Sreenivasulu, > >> >> > >> >> > Please suggest any clue to tune the query. > >> >> > >> >> Which query? > >> >> > >> >> Best, > >> >> Christian > >> >> > >> >> > >> >> > >> >> > > >> >> > Regards, > >> >> > YSL > >> >> > > >> >> > > >> >> > > >> >> > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün < > christian.gruen at gmail.com> wrote: > >> >> >> > >> >> >> Dear Sreenivasulu (cc to the list), > >> >> >> > >> >> >> Thanks for the Java code; I could reproduce the behavior. > >> >> >> > >> >> >> A new stable snapshot will be available in a few minutes [1]; > could > >> >> >> you check for us if it resolves the problem? > >> >> >> > >> >> >> Thanks, > >> >> >> Christian > >> >> >> > >> >> >> [1] http://files.basex.org/releases/latest/ > >> >> >> > >> >> >> > >> >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli > >> >> >> > <srinivasuluy75 at gmail.com> wrote: > >> >> >> > > > >> >> >> > > Hi Christian, > >> >> >> > > > >> >> >> > > Currently, as part of our business use case we are receiving > some XML file, which contains special characters like semicolon ": " in the > file names. > >> >> >> > > > >> >> >> > > We're able to get process using the memory file stream, > >> >> >> > > File ofile= new File("<<FILEPATH"); > >> >> >> > > FileInputStream xmlFile = new FileInputStream(ofile); > >> >> >> > > clientSession.add(<<FILE_NAME>>, xmlFile); > >> >> >> > > > >> >> >> > > We're not able to process these files using the command ADD > >> >> >> > > clientSession.execute("ADD <<FILE_PATH>>"); > >> >> >> > > > >> >> >> > > Kindly suggest us the way forward \ alternative approach to > the resolve the concern at earliest. > >> >> >> > > > >> >> >> > > > >> >> >> > > Regards, > >> >> >> > > YSL > >> >> >> > > > >> >> >> > > > >> >> >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < > srinivasuluy75 at gmail.com> wrote: > >> >> >> > >> > >> >> >> > >> Hi Team, > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> We have a big problem for creating the collection bcoz of > large no of xmls. > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> Every day we are moving collections around 55k to 60k no of > xml files large account. Its taking more than 18 hours. At that time we > want access the collection for generating report its on lock mode and scrip > that collection. > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> Please help and do needful. > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > >> >> >> > >> Regards, > >> >> >> > >> > >> >> >> > >> YSL >
CG
Christian Grün3 Mar 2020 07:48 #
Feel free to send us the output of the info view, it might contain some other valuable information. Sreenivasulu Yadavalli <srinivasuluy75 at gmail.com> schrieb am Di., 3. März 2020, 00:55:
Quoted text · Sreenivasulu Yadavalli · 165 lines
> Hi Christian, > > We checked with the latest snapshot and we couldn't get an improved > results. > > We're looking for the possible index creations.and check the execution > time on old snapshot . > > Regards, > YSL > > > > On Mon, Mar 2, 2020 at 6:45 PM Christian Grün <christian.gruen at gmail.com> > wrote: > >> Hi YSL, >> >> What about my last response: Did you check the query plans, could you >> observe any improvements? >> >> Best, >> Christian >> >> >> >> >> On Mon, Mar 2, 2020 at 7:44 AM Sreenivasulu Yadavalli >> <srinivasuluy75 at gmail.com> wrote: >> > >> > Hi Christian, >> > >> > We're having a requirement to move Our application to AWS cloud. >> > >> > Please suggest on how to use Base-X DB on AWS cloud environment ? >> > >> > Regards, >> > YSL >> > >> > On Tue, Feb 11, 2020 at 12:31 AM Christian Grün < >> christian.gruen at gmail.com> wrote: >> >> >> >> Hi YSL, >> >> >> >> You could try to rewrite the number test to a string comparison: >> >> >> >> OLD: >> >> ....[AccNo[@NO = (12345)]] >> >> >> >> NEW: >> >> ....[AccNo/@NO = '12345'] >> >> >> >> If you run your query, are some expressions rewritten for index >> >> access? Just have a closer look at the Info View output and look for >> >> "applying... index" strings. >> >> >> >> Best, >> >> Christian >> >> >> >> >> >> >> >> On Mon, Feb 10, 2020 at 11:49 AM Sreenivasulu Yadavalli >> >> <srinivasuluy75 at gmail.com> wrote: >> >> > >> >> > Dear Christian, >> >> > Please find the attachment for query >> >> > >> >> > >> >> > >> >> > Regards, >> >> > YSL >> >> > >> >> > >> >> > On Mon, Feb 10, 2020 at 3:59 PM Christian Grün < >> christian.gruen at gmail.com> wrote: >> >> >> >> >> >> Hi Sreenivasulu, >> >> >> >> >> >> > Please suggest any clue to tune the query. >> >> >> >> >> >> Which query? >> >> >> >> >> >> Best, >> >> >> Christian >> >> >> >> >> >> >> >> >> >> >> >> > >> >> >> > Regards, >> >> >> > YSL >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Mon, Dec 30, 2019 at 1:07 AM Christian Grün < >> christian.gruen at gmail.com> wrote: >> >> >> >> >> >> >> >> Dear Sreenivasulu (cc to the list), >> >> >> >> >> >> >> >> Thanks for the Java code; I could reproduce the behavior. >> >> >> >> >> >> >> >> A new stable snapshot will be available in a few minutes [1]; >> could >> >> >> >> you check for us if it resolves the problem? >> >> >> >> >> >> >> >> Thanks, >> >> >> >> Christian >> >> >> >> >> >> >> >> [1] http://files.basex.org/releases/latest/ >> >> >> >> >> >> >> >> >> >> >> >> > On Fri, Dec 20, 2019 at 10:59 AM Sreenivasulu Yadavalli >> >> >> >> > <srinivasuluy75 at gmail.com> wrote: >> >> >> >> > > >> >> >> >> > > Hi Christian, >> >> >> >> > > >> >> >> >> > > Currently, as part of our business use case we are receiving >> some XML file, which contains special characters like semicolon ": " in the >> file names. >> >> >> >> > > >> >> >> >> > > We're able to get process using the memory file stream, >> >> >> >> > > File ofile= new File("<<FILEPATH"); >> >> >> >> > > FileInputStream xmlFile = new FileInputStream(ofile); >> >> >> >> > > clientSession.add(<<FILE_NAME>>, xmlFile); >> >> >> >> > > >> >> >> >> > > We're not able to process these files using the command ADD >> >> >> >> > > clientSession.execute("ADD <<FILE_PATH>>"); >> >> >> >> > > >> >> >> >> > > Kindly suggest us the way forward \ alternative approach to >> the resolve the concern at earliest. >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > Regards, >> >> >> >> > > YSL >> >> >> >> > > >> >> >> >> > > >> >> >> >> > > On Tue, Dec 17, 2019 at 11:48 AM Sreenivasulu Yadavalli < >> srinivasuluy75 at gmail.com> wrote: >> >> >> >> > >> >> >> >> >> > >> Hi Team, >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> We have a big problem for creating the collection bcoz of >> large no of xmls. >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> Every day we are moving collections around 55k to 60k no of >> xml files large account. Its taking more than 18 hours. At that time we >> want access the collection for generating report its on lock mode and scrip >> that collection. >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> Please help and do needful. >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> Regards, >> >> >> >> > >> >> >> >> >> > >> YSL >> >

Written for BaseX 9.3. BaseX may have changed since: please check the current documentation.

« Text indexHow many QueryModule instances can be created? »
⚡Generated with XQuery