Multiple db:replace() = slower updates with 7.8

Wednesday, 19 March 2014 · 8 messages · last reply 4 Apr 2014
FB
France Baril19 Mar 2014 20:00 #
Hi, We run batch processes on our files in order to send them out to translation in xliff format (xml format for translations). I run a piece of code that: 1. applies transformations 2. file:writes the resulting files (for debug purposes) 3. and then db:replaces the resulting files. If I send 5 files into the process, it takes a few seconds before the files appear on the file system. Yet full process with the db:replace takes 2 minutes to complete. When I send 60 files... the process never ends. All files are available on the file system and are valid, which means I am fairly certain that the freeze doesn't occur because of an error in one of the files. There is no error message in the terminal window. Is possible that something was changed after 7.7 that would affect performance? On our production server, which is on 7.7 we can run 100+ files in just a few minutes. Maybe there are new server settings that we missed? For reference, here is the code. I understand that updates are all performed at the end of the process, which is why we get the debug files first, but the lag time between the time where we get all files on the file system and the db:replace seems huge.
*declare* *updating* *function* *translation:transform-to-xliff*(*$menu-id* *as
**xs:string*, *$lang* *as **xs:string**, *$package-number* *as **xs:string*
){

    *let **$menu* := *menu:xml-get-menu*(*$menu-id*, 'en-us')

    (: Need to translate menu and its topics without getting duplicated
topics twice:)
    (: Always send the reusable strings from writers and from dev...
doesn't cost anything if no new string anyway :)
    *let* *$file-ids* := (*$menu-id*, *for **$id* *in **distinct-values*(
*$menu*/descendant::*[*name*()='topicref']/*data*(*@id-ref*))[1] *return *
*$id*,
                      'stringreferences', 'stringreferencestransform')

    *return  *(*for **$file-id* *in **distinct-values*(*$file-ids*)
             *let* *$file* := *db:open*('en-us')/*[*@id*=*$file-id*]
             *let* *$skl-href* := 'skeletons/' || *$package-number* || '/'||
*substring-after*(*$file*/*base-uri*(), 'en-us/')
             *let* *$file-with-preserve* :=
                     *copy* *$copy* := *$file*
                     *modify*(*if* (*exists*(*$copy*/*@xml:space*))
                            *then *()
                            *else **insert* *node* *attribute* {'xml:space'
}{'preserve'} *into* *$copy*)
                     *return **$copy*
             *let* *$all-trans-files* :=
*xliff-transformer:create-translation-files*(*$file-with-preserve*,
*$package-number*, *$menu-id*, *$lang*, *$skl-href*)

             *let* *$skeleton* := *$all-trans-files*[1]

             *let* *$xliffs* := *for **$xliff* *in **$all-trans-files*[
*position*() != 1]
                            *return **$xliff*

             *let* *$debug* := *file:write*('Debug-xliff/skls/' ||
*tokenize*(*$skl-href*, '/')[*last*()], *$skeleton*)

*return *(*db:replace*('Translation-management', *$skl-href*, *$skeleton*),
                     *for **$xliff* *in **$xliffs*
                      *let* *$target-lang* := *$xliff*/descendant::*[*name*
()='file']/*data*(*@target-language*)
                      *let* *$lang-pack-name* := *concat*(*$target-lang*,'_',
*$package-number*)
                      *let* *$file-loc-end* := *$menu-id* || '/' ||
*$lang-pack-name* ||'/' || *substring-after*(*$file*/*base-uri*(), 'en-us/')
                      (:let $debug := file:append('debug-ids.xml', '
'
|| $file-loc-end):)
                      *let* *$file-loc* := *if* (*exists*(*$xliff*/
descendant::*[*@translate*='yes']))
                                       *then *'xliff-to-translate/' ||
*$file-loc-end*
                                       *else *'xliff-no-new-segment/' ||
*$file-loc-end*
                      *let* *$debug* := *file:write*('Debug-xliff/' ||
*tokenize*(*$file-loc*, '/')[*last*()], *$xliff*)
                      *return **db:replace*('Translation-management',
*$file-loc*, *$xliff*)
                     )
             )
};
-- France Baril Architecte documentaire / Documentation architect france.baril at architextus.com (514) 572-0341
CG
Christian Grün25 Mar 2014 14:41 #
Hi France, sorry, I know that an answer to your e-mail is still pending. If you should manage to build a self-contained example, it could help us to find the culprit. Best, Christian
Quoted text · France Baril · 105 lines
On Wed, Mar 19, 2014 at 6:59 PM, France Baril <france.baril at architextus.com> wrote: > Hi, > > We run batch processes on our files in order to send them out to translation > in xliff format (xml format for translations). > > I run a piece of code that: > 1. applies transformations > 2. file:writes the resulting files (for debug purposes) > 3. and then db:replaces the resulting files. > > If I send 5 files into the process, it takes a few seconds before the files > appear on the file system. Yet full process with the db:replace takes 2 > minutes to complete. > > When I send 60 files... the process never ends. All files are available on > the file system and are valid, which means I am fairly certain that the > freeze doesn't occur because of an error in one of the files. > > There is no error message in the terminal window. Is possible that something > was changed after 7.7 that would affect performance? On our production > server, which is on 7.7 we can run 100+ files in just a few minutes. Maybe > there are new server settings that we missed? > > For reference, here is the code. I understand that updates are all performed > at the end of the process, which is why we get the debug files first, but > the lag time between the time where we get all files on the file system and > the db:replace seems huge. > > declare updating function translation:transform-to-xliff($menu-id as > xs:string, $lang as xs:string*, $package-number as xs:string){ > > let $menu := menu:xml-get-menu($menu-id, 'en-us') > > (: Need to translate menu and its topics without getting duplicated > topics twice:) > (: Always send the reusable strings from writers and from dev... doesn't > cost anything if no new string anyway :) > let $file-ids := ($menu-id, for $id in > distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] > return $id, > 'stringreferences', 'stringreferencestransform') > > return (for $file-id in distinct-values($file-ids) > let $file := db:open('en-us')/*[@id=$file-id] > let $skl-href := 'skeletons/' || $package-number || '/' || > substring-after($file/base-uri(), 'en-us/') > let $file-with-preserve := > copy $copy := $file > modify(if (exists($copy/@xml:space)) > then () > else insert node attribute > {'xml:space'}{'preserve'} into $copy) > return $copy > let $all-trans-files := > xliff-transformer:create-translation-files($file-with-preserve, > $package-number, $menu-id, $lang, $skl-href) > > let $skeleton := $all-trans-files[1] > > let $xliffs := for $xliff in $all-trans-files[position() != 1] > return $xliff > > let $debug := file:write('Debug-xliff/skls/' || > tokenize($skl-href, '/')[last()], $skeleton) > > > return (db:replace('Translation-management', $skl-href, $skeleton), > for $xliff in $xliffs > let $target-lang := > $xliff/descendant::*[name()='file']/data(@target-language) > let $lang-pack-name := concat($target-lang,'_', > $package-number) > let $file-loc-end := $menu-id || '/' || > $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') > (:let $debug := file:append('debug-ids.xml', '&#10;' > || $file-loc-end):) > let $file-loc := if > (exists($xliff/descendant::*[@translate='yes'])) > then 'xliff-to-translate/' || > $file-loc-end > else 'xliff-no-new-segment/' || > $file-loc-end > let $debug := file:write('Debug-xliff/' || > tokenize($file-loc, '/')[last()], $xliff) > return db:replace('Translation-management', $file-loc, > $xliff) > ) > ) > }; > > > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com > (514) 572-0341 > > _______________________________________________ > BaseX-Talk mailing list > BaseX-Talk at mailman.uni-konstanz.de > https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >
CG
Christian Grün25 Mar 2014 19:29 #
We may have found a first evidence for the slow down. Do your documents use namespaces?
Quoted text · Christian Grün · 117 lines
On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi France, > > sorry, I know that an answer to your e-mail is still pending. If you > should manage to build a self-contained example, it could help us to > find the culprit. > > Best, > Christian > > > On Wed, Mar 19, 2014 at 6:59 PM, France Baril > <france.baril at architextus.com> wrote: >> Hi, >> >> We run batch processes on our files in order to send them out to translation >> in xliff format (xml format for translations). >> >> I run a piece of code that: >> 1. applies transformations >> 2. file:writes the resulting files (for debug purposes) >> 3. and then db:replaces the resulting files. >> >> If I send 5 files into the process, it takes a few seconds before the files >> appear on the file system. Yet full process with the db:replace takes 2 >> minutes to complete. >> >> When I send 60 files... the process never ends. All files are available on >> the file system and are valid, which means I am fairly certain that the >> freeze doesn't occur because of an error in one of the files. >> >> There is no error message in the terminal window. Is possible that something >> was changed after 7.7 that would affect performance? On our production >> server, which is on 7.7 we can run 100+ files in just a few minutes. Maybe >> there are new server settings that we missed? >> >> For reference, here is the code. I understand that updates are all performed >> at the end of the process, which is why we get the debug files first, but >> the lag time between the time where we get all files on the file system and >> the db:replace seems huge. >> >> declare updating function translation:transform-to-xliff($menu-id as >> xs:string, $lang as xs:string*, $package-number as xs:string){ >> >> let $menu := menu:xml-get-menu($menu-id, 'en-us') >> >> (: Need to translate menu and its topics without getting duplicated >> topics twice:) >> (: Always send the reusable strings from writers and from dev... doesn't >> cost anything if no new string anyway :) >> let $file-ids := ($menu-id, for $id in >> distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] >> return $id, >> 'stringreferences', 'stringreferencestransform') >> >> return (for $file-id in distinct-values($file-ids) >> let $file := db:open('en-us')/*[@id=$file-id] >> let $skl-href := 'skeletons/' || $package-number || '/' || >> substring-after($file/base-uri(), 'en-us/') >> let $file-with-preserve := >> copy $copy := $file >> modify(if (exists($copy/@xml:space)) >> then () >> else insert node attribute >> {'xml:space'}{'preserve'} into $copy) >> return $copy >> let $all-trans-files := >> xliff-transformer:create-translation-files($file-with-preserve, >> $package-number, $menu-id, $lang, $skl-href) >> >> let $skeleton := $all-trans-files[1] >> >> let $xliffs := for $xliff in $all-trans-files[position() != 1] >> return $xliff >> >> let $debug := file:write('Debug-xliff/skls/' || >> tokenize($skl-href, '/')[last()], $skeleton) >> >> >> return (db:replace('Translation-management', $skl-href, $skeleton), >> for $xliff in $xliffs >> let $target-lang := >> $xliff/descendant::*[name()='file']/data(@target-language) >> let $lang-pack-name := concat($target-lang,'_', >> $package-number) >> let $file-loc-end := $menu-id || '/' || >> $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') >> (:let $debug := file:append('debug-ids.xml', '&#10;' >> || $file-loc-end):) >> let $file-loc := if >> (exists($xliff/descendant::*[@translate='yes'])) >> then 'xliff-to-translate/' || >> $file-loc-end >> else 'xliff-no-new-segment/' || >> $file-loc-end >> let $debug := file:write('Debug-xliff/' || >> tokenize($file-loc, '/')[last()], $xliff) >> return db:replace('Translation-management', $file-loc, >> $xliff) >> ) >> ) >> }; >> >> >> >> -- >> France Baril >> Architecte documentaire / Documentation architect >> france.baril at architextus.com >> (514) 572-0341 >> >> _______________________________________________ >> BaseX-Talk mailing list >> BaseX-Talk at mailman.uni-konstanz.de >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>
FB
France Baril25 Mar 2014 20:35 #
Yes, we do use namespaces. On Tue, Mar 25, 2014 at 2:29 PM, Christian Grün <christian.gruen at gmail.com>wrote:
Quoted text · Christian Grün · 134 lines
> We may have found a first evidence for the slow down. Do your > documents use namespaces? > > > On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün > <christian.gruen at gmail.com> wrote: > > Hi France, > > > > sorry, I know that an answer to your e-mail is still pending. If you > > should manage to build a self-contained example, it could help us to > > find the culprit. > > > > Best, > > Christian > > > > > > On Wed, Mar 19, 2014 at 6:59 PM, France Baril > > <france.baril at architextus.com> wrote: > >> Hi, > >> > >> We run batch processes on our files in order to send them out to > translation > >> in xliff format (xml format for translations). > >> > >> I run a piece of code that: > >> 1. applies transformations > >> 2. file:writes the resulting files (for debug purposes) > >> 3. and then db:replaces the resulting files. > >> > >> If I send 5 files into the process, it takes a few seconds before the > files > >> appear on the file system. Yet full process with the db:replace takes 2 > >> minutes to complete. > >> > >> When I send 60 files... the process never ends. All files are available > on > >> the file system and are valid, which means I am fairly certain that the > >> freeze doesn't occur because of an error in one of the files. > >> > >> There is no error message in the terminal window. Is possible that > something > >> was changed after 7.7 that would affect performance? On our production > >> server, which is on 7.7 we can run 100+ files in just a few minutes. > Maybe > >> there are new server settings that we missed? > >> > >> For reference, here is the code. I understand that updates are all > performed > >> at the end of the process, which is why we get the debug files first, > but > >> the lag time between the time where we get all files on the file system > and > >> the db:replace seems huge. > >> > >> declare updating function translation:transform-to-xliff($menu-id as > >> xs:string, $lang as xs:string*, $package-number as xs:string){ > >> > >> let $menu := menu:xml-get-menu($menu-id, 'en-us') > >> > >> (: Need to translate menu and its topics without getting duplicated > >> topics twice:) > >> (: Always send the reusable strings from writers and from dev... > doesn't > >> cost anything if no new string anyway :) > >> let $file-ids := ($menu-id, for $id in > >> distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] > >> return $id, > >> 'stringreferences', 'stringreferencestransform') > >> > >> return (for $file-id in distinct-values($file-ids) > >> let $file := db:open('en-us')/*[@id=$file-id] > >> let $skl-href := 'skeletons/' || $package-number || '/' || > >> substring-after($file/base-uri(), 'en-us/') > >> let $file-with-preserve := > >> copy $copy := $file > >> modify(if (exists($copy/@xml:space)) > >> then () > >> else insert node attribute > >> {'xml:space'}{'preserve'} into $copy) > >> return $copy > >> let $all-trans-files := > >> xliff-transformer:create-translation-files($file-with-preserve, > >> $package-number, $menu-id, $lang, $skl-href) > >> > >> let $skeleton := $all-trans-files[1] > >> > >> let $xliffs := for $xliff in $all-trans-files[position() > != 1] > >> return $xliff > >> > >> let $debug := file:write('Debug-xliff/skls/' || > >> tokenize($skl-href, '/')[last()], $skeleton) > >> > >> > >> return (db:replace('Translation-management', $skl-href, $skeleton), > >> for $xliff in $xliffs > >> let $target-lang := > >> $xliff/descendant::*[name()='file']/data(@target-language) > >> let $lang-pack-name := concat($target-lang,'_', > >> $package-number) > >> let $file-loc-end := $menu-id || '/' || > >> $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') > >> (:let $debug := file:append('debug-ids.xml', > '&#10;' > >> || $file-loc-end):) > >> let $file-loc := if > >> (exists($xliff/descendant::*[@translate='yes'])) > >> then 'xliff-to-translate/' || > >> $file-loc-end > >> else 'xliff-no-new-segment/' || > >> $file-loc-end > >> let $debug := file:write('Debug-xliff/' || > >> tokenize($file-loc, '/')[last()], $xliff) > >> return db:replace('Translation-management', > $file-loc, > >> $xliff) > >> ) > >> ) > >> }; > >> > >> > >> > >> -- > >> France Baril > >> Architecte documentaire / Documentation architect > >> france.baril at architextus.com > >> (514) 572-0341 > >> > >> _______________________________________________ > >> BaseX-Talk mailing list > >> BaseX-Talk at mailman.uni-konstanz.de > >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >> >
-- France Baril Architecte documentaire / Documentation architect france.baril at architextus.com (514) 572-0341
CG
Christian Grünin reply to France Baril, 1 Apr 2014, 19:261 Apr 2014 17:43 #
Hi France, sorry for lettting you wait. We've already solved part of the problem, but we would first like to shed some more light on another related issue before relasing a new patch version. Christian
Quoted text · France Baril · 170 lines
On Tue, Apr 1, 2014 at 5:26 PM, France Baril <france.baril at architextus.com> wrote: > Any news on this issue? > > > On Tue, Mar 25, 2014 at 2:30 PM, France Baril <france.baril at architextus.com> > wrote: >> >> Yes, we do use namespaces. >> >> >> On Tue, Mar 25, 2014 at 2:29 PM, Christian Grün >> <christian.gruen at gmail.com> wrote: >>> >>> We may have found a first evidence for the slow down. Do your >>> documents use namespaces? >>> >>> >>> On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün >>> <christian.gruen at gmail.com> wrote: >>> > Hi France, >>> > >>> > sorry, I know that an answer to your e-mail is still pending. If you >>> > should manage to build a self-contained example, it could help us to >>> > find the culprit. >>> > >>> > Best, >>> > Christian >>> > >>> > >>> > On Wed, Mar 19, 2014 at 6:59 PM, France Baril >>> > <france.baril at architextus.com> wrote: >>> >> Hi, >>> >> >>> >> We run batch processes on our files in order to send them out to >>> >> translation >>> >> in xliff format (xml format for translations). >>> >> >>> >> I run a piece of code that: >>> >> 1. applies transformations >>> >> 2. file:writes the resulting files (for debug purposes) >>> >> 3. and then db:replaces the resulting files. >>> >> >>> >> If I send 5 files into the process, it takes a few seconds before the >>> >> files >>> >> appear on the file system. Yet full process with the db:replace takes >>> >> 2 >>> >> minutes to complete. >>> >> >>> >> When I send 60 files... the process never ends. All files are >>> >> available on >>> >> the file system and are valid, which means I am fairly certain that >>> >> the >>> >> freeze doesn't occur because of an error in one of the files. >>> >> >>> >> There is no error message in the terminal window. Is possible that >>> >> something >>> >> was changed after 7.7 that would affect performance? On our production >>> >> server, which is on 7.7 we can run 100+ files in just a few minutes. >>> >> Maybe >>> >> there are new server settings that we missed? >>> >> >>> >> For reference, here is the code. I understand that updates are all >>> >> performed >>> >> at the end of the process, which is why we get the debug files first, >>> >> but >>> >> the lag time between the time where we get all files on the file >>> >> system and >>> >> the db:replace seems huge. >>> >> >>> >> declare updating function translation:transform-to-xliff($menu-id as >>> >> xs:string, $lang as xs:string*, $package-number as xs:string){ >>> >> >>> >> let $menu := menu:xml-get-menu($menu-id, 'en-us') >>> >> >>> >> (: Need to translate menu and its topics without getting >>> >> duplicated >>> >> topics twice:) >>> >> (: Always send the reusable strings from writers and from dev... >>> >> doesn't >>> >> cost anything if no new string anyway :) >>> >> let $file-ids := ($menu-id, for $id in >>> >> >>> >> distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] >>> >> return $id, >>> >> 'stringreferences', 'stringreferencestransform') >>> >> >>> >> return (for $file-id in distinct-values($file-ids) >>> >> let $file := db:open('en-us')/*[@id=$file-id] >>> >> let $skl-href := 'skeletons/' || $package-number || '/' >>> >> || >>> >> substring-after($file/base-uri(), 'en-us/') >>> >> let $file-with-preserve := >>> >> copy $copy := $file >>> >> modify(if (exists($copy/@xml:space)) >>> >> then () >>> >> else insert node attribute >>> >> {'xml:space'}{'preserve'} into $copy) >>> >> return $copy >>> >> let $all-trans-files := >>> >> xliff-transformer:create-translation-files($file-with-preserve, >>> >> $package-number, $menu-id, $lang, $skl-href) >>> >> >>> >> let $skeleton := $all-trans-files[1] >>> >> >>> >> let $xliffs := for $xliff in $all-trans-files[position() >>> >> != 1] >>> >> return $xliff >>> >> >>> >> let $debug := file:write('Debug-xliff/skls/' || >>> >> tokenize($skl-href, '/')[last()], $skeleton) >>> >> >>> >> >>> >> return (db:replace('Translation-management', $skl-href, $skeleton), >>> >> for $xliff in $xliffs >>> >> let $target-lang := >>> >> $xliff/descendant::*[name()='file']/data(@target-language) >>> >> let $lang-pack-name := concat($target-lang,'_', >>> >> $package-number) >>> >> let $file-loc-end := $menu-id || '/' || >>> >> $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') >>> >> (:let $debug := file:append('debug-ids.xml', >>> >> '&#10;' >>> >> || $file-loc-end):) >>> >> let $file-loc := if >>> >> (exists($xliff/descendant::*[@translate='yes'])) >>> >> then 'xliff-to-translate/' || >>> >> $file-loc-end >>> >> else 'xliff-no-new-segment/' || >>> >> $file-loc-end >>> >> let $debug := file:write('Debug-xliff/' || >>> >> tokenize($file-loc, '/')[last()], $xliff) >>> >> return db:replace('Translation-management', >>> >> $file-loc, >>> >> $xliff) >>> >> ) >>> >> ) >>> >> }; >>> >> >>> >> >>> >> >>> >> -- >>> >> France Baril >>> >> Architecte documentaire / Documentation architect >>> >> france.baril at architextus.com >>> >> (514) 572-0341 >>> >> >>> >> _______________________________________________ >>> >> BaseX-Talk mailing list >>> >> BaseX-Talk at mailman.uni-konstanz.de >>> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >>> >> >> >> >> >> >> -- >> France Baril >> Architecte documentaire / Documentation architect >> france.baril at architextus.com >> (514) 572-0341 > > > > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com > (514) 572-0341
FB
France Barilin reply to France Baril, 25 Mar 2014, 20:351 Apr 2014 19:26 #
Any news on this issue? On Tue, Mar 25, 2014 at 2:30 PM, France Baril <france.baril at architextus.com>wrote:
Quoted text · France Baril · 150 lines
> Yes, we do use namespaces. > > > On Tue, Mar 25, 2014 at 2:29 PM, Christian Grün <christian.gruen at gmail.com > > wrote: > >> We may have found a first evidence for the slow down. Do your >> documents use namespaces? >> >> >> On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün >> <christian.gruen at gmail.com> wrote: >> > Hi France, >> > >> > sorry, I know that an answer to your e-mail is still pending. If you >> > should manage to build a self-contained example, it could help us to >> > find the culprit. >> > >> > Best, >> > Christian >> > >> > >> > On Wed, Mar 19, 2014 at 6:59 PM, France Baril >> > <france.baril at architextus.com> wrote: >> >> Hi, >> >> >> >> We run batch processes on our files in order to send them out to >> translation >> >> in xliff format (xml format for translations). >> >> >> >> I run a piece of code that: >> >> 1. applies transformations >> >> 2. file:writes the resulting files (for debug purposes) >> >> 3. and then db:replaces the resulting files. >> >> >> >> If I send 5 files into the process, it takes a few seconds before the >> files >> >> appear on the file system. Yet full process with the db:replace takes 2 >> >> minutes to complete. >> >> >> >> When I send 60 files... the process never ends. All files are >> available on >> >> the file system and are valid, which means I am fairly certain that the >> >> freeze doesn't occur because of an error in one of the files. >> >> >> >> There is no error message in the terminal window. Is possible that >> something >> >> was changed after 7.7 that would affect performance? On our production >> >> server, which is on 7.7 we can run 100+ files in just a few minutes. >> Maybe >> >> there are new server settings that we missed? >> >> >> >> For reference, here is the code. I understand that updates are all >> performed >> >> at the end of the process, which is why we get the debug files first, >> but >> >> the lag time between the time where we get all files on the file >> system and >> >> the db:replace seems huge. >> >> >> >> declare updating function translation:transform-to-xliff($menu-id as >> >> xs:string, $lang as xs:string*, $package-number as xs:string){ >> >> >> >> let $menu := menu:xml-get-menu($menu-id, 'en-us') >> >> >> >> (: Need to translate menu and its topics without getting duplicated >> >> topics twice:) >> >> (: Always send the reusable strings from writers and from dev... >> doesn't >> >> cost anything if no new string anyway :) >> >> let $file-ids := ($menu-id, for $id in >> >> >> distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] >> >> return $id, >> >> 'stringreferences', 'stringreferencestransform') >> >> >> >> return (for $file-id in distinct-values($file-ids) >> >> let $file := db:open('en-us')/*[@id=$file-id] >> >> let $skl-href := 'skeletons/' || $package-number || '/' || >> >> substring-after($file/base-uri(), 'en-us/') >> >> let $file-with-preserve := >> >> copy $copy := $file >> >> modify(if (exists($copy/@xml:space)) >> >> then () >> >> else insert node attribute >> >> {'xml:space'}{'preserve'} into $copy) >> >> return $copy >> >> let $all-trans-files := >> >> xliff-transformer:create-translation-files($file-with-preserve, >> >> $package-number, $menu-id, $lang, $skl-href) >> >> >> >> let $skeleton := $all-trans-files[1] >> >> >> >> let $xliffs := for $xliff in $all-trans-files[position() >> != 1] >> >> return $xliff >> >> >> >> let $debug := file:write('Debug-xliff/skls/' || >> >> tokenize($skl-href, '/')[last()], $skeleton) >> >> >> >> >> >> return (db:replace('Translation-management', $skl-href, $skeleton), >> >> for $xliff in $xliffs >> >> let $target-lang := >> >> $xliff/descendant::*[name()='file']/data(@target-language) >> >> let $lang-pack-name := concat($target-lang,'_', >> >> $package-number) >> >> let $file-loc-end := $menu-id || '/' || >> >> $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') >> >> (:let $debug := file:append('debug-ids.xml', >> '&#10;' >> >> || $file-loc-end):) >> >> let $file-loc := if >> >> (exists($xliff/descendant::*[@translate='yes'])) >> >> then 'xliff-to-translate/' || >> >> $file-loc-end >> >> else 'xliff-no-new-segment/' || >> >> $file-loc-end >> >> let $debug := file:write('Debug-xliff/' || >> >> tokenize($file-loc, '/')[last()], $xliff) >> >> return db:replace('Translation-management', >> $file-loc, >> >> $xliff) >> >> ) >> >> ) >> >> }; >> >> >> >> >> >> >> >> -- >> >> France Baril >> >> Architecte documentaire / Documentation architect >> >> france.baril at architextus.com >> >> (514) 572-0341 >> >> >> >> _______________________________________________ >> >> BaseX-Talk mailing list >> >> BaseX-Talk at mailman.uni-konstanz.de >> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> > > > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com > (514) 572-0341 >
-- France Baril Architecte documentaire / Documentation architect france.baril at architextus.com (514) 572-0341
FB
France Barilin reply to Christian Grün, 1 Apr 2014, 17:432 Apr 2014 20:26 #
Ok, thanks for the update. We are dependent on this to make the next release to our production server, so if you have an idea on timing, it would really help us adapt. France On Tue, Apr 1, 2014 at 11:43 AM, Christian Grün <christian.gruen at gmail.com>wrote:
Quoted text · Christian Grün · 190 lines
> Hi France, > > sorry for lettting you wait. We've already solved part of the problem, > but we would first like to shed some more light on another related > issue before relasing a new patch version. > > Christian > ____________________________________ > > On Tue, Apr 1, 2014 at 5:26 PM, France Baril > <france.baril at architextus.com> wrote: > > Any news on this issue? > > > > > > On Tue, Mar 25, 2014 at 2:30 PM, France Baril < > france.baril at architextus.com> > > wrote: > >> > >> Yes, we do use namespaces. > >> > >> > >> On Tue, Mar 25, 2014 at 2:29 PM, Christian Grün > >> <christian.gruen at gmail.com> wrote: > >>> > >>> We may have found a first evidence for the slow down. Do your > >>> documents use namespaces? > >>> > >>> > >>> On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün > >>> <christian.gruen at gmail.com> wrote: > >>> > Hi France, > >>> > > >>> > sorry, I know that an answer to your e-mail is still pending. If you > >>> > should manage to build a self-contained example, it could help us to > >>> > find the culprit. > >>> > > >>> > Best, > >>> > Christian > >>> > > >>> > > >>> > On Wed, Mar 19, 2014 at 6:59 PM, France Baril > >>> > <france.baril at architextus.com> wrote: > >>> >> Hi, > >>> >> > >>> >> We run batch processes on our files in order to send them out to > >>> >> translation > >>> >> in xliff format (xml format for translations). > >>> >> > >>> >> I run a piece of code that: > >>> >> 1. applies transformations > >>> >> 2. file:writes the resulting files (for debug purposes) > >>> >> 3. and then db:replaces the resulting files. > >>> >> > >>> >> If I send 5 files into the process, it takes a few seconds before > the > >>> >> files > >>> >> appear on the file system. Yet full process with the db:replace > takes > >>> >> 2 > >>> >> minutes to complete. > >>> >> > >>> >> When I send 60 files... the process never ends. All files are > >>> >> available on > >>> >> the file system and are valid, which means I am fairly certain that > >>> >> the > >>> >> freeze doesn't occur because of an error in one of the files. > >>> >> > >>> >> There is no error message in the terminal window. Is possible that > >>> >> something > >>> >> was changed after 7.7 that would affect performance? On our > production > >>> >> server, which is on 7.7 we can run 100+ files in just a few minutes. > >>> >> Maybe > >>> >> there are new server settings that we missed? > >>> >> > >>> >> For reference, here is the code. I understand that updates are all > >>> >> performed > >>> >> at the end of the process, which is why we get the debug files > first, > >>> >> but > >>> >> the lag time between the time where we get all files on the file > >>> >> system and > >>> >> the db:replace seems huge. > >>> >> > >>> >> declare updating function translation:transform-to-xliff($menu-id as > >>> >> xs:string, $lang as xs:string*, $package-number as xs:string){ > >>> >> > >>> >> let $menu := menu:xml-get-menu($menu-id, 'en-us') > >>> >> > >>> >> (: Need to translate menu and its topics without getting > >>> >> duplicated > >>> >> topics twice:) > >>> >> (: Always send the reusable strings from writers and from dev... > >>> >> doesn't > >>> >> cost anything if no new string anyway :) > >>> >> let $file-ids := ($menu-id, for $id in > >>> >> > >>> >> > distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] > >>> >> return $id, > >>> >> 'stringreferences', > 'stringreferencestransform') > >>> >> > >>> >> return (for $file-id in distinct-values($file-ids) > >>> >> let $file := db:open('en-us')/*[@id=$file-id] > >>> >> let $skl-href := 'skeletons/' || $package-number || '/' > >>> >> || > >>> >> substring-after($file/base-uri(), 'en-us/') > >>> >> let $file-with-preserve := > >>> >> copy $copy := $file > >>> >> modify(if (exists($copy/@xml:space)) > >>> >> then () > >>> >> else insert node attribute > >>> >> {'xml:space'}{'preserve'} into $copy) > >>> >> return $copy > >>> >> let $all-trans-files := > >>> >> xliff-transformer:create-translation-files($file-with-preserve, > >>> >> $package-number, $menu-id, $lang, $skl-href) > >>> >> > >>> >> let $skeleton := $all-trans-files[1] > >>> >> > >>> >> let $xliffs := for $xliff in > $all-trans-files[position() > >>> >> != 1] > >>> >> return $xliff > >>> >> > >>> >> let $debug := file:write('Debug-xliff/skls/' || > >>> >> tokenize($skl-href, '/')[last()], $skeleton) > >>> >> > >>> >> > >>> >> return (db:replace('Translation-management', $skl-href, $skeleton), > >>> >> for $xliff in $xliffs > >>> >> let $target-lang := > >>> >> $xliff/descendant::*[name()='file']/data(@target-language) > >>> >> let $lang-pack-name := > concat($target-lang,'_', > >>> >> $package-number) > >>> >> let $file-loc-end := $menu-id || '/' || > >>> >> $lang-pack-name ||'/' || substring-after($file/base-uri(), 'en-us/') > >>> >> (:let $debug := file:append('debug-ids.xml', > >>> >> '&#10;' > >>> >> || $file-loc-end):) > >>> >> let $file-loc := if > >>> >> (exists($xliff/descendant::*[@translate='yes'])) > >>> >> then 'xliff-to-translate/' || > >>> >> $file-loc-end > >>> >> else 'xliff-no-new-segment/' > || > >>> >> $file-loc-end > >>> >> let $debug := file:write('Debug-xliff/' || > >>> >> tokenize($file-loc, '/')[last()], $xliff) > >>> >> return db:replace('Translation-management', > >>> >> $file-loc, > >>> >> $xliff) > >>> >> ) > >>> >> ) > >>> >> }; > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> France Baril > >>> >> Architecte documentaire / Documentation architect > >>> >> france.baril at architextus.com > >>> >> (514) 572-0341 > >>> >> > >>> >> _______________________________________________ > >>> >> BaseX-Talk mailing list > >>> >> BaseX-Talk at mailman.uni-konstanz.de > >>> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk > >>> >> > >> > >> > >> > >> > >> -- > >> France Baril > >> Architecte documentaire / Documentation architect > >> france.baril at architextus.com > >> (514) 572-0341 > > > > > > > > > > -- > > France Baril > > Architecte documentaire / Documentation architect > > france.baril at architextus.com > > (514) 572-0341 >
-- France Baril Architecte documentaire / Documentation architect france.baril at architextus.com (514) 572-0341
CG
Christian Grün4 Apr 2014 13:47 #
Hi France, our latest release includes the fix for the slow-downs you encountered. Best, Christian
Quoted text · France Baril · 467 lines
On Wed, Apr 2, 2014 at 10:29 PM, France Baril <france.baril at architextus.com> wrote: > Performance is definitely back. This is great. The big drawback is that Java > 7 will only work with 64 bits applications on Mac OS and Antenna House that > we use for PDF in xquery through the java API only run on 32 bits on Mac OS > X. > > Anyhow, the end-client runs a Windows shop, so we'll manage. > > Big thanks! > > > On Wed, Apr 2, 2014 at 2:56 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> > I can't think of any issue. I was looking in the documentation. Might be >> > interesting to add a note somewhere about Java 7 :-). >> > >> > Thanks! >> >> Yes, this will definitely be documented soon ;) >> >> >> > >> > >> > On Wed, Apr 2, 2014 at 2:47 PM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> > Does Basex 8 use Java 7? >> >> >> >> Exactly. As Java 6 is not supported by Oracle anymore, and as we >> >> didn't get much votes against this consideration one year ago [1], we >> >> felt that it's finally time to switch. Beside that, Java 7 gives us >> >> much better native OS support for handling files on disk. This is why >> >> the BaseX 7.8.x versions will be the last that will be based on Java >> >> 6. >> >> >> >> Do you think you'll encounter problems migrating to Java 7? >> >> >> >> Christian >> >> >> >> [1] >> >> >> >> https://mailman.uni-konstanz.de/pipermail/basex-talk/2013-February/004724.html >> >> >> >> >> >> >> >> > On Wed, Apr 2, 2014 at 2:31 PM, Christian Grün >> >> > <christian.gruen at gmail.com> >> >> > wrote: >> >> >> >> >> >> > basexhttp returns an error and logs out: >> >> >> >> >> >> Are you using Java 7? >> >> >> >> >> >> >> >> >> > >> >> >> > NoName:~ archie$ /Applications/basex/bin/basexhttp ; exit; >> >> >> > >> >> >> > Exception in thread "main" java.lang.UnsupportedClassVersionError: >> >> >> > org/basex/BaseXHTTP : Unsupported major.minor version 51.0 >> >> >> > >> >> >> > at java.lang.ClassLoader.defineClass1(Native Method) >> >> >> > >> >> >> > at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637) >> >> >> > >> >> >> > at java.lang.ClassLoader.defineClass(ClassLoader.java:621) >> >> >> > >> >> >> > at >> >> >> > >> >> >> > >> >> >> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) >> >> >> > >> >> >> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) >> >> >> > >> >> >> > at java.net.URLClassLoader.access$000(URLClassLoader.java:58) >> >> >> > >> >> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:197) >> >> >> > >> >> >> > at java.security.AccessController.doPrivileged(Native Method) >> >> >> > >> >> >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> >> >> > >> >> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >> >> >> > >> >> >> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> >> >> > >> >> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >> >> >> > >> >> >> > logout >> >> >> > >> >> >> > >> >> >> > [Process completed] >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Wed, Apr 2, 2014 at 1:49 PM, France Baril >> >> >> > <france.baril at architextus.com> >> >> >> > wrote: >> >> >> >> >> >> >> >> Timing is good, I should be able to try the new snapshot today. >> >> >> >> >> >> >> >> >> >> >> >> On Wed, Apr 2, 2014 at 12:55 PM, Christian Grün >> >> >> >> <christian.gruen at gmail.com> wrote: >> >> >> >>> >> >> >> >>> Hi France, >> >> >> >>> >> >> >> >>> we are also working towards some important commercial deadlines, >> >> >> >>> which >> >> >> >>> is why our mailing list feedback takes more time than expected >> >> >> >>> (open >> >> >> >>> source life would be so comfortable without financial pressure.. >> >> >> >>> ;). >> >> >> >>> However, I have now uploaded a new 8.0 snapshot [1], and I would >> >> >> >>> be >> >> >> >>> glad to get your feedback if it fixes the problems you >> >> >> >>> encountered. >> >> >> >>> You may as well think about exporting and reimporting your >> >> >> >>> database >> >> >> >>> (or applying OPTIMIZE ALL on it), as we have also minimized the >> >> >> >>> storage of redundant namespaces in the latest version. >> >> >> >>> >> >> >> >>> Looking forward to your response, >> >> >> >>> Christian >> >> >> >>> >> >> >> >>> [1] http://files.basex.org/releases/latest/ >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> On Wed, Apr 2, 2014 at 6:25 PM, France Baril >> >> >> >>> <france.baril at architextus.com> wrote: >> >> >> >>> > Ok, thanks for the update. >> >> >> >>> > >> >> >> >>> > We are dependent on this to make the next release to our >> >> >> >>> > production >> >> >> >>> > server, >> >> >> >>> > so if you have an idea on timing, it would really help us >> >> >> >>> > adapt. >> >> >> >>> > >> >> >> >>> > France >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > On Tue, Apr 1, 2014 at 11:43 AM, Christian Grün >> >> >> >>> > <christian.gruen at gmail.com> >> >> >> >>> > wrote: >> >> >> >>> >> >> >> >> >>> >> Hi France, >> >> >> >>> >> >> >> >> >>> >> sorry for lettting you wait. We've already solved part of the >> >> >> >>> >> problem, >> >> >> >>> >> but we would first like to shed some more light on another >> >> >> >>> >> related >> >> >> >>> >> issue before relasing a new patch version. >> >> >> >>> >> >> >> >> >>> >> Christian >> >> >> >>> >> ____________________________________ >> >> >> >>> >> >> >> >> >>> >> On Tue, Apr 1, 2014 at 5:26 PM, France Baril >> >> >> >>> >> <france.baril at architextus.com> wrote: >> >> >> >>> >> > Any news on this issue? >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> >> > On Tue, Mar 25, 2014 at 2:30 PM, France Baril >> >> >> >>> >> > <france.baril at architextus.com> >> >> >> >>> >> > wrote: >> >> >> >>> >> >> >> >> >> >>> >> >> Yes, we do use namespaces. >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> On Tue, Mar 25, 2014 at 2:29 PM, Christian Grün >> >> >> >>> >> >> <christian.gruen at gmail.com> wrote: >> >> >> >>> >> >>> >> >> >> >>> >> >>> We may have found a first evidence for the slow down. Do >> >> >> >>> >> >>> your >> >> >> >>> >> >>> documents use namespaces? >> >> >> >>> >> >>> >> >> >> >>> >> >>> >> >> >> >>> >> >>> On Tue, Mar 25, 2014 at 2:41 PM, Christian Grün >> >> >> >>> >> >>> <christian.gruen at gmail.com> wrote: >> >> >> >>> >> >>> > Hi France, >> >> >> >>> >> >>> > >> >> >> >>> >> >>> > sorry, I know that an answer to your e-mail is still >> >> >> >>> >> >>> > pending. >> >> >> >>> >> >>> > If >> >> >> >>> >> >>> > you >> >> >> >>> >> >>> > should manage to build a self-contained example, it >> >> >> >>> >> >>> > could >> >> >> >>> >> >>> > help >> >> >> >>> >> >>> > us to >> >> >> >>> >> >>> > find the culprit. >> >> >> >>> >> >>> > >> >> >> >>> >> >>> > Best, >> >> >> >>> >> >>> > Christian >> >> >> >>> >> >>> > >> >> >> >>> >> >>> > >> >> >> >>> >> >>> > On Wed, Mar 19, 2014 at 6:59 PM, France Baril >> >> >> >>> >> >>> > <france.baril at architextus.com> wrote: >> >> >> >>> >> >>> >> Hi, >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> We run batch processes on our files in order to send >> >> >> >>> >> >>> >> them >> >> >> >>> >> >>> >> out >> >> >> >>> >> >>> >> to >> >> >> >>> >> >>> >> translation >> >> >> >>> >> >>> >> in xliff format (xml format for translations). >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> I run a piece of code that: >> >> >> >>> >> >>> >> 1. applies transformations >> >> >> >>> >> >>> >> 2. file:writes the resulting files (for debug >> >> >> >>> >> >>> >> purposes) >> >> >> >>> >> >>> >> 3. and then db:replaces the resulting files. >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> If I send 5 files into the process, it takes a few >> >> >> >>> >> >>> >> seconds >> >> >> >>> >> >>> >> before >> >> >> >>> >> >>> >> the >> >> >> >>> >> >>> >> files >> >> >> >>> >> >>> >> appear on the file system. Yet full process with the >> >> >> >>> >> >>> >> db:replace >> >> >> >>> >> >>> >> takes >> >> >> >>> >> >>> >> 2 >> >> >> >>> >> >>> >> minutes to complete. >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> When I send 60 files... the process never ends. All >> >> >> >>> >> >>> >> files >> >> >> >>> >> >>> >> are >> >> >> >>> >> >>> >> available on >> >> >> >>> >> >>> >> the file system and are valid, which means I am fairly >> >> >> >>> >> >>> >> certain >> >> >> >>> >> >>> >> that >> >> >> >>> >> >>> >> the >> >> >> >>> >> >>> >> freeze doesn't occur because of an error in one of the >> >> >> >>> >> >>> >> files. >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> There is no error message in the terminal window. Is >> >> >> >>> >> >>> >> possible >> >> >> >>> >> >>> >> that >> >> >> >>> >> >>> >> something >> >> >> >>> >> >>> >> was changed after 7.7 that would affect performance? >> >> >> >>> >> >>> >> On >> >> >> >>> >> >>> >> our >> >> >> >>> >> >>> >> production >> >> >> >>> >> >>> >> server, which is on 7.7 we can run 100+ files in just >> >> >> >>> >> >>> >> a >> >> >> >>> >> >>> >> few >> >> >> >>> >> >>> >> minutes. >> >> >> >>> >> >>> >> Maybe >> >> >> >>> >> >>> >> there are new server settings that we missed? >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> For reference, here is the code. I understand that >> >> >> >>> >> >>> >> updates >> >> >> >>> >> >>> >> are >> >> >> >>> >> >>> >> all >> >> >> >>> >> >>> >> performed >> >> >> >>> >> >>> >> at the end of the process, which is why we get the >> >> >> >>> >> >>> >> debug >> >> >> >>> >> >>> >> files >> >> >> >>> >> >>> >> first, >> >> >> >>> >> >>> >> but >> >> >> >>> >> >>> >> the lag time between the time where we get all files >> >> >> >>> >> >>> >> on >> >> >> >>> >> >>> >> the >> >> >> >>> >> >>> >> file >> >> >> >>> >> >>> >> system and >> >> >> >>> >> >>> >> the db:replace seems huge. >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> declare updating function >> >> >> >>> >> >>> >> translation:transform-to-xliff($menu-id >> >> >> >>> >> >>> >> as >> >> >> >>> >> >>> >> xs:string, $lang as xs:string*, $package-number as >> >> >> >>> >> >>> >> xs:string){ >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> let $menu := menu:xml-get-menu($menu-id, 'en-us') >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> (: Need to translate menu and its topics without >> >> >> >>> >> >>> >> getting >> >> >> >>> >> >>> >> duplicated >> >> >> >>> >> >>> >> topics twice:) >> >> >> >>> >> >>> >> (: Always send the reusable strings from writers >> >> >> >>> >> >>> >> and >> >> >> >>> >> >>> >> from >> >> >> >>> >> >>> >> dev... >> >> >> >>> >> >>> >> doesn't >> >> >> >>> >> >>> >> cost anything if no new string anyway :) >> >> >> >>> >> >>> >> let $file-ids := ($menu-id, for $id in >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> distinct-values($menu/descendant::*[name()='topicref']/data(@id-ref))[1] >> >> >> >>> >> >>> >> return $id, >> >> >> >>> >> >>> >> 'stringreferences', >> >> >> >>> >> >>> >> 'stringreferencestransform') >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> return (for $file-id in >> >> >> >>> >> >>> >> distinct-values($file-ids) >> >> >> >>> >> >>> >> let $file := >> >> >> >>> >> >>> >> db:open('en-us')/*[@id=$file-id] >> >> >> >>> >> >>> >> let $skl-href := 'skeletons/' || >> >> >> >>> >> >>> >> $package-number >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> '/' >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> substring-after($file/base-uri(), 'en-us/') >> >> >> >>> >> >>> >> let $file-with-preserve := >> >> >> >>> >> >>> >> copy $copy := $file >> >> >> >>> >> >>> >> modify(if >> >> >> >>> >> >>> >> (exists($copy/@xml:space)) >> >> >> >>> >> >>> >> then () >> >> >> >>> >> >>> >> else insert node attribute >> >> >> >>> >> >>> >> {'xml:space'}{'preserve'} into $copy) >> >> >> >>> >> >>> >> return $copy >> >> >> >>> >> >>> >> let $all-trans-files := >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> xliff-transformer:create-translation-files($file-with-preserve, >> >> >> >>> >> >>> >> $package-number, $menu-id, $lang, $skl-href) >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> let $skeleton := $all-trans-files[1] >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> let $xliffs := for $xliff in >> >> >> >>> >> >>> >> $all-trans-files[position() >> >> >> >>> >> >>> >> != 1] >> >> >> >>> >> >>> >> return $xliff >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> let $debug := >> >> >> >>> >> >>> >> file:write('Debug-xliff/skls/' >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> tokenize($skl-href, '/')[last()], $skeleton) >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> return (db:replace('Translation-management', >> >> >> >>> >> >>> >> $skl-href, >> >> >> >>> >> >>> >> $skeleton), >> >> >> >>> >> >>> >> for $xliff in $xliffs >> >> >> >>> >> >>> >> let $target-lang := >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> $xliff/descendant::*[name()='file']/data(@target-language) >> >> >> >>> >> >>> >> let $lang-pack-name := >> >> >> >>> >> >>> >> concat($target-lang,'_', >> >> >> >>> >> >>> >> $package-number) >> >> >> >>> >> >>> >> let $file-loc-end := $menu-id || >> >> >> >>> >> >>> >> '/' >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> $lang-pack-name ||'/' || >> >> >> >>> >> >>> >> substring-after($file/base-uri(), >> >> >> >>> >> >>> >> 'en-us/') >> >> >> >>> >> >>> >> (:let $debug := >> >> >> >>> >> >>> >> file:append('debug-ids.xml', >> >> >> >>> >> >>> >> '&#10;' >> >> >> >>> >> >>> >> || $file-loc-end):) >> >> >> >>> >> >>> >> let $file-loc := if >> >> >> >>> >> >>> >> (exists($xliff/descendant::*[@translate='yes'])) >> >> >> >>> >> >>> >> then >> >> >> >>> >> >>> >> 'xliff-to-translate/' >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> $file-loc-end >> >> >> >>> >> >>> >> else >> >> >> >>> >> >>> >> 'xliff-no-new-segment/' >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> $file-loc-end >> >> >> >>> >> >>> >> let $debug := >> >> >> >>> >> >>> >> file:write('Debug-xliff/' >> >> >> >>> >> >>> >> || >> >> >> >>> >> >>> >> tokenize($file-loc, '/')[last()], $xliff) >> >> >> >>> >> >>> >> return >> >> >> >>> >> >>> >> db:replace('Translation-management', >> >> >> >>> >> >>> >> $file-loc, >> >> >> >>> >> >>> >> $xliff) >> >> >> >>> >> >>> >> ) >> >> >> >>> >> >>> >> ) >> >> >> >>> >> >>> >> }; >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> -- >> >> >> >>> >> >>> >> France Baril >> >> >> >>> >> >>> >> Architecte documentaire / Documentation architect >> >> >> >>> >> >>> >> france.baril at architextus.com >> >> >> >>> >> >>> >> (514) 572-0341 >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> _______________________________________________ >> >> >> >>> >> >>> >> BaseX-Talk mailing list >> >> >> >>> >> >>> >> BaseX-Talk at mailman.uni-konstanz.de >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> >> >> >> >>> >> >>> >> https://mailman.uni-konstanz.de/mailman/listinfo/basex-talk >> >> >> >>> >> >>> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> >> >> >> >>> >> >> -- >> >> >> >>> >> >> France Baril >> >> >> >>> >> >> Architecte documentaire / Documentation architect >> >> >> >>> >> >> france.baril at architextus.com >> >> >> >>> >> >> (514) 572-0341 >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> >> > >> >> >> >>> >> > -- >> >> >> >>> >> > France Baril >> >> >> >>> >> > Architecte documentaire / Documentation architect >> >> >> >>> >> > france.baril at architextus.com >> >> >> >>> >> > (514) 572-0341 >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > >> >> >> >>> > -- >> >> >> >>> > France Baril >> >> >> >>> > Architecte documentaire / Documentation architect >> >> >> >>> > france.baril at architextus.com >> >> >> >>> > (514) 572-0341 >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> France Baril >> >> >> >> Architecte documentaire / Documentation architect >> >> >> >> france.baril at architextus.com >> >> >> >> (514) 572-0341 >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> > -- >> >> >> > France Baril >> >> >> > Architecte documentaire / Documentation architect >> >> >> > france.baril at architextus.com >> >> >> > (514) 572-0341 >> >> > >> >> > >> >> > >> >> > >> >> > -- >> >> > France Baril >> >> > Architecte documentaire / Documentation architect >> >> > france.baril at architextus.com >> >> > (514) 572-0341 >> > >> > >> > >> > >> > -- >> > France Baril >> > Architecte documentaire / Documentation architect >> > france.baril at architextus.com >> > (514) 572-0341 > > > > > -- > France Baril > Architecte documentaire / Documentation architect > france.baril at architextus.com > (514) 572-0341

⚡Generated with XQuery