Main Page » Mailing List » July 2014

UTF-8 charcaters

Friday, 4 July 2014 · 5 messages · last reply 7 Jul 2014
CG
Christian Grünin reply to Erol Akarsu, 4 Jul 2014, 23:144 Jul 2014 21:44 #
Hi Erol,
Quoted text · Erol Akarsu · 5 lines
> I am trying to fetch an url page > http://www.ruya-tabirleri.com/ruyada-adam-gormek.html that has Turkish > characters in it. When I pull it in basex, the pages returns in > charset=windows-1254 but as you see it doesn't render Turkish characters > like here:
Please give us more details on how you are trying to retrieve the web page. One solution could look as follows: html:parse(fetch:text('http://www.ruya-tabirleri.com/ruyada-adam-gormek.html', 'windows-1254'), map { 'encoding': 'UTF-8' }) Cheers, Christian
Quoted text · Erol Akarsu · 32 lines
> > content="r?a tabirleri, r?a yorumlar? r?a tabiri, r?a, r?atabirleri, haberci > r?a, r?a t?leri, r?a nedir, aksenov, forsa, atat?k, evliya ?lebi, bur?ar, > astroloji"/> > > > <html lang="tr-TR"> > <head profile="http://gmpg.org/xfn/11"> > <title>R?a Tabirleri, R?a Tabiri, R?a Yorumlar? R?a, r?atabirleri, ruya > tabirleri</title> > <meta name="keywords" content="r?a tabirleri, r?a yorumlar? r?a tabiri, > r?a, r?atabirleri, haberci r?a, r?a t?leri, r?a nedir, aksenov, forsa, > atat?k, evliya ?lebi, bur?ar, astroloji"/> > <meta name="description" content="r?a tabirleri, r?a tabiri, r?a > yorumlar? r?a, ruya tabirleri, r?atabirleri, r?alar?? bu sitede anlam > kazanacak."/> > <meta http-equiv="Content-Language" content="tr"/> > <meta http-equiv="Content-Type" content="text/html; > charset=windows-1254"/> > <meta name="Author" content="Ruya-tabirleri.com"/> > <meta name="robots" content="index, follow"/> > <meta name="ROBOTS" content="ALL"/> > <meta name="googlebot" content="index, follow"/> > <meta name="Revisit-After" content="1 Days"/> > <meta name="RATING" content="General"/> > <meta name="copyright" content="r?a tabirleri"/> > <link rel="stylesheet" > href="http://www.ruya-tabirleri.com/img/ruya1.css" type="text/css"/> > <script type="text/javascript" > src="http://apis.google.com/js/plusone.js"/> > </head> >
EA
Erol Akarsu4 Jul 2014 23:14 #
I am trying to fetch an url page http://www.ruya-tabirleri.com/ruyada-adam-gormek.html that has Turkish characters in it. When I pull it in basex, the pages returns in charset=windows-1254 but as you see it doesn't render Turkish characters like here: content="r?a tabirleri, r?a yorumlar? r?a tabiri, r?a, r?atabirleri, haberci r?a, r?a t?leri, r?a nedir, aksenov, forsa, atat?k, evliya ?lebi, bur?ar, astroloji"/>
<html lang="tr-TR">
  <head profile="http://gmpg.org/xfn/11">
    <title>R?a Tabirleri, R?a Tabiri, R?a Yorumlar? R?a, r?atabirleri, ruya
tabirleri</title>
    <meta name="keywords" content="r?a tabirleri, r?a yorumlar? r?a tabiri,
r?a, r?atabirleri, haberci r?a, r?a t?leri, r?a nedir, aksenov, forsa,
atat?k, evliya ?lebi, bur?ar, astroloji"/>
    <meta name="description" content="r?a tabirleri, r?a tabiri, r?a
yorumlar? r?a, ruya tabirleri, r?atabirleri, r?alar?? bu sitede anlam
kazanacak."/>
    <meta http-equiv="Content-Language" content="tr"/>
    <meta http-equiv="Content-Type" content="text/html;
charset=windows-1254"/>
    <meta name="Author" content="Ruya-tabirleri.com"/>
    <meta name="robots" content="index, follow"/>
    <meta name="ROBOTS" content="ALL"/>
    <meta name="googlebot" content="index, follow"/>
    <meta name="Revisit-After" content="1 Days"/>
    <meta name="RATING" content="General"/>
    <meta name="copyright" content="r?a tabirleri"/>
    <link rel="stylesheet" href="http://www.ruya-tabirleri.com/img/ruya1.css"
type="text/css"/>
    <script type="text/javascript" src="http://apis.google.com/js/plusone.js
"/>
  </head>
CG
Christian Grün5 Jul 2014 01:13 #
Hi Erol, thanks for the script. Do you think you could provide us with a minimized example? Christian
Quoted text · Erol Akarsu · 70 lines
On Fri, Jul 4, 2014 at 11:43 PM, Erol Akarsu <eakarsu at gmail.com> wrote: > Christian, > > My xquery script and result are attached. > You will see incorrect encoded elements as? > > > On Fri, Jul 4, 2014 at 3:44 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Erol, >> >> > I am trying to fetch an url page >> > http://www.ruya-tabirleri.com/ruyada-adam-gormek.html that has Turkish >> > characters in it. When I pull it in basex, the pages returns in >> > charset=windows-1254 but as you see it doesn't render Turkish characters >> > like here: >> >> Please give us more details on how you are trying to retrieve the web >> page. >> >> One solution could look as follows: >> >> >> html:parse(fetch:text('http://www.ruya-tabirleri.com/ruyada-adam-gormek.html', >> 'windows-1254'), map { 'encoding': 'UTF-8' }) >> >> Cheers, >> Christian >> >> >> >> > >> > content="r?a tabirleri, r?a yorumlar? r?a tabiri, r?a, r?atabirleri, >> > haberci >> > r?a, r?a t?leri, r?a nedir, aksenov, forsa, atat?k, evliya ?lebi, >> > bur?ar, >> > astroloji"/> >> > >> > >> > <html lang="tr-TR"> >> > <head profile="http://gmpg.org/xfn/11"> >> > <title>R?a Tabirleri, R?a Tabiri, R?a Yorumlar? R?a, r?atabirleri, >> > ruya >> > tabirleri</title> >> > <meta name="keywords" content="r?a tabirleri, r?a yorumlar? r?a >> > tabiri, >> > r?a, r?atabirleri, haberci r?a, r?a t?leri, r?a nedir, aksenov, forsa, >> > atat?k, evliya ?lebi, bur?ar, astroloji"/> >> > <meta name="description" content="r?a tabirleri, r?a tabiri, r?a >> > yorumlar? r?a, ruya tabirleri, r?atabirleri, r?alar?? bu sitede anlam >> > kazanacak."/> >> > <meta http-equiv="Content-Language" content="tr"/> >> > <meta http-equiv="Content-Type" content="text/html; >> > charset=windows-1254"/> >> > <meta name="Author" content="Ruya-tabirleri.com"/> >> > <meta name="robots" content="index, follow"/> >> > <meta name="ROBOTS" content="ALL"/> >> > <meta name="googlebot" content="index, follow"/> >> > <meta name="Revisit-After" content="1 Days"/> >> > <meta name="RATING" content="General"/> >> > <meta name="copyright" content="r?a tabirleri"/> >> > <link rel="stylesheet" >> > href="http://www.ruya-tabirleri.com/img/ruya1.css" type="text/css"/> >> > <script type="text/javascript" >> > src="http://apis.google.com/js/plusone.js"/> >> > </head> >> > > >
EA
Erol Akarsu5 Jul 2014 03:46 #
Christian, Slight modification I made to script: only 10 sites are pulled and produced output I attached new script and result you will get when you run it
Quoted text · Christian Grün · 85 lines
On Fri, Jul 4, 2014 at 7:13 PM, Christian Grün <christian.gruen at gmail.com> wrote: > Hi Erol, > > thanks for the script. Do you think you could provide us with a > minimized example? > > Christian > _____________________________________ > > On Fri, Jul 4, 2014 at 11:43 PM, Erol Akarsu <eakarsu at gmail.com> wrote: > > Christian, > > > > My xquery script and result are attached. > > You will see incorrect encoded elements as? > > > > > > On Fri, Jul 4, 2014 at 3:44 PM, Christian Grün < > christian.gruen at gmail.com> > > wrote: > >> > >> Hi Erol, > >> > >> > I am trying to fetch an url page > >> > http://www.ruya-tabirleri.com/ruyada-adam-gormek.html that has > Turkish > >> > characters in it. When I pull it in basex, the pages returns in > >> > charset=windows-1254 but as you see it doesn't render Turkish > characters > >> > like here: > >> > >> Please give us more details on how you are trying to retrieve the web > >> page. > >> > >> One solution could look as follows: > >> > >> > >> html:parse(fetch:text(' > http://www.ruya-tabirleri.com/ruyada-adam-gormek.html', > >> 'windows-1254'), map { 'encoding': 'UTF-8' }) > >> > >> Cheers, > >> Christian > >> > >> > >> > >> > > >> > content="r?a tabirleri, r?a yorumlar? r?a tabiri, r?a, r?atabirleri, > >> > haberci > >> > r?a, r?a t?leri, r?a nedir, aksenov, forsa, atat?k, evliya ?lebi, > >> > bur?ar, > >> > astroloji"/> > >> > > >> > > >> > <html lang="tr-TR"> > >> > <head profile="http://gmpg.org/xfn/11"> > >> > <title>R?a Tabirleri, R?a Tabiri, R?a Yorumlar? R?a, r?atabirleri, > >> > ruya > >> > tabirleri</title> > >> > <meta name="keywords" content="r?a tabirleri, r?a yorumlar? r?a > >> > tabiri, > >> > r?a, r?atabirleri, haberci r?a, r?a t?leri, r?a nedir, aksenov, forsa, > >> > atat?k, evliya ?lebi, bur?ar, astroloji"/> > >> > <meta name="description" content="r?a tabirleri, r?a tabiri, r?a > >> > yorumlar? r?a, ruya tabirleri, r?atabirleri, r?alar?? bu sitede anlam > >> > kazanacak."/> > >> > <meta http-equiv="Content-Language" content="tr"/> > >> > <meta http-equiv="Content-Type" content="text/html; > >> > charset=windows-1254"/> > >> > <meta name="Author" content="Ruya-tabirleri.com"/> > >> > <meta name="robots" content="index, follow"/> > >> > <meta name="ROBOTS" content="ALL"/> > >> > <meta name="googlebot" content="index, follow"/> > >> > <meta name="Revisit-After" content="1 Days"/> > >> > <meta name="RATING" content="General"/> > >> > <meta name="copyright" content="r?a tabirleri"/> > >> > <link rel="stylesheet" > >> > href="http://www.ruya-tabirleri.com/img/ruya1.css" type="text/css"/> > >> > <script type="text/javascript" > >> > src="http://apis.google.com/js/plusone.js"/> > >> > </head> > >> > > > > > >
CG
Christian Grün7 Jul 2014 10:47 #
Hi Erol, I probably don't have time to wait for the script to finish, and I think it does much more than we need to get down to the core of the problem, so.. If you manage to isolate the problem you want us to solve, feel free to give it another try. Christian
Quoted text · Erol Akarsu · 98 lines
On Sat, Jul 5, 2014 at 1:46 AM, Erol Akarsu <eakarsu at gmail.com> wrote: > Christian, > > Slight modification I made to script: only 10 sites are pulled and produced > output > I attached new script and result you will get when you run it > > > > On Fri, Jul 4, 2014 at 7:13 PM, Christian Grün <christian.gruen at gmail.com> > wrote: >> >> Hi Erol, >> >> thanks for the script. Do you think you could provide us with a >> minimized example? >> >> Christian >> _____________________________________ >> >> On Fri, Jul 4, 2014 at 11:43 PM, Erol Akarsu <eakarsu at gmail.com> wrote: >> > Christian, >> > >> > My xquery script and result are attached. >> > You will see incorrect encoded elements as? >> > >> > >> > On Fri, Jul 4, 2014 at 3:44 PM, Christian Grün >> > <christian.gruen at gmail.com> >> > wrote: >> >> >> >> Hi Erol, >> >> >> >> > I am trying to fetch an url page >> >> > http://www.ruya-tabirleri.com/ruyada-adam-gormek.html that has >> >> > Turkish >> >> > characters in it. When I pull it in basex, the pages returns in >> >> > charset=windows-1254 but as you see it doesn't render Turkish >> >> > characters >> >> > like here: >> >> >> >> Please give us more details on how you are trying to retrieve the web >> >> page. >> >> >> >> One solution could look as follows: >> >> >> >> >> >> >> >> html:parse(fetch:text('http://www.ruya-tabirleri.com/ruyada-adam-gormek.html', >> >> 'windows-1254'), map { 'encoding': 'UTF-8' }) >> >> >> >> Cheers, >> >> Christian >> >> >> >> >> >> >> >> > >> >> > content="r?a tabirleri, r?a yorumlar? r?a tabiri, r?a, r?atabirleri, >> >> > haberci >> >> > r?a, r?a t?leri, r?a nedir, aksenov, forsa, atat?k, evliya ?lebi, >> >> > bur?ar, >> >> > astroloji"/> >> >> > >> >> > >> >> > <html lang="tr-TR"> >> >> > <head profile="http://gmpg.org/xfn/11"> >> >> > <title>R?a Tabirleri, R?a Tabiri, R?a Yorumlar? R?a, >> >> > r?atabirleri, >> >> > ruya >> >> > tabirleri</title> >> >> > <meta name="keywords" content="r?a tabirleri, r?a yorumlar? r?a >> >> > tabiri, >> >> > r?a, r?atabirleri, haberci r?a, r?a t?leri, r?a nedir, aksenov, >> >> > forsa, >> >> > atat?k, evliya ?lebi, bur?ar, astroloji"/> >> >> > <meta name="description" content="r?a tabirleri, r?a tabiri, r?a >> >> > yorumlar? r?a, ruya tabirleri, r?atabirleri, r?alar?? bu sitede anlam >> >> > kazanacak."/> >> >> > <meta http-equiv="Content-Language" content="tr"/> >> >> > <meta http-equiv="Content-Type" content="text/html; >> >> > charset=windows-1254"/> >> >> > <meta name="Author" content="Ruya-tabirleri.com"/> >> >> > <meta name="robots" content="index, follow"/> >> >> > <meta name="ROBOTS" content="ALL"/> >> >> > <meta name="googlebot" content="index, follow"/> >> >> > <meta name="Revisit-After" content="1 Days"/> >> >> > <meta name="RATING" content="General"/> >> >> > <meta name="copyright" content="r?a tabirleri"/> >> >> > <link rel="stylesheet" >> >> > href="http://www.ruya-tabirleri.com/img/ruya1.css" type="text/css"/> >> >> > <script type="text/javascript" >> >> > src="http://apis.google.com/js/plusone.js"/> >> >> > </head> >> >> > >> > >> > > >

Written for BaseX 7.9. BaseX may have changed since: please check the current documentation.

⚡Generated with XQuery