Changes

Jump to navigation Jump to search
14 bytes removed ,  14:42, 25 May 2012
no edit summary
For retrieving the Twitter stream we connect with the Streaming API to the endpoint of Twitter and receive a never ending tweet stream. As Twitter delivers the tweets as [http://www.json.org/ JSON] objects the objects has to be
converted into XML fragments. For this purpose the parse function of the [[JSON Module|XQuery JSON Module]] is used. The examples [[#Example Tweet (JSON)|tweet as JSON]] and [[#Example Tweet (XML)|tweet as XML]] show that each tweet is streamed as an object containing the tweet message itself and over 60 data fields (for further information see the [https://dev.twitter.com/docs/platform-objects fields description]). For storing the tweets including the meta-data, we use the standard ''insert'' function of [[Updates|XQuery Update]].
=Twitters' Streaming Data=
Each tweet object in the data stream contains the tweet message itself and over 60 data fields (for further information see the [https://dev.twitter.com/docs/platform-objects fields description]). In the examples section ([[#Example Tweet (JSON)|tweet as JSON]] and [[#Example Tweet (XML)|tweet as XML]] show .The following figure section shows the amount of data, that is delivered by the Twitter Streaming API to the connected endpoints with the 10% gardenhose access per hour
on the 6th of the months February, March, April and May. It is the pure public live stream without any filtering applied.
 
==Statistics==
[[File:Tweets.png]]
 
==Statistics==
{| class="wikitable" width="50%"
|-
! TypeDay
! Description
! Example (native → hex integers)Amount|-| Mon, 6-Feb-2012| Total tweets| 30.824.976<br/>|-|-| | Average tweets per hour| 1.284.374<br/>|-| | Average tweets per minute| 21.406<br/>|-| | Average tweets per second| 356<br/>|-| Tue, 6-Mar-2012| Total tweets| 31.823.776<br/>|-| | Average tweets per hour| 1.325.990<br/>|-| | Average tweets per minute| 22.099<br/>|-| | Average tweets per second| 368<br/>|-| Fri, 6-Apr-2012| Total tweets| 34.638.976 million<br/>|-| | Average tweets per hour| 1.443.290<br/>
|-
| {{Type|Num}}Average tweets per minute| Compressed integer (1-5 bytes), specified in [https://github24.com/BaseXdb/basex/blob/master/src/main/java/org/basex/util/Num.java Num.java]| {{Mono|15}} → {{Mono|0F}}; {{Mono|511}} → {{Mono|41 FF}}054<br/>
|-
| {{Type|Token}}| Length ({{Type|Num}}) and bytes of UTF8 byte representationAverage tweets per second| {{Mono|Hello}} → {{Mono|05 48 65 6c 6c 6f}}400<br/>
|-
| {{Type|Double}}Sun, 6-May-2012| Number, stored as tokenTotal tweets| {{Mono|123}} → {{Mono|03 31 32 33}}35.982.976 million<br/>
|-
| {{Type|Boolean}}Average tweets per hour| Boolean (1 byte, {{Mono|00}} or {{Mono|01}})| {{Mono|true}} → {{Mono|01}}.499.290<br/>
|-
| {{Type|Nums}}, {{Type|Tokens}}, {{Type|Doubles}}| Arrays of values, introduced with the number of entriesAverage tweets per minute| {{Mono|1,2}} → {{Mono|02 01 31 01 32}}24.988<br/>
|-
| {{Type|TokenSet}}Average tweets per second| Key array ({{Type|Tokens}}), next/bucket416<br/size arrays (3x {{Type|Nums}})|>
|}
bueraucrat, Bureaucrats, editor, reviewer, Administrators
907

edits

Navigation menu