Difference between revisions of "Storage Layout"

From BaseX Documentation
Jump to navigation Jump to search
Line 1: Line 1:
 
+
=Data Types=
==Data Types==
 
  
 
The following data types are used for specifying the storage layout:
 
The following data types are used for specifying the storage layout:
Line 35: Line 34:
 
|}
 
|}
  
==Database Files==
+
=Database Files=
  
The following tables present the layout of BaseX database files. All files are suffixed with <code>.basex</code>.
+
The following tables illustrate the layout of the BaseX database files. All files are suffixed with <code>.basex</code>.
  
===Meta Data, Name/Path/Doc Indexes: <code>inf</code>===
+
==Meta Data, Name/Path/Doc Indexes: <code>inf</code>==
  
 
{| class="wikitable" width="100%"
 
{| class="wikitable" width="100%"
Line 72: Line 71:
 
|}
 
|}
  
===Node Table: <code>tbl</code>, <code>tbli</code>===
+
==Node Table: <code>tbl</code>, <code>tbli</code>==
  
 
* <code>tbl</code>: Main database table, stored in blocks.
 
* <code>tbl</code>: Main database table, stored in blocks.
 
* <code>tbli</code>: Database directory, organizing the database blocks.
 
* <code>tbli</code>: Database directory, organizing the database blocks.
  
===Texts: <code>txt</code>, <code>atv</code>===
+
==Texts: <code>txt</code>, <code>atv</code>==
  
 
* <code>txt</code>: Heap file for text values (document names, string values of texts, comments and processing instructions)
 
* <code>txt</code>: Heap file for text values (document names, string values of texts, comments and processing instructions)
 
* <code>atv</code>: Heap file for attribute values.
 
* <code>atv</code>: Heap file for attribute values.
  
===Value Indexes: <code>txtl</code>, <code>txtr</code>, <code>atvl</code>, <code>atvr</code>===
+
==Value Indexes: <code>txtl</code>, <code>txtr</code>, <code>atvl</code>, <code>atvr</code>==
  
 
'''Text Index:'''
 
'''Text Index:'''
Line 89: Line 88:
 
The '''Attribute Index''' is contained in the files <code>atvl</code> and <code>atvr</code>; it uses the same layout.
 
The '''Attribute Index''' is contained in the files <code>atvl</code> and <code>atvr</code>; it uses the same layout.
  
===Full-Text Fuzzy Index: <code>ftxx</code>, <code>ftxy</code>, <code>ftxz</code>===
+
==Full-Text Fuzzy Index: <code>ftxx</code>, <code>ftxy</code>, <code>ftxz</code>==
  
 
...will soon be reimplemented.
 
...will soon be reimplemented.
  
===Full-Text Trie Index: <code>ftxa</code>, <code>ftxb</code>, <code>ftxc</code>===
+
==Full-Text Trie Index: <code>ftxa</code>, <code>ftxb</code>, <code>ftxc</code>==
  
 
...will soon be dismissed.
 
...will soon be dismissed.

Revision as of 18:43, 26 October 2011

Data Types

The following data types are used for specifying the storage layout:

Type Description Example (native → hex integers)
Num Compressed integer (1-5 bytes), specified in Num.java 150F; 51141 FF
Token Length (Num) and bytes of UTF8 byte representation Hello05 48 65 6c 6c 6f
Double Number, stored as token 12303 31 32 33
Boolean Boolean (1 byte, 00 or 01) true01
Nums, Tokens, Doubles Arrays of values, introduced with the number of entries 1,202 01 31 01 32
TokenSet Key array (Tokens), next/bucket/size arrays (3x Nums)

Database Files

The following tables illustrate the layout of the BaseX database files. All files are suffixed with .basex.

Meta Data, Name/Path/Doc Indexes: inf

Description Format Method
1. Meta Data 1. Key/value pairs (Token/Token):
  • PERM → Number of users (Num), and name/password/permission values for each user (Token/Token/Num)
2. Empty key as finalizer
DiskData()
MetaData()
Users()
2. Main memory indexes 1. Key/value pairs (Token/Token):
  • TAGS → Tag Index
  • ATTS → Attribute Name Index
  • PATH → Path Index
  • NS → Namespaces
  • DOCS → Document Index
2. Empty key as finalizer
DiskData()
2 a) Name Index
Tag/attribute names
1. Token set, storing all names (TokenSet)
2. One StatsKey instance per entry:
2.1. Content kind (Num):
2.1.1. Number: min/max (Doubles)
2.1.2. Category: number of entries (Num), entries (Tokens)
2.2. Number of entries (Num)
2.3. Leaf flag (Boolean)
2.4. Maximum text length (Double; legacy, could be Num)
Names()
TokenSet.read()
StatsKey()
2 b) Path Index 1. Flag for path definition (Boolean, always true; legacy)
2. PathNode:
2.1. Name reference (Num)
2.2. Node kind (Num)
2.3. Number of occurrences (Num)
2.4. Number of children (Num)
2.5. Double; legacy, can be reused or discarded
2.6. Recursive generation of child nodes (→ 2)
PathSummary()
PathNode()
2 c) Namespaces 1. Token set, storing prefixes (TokenSet)
2. Token set, storing URIs (TokenSet)
3. NSNode:
3.1. pre value (Num)
3.2. References to prefix/URI pairs (Nums)
3.3. Number of children (Num)
3.4. Recursive generation of child nodes (→ 3)
Namespaces()
NSNode()
2 d) Document Index Array of integers, representing the distances between all document pre values (Nums) DocIndex()

Node Table: tbl, tbli

  • tbl: Main database table, stored in blocks.
  • tbli: Database directory, organizing the database blocks.

Texts: txt, atv

  • txt: Heap file for text values (document names, string values of texts, comments and processing instructions)
  • atv: Heap file for attribute values.

Value Indexes: txtl, txtr, atvl, atvr

Text Index:

  • txtl: Heap file with ID lists.
  • txtr: Index file with references to ID lists.

The Attribute Index is contained in the files atvl and atvr; it uses the same layout.

Full-Text Fuzzy Index: ftxx, ftxy, ftxz

...will soon be reimplemented.

Full-Text Trie Index: ftxa, ftxb, ftxc

...will soon be dismissed.