Difference between revisions of "Profiling Module"

From BaseX Documentation
Jump to navigation Jump to search
(One intermediate revision by the same user not shown)
Line 10: Line 10:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:track|$expression as item()|item()*}}<br/>{{Func|prof:track|$expression as item(), $options as map(*)?|item()*}}
 
|{{Func|prof:track|$expression as item()|item()*}}<br/>{{Func|prof:track|$expression as item(), $options as map(*)?|item()*}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Measures the execution time and memory consumption required for evaluating the specified {{Code|$expression}} and returns a map with the results. The following {{Code|$options}} are available:
 
|Measures the execution time and memory consumption required for evaluating the specified {{Code|$expression}} and returns a map with the results. The following {{Code|$options}} are available:
Line 24: Line 24:
 
* If a value has a compact internal representation, memory consumption will be very low, even if the serialized result may consume much more memory.
 
* If a value has a compact internal representation, memory consumption will be very low, even if the serialized result may consume much more memory.
 
* Please note that memory profiling is only approximative, so it can be quite misleading. If the memory option is enabled, main-memory will be garbage-collected before and after evaluation to improve the quality of the measurement.
 
* Please note that memory profiling is only approximative, so it can be quite misleading. If the memory option is enabled, main-memory will be garbage-collected before and after evaluation to improve the quality of the measurement.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
* Return a human-readable representation of the memory consumption caused by fetching an XML document (<code>fetch:xml</code> is used, as <code>fn:doc</code> may already be evaluated at compilation time):
+
* Return a human-readable representation of the memory consumption caused by fetching an XML document. {{Function||fetch:doc}} is used, as <code>fn:doc</code> may already be evaluated at compilation time:
 
<syntaxhighlight lang="xquery">
 
<syntaxhighlight lang="xquery">
prof:track(fetch:xml('factbook.xml'))?memory
+
prof:track(fetch:doc('factbook.xml'))?memory
 
=> prof:human()
 
=> prof:human()
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 47: Line 47:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:time|$expr as item()|item()*}}<br />{{Func|prof:time|$expr as item(), $label as xs:string|item()*}}
 
|{{Func|prof:time|$expr as item()|item()*}}<br />{{Func|prof:time|$expr as item(), $label as xs:string|item()*}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Measures the time needed to evaluate {{Code|$expr}} and outputs a string to standard error or, if the GUI is used, to the Info View. An optional {{Code|$label}} may be specified to tag the profiling result. See {{Function||prof:track}} for further notes.
 
|Measures the time needed to evaluate {{Code|$expr}} and outputs a string to standard error or, if the GUI is used, to the Info View. An optional {{Code|$label}} may be specified to tag the profiling result. See {{Function||prof:track}} for further notes.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
Line 65: Line 65:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:memory|$expr as item()|item()*}}<br/>{{Func|prof:memory|$expr as item(), $label as xs:string|item()*}}
 
|{{Func|prof:memory|$expr as item()|item()*}}<br/>{{Func|prof:memory|$expr as item(), $label as xs:string|item()*}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Measures the memory allocated by evaluating {{Code|$expr}} and outputs a string to standard error or, if the GUI is used, to the Info View. An optional {{Code|$label}} may be specified to tag the profiling result. See {{Function||prof:track}} for further notes.
 
|Measures the memory allocated by evaluating {{Code|$expr}} and outputs a string to standard error or, if the GUI is used, to the Info View. An optional {{Code|$label}} may be specified to tag the profiling result. See {{Function||prof:track}} for further notes.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
Line 83: Line 83:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:current-ms||xs:integer}}<br />
 
|{{Func|prof:current-ms||xs:integer}}<br />
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Returns the number of milliseconds passed since 1970/01/01 UTC. The granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.
 
|Returns the number of milliseconds passed since 1970/01/01 UTC. The granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|In contrast to {{Code|fn:current-time()}}, the function is ''non-deterministic'' and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
 
|In contrast to {{Code|fn:current-time()}}, the function is ''non-deterministic'' and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
Line 101: Line 101:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:current-ns||xs:integer}}<br />
 
|{{Func|prof:current-ns||xs:integer}}<br />
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Returns the current value of the most precise available system timer in nanoseconds.
 
|Returns the current value of the most precise available system timer in nanoseconds.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|In contrast to {{Code|fn:current-time()}}, the function is ''non-deterministic'' and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
 
|In contrast to {{Code|fn:current-time()}}, the function is ''non-deterministic'' and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
| Measures the time of an expression:
 
| Measures the time of an expression:
Line 130: Line 130:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:dump|$expr as item()*|empty-sequence()}}<br />{{Func|prof:dump|$expr as item()*, $label as xs:string|empty-sequence()}}<br />
 
|{{Func|prof:dump|$expr as item()*|empty-sequence()}}<br />{{Func|prof:dump|$expr as item()*, $label as xs:string|empty-sequence()}}<br />
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Dumps a serialized representation of {{Code|$expr}} to {{Code|STDERR}}, optionally prefixed with {{Code|$label}}, and returns an empty sequence. If the GUI is used, the dumped result is shown in the [[Graphical User Interface#Visualizations|Info View]].
 
|Dumps a serialized representation of {{Code|$expr}} to {{Code|STDERR}}, optionally prefixed with {{Code|$label}}, and returns an empty sequence. If the GUI is used, the dumped result is shown in the [[Graphical User Interface#Visualizations|Info View]].
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|In contrast to {{Code|fn:trace()}}, the consumed expression will not be passed on.
 
|In contrast to {{Code|fn:trace()}}, the consumed expression will not be passed on.
Line 144: Line 144:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:variables||empty-sequence()}}
 
|{{Func|prof:variables||empty-sequence()}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Prints a list of all current local and global variable assignments to standard error or, if the GUI is used, to the Info View.<br />As every query is optimized before being evaluated, not all of the original variables may be visible in the output. Moreover, many variables of function calls will disappear because functions are inlined. Function inlining can be turned off by setting {{Option|INLINELIMIT}} to <code>0</code>.
 
|Prints a list of all current local and global variable assignments to standard error or, if the GUI is used, to the Info View.<br />As every query is optimized before being evaluated, not all of the original variables may be visible in the output. Moreover, many variables of function calls will disappear because functions are inlined. Function inlining can be turned off by setting {{Option|INLINELIMIT}} to <code>0</code>.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
Line 162: Line 162:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:type|$expr as item()*|item()*}}
 
|{{Func|prof:type|$expr as item()*|item()*}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Similar to {{Code|fn:trace($expr, $msg)}}, but instead of a user-defined message, it emits the compile-time type and estimated result size of its argument.
 
|Similar to {{Code|fn:trace($expr, $msg)}}, but instead of a user-defined message, it emits the compile-time type and estimated result size of its argument.
Line 173: Line 173:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:gc||empty-sequence()}}<br/>{{Func|prof:gc|$count as xs:integer|empty-sequence()}}
 
|{{Func|prof:gc||empty-sequence()}}<br/>{{Func|prof:gc|$count as xs:integer|empty-sequence()}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Enforces Java garbage collection. If no {{Code|$count}} is supplied, garbage will be collected once. Please note that this function should only be used for debugging purposes; in productive code, it is best to trust the garbage collecting strategies of Java.
 
|Enforces Java garbage collection. If no {{Code|$count}} is supplied, garbage will be collected once. Please note that this function should only be used for debugging purposes; in productive code, it is best to trust the garbage collecting strategies of Java.
Line 184: Line 184:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:runtime|$name of xs:string|xs:integer}}
 
|{{Func|prof:runtime|$name of xs:string|xs:integer}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Returns the value of the specified runtime {{Code|$option}}. The following options exist:
 
|Returns the value of the specified runtime {{Code|$option}}. The following options exist:
Line 194: Line 194:
 
* {{Code|used}}: Currently used memory (varies over time, will shrink after garbage collection).
 
* {{Code|used}}: Currently used memory (varies over time, will shrink after garbage collection).
 
* {{Code|processors}}: number of processors available to the Java virtual machine.
 
* {{Code|processors}}: number of processors available to the Java virtual machine.
|-
+
|- valign="top"
 
|{{Code|option}}
 
|{{Code|option}}
 
|The specified option is unknown.
 
|The specified option is unknown.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
Line 208: Line 208:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:void|$value as item()*|empty-sequence()}}
 
|{{Func|prof:void|$value as item()*|empty-sequence()}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Swallows all items of the specified {{Code|$value}} and returns an empty sequence. This function is helpful if some code needs to be evaluated and if the actual result is irrelevant.
 
|Swallows all items of the specified {{Code|$value}} and returns an empty sequence. This function is helpful if some code needs to be evaluated and if the actual result is irrelevant.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
|-
+
|- valign="top"
 
| '''Examples'''
 
| '''Examples'''
 
|
 
|
Line 226: Line 226:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:sleep|$ms as xs:integer|empty-sequence()}}<br />
 
|{{Func|prof:sleep|$ms as xs:integer|empty-sequence()}}<br />
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Sleeps for the specified number of milliseconds.
 
|Sleeps for the specified number of milliseconds.
|-
+
|- valign="top"
 
| '''Properties'''
 
| '''Properties'''
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
 
|The function is ''non-deterministic'': evaluation order will be preserved by the compiler.
Line 240: Line 240:
  
 
{| width='100%'
 
{| width='100%'
|-
+
|- valign="top"
 
| width='120' | '''Signatures'''
 
| width='120' | '''Signatures'''
 
|{{Func|prof:human|$number as xs:integer|xs:string}}
 
|{{Func|prof:human|$number as xs:integer|xs:string}}
|-
+
|- valign="top"
 
| '''Summary'''
 
| '''Summary'''
 
|Returns a human-readable representation of the specified {{Code|$number}}.
 
|Returns a human-readable representation of the specified {{Code|$number}}.
|-
+
|- valign="top"
 
| '''Example'''
 
| '''Example'''
 
|
 
|
Line 257: Line 257:
 
! width="110"|Code
 
! width="110"|Code
 
|Description
 
|Description
|-
+
|- valign="top"
 
|{{Code|option}}
 
|{{Code|option}}
 
|The specified option is unknown.
 
|The specified option is unknown.

Revision as of 14:19, 20 July 2022

This XQuery Module contains various functions to test and profile code, and to dump information to standard output.

Conventions

All functions and errors in this module are assigned to the http://basex.org/modules/prof namespace, which is statically bound to the prof prefix.

Performance Functions

prof:track

Signatures prof:track($expression as item()) as item()*
prof:track($expression as item(), $options as map(*)?) as item()*
Summary Measures the execution time and memory consumption required for evaluating the specified $expression and returns a map with the results. The following $options are available:
  • time: Include execution time in result as xs:decimal (unit: milliseconds; default: true).
  • memory: Include memory consumption in result as xs:integer (unit: bytes; default: false).
  • value: Include value in result (default: true).

Helpful notes:

  • If you are not interested in some of the returned results, you should disable them to save time and memory.
  • Profiling might change the execution behavior of your code: An expression that might be executed iteratively will be cached by the profiling function.
  • If a value has a compact internal representation, memory consumption will be very low, even if the serialized result may consume much more memory.
  • Please note that memory profiling is only approximative, so it can be quite misleading. If the memory option is enabled, main-memory will be garbage-collected before and after evaluation to improve the quality of the measurement.
Properties The function is non-deterministic: evaluation order will be preserved by the compiler.
Examples
  • Return a human-readable representation of the memory consumption caused by fetching an XML document. fetch:doc is used, as fn:doc may already be evaluated at compilation time:

<syntaxhighlight lang="xquery"> prof:track(fetch:doc('factbook.xml'))?memory => prof:human() </syntaxhighlight>

  • The function call prof:track((1 to 1000000)[. mod 2 = 0], map { 'time': false() }) will return something similar to:

<syntaxhighlight lang="xquery"> map {

 "memory": 21548400,
 "value": (2, 4, 6, 8, 10, ...)

} </syntaxhighlight>

prof:time

Signatures prof:time($expr as item()) as item()*
prof:time($expr as item(), $label as xs:string) as item()*
Summary Measures the time needed to evaluate $expr and outputs a string to standard error or, if the GUI is used, to the Info View. An optional $label may be specified to tag the profiling result. See prof:track for further notes.
Properties The function is non-deterministic: evaluation order will be preserved by the compiler.
Examples
  • prof:time(prof:sleep(1000)) outputs something similar to 1000.99 ms.

prof:memory

Signatures prof:memory($expr as item()) as item()*
prof:memory($expr as item(), $label as xs:string) as item()*
Summary Measures the memory allocated by evaluating $expr and outputs a string to standard error or, if the GUI is used, to the Info View. An optional $label may be specified to tag the profiling result. See prof:track for further notes.
Properties The function is non-deterministic: evaluation order will be preserved by the compiler.
Examples
  • prof:memory((1 to 100000) ! <a/>) will output something similar to 5620 kB.

prof:current-ms

Signatures prof:current-ms() as xs:integer
Summary Returns the number of milliseconds passed since 1970/01/01 UTC. The granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.
Properties In contrast to fn:current-time(), the function is non-deterministic and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
Examples
  • convert:integer-to-dateTime(prof:current-ms()) returns the current miliseconds in the xs:dateTime format.

prof:current-ns

Signatures prof:current-ns() as xs:integer
Summary Returns the current value of the most precise available system timer in nanoseconds.
Properties In contrast to fn:current-time(), the function is non-deterministic and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
Examples Measures the time of an expression:

<syntaxhighlight lang="xquery"> let $ns1 := prof:current-ns() return (

 (: process to measure :)
 (1 to 1000000)[. = 0],
 let $ns2 := prof:current-ns()
 let $ms := ((($ns2 - $ns1) idiv 10000) div 100)
 return $ms || ' ms'

) </syntaxhighlight>

Debugging Functions

prof:dump

Signatures prof:dump($expr as item()*) as empty-sequence()
prof:dump($expr as item()*, $label as xs:string) as empty-sequence()
Summary Dumps a serialized representation of $expr to STDERR, optionally prefixed with $label, and returns an empty sequence. If the GUI is used, the dumped result is shown in the Info View.
Properties In contrast to fn:trace(), the consumed expression will not be passed on.

prof:variables

Signatures prof:variables() as empty-sequence()
Summary Prints a list of all current local and global variable assignments to standard error or, if the GUI is used, to the Info View.
As every query is optimized before being evaluated, not all of the original variables may be visible in the output. Moreover, many variables of function calls will disappear because functions are inlined. Function inlining can be turned off by setting INLINELIMIT to 0.
Properties The function is non-deterministic: evaluation order will be preserved by the compiler.
Examples
  • for $x in 1 to 2 return ($x, prof:variables()) will dump the values of $x to standard error.

prof:type

Signatures prof:type($expr as item()*) as item()*
Summary Similar to fn:trace($expr, $msg), but instead of a user-defined message, it emits the compile-time type and estimated result size of its argument.

prof:gc

Signatures prof:gc() as empty-sequence()
prof:gc($count as xs:integer) as empty-sequence()
Summary Enforces Java garbage collection. If no $count is supplied, garbage will be collected once. Please note that this function should only be used for debugging purposes; in productive code, it is best to trust the garbage collecting strategies of Java.

prof:runtime

Signatures prof:runtime($name of xs:string) as xs:integer
Summary Returns the value of the specified runtime $option. The following options exist:
  • max: Maximum memory that the Java virtual machine will attempt to use.
  • total: Total memory in the Java virtual machine (varies over time).
  • used: Currently used memory (varies over time, will shrink after garbage collection).
  • processors: number of processors available to the Java virtual machine.
option The specified option is unknown.
Examples
  • prof:gc(3), prof:human(prof:runtime('used')) performs some garbage collection and returns the currently used amount of memory in a user-friendly format.

Helper Functions

prof:void

Signatures prof:void($value as item()*) as empty-sequence()
Summary Swallows all items of the specified $value and returns an empty sequence. This function is helpful if some code needs to be evaluated and if the actual result is irrelevant.
Properties The function is non-deterministic: evaluation order will be preserved by the compiler.
Examples

prof:sleep

Signatures prof:sleep($ms as xs:integer) as empty-sequence()
Summary Sleeps for the specified number of milliseconds.
Properties The function is non-deterministic: evaluation order will be preserved by the compiler.

prof:human

Signatures prof:human($number as xs:integer) as xs:string
Summary Returns a human-readable representation of the specified $number.
Example
  • prof:human(16384) returns 16K.

Errors

Code Description
option The specified option is unknown.

Changelog

Version 9.2
Version 9.0
Version 8.5
Version 8.1
Version 7.7
Version 7.6
Version 7.5

This module was introduced with Version 7.3.