Difference between revisions of "Profiling Module"
Jump to navigation
Jump to search
m (Text replacement - "<syntaxhighlight lang="xquery">" to "<pre lang='xquery'>") |
|||
(64 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
− | This [[Module Library|XQuery Module]] contains various | + | This [[Module Library|XQuery Module]] contains various functions to test and profile code, and to dump information to standard output. |
=Conventions= | =Conventions= | ||
− | All functions in this module are assigned to the | + | All functions and errors in this module are assigned to the <code><nowiki>http://basex.org/modules/prof</nowiki></code> namespace, which is statically bound to the {{Code|prof}} prefix.<br/> |
− | |||
− | =Functions= | + | =Performance Functions= |
+ | |||
+ | ==prof:track== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- valign="top" | ||
+ | | width='120' | '''Signature''' | ||
+ | |<pre>prof:track( | ||
+ | $input as item()*, | ||
+ | $options as map(*)? := map { } | ||
+ | ) as item()*</pre> | ||
+ | |- valign="top" | ||
+ | | '''Summary''' | ||
+ | |Measures the execution time and memory consumption required for evaluating the specified {{Code|$input}} and returns a map with the results. The following {{Code|$options}} are available: | ||
+ | * {{Code|time}}: Include execution time in result as {{Code|xs:decimal}} (unit: milliseconds; default: true). | ||
+ | * {{Code|memory}}: Include memory consumption in result as {{Code|xs:integer}} (unit: bytes; default: false). | ||
+ | * {{Code|value}}: Include value in result (default: true). | ||
+ | Helpful notes: | ||
+ | * If you are not interested in some of the returned results, you should disable them to save time and memory. | ||
+ | * Profiling might change the execution behavior of your code: An expression that might be executed iteratively will be cached by the profiling function. | ||
+ | * If a value has a compact internal representation, memory consumption will be very low, even if the serialized result may consume much more memory. | ||
+ | * Please note that memory profiling is only approximative, so it can be quite misleading. If the memory option is enabled, main-memory will be garbage-collected before and after evaluation to improve the quality of the measurement. | ||
+ | |- valign="top" | ||
+ | | '''Properties''' | ||
+ | |The function is ''nondeterministic'': evaluation order will be preserved by the compiler. | ||
+ | |- valign="top" | ||
+ | | '''Examples''' | ||
+ | | | ||
+ | * Return a human-readable representation of the memory consumption caused by fetching an XML document. {{Function||fetch:doc}} is used, as <code>fn:doc</code> may already be evaluated at compilation time: | ||
+ | <pre lang='xquery'> | ||
+ | prof:track(fetch:doc('factbook.xml'))?memory | ||
+ | => prof:human() | ||
+ | </pre> | ||
+ | * The function call <code>prof:track((1 to 1000000)[. mod 2 = 0], map { 'time': false() })</code> will return something similar to: | ||
+ | <pre lang='xquery'> | ||
+ | map { | ||
+ | "memory": 21548400, | ||
+ | "value": (2, 4, 6, 8, 10, ...) | ||
+ | } | ||
+ | </pre> | ||
+ | |} | ||
==prof:time== | ==prof:time== | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:time( |
− | |- | + | $input as item(), |
+ | $label as xs:string? := () | ||
+ | ) as item()*</pre> | ||
+ | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
− | |Measures the time needed to evaluate {{Code|$ | + | |Measures the time needed to evaluate {{Code|$input}} and outputs a string to standard error or, if the GUI is used, to the Info View. An optional {{Code|$label}} may be specified to tag the profiling result. See {{Function||prof:track}} for further notes. |
− | |- | + | |- valign="top" |
| '''Properties''' | | '''Properties''' | ||
− | |The function is '' | + | |The function is ''nondeterministic'': evaluation order will be preserved by the compiler. |
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
− | * {{Code|prof:time( | + | * {{Code|prof:time(prof:sleep(1000))}} outputs something similar to {{Code|1000.99 ms}}. |
− | |||
|} | |} | ||
− | ==prof: | + | ==prof:memory== |
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:memory( |
− | |- | + | $input as item(), |
+ | $label as xs:string? := () | ||
+ | ) as item()*</pre> | ||
+ | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
− | |Measures the memory allocated by evaluating {{Code|$ | + | |Measures the memory allocated by evaluating {{Code|$input}} and outputs a string to standard error or, if the GUI is used, to the Info View. An optional {{Code|$label}} may be specified to tag the profiling result. See {{Function||prof:track}} for further notes. |
− | |- | + | |- valign="top" |
| '''Properties''' | | '''Properties''' | ||
− | |The function is '' | + | |The function is ''nondeterministic'': evaluation order will be preserved by the compiler. |
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
− | * {{Code|prof: | + | * {{Code|prof:memory((1 to 100000) ! <a/>)}} will output something similar to {{Code|5620 kB}}. |
− | |||
|} | |} | ||
− | ==prof: | + | ==prof:current-ms== |
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:current-ms() as xs:integer</pre> |
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
− | | | + | |Returns the number of milliseconds passed since 1970/01/01 UTC. The granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds. |
− | |- | + | |- valign="top" |
| '''Properties''' | | '''Properties''' | ||
− | | | + | |In contrast to {{Code|fn:current-time()}}, the function is ''nondeterministic'' and returns different values every time it is called. Its evaluation order will be preserved by the compiler. |
+ | |- valign="top" | ||
+ | | '''Examples''' | ||
+ | | | ||
+ | * {{Code|convert:integer-to-dateTime(prof:current-ms())}} returns the current miliseconds in the {{Code|xs:dateTime}} format. | ||
|} | |} | ||
− | ==prof: | + | ==prof:current-ns== |
− | |||
− | |||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:current-ns() as xs:integer</pre> |
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
− | |Returns | + | |Returns the current value of the most precise available system timer in nanoseconds. |
− | |- | + | |- valign="top" |
− | | ''' | + | | '''Properties''' |
− | | | + | |In contrast to {{Code|fn:current-time()}}, the function is ''nondeterministic'' and returns different values every time it is called. Its evaluation order will be preserved by the compiler. |
− | + | |- valign="top" | |
+ | | '''Examples''' | ||
+ | | Measures the time of an expression: | ||
+ | <pre lang='xquery'> | ||
+ | let $ns1 := prof:current-ns() | ||
+ | return ( | ||
+ | (: process to measure :) | ||
+ | (1 to 1000000)[. = 0], | ||
+ | let $ns2 := prof:current-ns() | ||
+ | let $ms := ((($ns2 - $ns1) idiv 10000) div 100) | ||
+ | return $ms || ' ms' | ||
+ | ) | ||
+ | </pre> | ||
|} | |} | ||
+ | |||
+ | =Debugging Functions= | ||
==prof:dump== | ==prof:dump== | ||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:dump( |
− | |- | + | $input as item()*, |
+ | $label as xs:string? := () | ||
+ | ) as empty-sequence()</pre> | ||
+ | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
− | |Dumps a serialized representation of {{Code|$ | + | |Dumps a serialized representation of {{Code|$input}} to {{Code|STDERR}}, optionally prefixed with {{Code|$label}}, and returns an empty sequence. If the GUI is used, the dumped result is shown in the [[Graphical User Interface#Visualizations|Info View]]. |
− | |- | + | |- valign="top" |
| '''Properties''' | | '''Properties''' | ||
|In contrast to {{Code|fn:trace()}}, the consumed expression will not be passed on. | |In contrast to {{Code|fn:trace()}}, the consumed expression will not be passed on. | ||
|} | |} | ||
− | ==prof: | + | ==prof:variables== |
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:variables() as empty-sequence()</pre> |
− | |- | + | |- valign="top" |
| '''Summary''' | | '''Summary''' | ||
− | | | + | |Prints a list of all current local and global variable assignments to standard error or, if the GUI is used, to the Info View.<br/>As every query is optimized before being evaluated, not all of the original variables may be visible in the output. Moreover, many variables of function calls will disappear because functions are inlined. Function inlining can be turned off by setting {{Option|INLINELIMIT}} to <code>0</code>. |
− | |- | + | |- valign="top" |
| '''Properties''' | | '''Properties''' | ||
− | | | + | |The function is ''nondeterministic'': evaluation order will be preserved by the compiler. |
+ | |- valign="top" | ||
+ | | '''Examples''' | ||
+ | | | ||
+ | * {{Code|for $x in 1 to 2 return ($x, prof:variables())}} will dump the values of <code>$x</code> to standard error. | ||
+ | |} | ||
+ | |||
+ | ==prof:type== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- valign="top" | ||
+ | | width='120' | '''Signature''' | ||
+ | |<pre>prof:type( | ||
+ | $expr as item()* | ||
+ | ) as item()*</pre> | ||
+ | |- valign="top" | ||
+ | | '''Summary''' | ||
+ | |Similar to {{Code|fn:trace($expr, $msg)}}, but instead of a user-defined message, it emits the compile-time type and estimated result size of its argument. | ||
+ | |} | ||
+ | |||
+ | ==prof:gc== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- valign="top" | ||
+ | | width='120' | '''Signature''' | ||
+ | |<pre>prof:gc( | ||
+ | $count as xs:integer := () | ||
+ | ) as empty-sequence()</pre> | ||
+ | |- valign="top" | ||
+ | | '''Summary''' | ||
+ | |Enforces Java garbage collection. If no {{Code|$count}} is supplied, garbage will be collected once. Please note that this function should only be used for debugging purposes; in productive code, it is best to trust the garbage collecting strategies of Java. | ||
|} | |} | ||
− | ==prof: | + | ==prof:runtime== |
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:runtime( |
− | |- | + | $option as xs:string |
+ | ) as xs:integer</pre> | ||
+ | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
− | |Returns the | + | |Returns the value of the specified runtime {{Code|$option}}. The following options are available: |
− | |- | + | * {{Code|max}}: Maximum memory that the Java virtual machine will attempt to use. |
− | | ''' | + | * {{Code|total}}: Total memory in the Java virtual machine (varies over time). |
− | | | + | * {{Code|used}}: Currently used memory (varies over time, will shrink after garbage collection). |
+ | * {{Code|processors}}: number of processors available to the Java virtual machine. | ||
+ | |- valign="top" | ||
+ | |{{Code|option}} | ||
+ | |The specified option is unknown. | ||
+ | |- valign="top" | ||
+ | | '''Examples''' | ||
+ | | | ||
+ | * <code>prof:gc(3), prof:human(prof:runtime('used'))</code> performs some garbage collection and returns the currently used amount of memory in a user-friendly format. | ||
|} | |} | ||
+ | |||
+ | =Helper Functions= | ||
==prof:void== | ==prof:void== | ||
− | |||
− | |||
{| width='100%' | {| width='100%' | ||
− | |- | + | |- valign="top" |
− | | width=' | + | | width='120' | '''Signature''' |
− | | | + | |<pre>prof:void( |
− | |- | + | $input as item()* |
+ | ) as empty-sequence()</pre> | ||
+ | |- valign="top" | ||
| '''Summary''' | | '''Summary''' | ||
− | | | + | |Absorbs the specified {{Code|$input}} and returns an empty sequence. This function is helpful if some (often nondeterministic or side-effecting) code needs to be evaluated and if the resulting value is not required. |
− | |- | + | |- valign="top" |
| '''Properties''' | | '''Properties''' | ||
− | |The function is '' | + | |The function is ''nondeterministic'': evaluation order will be preserved by the compiler. |
− | |- | + | |- valign="top" |
| '''Examples''' | | '''Examples''' | ||
| | | | ||
* {{Code|prof:void(fetch:binary('http://my.rest.service'))}} performs an HTTP request and ignores the result. | * {{Code|prof:void(fetch:binary('http://my.rest.service'))}} performs an HTTP request and ignores the result. | ||
+ | |} | ||
+ | |||
+ | ==prof:sleep== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- valign="top" | ||
+ | | width='120' | '''Signature''' | ||
+ | |<pre>prof:sleep( | ||
+ | $ms as xs:integer | ||
+ | ) as empty-sequence()</pre> | ||
+ | |- valign="top" | ||
+ | | '''Summary''' | ||
+ | |Sleeps for the specified number of milliseconds. | ||
+ | |- valign="top" | ||
+ | | '''Properties''' | ||
+ | |The function is ''nondeterministic'': evaluation order will be preserved by the compiler. | ||
+ | |} | ||
+ | |||
+ | ==prof:human== | ||
+ | |||
+ | {| width='100%' | ||
+ | |- valign="top" | ||
+ | | width='120' | '''Signature''' | ||
+ | |<pre>prof:human( | ||
+ | $number as xs:integer | ||
+ | ) as xs:string</pre> | ||
+ | |- valign="top" | ||
+ | | '''Summary''' | ||
+ | |Returns a human-readable representation of the specified {{Code|$number}}. | ||
+ | |- valign="top" | ||
+ | | '''Example''' | ||
+ | | | ||
+ | * {{Code|prof:human(16384)}} returns {{Code|16K}}. | ||
+ | |} | ||
+ | |||
+ | =Errors= | ||
+ | |||
+ | {| class="wikitable" width="100%" | ||
+ | ! width="110"|Code | ||
+ | |Description | ||
+ | |- valign="top" | ||
+ | |{{Code|option}} | ||
+ | |The specified option is unknown. | ||
|} | |} | ||
=Changelog= | =Changelog= | ||
+ | |||
+ | ;Version 9.2 | ||
+ | |||
+ | * Added: {{Function||prof:gc}}, {{Function||prof:runtime}} | ||
+ | * Updated: {{Function||prof:track}}: decimal timing results; by default no memory profiling | ||
+ | |||
+ | ;Version 9.0 | ||
+ | |||
+ | * Added: {{Function||prof:track}} | ||
+ | * Updated: renamed prof:mem to {{Function||prof:memory}}, {{Function||prof:time}}: <code>$cache</code> argument removed | ||
+ | |||
+ | ;Version 8.5 | ||
+ | |||
+ | * Added: {{Function||prof:type}} (moved from [[XQuery Module]]) | ||
+ | |||
+ | ;Version 8.1 | ||
+ | |||
+ | * Added: {{Function||prof:variables}} | ||
;Version 7.7 | ;Version 7.7 | ||
− | * Added: | + | * Added: {{Function||prof:void}} |
;Version 7.6 | ;Version 7.6 | ||
− | * Added: | + | * Added: {{Function||prof:human}} |
;Version 7.5 | ;Version 7.5 | ||
− | * Added: | + | * Added: {{Function||prof:dump}}, {{Function||prof:current-ms}}, {{Function||prof:current-ns}} |
This module was introduced with Version 7.3. | This module was introduced with Version 7.3. | ||
− | |||
− |
Latest revision as of 18:37, 1 December 2023
This XQuery Module contains various functions to test and profile code, and to dump information to standard output.
Contents
Conventions[edit]
All functions and errors in this module are assigned to the http://basex.org/modules/prof
namespace, which is statically bound to the prof
prefix.
Performance Functions[edit]
prof:track[edit]
Signature | prof:track( $input as item()*, $options as map(*)? := map { } ) as item()* |
Summary | Measures the execution time and memory consumption required for evaluating the specified $input and returns a map with the results. The following $options are available:
Helpful notes:
|
Properties | The function is nondeterministic: evaluation order will be preserved by the compiler. |
Examples |
prof:track(fetch:doc('factbook.xml'))?memory
=> prof:human()
map {
"memory": 21548400,
"value": (2, 4, 6, 8, 10, ...)
}
|
prof:time[edit]
Signature | prof:time( $input as item(), $label as xs:string? := () ) as item()* |
Summary | Measures the time needed to evaluate $input and outputs a string to standard error or, if the GUI is used, to the Info View. An optional $label may be specified to tag the profiling result. See prof:track for further notes.
|
Properties | The function is nondeterministic: evaluation order will be preserved by the compiler. |
Examples |
|
prof:memory[edit]
Signature | prof:memory( $input as item(), $label as xs:string? := () ) as item()* |
Summary | Measures the memory allocated by evaluating $input and outputs a string to standard error or, if the GUI is used, to the Info View. An optional $label may be specified to tag the profiling result. See prof:track for further notes.
|
Properties | The function is nondeterministic: evaluation order will be preserved by the compiler. |
Examples |
|
prof:current-ms[edit]
Signature | prof:current-ms() as xs:integer |
Summary | Returns the number of milliseconds passed since 1970/01/01 UTC. The granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds. |
Properties | In contrast to fn:current-time() , the function is nondeterministic and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
|
Examples |
|
prof:current-ns[edit]
Signature | prof:current-ns() as xs:integer |
Summary | Returns the current value of the most precise available system timer in nanoseconds. |
Properties | In contrast to fn:current-time() , the function is nondeterministic and returns different values every time it is called. Its evaluation order will be preserved by the compiler.
|
Examples | Measures the time of an expression:
let $ns1 := prof:current-ns()
return (
(: process to measure :)
(1 to 1000000)[. = 0],
let $ns2 := prof:current-ns()
let $ms := ((($ns2 - $ns1) idiv 10000) div 100)
return $ms || ' ms'
)
|
Debugging Functions[edit]
prof:dump[edit]
Signature | prof:dump( $input as item()*, $label as xs:string? := () ) as empty-sequence() |
Summary | Dumps a serialized representation of $input to STDERR , optionally prefixed with $label , and returns an empty sequence. If the GUI is used, the dumped result is shown in the Info View.
|
Properties | In contrast to fn:trace() , the consumed expression will not be passed on.
|
prof:variables[edit]
Signature | prof:variables() as empty-sequence() |
Summary | Prints a list of all current local and global variable assignments to standard error or, if the GUI is used, to the Info View. As every query is optimized before being evaluated, not all of the original variables may be visible in the output. Moreover, many variables of function calls will disappear because functions are inlined. Function inlining can be turned off by setting INLINELIMIT to 0 .
|
Properties | The function is nondeterministic: evaluation order will be preserved by the compiler. |
Examples |
|
prof:type[edit]
Signature | prof:type( $expr as item()* ) as item()* |
Summary | Similar to fn:trace($expr, $msg) , but instead of a user-defined message, it emits the compile-time type and estimated result size of its argument.
|
prof:gc[edit]
Signature | prof:gc( $count as xs:integer := () ) as empty-sequence() |
Summary | Enforces Java garbage collection. If no $count is supplied, garbage will be collected once. Please note that this function should only be used for debugging purposes; in productive code, it is best to trust the garbage collecting strategies of Java.
|
prof:runtime[edit]
Signature | prof:runtime( $option as xs:string ) as xs:integer |
Summary | Returns the value of the specified runtime $option . The following options are available:
|
option
|
The specified option is unknown. |
Examples |
|
Helper Functions[edit]
prof:void[edit]
Signature | prof:void( $input as item()* ) as empty-sequence() |
Summary | Absorbs the specified $input and returns an empty sequence. This function is helpful if some (often nondeterministic or side-effecting) code needs to be evaluated and if the resulting value is not required.
|
Properties | The function is nondeterministic: evaluation order will be preserved by the compiler. |
Examples |
|
prof:sleep[edit]
Signature | prof:sleep( $ms as xs:integer ) as empty-sequence() |
Summary | Sleeps for the specified number of milliseconds. |
Properties | The function is nondeterministic: evaluation order will be preserved by the compiler. |
prof:human[edit]
Signature | prof:human( $number as xs:integer ) as xs:string |
Summary | Returns a human-readable representation of the specified $number .
|
Example |
|
Errors[edit]
Code | Description |
---|---|
option
|
The specified option is unknown. |
Changelog[edit]
- Version 9.2
- Added:
prof:gc
,prof:runtime
- Updated:
prof:track
: decimal timing results; by default no memory profiling
- Version 9.0
- Added:
prof:track
- Updated: renamed prof:mem to
prof:memory
,prof:time
:$cache
argument removed
- Version 8.5
- Added:
prof:type
(moved from XQuery Module)
- Version 8.1
- Added:
prof:variables
- Version 7.7
- Added:
prof:void
- Version 7.6
- Added:
prof:human
- Version 7.5
- Added:
prof:dump
,prof:current-ms
,prof:current-ns
This module was introduced with Version 7.3.