base/bif/top-k.bif.bro

GLOBAL

Functions to probabilistically determine top-k elements.

Namespace:GLOBAL Source File:/scripts/base/bif/top-k.bif.bro

Summary

Functions

topk_add: function	Add a new observed object to the data structure.
topk_count: function	Get an overestimated count of how often a value has been encountered.
topk_epsilon: function	Get the maximal overestimation for count.
topk_get_top: function	Get the first k elements of the top-k data structure.
topk_init: function	Creates a top-k data structure which tracks size elements.
topk_merge: function	Merge the second top-k data structure into the first.
topk_merge_prune: function	Merge the second top-k data structure into the first and prunes the final data structure back to the size given on initialization.
topk_size: function	Get the number of elements this data structure is supposed to track (given on init).
topk_sum: function	Get the sum of all counts of all elements in the data structure.

Detailed Interface

Functions

topk_add

Type:function (handle: opaque of topk, value: any) : any

Add a new observed object to the data structure.

Note

The first added object sets the type of data tracked by the top-k data structure. All following values have to be of the same type.

Handle:the TopK handle. Value:observed value.

See also: topk_init, topk_get_top, topk_count, topk_epsilon, topk_size, topk_sum, topk_merge, topk_merge_prune

topk_count

Type:function (handle: opaque of topk, value: any) : count

Get an overestimated count of how often a value has been encountered.

Note

The value has to be part of the currently tracked elements, otherwise 0 will be returned and an error message will be added to reporter.

Handle:the TopK handle. Value:Value to look up count for. Returns:Overestimated number for how often the element has been encountered.

See also: topk_init, topk_add, topk_get_top, topk_epsilon, topk_size, topk_sum, topk_merge, topk_merge_prune

topk_epsilon

Type:function (handle: opaque of topk, value: any) : count

Get the maximal overestimation for count.

Note

Same restrictions as for topk_count apply.

Handle:the TopK handle. Value:Value to look up epsilon for. Returns:Number which represents the maximal overestimation for the count of this element.

See also: topk_init, topk_add, topk_get_top, topk_count, topk_size, topk_sum, topk_merge, topk_merge_prune

topk_get_top

Type:function (handle: opaque of topk, k: count) : any_vec

Get the first k elements of the top-k data structure.

Handle:the TopK handle. K:number of elements to return. Returns:vector of the first k elements.

See also: topk_init, topk_add, topk_count, topk_epsilon, topk_size, topk_sum, topk_merge, topk_merge_prune

topk_init

Type:function (size: count) : opaque of topk

Creates a top-k data structure which tracks size elements.

Size:number of elements to track. Returns:Opaque pointer to the data structure.

See also: topk_add, topk_get_top, topk_count, topk_epsilon, topk_size, topk_sum, topk_merge, topk_merge_prune

topk_merge

Type:function (handle1: opaque of topk, handle2: opaque of topk) : any

Merge the second top-k data structure into the first.

Handle1:the first TopK handle. Handle2:the second TopK handle.

Note

This does not remove any elements, the resulting data structure can be bigger than the maximum size given on initialization.

See also: topk_init, topk_add, topk_get_top, topk_count, topk_epsilon, topk_size, topk_sum, topk_merge_prune

topk_merge_prune

Type:function (handle1: opaque of topk, handle2: opaque of topk) : any

Merge the second top-k data structure into the first and prunes the final data structure back to the size given on initialization.

Note

Use with care and only when being aware of the restrictions this entails. Do not call topk_size or topk_add afterwards, results will probably not be what you expect.

Handle1:the TopK handle in which the second TopK structure is merged. Handle2:the TopK handle in which is merged into the first TopK structure.

See also: topk_init, topk_add, topk_get_top, topk_count, topk_epsilon, topk_size, topk_sum, topk_merge

topk_size

Type:function (handle: opaque of topk) : count

Get the number of elements this data structure is supposed to track (given on init).

Note

Note that the actual number of elements in the data structure can be lower or higher (due to non-pruned merges) than this.

Handle:the TopK handle. Returns:size given during initialization.

See also: topk_init, topk_add, topk_get_top, topk_count, topk_epsilon, topk_sum, topk_merge, topk_merge_prune

topk_sum

Type:function (handle: opaque of topk) : count

Get the sum of all counts of all elements in the data structure.

Note

This is equal to the number of all inserted objects if the data structure never has been pruned. Do not use after calling topk_merge_prune (will throw a warning message if used afterwards).

Handle:the TopK handle. Returns:sum of all counts.

See also: topk_init, topk_add, topk_get_top, topk_count, topk_epsilon, topk_size, topk_merge, topk_merge_prune