Storm Libraries

Storm Libraries represent powerful tools available inside of the Storm query language.

$lib

The Base Storm Library. This mainly contains utility functionality.

$lib.cast(name, valu)

Normalize a value as a Synapse Data Model Type.

Args:

name (str): The name of the model type to normalize the value as.

valu (any): The value to normalize.

Returns:

The normalized value. The return type is prim.

$lib.copy(item)

Create and return a deep copy of the given storm object.

Note:

This is currently limited to msgpack compatible primitives.

Examples:

Make a copy of a list or dict:

$copy = $lib.copy($item)

Args:

item (prim): The item to make a copy of.

Returns:

A deep copy of the primitive object. The return type is prim.

$lib.debug

True if the current runtime has debugging enabled.

Note:

The debug state is inherited by sub-runtimes at instantiation time. Any changes to a runtime’s debug state do not percolate automatically.

Examples:

Check if the runtime is in debug and print a message:

if $lib.debug {
    $lib.print('Doing stuff!')
}

Update the current runtime to enable debugging:

$lib.debug = $lib.true

Returns:

The return type is boolean. When this is used to set the value, it does not have a return type.

$lib.exit(mesg=$lib.null, **kwargs)

Cause a Storm Runtime to stop running.

Args:

mesg (str): Optional string to warn.

**kwargs (any): Keyword arguments to substitute into the mesg.

Returns:

The return type is null.

$lib.false

This constant represents a value of False that can be used in Storm.

Examples:

Conditionally print a statement based on the constant value:

cli> storm if $lib.false { $lib.print('Is True') } else { $lib.print('Is False') }
Is False

Returns:

The type is boolean.

$lib.fire(name, **info)

Fire an event onto the runtime.

Notes:

This fires events as storm:fire event types. The name of the event is placed into a type key, and any additional keyword arguments are added to a dictionary under the data key.

Examples:

Fire an event called demo with some data:

cli> storm $foo='bar' $lib.fire('demo', foo=$foo, knight='ni')
...
('storm:fire', {'type': 'demo', 'data': {'foo': 'bar', 'knight': 'ni'}})
...

Args:

name (str): The name of the event to fire.

**info (any): Additional keyword arguments containing data to add to the event.

Returns:

The return type is null.

$lib.guid(*args, valu=$lib.undef)

Get a random guid, or generate a guid from the arguments.

Args:

*args (prim): Arguments which are hashed to create a guid.

valu (prim): Create a guid from a single value (no positional arguments can be specified).

Returns:

A guid. The return type is str.

$lib.import(name, debug=$lib.false, reqvers=$lib.null)

Import a Storm module.

Args:

name (str): Name of the module to import.

debug (boolean): Enable debugging in the module.

reqvers (str): Version requirement for the imported module.

Returns:

A lib instance representing the imported package. The return type is lib.

$lib.len(item)

Get the length of a item.

This could represent the size of a string, or the number of keys in a dictionary, or the number of elements in an array. It may also be used to iterate an emitter or yield function and count the total.

Args:: item (prim): The item to get the length of.
Returns:: The length of the item. The return type is int.

$lib.list(*vals)

Get a Storm List object.

Args:: *vals (any): Initial values to place in the list.
Returns:: A new list object. The return type is list.

$lib.max(*args)

Get the maximum value in a list of arguments.

Args:: *args (any): List of arguments to evaluate.
Returns:: The largest argument. The return type is int.

$lib.min(*args)

Get the minimum value in a list of arguments.

Args:: *args (any): List of arguments to evaluate.
Returns:: The smallest argument. The return type is int.

$lib.null

This constant represents a value of None that can be used in Storm.

Examples:

Create a dictionary object with a key whose value is null, and call $lib.fire() with it:

cli> storm $d=({"key": $lib.null}) $lib.fire('demo', d=$d)
('storm:fire', {'type': 'demo', 'data': {'d': {'key': None}}})

Returns:

The type is null.

$lib.pprint(item, prefix=, clamp=$lib.null)

The pprint API should not be considered a stable interface.

Args:

item (any): Item to pprint

prefix (str): Line prefix.

clamp (int): Line clamping length.

Returns:

The return type is null.

$lib.print(mesg, **kwargs)

Print a message to the runtime.

Examples:

Print a simple string:

cli> storm $lib.print("Hello world!")
Hello world!

Format and print string based on variables:

cli> storm $d=({"key1": (1), "key2": "two"})
     for ($key, $value) in $d { $lib.print('{k} => {v}', k=$key, v=$value) }
key1 => 1
key2 => two

Use values off of a node to format and print string:

cli> storm inet:ipv4:asn
     $lib.print("node: {ndef}, asn: {asn}", ndef=$node.ndef(), asn=:asn) | spin
node: ('inet:ipv4', 16909060), asn: 1138

Notes:

Arbitrary objects can be printed as well. They will have their Python __repr()__ printed.

Args:

mesg (str): String to print.

**kwargs (any): Keyword arguments to substitute into the mesg.

Returns:

The return type is null.

$lib.raise(name, mesg, **info)

Raise an exception in the storm runtime.

Args:

name (str): The name of the error condition to raise.

mesg (str): A friendly description of the specific error.

**info (any): Additional metadata to include in the exception.

Returns:

This function does not return. The return type is null.

$lib.range(stop, start=$lib.null, step=$lib.null)

Generate a range of integers.

Examples:

Generate a sequence of integers based on the size of an array:

cli> storm $a=(foo,bar,(2)) for $i in $lib.range($lib.len($a)) {$lib.fire('test', indx=$i, valu=$a.$i)}
Executing query at 2021/03/22 19:25:48.835
('storm:fire', {'type': 'test', 'data': {'index': 0, 'valu': 'foo'}})
('storm:fire', {'type': 'test', 'data': {'index': 1, 'valu': 'bar'}})
('storm:fire', {'type': 'test', 'data': {'index': 2, 'valu': 2}})

Notes:

The range behavior is the same as the Python3 range() builtin Sequence type.

Args:

stop (int): The value to stop at.

start (int): The value to start at.

step (int): The range step size.

Yields:

The sequence of integers. The return type is int.

$lib.repr(name, valu)

Attempt to convert a system mode value to a display mode string.

Examples:

Print the Synapse user name for an iden:

$lib.print($lib.repr(syn:user, $iden))

Args:

name (str): The name of the model type.

valu (any): The value to convert.

Returns:

A display mode representation of the value. The return type is str.

$lib.set(*vals)

Get a Storm Set object.

Args:: *vals (any): Initial values to place in the set.
Returns:: The new set. The return type is set.

$lib.sorted(valu, reverse=$lib.false)

Yield sorted values.

Args:

valu (any): An iterable object to sort.

reverse (boolean): Reverse the sort order.

Yields:

Yields the sorted output. The return type is any.

$lib.text(*args)

Get a Storm Text object.

Args:: *args (str): An initial set of values to place in the Text. These values are joined together with an empty string.
Returns:: The new Text object. The return type is text.

$lib.true

This constant represents a value of True that can be used in Storm.

Examples:

Conditionally print a statement based on the constant value:

cli> storm if $lib.true { $lib.print('Is True') } else { $lib.print('Is False') }
Is True

Returns:

The type is boolean.

$lib.trycast(name, valu)

Attempt to normalize a value and return status and the normalized value.

Examples:

Do something if the value is a valid IPV4:

($ok, $ipv4) = $lib.trycast(inet:ipv4, 1.2.3.4)
if $ok { $dostuff($ipv4) }

Args:

name (str): The name of the model type to normalize the value as.

valu (any): The value to normalize.

Returns:

A list of (<bool>, <prim>) for status and normalized value. The return type is list.

$lib.undef

This constant can be used to unset variables and derefs.

Examples:

Unset the variable $foo:

$foo = $lib.undef

Remove a dictionary key bar:

$foo.bar = $lib.undef

Remove a list index of 0:

$foo.0 = $lib.undef

Returns:

The type is undef.

$lib.warn(mesg, **kwargs)

Print a warning message to the runtime.

Notes:

Arbitrary objects can be warned as well. They will have their Python __repr()__ printed.

Args:

mesg (str): String to warn.

**kwargs (any): Keyword arguments to substitute into the mesg.

Returns:

The return type is null.

$lib.aha

A Storm Library for interacting with AHA.

$lib.aha.del(svcname)

Delete a service from AHA.

Examples:
Deleting a service with its relative name:
$lib.aha.del(00.mysvc...)
Deleting a service with its full name:
$lib.aha.del(00.mysvc.loop.vertex.link)

Args:: svcname (str): The name of the service to delete. It is easiest to use the relative name of a service, ending with “…”.
Returns:: The return type is null.

$lib.aha.get(svcname, filters=$lib.null)

Get information about an AHA service.

Examples:
Getting service information with a relative name:
$lib.aha.get(00.cortex...)
Getting service information with its full name:
$lib.aha.get(00.cortex.loop.vertex.link)

Args:

svcname (str): The name of the AHA service to look up. It is easiest to use the relative name of a service, ending with “…”.

filters (dict): An optional dictionary of filters to use when resolving the AHA service.

Returns:

The AHA service information dictionary, or $lib.null. The return type may be one of the following: null, dict.

$lib.aha.list()

Enumerate all of the AHA services.

Yields:: The AHA service dictionaries. The return type is list.

$lib.aha.pool

A Storm Library for interacting with AHA service pools.

$lib.aha.pool.add(name)

Add a new AHA service pool.

Examples:
Add a pool via its relative name:
$lib.aha.pool.add(pool00.cortex...)

Args:: name (str): The name of the pool to add. It is easiest to use the relative name of a pool, ending with “…”.
Returns:: The return type is aha:pool.

$lib.aha.pool.del(name)

Delete an existing AHA service pool.

Examples:
Delete a pool via its relative name:
$lib.aha.pool.del(pool00.cortex...)

Args:: name (str): The name of the pool to delete. It is easiest to use the relative name of a pool, ending with “…”.
Returns:: The AHA pool definition that was deleted. The return type is dict.

$lib.aha.pool.get(name)

Get an existing AHA service pool.

Args:: name (str): The name of the pool to get. It is easiest to use the relative name of a pool, ending with “…”.
Returns:: The pool if it exists, or $lib.null. The return type may be one of the following: null, aha:pool.

$lib.aha.pool.list()

Enumerate all of the AHA service pools.

Yields:: The return type is aha:pool.

$lib.auth

A Storm Library for interacting with Auth in the Cortex.

$lib.auth.getPermDef(perm)

Return a single permission definition.

Args:: perm (list): A permission tuple.
Returns:: A permission definition or null. The return type is dict.

$lib.auth.getPermDefs()

Return a list of permission definitions.

Returns:: The list of permission definitions. The return type is list.

$lib.auth.ruleFromText(text)

Get a rule tuple from a text string.

Args:: text (str): The string to process.
Returns:: A tuple containing a bool and a list of permission parts. The return type is list.

$lib.auth.textFromRule(rule)

Return a text string from a rule tuple.

Args:: rule (list): A rule tuple.
Returns:: The rule text. The return type is str.

$lib.auth.easyperm

A Storm Library for interacting with easy perm dictionaries.

$lib.auth.easyperm.allowed(edef, level)

Check if the current user has a permission level in an easy perm dictionary.

Args:

edef (dict): The easy perm dictionary to check.

level (int): The required permission level number.

Returns:

True if the user meets the requirement, false otherwise. The return type is boolean.

$lib.auth.easyperm.confirm(edef, level, mesg=$lib.null)

Require that the current user has a permission level in an easy perm dictionary.

Args:

edef (dict): The easy perm dictionary to check.

level (int): The required permission level number.

mesg (str): Optional error message to present if user does not have required permission level.

Returns:

The return type is null.

$lib.auth.easyperm.init(edef=$lib.null, default=(1))

Add the easy perm structure to a new or existing dictionary.

Note:

The current user will be given admin permission in the new easy perm structure.

Args:

edef (dict): A dictionary to add easy perms to.

default (int): Specify the default permission level for this item.

Returns:

Dictionary with the easy perm structure. The return type is dict.

$lib.auth.easyperm.level.admin

Constant for admin permission.

Returns:: The type is int.

$lib.auth.easyperm.level.deny

Constant for deny permission.

Returns:: The type is int.

$lib.auth.easyperm.level.edit

Constant for edit permission.

Returns:: The type is int.

$lib.auth.easyperm.level.read

Constant for read permission.

Returns:: The type is int.

$lib.auth.easyperm.set(edef, scope, iden, level)

Set the permission level for a user or role in an easy perm dictionary.

Args:

edef (dict): The easy perm dictionary to modify.

scope (str): The scope, either “users” or “roles”.

iden (str): The user/role iden depending on scope.

level (int): The permission level number, or None to remove the permission.

Returns:

Dictionary with the updated easy perm structure. The return type is dict.

$lib.auth.gates

A Storm Library for interacting with Auth Gates in the Cortex.

$lib.auth.gates.get(iden)

Get a specific Gate by iden.

Args:: iden (str): The iden of the gate to retrieve.
Returns:: The auth:gate if it exists, otherwise null. The return type may be one of the following: null, auth:gate.

$lib.auth.gates.list()

Get a list of Gates in the Cortex.

Returns:: A list of auth:gate objects. The return type is list.

$lib.auth.roles

A Storm Library for interacting with Auth Roles in the Cortex.

$lib.auth.roles.add(name, iden=$lib.null)

Add a Role to the Cortex.

Args:

name (str): The name of the role.

iden (str): The iden to assign to the new role.

Returns:

The new role object. The return type is auth:role.

$lib.auth.roles.byname(name)

Get a specific Role by name.

Args:: name (str): The name of the role to retrieve.
Returns:: The role by name, or null if it does not exist. The return type may be one of the following: null, auth:role.

$lib.auth.roles.del(iden)

Delete a Role from the Cortex.

Args:: iden (str): The iden of the role to delete.
Returns:: The return type is null.

$lib.auth.roles.get(iden)

Get a specific Role by iden.

Args:: iden (str): The iden of the role to retrieve.
Returns:: The auth:role object; or null if the role does not exist. The return type may be one of the following: null, auth:role.

$lib.auth.roles.list()

Get a list of Roles in the Cortex.

Returns:: A list of auth:role objects. The return type is list.

$lib.auth.users

A Storm Library for interacting with Auth Users in the Cortex.

$lib.auth.users.add(name, passwd=$lib.null, email=$lib.null, iden=$lib.null)

Add a User to the Cortex.

Args:

name (str): The name of the user.

passwd (str): The user’s password.

email (str): The user’s email address.

iden (str): The iden to use to create the user.

Returns:

The auth:user object for the new user. The return type is auth:user.

$lib.auth.users.byname(name)

Get a specific user by name.

Args:: name (str): The name of the user to retrieve.
Returns:: The auth:user object, or none if the user does not exist. The return type may be one of the following: null, auth:user.

$lib.auth.users.del(iden)

Delete a User from the Cortex.

Args:: iden (str): The iden of the user to delete.
Returns:: The return type is null.

$lib.auth.users.get(iden)

Get a specific User by iden.

Args:: iden (str): The iden of the user to retrieve.
Returns:: The auth:user object, or none if the user does not exist. The return type may be one of the following: null, auth:user.

$lib.auth.users.list()

Get a list of Users in the Cortex.

Returns:: A list of auth:user objects. The return type is list.

$lib.axon

A Storm library for interacting with the Cortex’s Axon.

For APIs that accept an ssl_opts argument, the dictionary may contain the following values:

{
    'verify': <bool> - Perform SSL/TLS verification. Is overridden by the ssl argument.
    'client_cert': <str> - PEM encoded full chain certificate for use in mTLS.
    'client_key': <str> - PEM encoded key for use in mTLS. Alternatively, can be included in client_cert.
}

$lib.axon.csvrows(sha256, dialect=excel, errors=ignore, **fmtparams)

Yields CSV rows from a CSV file stored in the Axon.

Notes:

The dialect and fmtparams expose the Python csv.reader() parameters.

Example:

Get the rows from a given csv file:

for $row in $lib.axon.csvrows($sha256) {
    $dostuff($row)
}

Get the rows from a given tab separated file:

for $row in $lib.axon.csvrows($sha256, delimiter="\t") {
    $dostuff($row)
}

Args:

sha256 (str): The SHA256 hash of the file.

dialect (str): The default CSV dialect to use.

errors (str): Specify how encoding errors should handled.

**fmtparams (any): Format arguments.

Yields:

A list of strings from the CSV file. The return type is list.

$lib.axon.del(sha256)

Remove the bytes from the Cortex’s Axon by sha256.

Example:

Delete files from the axon based on a tag:

file:bytes#foo +:sha256 $lib.axon.del(:sha256)

Args:

sha256 (hash:sha256): The sha256 of the bytes to remove from the Axon.

Returns:

True if the bytes were found and removed. The return type is boolean.

$lib.axon.dels(sha256s)

Remove multiple byte blobs from the Cortex’s Axon by a list of sha256 hashes.

Example:

Delete a list of files (by hash) from the Axon:

$list = ($hash0, $hash1, $hash2)
$lib.axon.dels($list)

Args:

sha256s (list): A list of sha256 hashes to remove from the Axon.

Returns:

A list of boolean values that are True if the bytes were found. The return type is list.

$lib.axon.has(sha256)

Check if the Axon the Cortex is configured to use has a given sha256 value.

Examples:

Check if the Axon has a given file:

# This example assumes the Axon does have the bytes
cli> storm if $lib.axon.has(9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08) {
        $lib.print("Has bytes")
    } else {
        $lib.print("Does not have bytes")
    }

Has bytes

Args:

sha256 (str): The sha256 value to check.

Returns:

True if the Axon has the file, false if it does not. The return type is boolean.

$lib.axon.hashset(sha256)

Return additional hashes of the bytes stored in the Axon for the given sha256.

Examples:

Get the md5 hash for a file given a variable named $sha256:

$hashset = $lib.axon.hashset($sha256)
$md5 = $hashset.md5

Args:

sha256 (str): The sha256 value to calculate hashes for.

Returns:

A dictionary of additional hashes. The return type is dict.

$lib.axon.jsonlines(sha256, errors=ignore)

Yields JSON objects from a JSON-lines file stored in the Axon.

Example:

Get the JSON objects from a given JSONL file:

for $item in $lib.axon.jsonlines($sha256) {
    $dostuff($item)
}

Args:

sha256 (str): The SHA256 hash of the file.

errors (str): Specify how encoding errors should handled.

Yields:

A JSON object parsed from a line of text. The return type is any.

$lib.axon.list(offs=(0), wait=$lib.false, timeout=$lib.null)

List (offset, sha256, size) tuples for files in the Axon in added order.

Example:

List files:

for ($offs, $sha256, $size) in $lib.axon.list() {
    $lib.print($sha256)
}

Start list from offset 10:

for ($offs, $sha256, $size) in $lib.axon.list(10) {
    $lib.print($sha256)
}

Args:

offs (int): The offset to start from.

wait (boolean): Wait for new results and yield them in realtime.

timeout (int): The maximum time to wait for a new result before returning.

Yields:

Tuple of (offset, sha256, size) in added order. The return type is list.

$lib.axon.metrics()

Get runtime metrics of the Axon.

Example:

Print the total number of files stored in the Axon:

$data = $lib.axon.metrics()
$lib.print("The Axon has {n} files", n=$data."file:count")

Returns:

A dictionary containing runtime data about the Axon. The return type is dict.

$lib.axon.put(byts)

Save the given bytes variable to the Axon the Cortex is configured to use.

Examples:

Save a base64 encoded buffer to the Axon:

cli> storm $s='dGVzdA==' $buf=$lib.base64.decode($s) ($size, $sha256)=$lib.axon.put($buf)
     $lib.print('size={size} sha256={sha256}', size=$size, sha256=$sha256)

size=4 sha256=9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

Args:

byts (bytes): The bytes to save.

Returns:

A tuple of the file size and sha256 value. The return type is list.

$lib.axon.readlines(sha256, errors=ignore)

Yields lines of text from a plain-text file stored in the Axon.

Examples:

// Get the lines for a given file. for $line in $lib.axon.readlines($sha256) {

$dostuff($line)

}

Args:

sha256 (str): The SHA256 hash of the file.

errors (str): Specify how encoding errors should handled.

Yields:

A line of text from the file. The return type is str.

$lib.axon.size(sha256)

Return the size of the bytes stored in the Axon for the given sha256.

Examples:

Get the size for a file given a variable named $sha256:

$size = $lib.axon.size($sha256)

Args:

sha256 (str): The sha256 value to check.

Returns:

The size of the file or null if the file is not found. The return type may be one of the following: int, null.

$lib.axon.upload(genr)

Upload a stream of bytes to the Axon as a file.

Examples:

Upload bytes from a generator:

($size, $sha256) = $lib.axon.upload($getBytesChunks())

Args:

genr (generator): A generator which yields bytes.

Returns:

A tuple of the file size and sha256 value. The return type is list.

$lib.axon.urlfile(*args, **kwargs)

Retrieve the target URL using the wget() function and construct an inet:urlfile node from the response.

Notes:

This accepts the same arguments as $lib.axon.wget().

Args:

*args (any): Args from $lib.axon.wget().

**kwargs (any): Args from $lib.axon.wget().

Returns:

The inet:urlfile node on success, null on error. The return type may be one of the following: node, null.

$lib.axon.wget(url, headers=$lib.null, params=$lib.null, method=GET, json=$lib.null, body=$lib.null, ssl=$lib.true, timeout=$lib.null, proxy=$lib.null, ssl_opts=$lib.null)

A method to download an HTTP(S) resource into the Cortex’s Axon.

Notes:

The response body will be stored regardless of the status code. See the Axon.wget() API documentation to see the complete structure of the response dictionary.

Example:

Get the Vertex Project website:

$headers = ({})
$headers."User-Agent" = Foo/Bar

$resp = $lib.axon.wget("http://vertex.link", method=GET, headers=$headers)
if $resp.ok { $lib.print("Downloaded: {size} bytes", size=$resp.size) }

Args:

url (str): The URL to download

headers (dict): An optional dictionary of HTTP headers to send.

params (dict): An optional dictionary of URL parameters to add.

method (str): The HTTP method to use.

json (dict): A JSON object to send as the body.

body (bytes): Bytes to send as the body.

ssl (boolean): Set to False to disable SSL/TLS certificate verification.

timeout (int): Timeout for the download operation.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: boolean, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.axon help for additional details.

Returns:

A status dictionary of metadata. The return type is dict.

$lib.axon.wput(sha256, url, headers=$lib.null, params=$lib.null, method=PUT, ssl=$lib.true, timeout=$lib.null, proxy=$lib.null, ssl_opts=$lib.null)

A method to upload a blob from the axon to an HTTP(S) endpoint.

Args:

sha256 (str): The sha256 of the file blob to upload.

url (str): The URL to upload the file to.

headers (dict): An optional dictionary of HTTP headers to send.

params (dict): An optional dictionary of URL parameters to add.

method (str): The HTTP method to use.

ssl (boolean): Set to False to disable SSL/TLS certificate verification.

timeout (int): Timeout for the download operation.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: boolean, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.axon help for additional details.

Returns:

A status dictionary of metadata. The return type is dict.

$lib.backup

A Storm Library for interacting with the backup APIs in the Cortex.

$lib.backup.del(name)

Remove a backup by name.

Args:: name (str): The name of the backup to remove.
Returns:: The return type is null.

$lib.backup.list()

Get a list of backup names.

Returns:: A list of backup names. The return type is list.

$lib.backup.run(name=$lib.null, wait=$lib.true)

Run a Cortex backup.

Args:

name (str): The name of the backup to generate.

wait (boolean): If true, wait for the backup to complete before returning.

Returns:

The name of the newly created backup. The return type is str.

$lib.base64

A Storm Library for encoding and decoding base64 data.

$lib.base64.decode(valu, urlsafe=$lib.true)

Decode a base64 string into a bytes object.

Args:

valu (str): The string to decode.

urlsafe (boolean): Perform the decoding in a urlsafe manner if true.

Returns:

A bytes object for the decoded data. The return type is bytes.

$lib.base64.encode(valu, urlsafe=$lib.true)

Encode a bytes object to a base64 encoded string.

Args:

valu (bytes): The object to encode.

urlsafe (boolean): Perform the encoding in a urlsafe manner if true.

Returns:

A base64 encoded string. The return type is str.

$lib.basex

A Storm library which implements helpers for encoding and decoding strings using an arbitrary charset.

$lib.basex.decode(text, charset)

Decode a baseX string into bytes.

Args:

text (str): The hex string to be decoded into bytes.

charset (str): The charset used to decode the string.

Returns:

The decoded bytes. The return type is bytes.

$lib.basex.encode(byts, charset)

Encode bytes into a baseX string.

Args:

byts (bytes): The bytes to be encoded into a string.

charset (str): The charset used to encode the bytes.

Returns:

The encoded string. The return type is str.

$lib.bytes

A Storm Library for interacting with bytes storage. This Library is deprecated; use $lib.axon.* instead.

$lib.bytes.has(sha256)

Warning

$lib.bytes.has has been deprecated and will be removed in version v3.0.0. Use the corresponding $lib.axon function.

Check if the Axon the Cortex is configured to use has a given sha256 value.

Examples:

Check if the Axon has a given file:

# This example assumes the Axon does have the bytes
cli> storm if $lib.bytes.has(9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08) {
        $lib.print("Has bytes")
    } else {
        $lib.print("Does not have bytes")
    }

Has bytes

Args:

sha256 (str): The sha256 value to check.

Returns:

True if the Axon has the file, false if it does not. The return type is boolean.

$lib.bytes.hashset(sha256)

Warning

$lib.bytes.hashset has been deprecated and will be removed in version v3.0.0. Use the corresponding $lib.axon function.

Return additional hashes of the bytes stored in the Axon for the given sha256.

Examples:

Get the md5 hash for a file given a variable named $sha256:

$hashset = $lib.bytes.hashset($sha256)
$md5 = $hashset.md5

Args:

sha256 (str): The sha256 value to calculate hashes for.

Returns:

A dictionary of additional hashes. The return type is dict.

$lib.bytes.put(byts)

Warning

$lib.bytes.put has been deprecated and will be removed in version v3.0.0. Use the corresponding $lib.axon function.

Save the given bytes variable to the Axon the Cortex is configured to use.

Examples:

Save a base64 encoded buffer to the Axon:

cli> storm $s='dGVzdA==' $buf=$lib.base64.decode($s) ($size, $sha256)=$lib.bytes.put($buf)
     $lib.print('size={size} sha256={sha256}', size=$size, sha256=$sha256)

size=4 sha256=9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

Args:

byts (bytes): The bytes to save.

Returns:

A tuple of the file size and sha256 value. The return type is list.

$lib.bytes.size(sha256)

Warning

$lib.bytes.size has been deprecated and will be removed in version v3.0.0. Use the corresponding $lib.axon function.

Return the size of the bytes stored in the Axon for the given sha256.

Examples:

Get the size for a file given a variable named $sha256:

$size = $lib.bytes.size($sha256)

Args:

sha256 (str): The sha256 value to check.

Returns:

The size of the file or null if the file is not found. The return type may be one of the following: int, null.

$lib.bytes.upload(genr)

Warning

$lib.bytes.upload has been deprecated and will be removed in version v3.0.0. Use the corresponding $lib.axon function.

Upload a stream of bytes to the Axon as a file.

Examples:

Upload bytes from a generator:

($size, $sha256) = $lib.bytes.upload($getBytesChunks())

Args:

genr (generator): A generator which yields bytes.

Returns:

A tuple of the file size and sha256 value. The return type is list.

$lib.cache

A Storm Library for interacting with Cache Objects.

$lib.cache.fixed(callback, size=(10000))

Get a new Fixed Cache object.

On a cache-miss when calling .get(), the callback Storm query is executed in a sub-runtime in the current execution context. A special variable, $cache_key, will be set to the key argument provided to .get().

The callback Storm query must contain a return statement, and if it does not return a value when executed with the input, $lib.null will be set as the value.

The fixed cache uses FIFO to evict items once the maximum size is reached.

Examples:

// Use a callback query with a function that modifies the outer runtime, // since it will run in the scope where it was defined. $test = foo

function callback(key) {
$test = $key // this will modify $test in the outer runtime return({$key}-val)

}

$cache = $lib.cache.fixed(${ return($callback($cache_key)) }) $value = $cache.get(bar) $lib.print($test) // this will equal “bar”

// Use a callback query that will not modify the outer runtime, // except for variables accessible as references. $test = foo $tests = ([])

$cache = $lib.cache.fixed(${
$test = $cache_key // this will not modify $test in the outer runtime $tests.append($cache_key) // this will modify $tests in the outer runtime return({$cache_key}-val)

})

$value = $cache.get(bar) $lib.print($test) // this will equal “foo” $lib.print($tests) // this will equal (foo,)

Args:

callback: A Storm query that will return a value for $cache_key on a cache miss. The input type may be one of the following: str, storm:query.

size (int): The maximum size of the cache.

Returns:

A new cache:fixed object. The return type is cache:fixed.

$lib.cell

A Storm Library for interacting with the Cortex.

$lib.cell.getBackupInfo()

Get information about recent backup activity.

Returns:: A dictionary containing backup information. The return type is dict.

$lib.cell.getCellInfo()

Return metadata specific for the Cortex.

Returns:: A dictionary containing metadata. The return type is dict.

$lib.cell.getHealthCheck()

Get healthcheck information about the Cortex.

Returns:: A dictionary containing healthcheck information. The return type is dict.

$lib.cell.getMirrorUrls(name=$lib.null)

Get mirror Telepath URLs for an AHA configured service.

Args:: name (str): The name, or iden, of the service to get mirror URLs for (defaults to the Cortex if not provided).
Returns:: A list of Telepath URLs. The return type is list.

$lib.cell.getSystemInfo()

Get info about the system in which the Cortex is running.

Returns:: A dictionary containing system information. The return type is dict.

$lib.cell.hotFixesApply()

Apply known data migrations and fixes via storm.

Returns:: Tuple containing the current version after applying the fixes. The return type is list.

$lib.cell.hotFixesCheck()

Check to see if there are known hot fixes to apply.

Returns:: Bool indicating if there are hot fixes to apply or not. The return type is boolean.

$lib.cell.trimNexsLog(consumers=$lib.null, timeout=(30))

Rotate and cull the Nexus log (and any consumers) at the current offset.

If the consumers argument is provided they will first be checked if online before rotating and raise otherwise. After rotation, all consumers provided must catch-up to the offset to cull at within the specified timeout before executing the cull, and will raise otherwise.

Args:

consumers (array): List of Telepath URLs for consumers of the Nexus log.

timeout (int): Time (in seconds) to wait for consumers to catch-up before culling.

Returns:

The offset that was culled (up to and including). The return type is int.

$lib.cell.uptime(name=$lib.null)

Get update data for the Cortex or a connected Service.

Args:: name (str): The name, or iden, of the service to get uptime data for (defaults to the Cortex if not provided).
Returns:: A dictionary containing uptime data. The return type is dict.

$lib.compression.bzip2

A Storm library which implements helpers for bzip2 compression.

$lib.compression.bzip2.en(valu)

Compress bytes using bzip2 and return them.

Example:

Compress bytes with bzip2:

$foo = $lib.compression.bzip2.en($mybytez)

Args:

valu (bytes): The bytes to be compressed.

Returns:

The bzip2 compressed bytes. The return type is bytes.

$lib.compression.bzip2.un(valu)

Decompress bytes using bzip2 and return them.

Example:

Decompress bytes with bzip2:

$foo = $lib.compression.bzip2.un($mybytez)

Args:

valu (bytes): The bytes to be decompressed.

Returns:

Decompressed bytes. The return type is bytes.

$lib.compression.gzip

A Storm library which implements helpers for gzip compression.

$lib.compression.gzip.en(valu)

Compress bytes using gzip and return them.

Example:

Compress bytes with gzip:

$foo = $lib.compression.gzip.en($mybytez)

Args:

valu (bytes): The bytes to be compressed.

Returns:

The gzip compressed bytes. The return type is bytes.

$lib.compression.gzip.un(valu)

Decompress bytes using gzip and return them.

Example:

Decompress bytes with gzip:

$foo = $lib.compression.gzip.un($mybytez)

Args:

valu (bytes): The bytes to be decompressed.

Returns:

Decompressed bytes. The return type is bytes.

$lib.compression.zlib

A Storm library which implements helpers for zlib compression.

$lib.compression.zlib.en(valu)

Compress bytes using zlib and return them.

Example:

Compress bytes with zlib:

$foo = $lib.compression.zlib.en($mybytez)

Args:

valu (bytes): The bytes to be compressed.

Returns:

The zlib compressed bytes. The return type is bytes.

$lib.compression.zlib.un(valu)

Decompress bytes using zlib and return them.

Example:

Decompress bytes with zlib:

$foo = $lib.compression.zlib.un($mybytez)

Args:

valu (bytes): The bytes to be decompressed.

Returns:

Decompressed bytes. The return type is bytes.

$lib.cortex.httpapi

Library for interacting with the Extended HTTP API.

$lib.cortex.httpapi.add(path, name=, desc=, runas=owner, authenticated=$lib.true, readonly=$lib.false, iden=$lib.null)

Add an Extended HTTP API endpoint to the Cortex.

This can be used to add an API endpoint which will be resolved under the API path “/api/ext/”. New API endpoint objects are appended to a list of APIs to resolve in order.

Notes:

The Cortex does not make any attempt to do any inspection of path values which may conflict between one another. This is because the paths for a given endpoint may be changed, they can contain regular expressions, and they may have their resolution order changed. Cortex administrators are responsible for configuring their Extended HTTP API endpoints with correct paths and order to meet their use cases.

Example:

Add a simple API handler:

// Create an endpoint for /api/ext/foo/bar
$api = $lib.cortex.httpapi.add('foo/bar')

// Define a GET response handler via storm that makes a simple reply.
$api.methods.get = ${ $request.reply(200, body=({"some": "data}) }

Add a wildcard handler:

// Create a wildcard endpoint for /api/ext/some/thing([a-zA-Z0-9]*)/([a-zA-Z0-9]*)
$api = $lib.cortex.httpapi.add('some/thing([a-zA-Z0-9]*)/([a-zA-Z0-9]*)')

// The capture groups are exposed as request arguments.
// Echo them back to the caller.
$api.methods.get = ${
    $request.reply(200, body=({"args": $request.args})
}

Args:

path (string): The extended HTTP API path.

name (string): Friendly name for the Extended HTTP API.

desc (string): Description for the Extended HTTP API.

runas (string): Run the storm query as the API “owner” or as the authenticated “user”.

authenticated (boolean): Require the API endpoint to be authenticated.

readonly (boolean): Run the Extended HTTP Storm methods in readonly mode.

iden (str): An iden for the new Extended HTTP API.

Returns:

A new http:api object. The return type is http:api.

$lib.cortex.httpapi.del(iden)

Delete an Extended HTTP API endpoint.

Args:: iden (string): The iden of the API to delete.
Returns:: The return type is null.

$lib.cortex.httpapi.get(iden)

Get an Extended http:api object.

Args:: iden (string): The iden of the API to retrieve.
Returns:: The http:api object. The return type is http:api.

$lib.cortex.httpapi.getByPath(path)

Get an Extended http:api object by path.

Notes:: The path argument is evaluated as a regular expression input, and will be used to get the first HTTP API handler whose path value has a match.
Args:: path (string): Path to use to retrieve an object.
Returns:: The http:api object or $lib.null if there is no match. The return type may be one of the following: http:api, null.

$lib.cortex.httpapi.index(iden, index=(0))

Set the index for a given Extended HTTP API.

Args:

iden (string): The iden of the API to modify.

index (int): The new index of the API. Uses zero based indexing.

Returns:

The new index location of the API. The return type is int.

$lib.cortex.httpapi.list()

Get all the Extended HTTP APIs on the Cortex

Returns:: A list of http:api objects The return type is list.

$lib.cortex.httpapi.response(requestinfo)

Make a response object. Used by API handlers automatically.

Args:: requestinfo (dict): Request info dictionary. This is an opaque data structure which may change.
Returns:: The return type is http:api:request.

$lib.cron

A Storm Library for interacting with Cron Jobs in the Cortex.

$lib.cron.add(**kwargs)

Add a recurring Cron Job to the Cortex.

Args:: **kwargs (any): Key-value parameters used to add the cron job.
Returns:: The new Cron Job. The return type is cronjob.

$lib.cron.at(**kwargs)

Add a non-recurring Cron Job to the Cortex.

Args:: **kwargs (any): Key-value parameters used to add the cron job.
Returns:: The new Cron Job. The return type is cronjob.

$lib.cron.del(prefix)

Delete a CronJob from the Cortex.

Args:: prefix (str): A prefix to match in order to identify a cron job to delete. Only a single matching prefix will be deleted.
Returns:: The return type is null.

$lib.cron.disable(prefix)

Disable a CronJob in the Cortex.

Args:: prefix (str): A prefix to match in order to identify a cron job to disable. Only a single matching prefix will be disabled.
Returns:: The iden of the CronJob which was disabled. The return type is str.

$lib.cron.enable(prefix)

Enable a CronJob in the Cortex.

Args:: prefix (str): A prefix to match in order to identify a cron job to enable. Only a single matching prefix will be enabled.
Returns:: The iden of the CronJob which was enabled. The return type is str.

$lib.cron.get(prefix)

Get a CronJob in the Cortex.

Args:: prefix (str): A prefix to match in order to identify a cron job to get. Only a single matching prefix will be retrieved.
Returns:: The requested cron job. The return type is cronjob.

$lib.cron.list()

List CronJobs in the Cortex.

Returns:: A list of cronjob objects. The return type is list.

$lib.cron.mod(prefix, query)

Modify the Storm query for a CronJob in the Cortex.

Args:

prefix (str): A prefix to match in order to identify a cron job to modify. Only a single matching prefix will be modified.

query: The new Storm query for the Cron Job. The input type may be one of the following: str, query.

Returns:

The iden of the CronJob which was modified. The return type is str.

$lib.cron.move(prefix, view)

Move a cron job to a new view.

Args:

prefix (str): A prefix to match in order to identify a cron job to move. Only a single matching prefix will be modified.

view (str): The iden of the view to move the CrobJob to

Returns:

The iden of the CronJob which was moved. The return type is str.

$lib.crypto.coin.ethereum

A Storm library which implements helpers for Ethereum.

$lib.crypto.coin.ethereum.eip55(addr)

Convert an Ethereum address to a checksummed address.

Args:: addr (str): The Ethereum address to be converted.
Returns:: A list of (<bool>, <addr>) for status and checksummed address. The return type is list.

$lib.crypto.hashes

A Storm Library for hashing bytes

$lib.crypto.hashes.md5(byts)

Retrieve an MD5 hash of a byte string.

Args:: byts (bytes): The bytes to hash.
Returns:: The hex digest of the MD5 hash of the input bytes. The return type is str.

$lib.crypto.hashes.sha1(byts)

Retrieve a SHA1 hash of a byte string.

Args:: byts (bytes): The bytes to hash.
Returns:: The hex digest of the SHA1 hash of the input bytes. The return type is str.

$lib.crypto.hashes.sha256(byts)

Retrieve a SHA256 hash of a byte string.

Args:: byts (bytes): The bytes to hash.
Returns:: The hex digest of the SHA256 hash of the input bytes. The return type is str.

$lib.crypto.hashes.sha512(byts)

Retrieve a SHA512 hash of a byte string.

Args:: byts (bytes): The bytes to hash.
Returns:: The hex digest of the SHA512 hash of the input bytes. The return type is str.

$lib.crypto.hmac

A Storm library for computing RFC2104 HMAC values.

$lib.crypto.hmac.digest(key, mesg, alg=sha256)

Compute the digest value of a message using RFC2104 HMAC.

Examples:

Compute the HMAC-SHA256 digest for a message with a secret key:

$digest = $lib.crypto.hmac.digest(key=$secretKey.encode(), mesg=$mesg.encode())

Args:

key (bytes): The key to use for the HMAC calculation.

mesg (bytes): The message to use for the HMAC calculation.

alg (str): The digest algorithm to use.

Returns:

The binary digest of the HMAC value. The return type is bytes.

$lib.csv

A Storm Library for interacting with csvtool.

$lib.csv.emit(*args, table=$lib.null)

Emit a csv:row event to the Storm runtime for the given args.

Args:

*args (any): Items which are emitted as a csv:row event.

table (str): The name of the table to emit data too. Optional.

Returns:

The return type is null.

$lib.dict

A Storm Library for interacting with dictionaries.

$lib.dict.has(valu, key)

Check a dictionary has a specific key.

Args:

valu (dict): The dictionary being checked.

key (any): The key to check.

Returns:

True if the key is present, false if the key is not present. The return type is boolean.

$lib.dict.keys(valu)

Retrieve a list of keys in the specified dictionary.

Args:: valu (dict): The dictionary to operate on.
Returns:: List of keys in the specified dictionary. The return type is list.

$lib.dict.pop(valu, key, default=$lib.undef)

Remove specified key and return the corresponding value.

Args:

valu (dict): The dictionary to operate on.

key (any): The key to pop.

default (any): Optional default value to return if the key does not exist in the dictionary.

Returns:

The popped value. The return type is any.

$lib.dict.update(valu, other)

Update the specified dictionary with keys/values from another dictionary.

Args:

valu (dict): The target dictionary (update to).

other (dict): The source dictionary (update from).

Returns:

The return type is null.

$lib.dict.values(valu)

Retrieve a list of values in the specified dictionary.

Args:: valu (dict): The dictionary to operate on.
Returns:: List of values in the specified dictionary. The return type is list.

$lib.dmon

A Storm Library for interacting with StormDmons.

$lib.dmon.add(text, name=noname, ddef=$lib.null)

Add a Storm Dmon to the Cortex.

Examples:

Add a dmon that executes a query:

$lib.dmon.add(${ myquery }, name='example dmon')

Args:

text: The Storm query to execute in the Dmon loop. The input type may be one of the following: str, storm:query.

name (str): The name of the Dmon.

ddef (dict): Additional daemon definition fields.

Returns:

The iden of the newly created Storm Dmon. The return type is str.

$lib.dmon.bump(iden)

Restart the Dmon.

Args:: iden (str): The GUID of the dmon to restart.
Returns:: True if the Dmon is restarted; False if the iden does not exist. The return type is boolean.

$lib.dmon.del(iden)

Delete a Storm Dmon by iden.

Args:: iden (str): The iden of the Storm Dmon to delete.
Returns:: The return type is null.

$lib.dmon.get(iden)

Get a Storm Dmon definition by iden.

Args:: iden (str): The iden of the Storm Dmon to get.
Returns:: A Storm Dmon definition dict. The return type is dict.

$lib.dmon.list()

Get a list of Storm Dmons.

Returns:: A list of Storm Dmon definitions. The return type is list.

$lib.dmon.log(iden)

Get the messages from a Storm Dmon.

Args:: iden (str): The iden of the Storm Dmon to get logs for.
Returns:: A list of messages from the StormDmon. The return type is list.

$lib.dmon.start(iden)

Start a storm dmon.

Args:: iden (str): The GUID of the dmon to start.
Returns:: $lib.true unless the dmon does not exist or was already started. The return type is boolean.

$lib.dmon.stop(iden)

Stop a Storm Dmon.

Args:: iden (str): The GUID of the Dmon to stop.
Returns:: $lib.true unless the dmon does not exist or was already stopped. The return type is boolean.

$lib.env

A Storm Library for accessing environment vars.

$lib.env.get(name, default=$lib.null)

Retrieve an environment variable.

Notes:

Environment variables must begin with SYN_STORM_ENV_ in order to be accessed by this API.

Args:

name (str): The name of the environment variable.

default (obj): The value to return if the environment variable is not set.

Returns:

The environment variable string. The return type is str.

$lib.export

A Storm Library for exporting data.

$lib.export.toaxon(query, opts=$lib.null)

Run a query as an export (fully resolving relationships between nodes in the output set) and save the resulting stream of packed nodes to the axon.

Args:

query (str): A query to run as an export.

opts (dict): Storm runtime query option params.

Returns:

Returns a tuple of (size, sha256). The return type is list.

$lib.feed

A Storm Library for interacting with Cortex feed functions.

$lib.feed.genr(name, data)

Yield nodes being added to the graph by adding data with a given ingest type.

Notes:

This is using the Runtimes’s Snap to call addFeedNodes(). This only yields nodes if the feed function yields nodes. If the generator is not entirely consumed there is no guarantee that all of the nodes which should be made by the feed function will be made.

Args:

name (str): Name of the ingest function to send data too.

data (prim): Data to send to the ingest function.

Yields:

Yields Nodes as they are created by the ingest function. The return type is node.

$lib.feed.ingest(name, data)

Add nodes to the graph with a given ingest type.

Notes:

This is using the Runtimes’s Snap to call addFeedData(), after setting the snap.strict mode to False. This will cause node creation and property setting to produce warning messages, instead of causing the Storm Runtime to be torn down.

Args:

name (str): Name of the ingest function to send data too.

data (prim): Data to send to the ingest function.

Returns:

The return type is null.

$lib.feed.list()

Get a list of feed functions.

Returns:: A list of feed functions. The return type is list.

$lib.gen

A Storm Library for secondary property based deconfliction.

$lib.gen.campaign(name, reporter)

Returns an ou:campaign node based on the campaign and reporter names, adding the node if it does not exist.

Args:

name (str): The reported name of the campaign.

reporter (str): The name of the organization which reported the campaign.

Returns:

An ou:campaign node. The return type is node.

$lib.gen.cryptoX509CertBySha256(sha256, try=$lib.false)

Returns a crypto:x509:cert node by SHA256, adding the node if it does not exist.

Args:

sha256: The SHA256 fingerprint for the certificate. The input type may be one of the following: str, hash:sha256.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

A crypto:x509:cert node with the given SHA256. The return type is node.

$lib.gen.fileBytesBySha256(sha256, try=$lib.false)

Returns a file:bytes node by SHA256, adding the node if it does not exist.

Args:

sha256: The SHA256 fingerprint for the file:bytes node. The input type may be one of the following: str, hash:sha256.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

A file:bytes node with the given SHA256. The return type is node.

$lib.gen.geoPlaceByName(name)

Returns a geo:place node by name, adding the node if it does not exist.

Args:: name (str): The name of the place.
Returns:: A geo:place node with the given name. The return type is node.

$lib.gen.industryByName(name)

Returns an ou:industry by name, adding the node if it does not exist.

Args:: name (str): The name of the industry.
Returns:: An ou:industry node with the given name. The return type is node.

$lib.gen.inetTlsServerCertByServerAndSha256(server, sha256, try=$lib.false)

Returns an inet:tls:servercert node by server and SHA256, adding the node if it does not exist.

Args:

server: The server associated with the x509 certificate. The input type may be one of the following: str, inet:server.

sha256: The SHA256 fingerprint for the certificate. The input type may be one of the following: str, hash:sha256.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

An inet:tls:servercert node with the given server and SHA256. The return type is node.

$lib.gen.itAvScanResultByTarget(form, value, signame, scanner=$lib.null, time=$lib.null, try=$lib.false)

Returns an it:av:scan:result node by deconflicting with a target and signature name, adding the node if it does not exist.

Args:

form (str): The target form.

value (str): The target value.

signame (str): The signature name.

scanner (str): An optional scanner software name to include in deconfliction.

time (time): An optional time when the scan was run to include in the deconfliction.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

An it:av:scan:result node. The return type is node.

$lib.gen.langByCode(name, try=$lib.false)

Returns a lang:language node by language code, adding the node if it does not exist.

Args:

name (str): The language code for the language.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

A lang:language node with the given code. The return type is node.

$lib.gen.langByName(name)

Returns a lang:language node by name, adding the node if it does not exist.

Args:: name (str): The name of the language.
Returns:: A lang:language node with the given name. The return type is node.

$lib.gen.newsByUrl(url, try=$lib.false)

Returns a media:news node by URL, adding the node if it does not exist.

Args:

url (inet:url): The URL where the news is published.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

A media:news node with the given URL. The return type is node.

$lib.gen.orgByFqdn(fqdn, try=$lib.false)

Returns an ou:org node by FQDN, adding the node if it does not exist.

Args:

fqdn (str): The FQDN of the org.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

An ou:org node with the given FQDN. The return type is node.

$lib.gen.orgByName(name)

Returns an ou:org by name, adding the node if it does not exist.

Args:: name (str): The name of the org.
Returns:: An ou:org node with the given name. The return type is node.

$lib.gen.orgHqByName(name)

Returns a ps:contact node for the ou:org, adding the node if it does not exist.

Args:: name (str): The name of the org.
Returns:: A ps:contact node for the ou:org with the given name. The return type is node.

$lib.gen.polCountryByIso2(iso2, try=$lib.false)

Returns a pol:country node by deconflicting the :iso2 property.

Args:

iso2 (str): The pol:country:iso2 property.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

A pol:country node. The return type is node.

$lib.gen.psContactByEmail(type, email, try=$lib.false)

Returns a ps:contact by deconflicting the type and email address.

Args:

type (str): The ps:contact:type property.

email (str): The ps:contact:email property.

try (boolean): Type normalization will fail silently instead of raising an exception.

Returns:

A ps:contact node. The return type is node.

$lib.gen.riskThreat(name, reporter)

Returns a risk:threat node based on the threat and reporter names, adding the node if it does not exist.

Args:

name (str): The reported name of the threat cluster.

reporter (str): The name of the organization which reported the threat cluster.

Returns:

A risk:threat node. The return type is node.

$lib.gen.riskToolSoftware(name, reporter)

Returns a risk:tool:software node based on the tool and reporter names, adding the node if it does not exist.

Args:

name (str): The reported name of the tool.

reporter (str): The name of the organization which reported the tool.

Returns:

A risk:tool:software node. The return type is node.

$lib.gen.softByName(name)

Returns it:prod:soft node by name, adding the node if it does not exist.

Args:: name (str): The name of the software.
Returns:: An it:prod:soft node with the given name. The return type is node.

$lib.gen.vulnByCve(cve, try=$lib.false, reporter=$lib.null)

Returns risk:vuln node by CVE and reporter, adding the node if it does not exist.

Args:

cve (str): The CVE id.

try (boolean): Type normalization will fail silently instead of raising an exception.

reporter (str): The name of the organization which reported the vulnerability.

Returns:

A risk:vuln node with the given CVE. The return type is node.

$lib.gis

A Storm library which implements helpers for earth based geospatial calculations.

$lib.gis.bbox(lon, lat, dist)

Calculate a min/max bounding box for the specified circle.

Args:

lon (float): The longitude in degrees.

lat (float): The latitude in degrees.

dist (int): A distance in geo:dist base units (mm).

Returns:

A tuple of (lonmin, lonmax, latmin, latmax). The return type is list.

$lib.globals

A Storm Library for interacting with global variables which are persistent across the Cortex.

$lib.globals.get(name, default=$lib.null)

Get a Cortex global variables.

Args:

name (str): Name of the variable.

default (prim): Default value to return if the variable is not set.

Returns:

The variable value. The return type is prim.

$lib.globals.list()

Get a list of variable names and values.

Returns:: A list of tuples with variable names and values that the user can access. The return type is list.

$lib.globals.pop(name, default=$lib.null)

Delete a variable value from the Cortex.

Args:

name (str): Name of the variable.

default (prim): Default value to return if the variable is not set.

Returns:

The variable value. The return type is prim.

$lib.globals.set(name, valu)

Set a variable value in the Cortex.

Args:

name (str): The name of the variable to set.

valu (prim): The value to set.

Returns:

The variable value. The return type is prim.

$lib.graph

A Storm Library for interacting with graph projections in the Cortex.

$lib.graph.activate(iden)

Set the graph projection to use for the top level Storm Runtime.

Args:: iden (str): The iden of the graph projection to use.
Returns:: The return type is null.

$lib.graph.add(gdef)

Add a graph projection to the Cortex.

Example:

Add a graph projection named “Test Projection”:

$rules = ({
    "name": "Test Projection",
    "desc": "My test projection",
    "degrees": 2,
    "pivots": ["-> meta:seen"],
    "filters": ["-#nope"],
    "forms": {
        "inet:fqdn": {
            "pivots": ["<- *", "-> *"],
            "filters": ["-inet:fqdn:issuffix=1"]
        },
        "*": {
            "pivots": ["-> #"],
        }
    }
})
$lib.graph.add($rules)

Args:

gdef (dict): A graph projection definition.

Returns:

The return type is null.

$lib.graph.del(iden)

Delete a graph projection from the Cortex.

Args:: iden (str): The iden of the graph projection to delete.
Returns:: The return type is null.

$lib.graph.get(iden=$lib.null)

Get a graph projection definition from the Cortex.

Args:: iden (str): The iden of the graph projection to get. If not specified, returns the current graph projection.
Returns:: A graph projection definition, or None if no iden was specified and there is currently no graph projection set. The return type is dict.

$lib.graph.grant(gden, scope, iden, level)

Modify permissions granted to users/roles on a graph projection.

Args:

gden (str): Iden of the graph projection to modify.

scope (str): The scope, either “users” or “roles”.

iden (str): The user/role iden depending on scope.

level (int): The permission level number.

Returns:

The return type is null.

$lib.graph.list()

List the graph projections available in the Cortex.

Returns:: A list of graph projection definitions. The return type is list.

$lib.graph.mod(iden, info)

Modify user editable properties of a graph projection.

Args:

iden (str): The iden of the graph projection to modify.

info (dict): A dictionary of the properties to edit.

Returns:

The return type is null.

$lib.graph.revoke(gden, scope, iden)

Revoke permissions granted to users/roles on a graph projection.

Args:

gden (str): Iden of the graph projection to modify.

scope (str): The scope, either “users” or “roles”.

iden (str): The user/role iden depending on scope.

Returns:

The return type is null.

$lib.hex

A Storm library which implements helpers for hexadecimal encoded strings.

$lib.hex.decode(valu)

Decode a hexadecimal string into bytes.

Args:: valu (str): The hex string to be decoded into bytes.
Returns:: The decoded bytes. The return type is bytes.

$lib.hex.encode(valu)

Encode bytes into a hexadecimal string.

Args:: valu (bytes): The bytes to be encoded into a hex string.
Returns:: The hex encoded string. The return type is str.

$lib.hex.fromint(valu, length, signed=$lib.false)

Convert an integer to a big endian hexadecimal string.

Args:

valu (int): The integer to be converted.

length (int): The number of bytes to use to represent the integer.

signed (bool): If true, convert as a signed value.

Returns:

The resulting hex string. The return type is str.

$lib.hex.signext(valu, length)

Sign extension pad a hexadecimal encoded signed integer.

Args:

valu (str): The hex string to pad.

length (int): The number of characters to pad the string to.

Returns:

The sign extended hex string. The return type is str.

$lib.hex.toint(valu, signed=$lib.false)

Convert a big endian hexadecimal string to an integer.

Args:

valu (str): The hex string to be converted.

signed (bool): If true, convert to a signed integer.

Returns:

The resulting integer. The return type is int.

$lib.hex.trimext(valu)

Trim sign extension bytes from a hexadecimal encoded signed integer.

Args:: valu (str): The hex string to trim.
Returns:: The trimmed hex string. The return type is str.

$lib.inet.http

A Storm Library exposing an HTTP client API.

For APIs that accept an ssl_opts argument, the dictionary may contain the following values:

{
    'verify': <bool> - Perform SSL/TLS verification. Is overridden by the ssl_verify argument.
    'client_cert': <str> - PEM encoded full chain certificate for use in mTLS.
    'client_key': <str> - PEM encoded key for use in mTLS. Alternatively, can be included in client_cert.
    'ca_cert': <str> - A PEM encoded full chain CA certificate for use when verifying the request.
}

$lib.inet.http.codereason(code)

Get the reason phrase for an HTTP status code.

Examples:

Get the reason for a 404 status code:

$str=$lib.inet.http.codereason(404)

Args:

code (int): The HTTP status code.

Returns:

The reason phrase for the status code. The return type is str.

$lib.inet.http.connect(url, headers=$lib.null, ssl_verify=$lib.true, timeout=(300), params=$lib.null, proxy=$lib.null, ssl_opts=$lib.null)

Connect a web socket to tx/rx JSON messages.

Args:

url (str): The URL to retrieve.

headers (dict): HTTP headers to send with the request.

ssl_verify (boolean): Perform SSL/TLS verification.

timeout (int): Total timeout for the request in seconds.

params (dict): Optional parameters which may be passed to the connection request.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: bool, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.inet.http help for additional details.

Returns:

A websocket object. The return type is inet:http:socket.

$lib.inet.http.get(url, headers=$lib.null, ssl_verify=$lib.true, params=$lib.null, timeout=(300), allow_redirects=$lib.true, proxy=$lib.null, ssl_opts=$lib.null)

Get the contents of a given URL.

Args:

url (str): The URL to retrieve.

headers (dict): HTTP headers to send with the request.

ssl_verify (boolean): Perform SSL/TLS verification.

params (dict): Optional parameters which may be passed to the request.

timeout (int): Total timeout for the request in seconds.

allow_redirects (bool): If set to false, do not follow redirects.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: bool, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.inet.http help for additional details.

Returns:

The response object. The return type is inet:http:resp.

$lib.inet.http.head(url, headers=$lib.null, ssl_verify=$lib.true, params=$lib.null, timeout=(300), allow_redirects=$lib.false, proxy=$lib.null, ssl_opts=$lib.null)

Get the HEAD response for a URL.

Args:

url (str): The URL to retrieve.

headers (dict): HTTP headers to send with the request.

ssl_verify (boolean): Perform SSL/TLS verification.

params (dict): Optional parameters which may be passed to the request.

timeout (int): Total timeout for the request in seconds.

allow_redirects (bool): If set to true, follow redirects.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: bool, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.inet.http help for additional details.

Returns:

The response object. The return type is inet:http:resp.

$lib.inet.http.post(url, headers=$lib.null, json=$lib.null, body=$lib.null, ssl_verify=$lib.true, params=$lib.null, timeout=(300), allow_redirects=$lib.true, fields=$lib.null, proxy=$lib.null, ssl_opts=$lib.null)

Post data to a given URL.

Args:

url (str): The URL to post to.

headers (dict): HTTP headers to send with the request.

json (prim): The data to post, as JSON object.

body (bytes): The data to post, as binary object.

ssl_verify (boolean): Perform SSL/TLS verification.

params (dict): Optional parameters which may be passed to the request.

timeout (int): Total timeout for the request in seconds.

allow_redirects (bool): If set to false, do not follow redirects.

fields (list): A list of info dictionaries containing the name, value or sha256, and additional parameters for fields to post, as multipart/form-data. If a sha256 is specified, the request will be sent from the axon and the corresponding file will be uploaded as the value for the field.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: bool, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.inet.http help for additional details.

Returns:

The response object. The return type is inet:http:resp.

$lib.inet.http.request(meth, url, headers=$lib.null, json=$lib.null, body=$lib.null, ssl_verify=$lib.true, params=$lib.null, timeout=(300), allow_redirects=$lib.true, fields=$lib.null, proxy=$lib.null, ssl_opts=$lib.null)

Make an HTTP request using the given HTTP method to the url.

Args:

meth (str): The HTTP method. (ex. PUT)

url (str): The URL to send the request to.

headers (dict): HTTP headers to send with the request.

json (prim): The data to include in the body, as JSON object.

body (bytes): The data to include in the body, as binary object.

ssl_verify (boolean): Perform SSL/TLS verification.

params (dict): Optional parameters which may be passed to the request.

timeout (int): Total timeout for the request in seconds.

allow_redirects (bool): If set to false, do not follow redirects.

fields (list): A list of info dictionaries containing the name, value or sha256, and additional parameters for fields to post, as multipart/form-data. If a sha256 is specified, the request will be sent from the axon and the corresponding file will be uploaded as the value for the field.

proxy: Set to a proxy URL string or $lib.false to disable proxy use. The input type may be one of the following: bool, null, str.

ssl_opts (dict): Optional SSL/TLS options. See $lib.inet.http help for additional details.

Returns:

The response object. The return type is inet:http:resp.

$lib.inet.http.urldecode(text)

Urldecode a text string.

This will replace %xx escape characters with the special characters they represent and replace plus signs with spaces.

Examples:

Urlencode a string:

$str=$lib.inet.http.urldecode("http%3A%2F%2Fgo+ogle.com")

Args:

text (str): The text string.

Returns:

The urldecoded string. The return type is str.

$lib.inet.http.urlencode(text)

Urlencode a text string.

This will replace special characters in a string using the %xx escape and replace spaces with plus signs.

Examples:

Urlencode a string:

$str=$lib.inet.http.urlencode("http://google.com")

Args:

text (str): The text string.

Returns:

The urlencoded string. The return type is str.

$lib.inet.http.oauth.v1

A Storm library to handle OAuth v1 authentication.

$lib.inet.http.oauth.v1.client(ckey, csecret, atoken, asecret, sigtype=QUERY)

Initialize an OAuthV1 Client to use for signing/authentication.

Args:

ckey (str): The OAuthV1 Consumer Key to store and use for signing requests.

csecret (str): The OAuthV1 Consumer Secret used to sign requests.

atoken (str): The OAuthV1 Access Token (or resource owner key) to use to sign requests.)

asecret (str): The OAuthV1 Access Token Secret (or resource owner secret) to use to sign requests.

sigtype (str): Where to populate the signature (in the HTTP body, in the query parameters, or in the header)

Returns:

An OAuthV1 client to be used to sign requests. The return type is inet:http:oauth:v1:client.

$lib.inet.http.oauth.v2

A Storm library for managing OAuth V2 clients.

$lib.inet.http.oauth.v2.addProvider(conf)

Add a new provider configuration.

Example:

Add a new provider which uses the authorization code flow:

$iden = $lib.guid(example, provider, oauth)
$conf = ({
    "iden": $iden,
    "name": "example_provider",
    "client_id": "yourclientid",
    "client_secret": "yourclientsecret",
    "scope": "first_scope second_scope",
    "auth_uri": "https://provider.com/auth",
    "token_uri": "https://provider.com/token",
    "redirect_uri": "https://local.redirect.com/oauth",
})

// Optionally enable PKCE
$conf.extensions = ({"pkce": $lib.true})

// Optionally disable SSL verification
$conf.ssl_verify = $lib.false

// Optionally provide additional key-val parameters
// to include when calling the auth URI
$conf.extra_auth_params = ({"customparam": "foo"})

$lib.inet.http.oauth.v2.addProvider($conf)

Args:

conf (dict): A provider configuration.

Returns:

The return type is null.

$lib.inet.http.oauth.v2.clearUserAccessToken(iden)

Clear the stored refresh data for the current user’s provider access token.

Args:: iden (str): The provider iden.
Returns:: The existing token state data or None if it did not exist. The return type is dict.

$lib.inet.http.oauth.v2.delProvider(iden)

Delete a provider configuration.

Args:: iden (str): The provider iden.
Returns:: The deleted provider configuration or None if it does not exist. The return type is dict.

$lib.inet.http.oauth.v2.getProvider(iden)

Get a provider configuration

Args:: iden (str): The provider iden.
Returns:: The provider configuration or None if it does not exist. The return type is dict.

$lib.inet.http.oauth.v2.getUserAccessToken(iden)

Get the provider access token for the current user.

Example:

Retrieve the token and handle needing an auth code:

$provideriden = $lib.globals.get("oauth:myprovider")

($ok, $data) = $lib.inet.http.oauth.v2.getUserAccessToken($provideriden)

if $ok {
    // $data is the token to be used in a request
else {
    // $data is a message stating why the token is not available
    // caller should now handle retrieving a new auth code for the user
}

Args:: iden (str): The provider iden.
Returns:: List of (<bool>, <token/mesg>) for status and data. The return type is list.

$lib.inet.http.oauth.v2.listProviders()

List provider configurations

Returns:: List of (iden, conf) tuples. The return type is list.

$lib.inet.http.oauth.v2.setUserAuthCode(iden, authcode, code_verifier=$lib.null)

Set the auth code for the current user.

Args:

iden (str): The provider iden.

authcode (str): The auth code for the user.

code_verifier (str): Optional PKCE code verifier.

Returns:

The return type is null.

$lib.inet.imap

A Storm library to connect to an IMAP server.

$lib.inet.imap.connect(host, port=(993), timeout=(30), ssl=$lib.true, ssl_verify=$lib.true)

Open a connection to an IMAP server.

This method will wait for a “hello” response from the server before returning the inet:imap:server instance.

Args:

host (str): The IMAP hostname.

port (int): The IMAP server port.

timeout (int): The time to wait for all commands on the server to execute.

ssl (bool): Use SSL to connect to the IMAP server.

ssl_verify (bool): Perform SSL/TLS verification.

Returns:

A new inet:imap:server instance. The return type is inet:imap:server.

$lib.inet.ipv6

A Storm Library for providing ipv6 helpers.

$lib.inet.ipv6.expand(valu)

Convert a IPv6 address to its expanded form.’

Notes:

The expanded form is also sometimes called the “long form” address.

Examples:

Expand a ipv6 address to its long form:

$expandedvalu = $lib.inet.ipv6.expand('2001:4860:4860::8888')

Args:

valu (str): IPv6 Address to expand

Returns:

The expanded form. The return type is str.

$lib.inet.smtp

A Storm Library for sending email messages via SMTP.

$lib.inet.smtp.message()

Construct a new email message.

Returns:: The newly constructed inet:smtp:message. The return type is inet:smtp:message.

$lib.inet.whois

A Storm Library for providing a consistent way to generate guids for WHOIS / Registration Data in Storm.

$lib.inet.whois.guid(props, form)

Warning

$lib.inet.whois.guid has been deprecated and will be removed in version v3.0.0. Please use the GUID constructor syntax.

Provides standard patterns for creating guids for certain inet:whois forms.

Raises:

StormRuntimeError: If form is not supported in this method.

Args:

props (dict): Dictionary of properties used to create the form.

form (str): The inet:whois form to create the guid for.

Returns:

A guid for creating a the node for. The return type is str.

$lib.infosec.cvss

A Storm library which implements CVSS score calculations.

$lib.infosec.cvss.calculate(node, save=$lib.true, vers=3.1)

Calculate the CVSS score values for an input risk:vuln node.

Args:

node (node): A risk:vuln node from the Storm runtime.

save (boolean): If true, save the computed scores to the node properties.

vers (str): The version of CVSS calculations to execute.

Returns:

A dictionary containing the computed score and subscores. The return type is dict.

$lib.infosec.cvss.calculateFromProps(props, vers=3.1)

Calculate the CVSS score values from a props dict.

Args:

props (dict): A props dictionary.

vers (str): The version of CVSS calculations to execute.

Returns:

A dictionary containing the computed score and subscores. The return type is dict.

$lib.infosec.cvss.saveVectToNode(node, text)

Warning

$lib.infosec.cvss.saveVectToNode has been deprecated and will be removed in version v3.0.0.

Parse a CVSS v3.1 vector and record properties on a risk:vuln node.

Args:

node (node): A risk:vuln node to record the CVSS properties on.

text (str): A CVSS vector string.

Returns:

The return type is null.

$lib.infosec.cvss.vectToProps(text)

Warning

$lib.infosec.cvss.vectToProps has been deprecated and will be removed in version v3.0.0.

Parse a CVSS v3.1 vector and return a dictionary of risk:vuln props.

Args:: text (str): A CVSS vector string.
Returns:: A dictionary of risk:vuln secondary props. The return type is dict.

$lib.infosec.cvss.vectToScore(vect, vers=$lib.null)

Compute CVSS scores from a vector string.

Takes a CVSS vector string, attempts to automatically detect the version (defaults to CVSS3.1 if it cannot), and calculates the base, temporal, and environmental scores.

Raises:

BadArg: An invalid vers string is provided
BadDataValu: The vector string is invalid in some way. Possible reasons are malformed string, duplicated metrics, missing mandatory metrics, and invalid metric values.

Args:

vect (str):

A valid CVSS vector string.

The following examples are valid formats:

CVSS 2 with version: CVSS2#AV:L/AC:L/Au:M/C:P/I:C/A:N

CVSS 2 with parentheses: (AV:L/AC:L/Au:M/C:P/I:C/A:N)

CVSS 2 without parentheses: AV:L/AC:L/Au:M/C:P/I:C/A:N

CVSS 3.0 with version: CVSS:3.0/AV:N/AC:H/PR:L/UI:R/S:U/C:L/I:L/A:L

CVSS 3.1 with version: CVSS:3.1/AV:N/AC:H/PR:L/UI:R/S:U/C:L/I:L/A:L

CVSS 3.0/3.1 with parentheses: (AV:N/AC:H/PR:L/UI:R/S:U/C:L/I:L/A:L)

CVSS 3.0/3.1 without parentheses: AV:N/AC:H/PR:L/UI:R/S:U/C:L/I:L/A:L

vers (str):

A valid version string or None to autodetect the version from the vector string. Accepted values are: 2, 3.0, 3.1, None.

Returns:

A dictionary with the detected version, base score, temporal score, environmental score, overall score, and normalized vector string. The normalized vector string will have metrics ordered in specification order and metrics with undefined values will be removed. Example:
{
    'version': '3.1',
    'score': 4.3,
    'base': 5.0,
    'temporal': 4.4,
    'environmental': 4.3,
    'normalized': 'AV:N/AC:H/PR:L/UI:R/S:U/C:L/I:L/A:L'
}
The return type is dict.

$lib.infosec.mitre.attack.flow

A Storm library which implements modeling MITRE ATT&CK Flow diagrams.

$lib.infosec.mitre.attack.flow.ingest(flow)

Ingest a MITRE ATT&CK Flow diagram in JSON format.

Args:: flow (data): The JSON data to ingest.
Returns:: The it:mitre:attack:flow node representing the ingested attack flow diagram. The return type may be one of the following: node, null.

$lib.infosec.mitre.attack.flow.norm(flow)

Normalize a MITRE ATT&CK Flow diagram in JSON format.

Args:: flow (data): The MITRE ATT&CK Flow diagram in JSON format to normalize (flatten and sort).
Returns:: The normalized MITRE ATT&CK Flow diagram. The return type is dict.

$lib.iters

A Storm library for providing iterator helpers.

$lib.iters.enum(genr)

Yield (<indx>, <item>) tuples from an iterable or generator.

Args:: genr (iter): An iterable or generator.
Yields:: Yields (<indx>, <item>) tuples. The return type is list.

$lib.iters.zip(*args)

Yield tuples created by iterating multiple iterables in parallel.

Args:: *args (iter): Iterables or generators.
Yields:: Yields tuples with an item from each iterable or generator. The return type is list.

$lib.json

A Storm Library for interacting with Json data.

$lib.json.load(text)

Parse a JSON string and return the deserialized data.

Args:: text (str): The string to be deserialized.
Returns:: The JSON deserialized object. The return type is prim.

$lib.json.save(item)

Save an object as a JSON string.

Args:: item (any): The item to be serialized as a JSON string.
Returns:: The JSON serialized object. The return type is str.

$lib.json.schema(schema, use_default=$lib.true)

Get a JS schema validation object.

Args:

schema (dict): The JsonSchema to use.

use_default (boolean): Whether to insert default schema values into the validated data structure.

Returns:

A validation object that can be used to validate data structures. The return type is json:schema.

$lib.jsonstor

Implements cortex JSON storage.

$lib.jsonstor.cachedel(path, key)

Remove cached data set with cacheset.

Args:

path (str|list): The base path to use for the cache key.

key (prim): The value to use for the GUID cache key.

Returns:

True if the del operation was successful. The return type is boolean.

$lib.jsonstor.cacheget(path, key, asof=now, envl=$lib.false)

Retrieve data stored with cacheset() if it was stored more recently than the asof argument.

Args:

path (str|list): The base path to use for the cache key.

key (prim): The value to use for the GUID cache key.

asof (time): The max cache age.

envl (boolean): Return the full cache envelope.

Returns:

The cached value (or envelope) or null. The return type is prim.

$lib.jsonstor.cacheset(path, key, valu)

Set cache data with an envelope that tracks time for cacheget() use.

Args:

path (str|list): The base path to use for the cache key.

key (prim): The value to use for the GUID cache key.

valu (prim): The data to store.

Returns:

The cached asof time and path. The return type is dict.

$lib.jsonstor.del(path, prop=$lib.null)

Delete a stored JSON object or object.

Args:

path (str|list): A path string or list of path parts.

prop (str|list): A property name or list of name parts.

Returns:

True if the del operation was successful. The return type is boolean.

$lib.jsonstor.get(path, prop=$lib.null)

Return a stored JSON object or object property.

Args:

path (str|list): A path string or list of path parts.

prop (str|list): A property name or list of name parts.

Returns:

The previously stored value or $lib.null The return type is prim.

$lib.jsonstor.iter(path=$lib.null)

Yield (<path>, <valu>) tuples for the JSON objects.

Args:: path (str|list): A path string or list of path parts.
Yields:: (<path>, <item>) tuples. The return type is list.

$lib.jsonstor.set(path, valu, prop=$lib.null)

Set a JSON object or object property.

Args:

path (str|list): A path string or list of path elements.

valu (prim): The value to set as the JSON object or object property.

prop (str|list): A property name or list of name parts.

Returns:

True if the set operation was successful. The return type is boolean.

$lib.layer

A Storm Library for interacting with Layers in the Cortex.

$lib.layer.add(ldef=$lib.null)

Add a layer to the Cortex.

Args:: ldef (dict): The layer definition dictionary.
Returns:: A layer object representing the new layer. The return type is layer.

$lib.layer.del(iden)

Delete a layer from the Cortex.

Args:: iden (str): The iden of the layer to delete.
Returns:: The return type is null.

$lib.layer.get(iden=$lib.null)

Get a Layer from the Cortex.

Args:: iden (str): The iden of the layer to get. If not set, this defaults to the top layer of the current View.
Returns:: The storm layer object. The return type is layer.

$lib.layer.list()

List the layers in a Cortex

Returns:: List of layer objects. The return type is list.

$lib.lift

A Storm Library for interacting with lift helpers.

$lib.lift.byNodeData(name)

Lift nodes which have a given nodedata name set on them.

Args:: name (str): The name to of the nodedata key to lift by.
Yields:: Yields nodes to the pipeline. This must be used in conjunction with the yield keyword. The return type is node.

$lib.log

A Storm library which implements server side logging. These messages are logged to the synapse.storm.log logger.

$lib.log.debug(mesg, extra=$lib.null)

Log a message to the Cortex at the debug log level.

Notes:

This requires the storm.lib.log.debug permission to use.

Examples:

Log a debug message:

$lib.log.debug('I am a debug message!')

Log a debug message with extra information:

$lib.log.debug('Extra information included here.', extra=({"key": $valu}))

Args:

mesg (str): The message to log.

extra (dict): Extra key / value pairs to include when structured logging is enabled on the Cortex.

Returns:

The return type is null.

$lib.log.error(mesg, extra=$lib.null)

Log a message to the Cortex at the error log level.

Notes:

This requires the storm.lib.log.error permission to use.

Examples:

Log an error message:

$lib.log.error('I am a error message!')

Log an error message with extra information:

$lib.log.error('Extra information included here.', extra=({"key": $valu}))

Args:

mesg (str): The message to log.

extra (dict): Extra key / value pairs to include when structured logging is enabled on the Cortex.

Returns:

The return type is null.

$lib.log.info(mesg, extra=$lib.null)

Log a message to the Cortex at the info log level.

Notes:

This requires the storm.lib.log.info permission to use.

Examples:

Log an info message:

$lib.log.info('I am a info message!')

Log an info message with extra information:

$lib.log.info('Extra information included here.', extra=({"key": $valu}))

Args:

mesg (str): The message to log.

extra (dict): Extra key / value pairs to include when structured logging is enabled on the Cortex.

Returns:

The return type is null.

$lib.log.warning(mesg, extra=$lib.null)

Log a message to the Cortex at the warning log level.

Notes:

This requires the storm.lib.log.warning permission to use.

Examples:

Log a warning message:

$lib.log.warning('I am a warning message!')

Log a warning message with extra information:

$lib.log.warning('Extra information included here.', extra=({"key": $valu}))

Args:

mesg (str): The message to log.

extra (dict): Extra key / value pairs to include when structured logging is enabled on the Cortex.

Returns:

The return type is null.

$lib.macro

A Storm Library for interacting with the Storm Macros in the Cortex.

$lib.macro.del(name)

Delete a Storm Macro by name from the Cortex.

Args:: name (str): The name of the macro to delete.
Returns:: The return type is null.

$lib.macro.get(name)

Get a Storm Macro definition by name from the Cortex.

Args:: name (str): The name of the macro to get.
Returns:: A macro definition. The return type is dict.

$lib.macro.grant(name, scope, iden, level)

Modify permissions granted to users/roles on a Storm Macro.

Args:

name (str): Name of the Storm Macro to modify.

scope (str): The scope, either “users” or “roles”.

iden (str): The user/role iden depending on scope.

level (int): The permission level number.

Returns:

The return type is null.

$lib.macro.list()

Get a list of Storm Macros in the Cortex.

Returns:: A list of dict objects containing Macro definitions. The return type is list.

$lib.macro.mod(name, info)

Modify user editable properties of a Storm Macro.

Args:

name (str): Name of the Storm Macro to modify.

info (dict): A dictionary of the properties to edit.

Returns:

The return type is null.

$lib.macro.set(name, storm)

Add or modify an existing Storm Macro in the Cortex.

Args:

name (str): Name of the Storm Macro to add or modify.

storm: The Storm query to add to the macro. The input type may be one of the following: str, storm:query.

Returns:

The return type is null.

$lib.math

A Storm library for performing math operations.

$lib.math.number(value)

Convert a value to a Storm Number object.

Storm Numbers are high precision fixed point decimals corresponding to the hugenum storage type.

This is not to be used for converting a string to an integer.

Args:: value (any): Value to convert.
Returns:: A Number object. The return type is number.

$lib.mime.html

A Storm library for manipulating HTML text.

$lib.mime.html.totext(html)

Return inner text from all tags within an HTML document.

Args:: html (str): The HTML text to be parsed.
Returns:: The newline-joined inner HTML text. The return type is str.

$lib.model

A Storm Library for interacting with the Data Model in the Cortex.

$lib.model.form(name)

Get a form object by name.

Args:: name (str): The name of the form to retrieve.
Returns:: The model:form instance if the form is present or null. The return type may be one of the following: model:form, null.

$lib.model.prop(name)

Get a prop object by name.

Args:: name (str): The name of the prop to retrieve.
Returns:: The model:property instance if the type if present or null. The return type may be one of the following: model:property, null.

$lib.model.tagprop(name)

Get a tag property object by name.

Args:: name (str): The name of the tag prop to retrieve.
Returns:: The model:tagprop instance of the tag prop if present or null. The return type may be one of the following: model:tagprop, null.

$lib.model.type(name)

Get a type object by name.

Args:: name (str): The name of the type to retrieve.
Returns:: The model:type instance if the type if present on the form or null. The return type may be one of the following: model:type, null.

$lib.model.deprecated

A storm library for interacting with the model deprecation mechanism.

$lib.model.deprecated.lock(name, locked)

Set the locked property for a deprecated model element.

Args:

name (str): The full path of the model element to lock.

locked (boolean): The lock status.

Returns:

The return type is null.

$lib.model.deprecated.locks()

Get a dictionary of the data model elements which are deprecated and their lock status in the Cortex.

Returns:: A dictionary of named elements to their boolean lock values. The return type is dict.

$lib.model.edge

A Storm Library for interacting with light edges and manipulating their key-value attributes. This Library is deprecated.

$lib.model.edge.del(verb, key)

Warning

$lib.model.edge.del has been deprecated and will be removed in version v3.0.0.

Delete a key from the key-value store for a verb.

Args:

verb (str): The name of the Edge verb to remove a key from.

key (str): The name of the key to remove from the key-value store.

Returns:

The return type is null.

$lib.model.edge.get(verb)

Warning

$lib.model.edge.get has been deprecated and will be removed in version v3.0.0.

Get the key-value data for a given Edge verb.

Args:: verb (str): The Edge verb to look up.
Returns:: A dictionary representing the key-value data set on a verb. The return type is dict.

$lib.model.edge.list()

Warning

$lib.model.edge.list has been deprecated and will be removed in version v3.0.0.

Get a list of (verb, key-value dictionary) pairs for Edge verbs in the current Cortex View.

Returns:: A list of (str, dict) tuples for each verb in the current Cortex View. The return type is list.

$lib.model.edge.set(verb, key, valu)

Warning

$lib.model.edge.set has been deprecated and will be removed in version v3.0.0.

Set a key-value for a given Edge verb.

Args:

verb (str): The Edge verb to set a value for.

key (str): The key to set.

valu (str): The value to set.

Returns:

The return type is null.

$lib.model.edge.validkeys()

Warning

$lib.model.edge.validkeys has been deprecated and will be removed in version v3.0.0.

Get a list of the valid keys that can be set on an Edge verb.

Returns:: A list of the valid keys. The return type is list.

$lib.model.ext

A Storm library for manipulating extended model elements.

$lib.model.ext.addEdge(n1form, verb, n2form, edgeinfo)

Add an extended edge definition to the data model.

Args:

n1form (str): The form of the n1 node. May be “*” or null to specify “any”.

verb (str): The edge verb, which must begin with “_”.

n2form (str): The form of the n2 node. May be “*” or null to specify “any”.

edgeinfo (dict): A Synapse edge info dictionary.

Returns:

The return type is null.

$lib.model.ext.addExtModel(model)

Add extended model elements to the Cortex from getExtModel().

Args:: model (dict): A model dictionary from getExtModel().
Returns:: The return type is boolean.

$lib.model.ext.addForm(formname, basetype, typeopts, typeinfo)

Add an extended form definition to the data model.

Args:

formname (str): The name of the form to add.

basetype (str): The base type the form is derived from.

typeopts (dict): A Synapse type opts dictionary.

typeinfo (dict): A Synapse form info dictionary.

Returns:

The return type is null.

$lib.model.ext.addFormProp(formname, propname, typedef, propinfo)

Add an extended property definition to the data model.

Args:

formname (str): The name of the form to add the property to.

propname (str): The name of the extended property.

typedef (list): A Synapse type definition tuple.

propinfo (dict): A synapse property definition dictionary.

Returns:

The return type is null.

$lib.model.ext.addTagProp(propname, typedef, propinfo)

Add an extended tag property definition to the data model.

Args:

propname (str): The name of the tag property.

typedef (list): A Synapse type definition tuple.

propinfo (dict): A synapse property definition dictionary.

Returns:

The return type is null.

$lib.model.ext.addType(typename, basetype, typeopts, typeinfo)

Add an extended type definition to the data model.

Args:

typename (str): The name of the type to add.

basetype (str): The base type the type is derived from.

typeopts (dict): A Synapse type opts dictionary.

typeinfo (dict): A Synapse type info dictionary.

Returns:

The return type is null.

$lib.model.ext.addUnivProp(propname, typedef, propinfo)

Add an extended universal property definition to the data model.

Args:

propname (str): The name of the universal property.

typedef (list): A Synapse type definition tuple.

propinfo (dict): A synapse property definition dictionary.

Returns:

The return type is null.

$lib.model.ext.delEdge(n1form, verb, n2form)

Remove an extended edge definition from the data model.

Args:

n1form (str): The form of the n1 node. May be “*” or null to specify “any”.

verb (str): The edge verb, which must begin with “_”.

n2form (str): The form of the n2 node. May be “*” or null to specify “any”.

Returns:

The return type is null.

$lib.model.ext.delForm(formname)

Remove an extended form definition from the model.

Args:: formname (str): The extended form to remove.
Returns:: The return type is null.

$lib.model.ext.delFormProp(formname, propname, force=$lib.false)

Remove an extended property definition from the model.

Args:

formname (str): The form with the extended property.

propname (str): The extended property to remove.

force (boolean): Delete the property from all nodes before removing the definition.

Returns:

The return type is null.

$lib.model.ext.delTagProp(propname, force=$lib.false)

Remove an extended tag property definition from the model.

Args:

propname (str): Name of the tag property to remove.

force (boolean): Delete the tag property from all nodes before removing the definition.

Returns:

The return type is null.

$lib.model.ext.delType(typename)

Remove an extended type definition from the model.

Args:: typename (str): The extended type to remove.
Returns:: The return type is null.

$lib.model.ext.delUnivProp(propname, force=$lib.false)

Remove an extended universal property definition from the model.

Args:

propname (str): Name of the universal property to remove.

force (boolean): Delete the property from all nodes before removing the definition.

Returns:

The return type is null.

$lib.model.ext.getExtModel()

Get all extended model elements.

Returns:: The return type is dict.

$lib.model.migration

A Storm library containing migration tools.

$lib.model.migration.copyData(src, dst, overwrite=$lib.false)

Copy node data from the src node to the dst node.

Args:

src (node): The node to copy data from.

dst (node): The node to copy data to.

overwrite (boolean): Copy data even if the key exists on the destination node.

Returns:

The return type is null.

$lib.model.migration.copyEdges(src, dst)

Copy edges from the src node to the dst node.

Args:

src (node): The node to copy edges from.

dst (node): The node to copy edges to.

Returns:

The return type is null.

$lib.model.migration.copyExtProps(src, dst)

Copy extended properties from the src node to the dst node.

Args:

src (node): The node to copy extended props from.

dst (node): The node to copy extended props to.

Returns:

The return type is null.

$lib.model.migration.copyTags(src, dst, overwrite=$lib.false)

Copy tags, tag timestamps, and tag props from the src node to the dst node.

Args:

src (node): The node to copy tags from.

dst (node): The node to copy tags to.

overwrite (boolean): Copy tag property value even if the property exists on the destination node.

Returns:

The return type is null.

$lib.model.migration.s

A Storm library for selectively migrating nodes in the current view.

$lib.model.migration.s.inetSslCertToTlsServerCert(n, nodata=$lib.false)

Create a inet:tls:servercert node from the provided inet:ssl:cert node.

Edits will be made to the inet:tls:servercert node in the current write layer.

Tags, tag properties, edges, and node data will be copied to the inet:tls:servercert node. However, existing tag properties and node data will not be overwritten.

Args:

n (node): The inet:ssl:cert node to migrate.

nodata (bool): Do not copy nodedata to the inet:tls:servercert node.

Returns:

The newly created inet:tls:servercert node. The return type is node.

$lib.model.migration.s.riskHasVulnToVulnerable(n, nodata=$lib.false)

Create a risk:vulnerable node from the provided risk:hasvuln node.

Edits will be made to the risk:vulnerable node in the current write layer.

If multiple vulnerable properties are set on the risk:hasvuln node multiple risk:vulnerable nodes will be created (each with a unique guid). Otherwise, a single risk:vulnerable node will be created with the same guid as the provided risk:hasvuln node. Extended properties will not be migrated.

Tags, tag properties, edges, and node data will be copied to the risk:vulnerable node. However, existing tag properties and node data will not be overwritten.

Args:

n (node): The risk:hasvuln node to migrate.

nodata (bool): Do not copy nodedata to the risk:vulnerable node.

Returns:

A list of idens for the risk:vulnerable nodes. The return type is list.

$lib.model.migration.s.model_0_2_31

A Storm library with helper functions for the 0.2.31 model it:sec:cpe migration.

$lib.model.migration.s.model_0_2_31.listNodes(form=$lib.null, source=$lib.null, offset=(0), size=$lib.null)

Yield queued nodes.

Args:

form (form): Only yield entries matching the specified form.

source (str): Only yield entries that were seen by the specified source.

offset (int): Skip this many entries.

size (int): Only yield up to this many entries.

Yields:

A tuple of (offset, form, valu, sources) values for the specified node. The return type is list.

$lib.model.migration.s.model_0_2_31.printNode(offset)

Print detailed queued node information.

Args:: offset (into): The offset of the queued node to print.
Returns:: The return type is null.

$lib.model.migration.s.model_0_2_31.repairNode(offset, newvalu, remove=$lib.false)

Repair a queued node.

Args:

offset (str): The node queue offset to repair.

newvalu (any): The new (corrected) node value.

remove (boolean): Specify whether to delete the repaired node from the queue.

Returns:

The queue node information The return type is dict.

$lib.model.tags

A Storm Library for interacting with tag specifications in the Cortex Data Model.

$lib.model.tags.del(tagname)

Delete a tag model specification.

Examples:

Delete the tag model specification for cno.threat:

$lib.model.tags.del(cno.threat)

Args:

tagname (str): The name of the tag.

Returns:

The return type is null.

$lib.model.tags.get(tagname)

Retrieve a tag model specification.

Examples:

Get the tag model specification for cno.threat:

$dict = $lib.model.tags.get(cno.threat)

Args:

tagname (str): The name of the tag.

Returns:

The tag model definition. The return type is dict.

$lib.model.tags.list()

List all tag model specifications.

Examples:

Iterate over the tag model specifications in the Cortex:

for ($name, $info) in $lib.model.tags.list() {
    ...
}

Returns:

List of tuples containing the tag name and model definition The return type is list.

$lib.model.tags.pop(tagname, propname)

Pop and return a tag model property.

Examples:

Remove the regex list from the cno.threat tag model:

$regxlist = $lib.model.tags.pop(cno.threat, regex)

Args:

tagname (str): The name of the tag.

propname (str): The name of the tag model property.

Returns:

The value of the property. The return type is prim.

$lib.model.tags.set(tagname, propname, propvalu)

Set a tag model property for a tag.

Examples:

Create a tag model for the cno.cve tag:

$regx = ($lib.null, $lib.null, "[0-9]{4}", "[0-9]{5}")
$lib.model.tags.set(cno.cve, regex, $regx)

Args:

tagname (str): The name of the tag.

propname (str): The name of the tag model property.

propvalu (prim): The value to set.

Returns:

The return type is null.

$lib.notifications

A Storm library for a user interacting with their notifications.

$lib.notifications.del(indx)

Delete a previously delivered notification.

Args:: indx (int): The index number of the notification to delete.
Retn:: Returns an ($ok, $valu) tuple. The return type is list.

$lib.notifications.get(indx)

Return a notification by ID (or $lib.null).

Args:: indx (int): The index number of the notification to return.
Retn:: The requested notification or $lib.null. The return type is dict.

$lib.notifications.list(size=$lib.null)

Yield (<indx>, <mesg>) tuples for a user’s notifications.

Args:: size (int): The max number of notifications to yield.
Yields:: Yields (useriden, time, mesgtype, msgdata) tuples. The return type is list.

$lib.pack

Packing / unpacking structured bytes.

$lib.pack.en(fmt, items)

Pack a sequence of items into an array of bytes.

Args:

fmt (str): A python struct.pack() format string.

items (list): A list of values to be packed.

Returns:

The packed byte structure. The return type is bytes.

$lib.pack.un(fmt, byts, offs=(0))

Unpack a sequence of items from an array of bytes.

Args:

fmt (str): A python struct.unpack() format string.

byts (bytes): Bytes to be unpacked

offs (int): The offset to begin unpacking from.

Returns:

The unpacked items. The return type is list.

$lib.pipe

A Storm library for interacting with non-persistent queues.

$lib.pipe.gen(filler, size=(10000))

Generate and return a Storm Pipe.

Notes:

The filler query is run in parallel with $pipe. This requires the permission storm.pipe.gen to use.

Examples:

Fill a pipe with a query and consume it with another:

$pipe = $lib.pipe.gen(${ $pipe.puts((1, 2, 3)) })

for $items in $pipe.slices(size=2) {
    $dostuff($items)
}

Args:

filler: A Storm query to fill the Pipe. The input type may be one of the following: str, storm:query.

size (int): Maximum size of the pipe.

Returns:

The pipe containing query results. The return type is pipe.

$lib.pkg

A Storm Library for interacting with Storm Packages.

$lib.pkg.add(pkgdef, verify=$lib.false)

Add a Storm Package to the Cortex.

Args:

pkgdef (dict): A Storm Package definition.

verify (boolean): Verify storm package signature.

Returns:

The return type is null.

$lib.pkg.del(name)

Delete a Storm Package from the Cortex.

Args:: name (str): The name of the package to delete.
Returns:: The return type is null.

$lib.pkg.deps(pkgdef)

Verify the dependencies for a Storm Package.

Args:: pkgdef (dict): A Storm Package definition.
Returns:: A dictionary listing dependencies and if they are met. The return type is dict.

$lib.pkg.get(name)

Get a Storm Package from the Cortex.

Args:: name (str): A Storm Package name.
Returns:: The Storm package definition. The return type is dict.

$lib.pkg.has(name)

Check if a Storm Package is available in the Cortex.

Args:: name (str): A Storm Package name to check for the existence of.
Returns:: True if the package exists in the Cortex, False if it does not. The return type is boolean.

$lib.pkg.list()

Get a list of Storm Packages loaded in the Cortex.

Returns:: A list of Storm Package definitions. The return type is list.

$lib.projects

A Storm Library for interacting with Projects in the Cortex.

$lib.projects.add(name, desc=)

Add a new project

Args:

name (str): The name of the Project to add

desc (str): A description of the overall project

Returns:

The newly created proj:project object The return type is proj:project.

$lib.projects.del(name)

Delete an existing project

Args:: name (str): The name of the Project to delete
Returns:: True if the project exists and gets deleted, otherwise False The return type is boolean.

$lib.projects.get(name)

Retrieve a project by name

Args:: name (str): The name of the Project to get
Returns:: The project object, if it exists, otherwise null The return type is proj:project.

$lib.ps

A Storm Library for interacting with running tasks on the Cortex.

$lib.ps.kill(prefix)

Stop a running task on the Cortex.

Args:: prefix (str): The prefix of the task to stop. Tasks will only be stopped if there is a single prefix match.
Returns:: True if the task was cancelled, False otherwise. The return type is boolean.

$lib.ps.list()

List tasks the current user can access.

Returns:: A list of task definitions. The return type is list.

$lib.queue

A Storm Library for interacting with persistent Queues in the Cortex.

$lib.queue.add(name)

Add a Queue to the Cortex with a given name.

Args:: name (str): The name of the queue to add.
Returns:: The return type is queue.

$lib.queue.del(name)

Delete a given named Queue.

Args:: name (str): The name of the queue to delete.
Returns:: The return type is null.

$lib.queue.gen(name)

Add or get a Storm Queue in a single operation.

Args:: name (str): The name of the Queue to add or get.
Returns:: The return type is queue.

$lib.queue.get(name)

Get an existing Storm Queue object.

Args:: name (str): The name of the Queue to get.
Returns:: A queue object. The return type is queue.

$lib.queue.list()

Get a list of the Queues in the Cortex.

Returns:: A list of queue definitions the current user is allowed to interact with. The return type is list.

$lib.random

A Storm library for generating random values.

$lib.random.generator(seed=$lib.null)

Make a random generator with a given seed.

Args:: seed (str): The seed value used for the random generator.
Returns:: The random generator object. The return type is random.

$lib.random.int(maxval, minval=(0))

Generate a random integer.

Args:

maxval (int): The maximum random value.

minval (int): The minimum random value.

Returns:

A random integer in the range min-max inclusive. The return type is int.

$lib.regex

A Storm library for searching/matching with regular expressions.

$lib.regex.escape(text)

Escape arbitrary strings for use in a regular expression pattern.

Example:

Escape node values for use in a regex pattern:
for $match in $lib.regex.findall($lib.regex.escape($node.repr()), $mydocument) {
    // do something with $match
}
Escape node values for use in regular expression filters:
it:dev:str~=$lib.regex.escape($node.repr())

Args:: text (str): The text to escape.
Returns:: Input string with special characters escaped. The return type is str.

$lib.regex.findall(pattern, text, flags=(0))

Search the given text for the patterns and return a list of matching strings.

Note:: If multiple matching groups are specified, the return value is a list of lists of strings.

Example:

Extract the matching strings from a piece of text:
for $x in $lib.regex.findall("G[0-9]{4}", "G0006 and G0001") {
    $dostuff($x)
}

Args:

pattern (str): The regular expression pattern.

text (str): The text to match.

flags (int): Regex flags to control the match behavior.

Returns:

A list of lists of strings for the matching groups in the pattern. The return type is list.

$lib.regex.flags.i

Regex flag to indicate that case insensitive matches are allowed.

Returns:: The type is int.

$lib.regex.flags.m

Regex flag to indicate that multiline matches are allowed.

Returns:: The type is int.

$lib.regex.matches(pattern, text, flags=(0))

Check if text matches a pattern. Returns $lib.true if the text matches the pattern, otherwise $lib.false.

Notes:

This API requires the pattern to match at the start of the string.

Example:

Check if the variable matches a expression:

if $lib.regex.matches("^[0-9]+.[0-9]+.[0-9]+$", $text) {
    $lib.print("It's semver! ...probably")
}

Args:

pattern (str): The regular expression pattern.

text (str): The text to match.

flags (int): Regex flags to control the match behavior.

Returns:

True if there is a match, False otherwise. The return type is boolean.

$lib.regex.replace(pattern, replace, text, flags=(0))

Replace any substrings that match the given regular expression with the specified replacement.

Example:

Replace a portion of a string with a new part based on a regex:

$norm = $lib.regex.replace("\sAND\s", " & ", "Ham and eggs!", $lib.regex.flags.i)

Args:

pattern (str): The regular expression pattern.

replace (str): The text to replace matching sub strings.

text (str): The input text to search/replace.

flags (int): Regex flags to control the match behavior.

Returns:

The new string with matches replaced. The return type is str.

$lib.regex.search(pattern, text, flags=(0))

Search the given text for the pattern and return the matching groups.

Note:

In order to get the matching groups, patterns must use parentheses to indicate the start and stop of the regex to return portions of. If groups are not used, a successful match will return a empty list and a unsuccessful match will return $lib.null.

Example:

Extract the matching groups from a piece of text:

$m = $lib.regex.search("^([0-9])+.([0-9])+.([0-9])+$", $text)
if $m {
    ($maj, $min, $pat) = $m
}

Args:

pattern (str): The regular expression pattern.

text (str): The text to match.

flags (int): Regex flags to control the match behavior.

Returns:

A list of strings for the matching groups in the pattern. The return type is list.

$lib.scrape

A Storm Library for providing helpers for scraping nodes from text.

$lib.scrape.context(text)

Attempt to scrape information from a blob of text, getting the context information about the values found.

Notes:

This does call the scrape Storm interface if that behavior is enabled on the Cortex.

Examples:

Scrape some text and make nodes out of it:

for ($form, $valu, $info) in $lib.scrape.context($text) {
    [ ( *$form ?= $valu ) ]
}

Args:

text (str): The text to scrape

Yields:

A dictionary of scraped values, rule types, and offsets scraped from the text. The return type is dict.

$lib.scrape.genMatches(text, pattern, fangs=$lib.null, flags=(2))

genMatches is a generic helper function for constructing scrape interfaces using pure Storm.

It accepts the text, a regex pattern, and produce results that can easily be used to create

Notes:

The pattern must have a named regular expression match for the key valu using the named group syntax. For example (somekey\s)(?P<valu>[a-z0-9]+)\s.

Examples:

A scrape implementation with a regex that matches name keys in text:

$re="(Name\:\s)(?P<valu>[a-z0-9]+)\s"
$form="ps:name"

function scrape(text, form) {
        $ret = $lib.list()
        for ($valu, $info) in $lib.scrape.genMatches($text, $re) {
            $ret.append(($form, $valu, $info))
        }
        return ( $ret )
    }

Args:

text (str): The text to scrape

pattern (str): The regular expression pattern to match against.

fangs (list): A list of (src, dst) pairs to refang from text. The src must be equal or larger than the dst in length.

flags (int): Regex flags to use (defaults to IGNORECASE).

Yields:

The return type is list.

$lib.scrape.ndefs(text)

Attempt to scrape node form, value tuples from a blob of text.

Examples:

Scrape some text and attempt to make nodes out of it:

for ($form, $valu) in $lib.scrape($text) {
    [ ( *$form ?= $valu ) ]
}

Args:

text (str): The text to scrape

Yields:

A list of (form, value) tuples scraped from the text. The return type is list.

$lib.service

A Storm Library for interacting with Storm Services.

$lib.service.add(name, url)

Add a Storm Service to the Cortex.

Args:

name (str): Name of the Storm Service to add.

url (str): The Telepath URL to the Storm Service.

Returns:

The Storm Service definition. The return type is dict.

$lib.service.del(iden)

Remove a Storm Service from the Cortex.

Args:: iden (str): The iden of the service to remove.
Returns:: The return type is null.

$lib.service.get(name)

Get a Storm Service definition.

Args:: name (str): The local name, local iden, or remote name, of the service to get the definition for.
Returns:: A Storm Service definition. The return type is dict.

$lib.service.has(name)

Check if a Storm Service is available in the Cortex.

Args:: name (str): The local name, local iden, or remote name, of the service to check for the existence of.
Returns:: True if the service exists in the Cortex, False if it does not. The return type is boolean.

$lib.service.list()

List the Storm Service definitions for the Cortex.

Notes:: The definition dictionaries have an additional ready key added to them to indicate if the Cortex is currently connected to the Storm Service or not.
Returns:: A list of Storm Service definitions. The return type is list.

$lib.service.wait(name, timeout=$lib.null)

Wait for a given service to be ready.

Notes:

If a timeout value is not specified, this will block a Storm query until the service is available.

Args:

name (str): The name, or iden, of the service to wait for.

timeout (int): Number of seconds to wait for the service.

Returns:

Returns true if the service is available, false on a timeout waiting for the service to be ready. The return type is boolean.

$lib.spooled

A Storm Library for interacting with Spooled Objects.

$lib.spooled.set(*vals)

Get a Spooled Storm Set object.

A Spooled Storm Set object is memory-safe to grow to extraordinarily large sizes, as it will fallback to file backed storage, with two restrictions. First is that all items in the set can be serialized to a file if the set grows too large, so all items added must be a serializable Storm primitive. Second is that when an item is added to the Set, because it could be immediately written disk, do not hold any references to it outside of the Set itself, as the two objects could differ.

Args:: *vals (any): Initial values to place in the set.
Returns:: The new set. The return type is set.

$lib.stats

A Storm Library for statistics related functionality.

$lib.stats.tally()

Get a Tally object.

Returns:: A new tally object. The return type is stat:tally.

$lib.stix

A Storm Library for interacting with Stix Version 2.1 CS02.

$lib.stix.lift(bundle)

Lift nodes from a STIX Bundle made by Synapse.

Notes:

This lifts nodes using the Node definitions embedded into the bundle when created by Synapse using custom extension properties.

Examples:

Lifting nodes from a STIX bundle:

yield $lib.stix($bundle)

Args:

bundle (dict): The STIX bundle to lift nodes from.

Yields:

Yields nodes The return type is node.

$lib.stix.validate(bundle)

Validate a STIX Bundle.

Notes:

This returns a dictionary containing the following values:

{
    'ok': <boolean> - False if bundle is invalid, True otherwise.
    'mesg': <str> - An error message if there was an error when validating the bundle.
    'results': The results of validating the bundle.
}

Args:

bundle (dict): The stix bundle to validate.

Returns:

Results dictionary. The return type is dict.

$lib.stix.export

A Storm Library for exporting to STIX version 2.1 CS02.

$lib.stix.export.bundle(config=$lib.null)

Return a new empty STIX bundle.

The config argument maps synapse forms to stix types and allows you to specify how to resolve STIX properties and relationships. The config expects to following format:

{
    "maxsize": 10000,

    "forms": {
        <formname>: {
            "default": <stixtype0>,
            "stix": {
                <stixtype0>: {
                    "props": {
                        <stix_prop_name>: <storm_with_return>,
                        ...
                    },
                    "rels": (
                        ( <relname>, <target_stixtype>, <storm> ),
                        ...
                    ),
                    "revs": (
                        ( <revname>, <source_stixtype>, <storm> ),
                        ...
                    )
                },
                <stixtype1>: ...
            },
        },
    },
},

For example, the default config includes the following entry to map ou:campaign nodes to stix campaigns:

{ "forms": {
    "ou:campaign": {
        "default": "campaign",
        "stix": {
            "campaign": {
                "props": {
                    "name": "{+:name return(:name)} return($node.repr())",
                    "description": "+:desc return(:desc)",
                    "objective": "+:goal :goal -> ou:goal +:name return(:name)",
                    "created": "return($lib.stix.export.timestamp(.created))",
                    "modified": "return($lib.stix.export.timestamp(.created))",
                },
                "rels": (
                    ("attributed-to", "threat-actor", ":org -> ou:org"),
                    ("originates-from", "location", ":org -> ou:org :hq -> geo:place"),
                    ("targets", "identity", "-> risk:attack :target:org -> ou:org"),
                    ("targets", "identity", "-> risk:attack :target:person -> ps:person"),
                ),
            },
        },
}},

You may also specify pivots on a per form+stixtype basis to automate pivoting to additional nodes to include in the bundle:

{"forms": {
    "inet:fqdn":
        ...
        "domain-name": {
            ...
            "pivots": [
                {"storm": "-> inet:dns:a -> inet:ipv4", "stixtype": "ipv4-addr"}
            ]
        {
    }
}

Note:: The default config is an evolving set of mappings. If you need to guarantee stable output please specify a config.
Args:: config (dict): The STIX bundle export config to use.
Returns:: A new stix:bundle instance. The return type is stix:bundle.

$lib.stix.export.config()

Construct a default STIX bundle export config.

Returns:: A default STIX bundle export config. The return type is dict.

$lib.stix.export.timestamp(tick)

Format an epoch milliseconds timestamp for use in STIX output.

Args:: tick (time): The epoch milliseconds timestamp.
Returns:: A STIX formatted timestamp string. The return type is str.

$lib.stix.import

A Storm Library for importing Stix Version 2.1 data.

$lib.stix.import.config()

Return an editable copy of the default STIX ingest config.

Returns:: A copy of the default STIX ingest configuration. The return type is dict.

$lib.stix.import.ingest(bundle, config=$lib.null)

Import nodes from a STIX bundle.

Args:

bundle (dict): The STIX bundle to ingest.

config (dict): An optional STIX ingest configuration.

Yields:

Yields nodes The return type is node.

$lib.storm

A Storm library for evaluating dynamic storm expressions.

$lib.storm.eval(text, cast=$lib.null)

Evaluate a Storm runtime value and optionally cast/coerce it.

Note:

If Storm logging is enabled, the expression being evaluated will be logged separately.

Args:

text (str): A storm expression string.

cast (str): A type to cast the result to.

Returns:

The value of the expression and optional cast. The return type is any.

$lib.storm.run(query, opts=$lib.null)

Run a Storm query and yield the messages output by the Storm interpreter.

Note:

If Storm logging is enabled, the query being run will be logged separately.

Args:

query (str): A Storm query string.

opts (dict): Storm options dictionary.

Yields:

The output messages from the Storm runtime. The return type is list.

$lib.str

A Storm Library for interacting with strings.

$lib.str.concat(*args)

Concatenate a set of strings together.

Args:: *args (any): Items to join together.
Returns:: The joined string. The return type is str.

$lib.str.format(text, **kwargs)

Format a text string.

Examples:

Format a string with a fixed argument and a variable:

cli> storm $list=(1,2,3,4)
     $str=$lib.str.format('Hello {name}, your list is {list}!', name='Reader', list=$list)
     $lib.print($str)

Hello Reader, your list is ['1', '2', '3', '4']!

Args:

text (str): The base text string.

**kwargs (any): Keyword values which are substituted into the string.

Returns:

The new string. The return type is str.

$lib.str.join(sepr, items)

Join items into a string using a separator.

Examples:

Join together a list of strings with a dot separator:

cli> storm $foo=$lib.str.join('.', ('rep', 'vtx', 'tag')) $lib.print($foo)

rep.vtx.tag

Args:

sepr (str): The separator used to join strings with.

items (list): A list of items to join together.

Returns:

The joined string. The return type is str.

$lib.tabular

A Storm Library for creating printable tables.

$lib.tabular.printer(conf)

Construct a new printer.

Examples:

Create a simple table using the default separators:

$conf = ({
    "columns": [
        {"name": "Year", "width": 4},
        {"name": "Author", "width": 20},
        {"name": "Title", "width": 12},
    ]
})

$printer = $lib.tabular.printer($conf)

$lib.print($printer.header())

for ($year, $author, $title, $publisher) in $data {
    $lib.print($printer.row(($year, $author, $title))
}

Create a configuration with custom separators and column options:

$conf = ({
    "separators": {
        "row:outline": true,
        "column:outline": true,
        "header:row": "#",
        "data:row": "*",
        "column": "+",
    },
    "columns": [
        {"name": "Year", "width": 4, "justify": "right"},
        {"name": "Author", "width": 20, "justify": "center"},
        {"name": "Title", "width": 12, "overflow": "wrap"},
    ]
})

$printer = $lib.tabular.printer($conf)

Args:

conf (dict): The table configuration dictionary.

Returns:

The newly constructed tabular:printer. The return type is tabular:printer.

$lib.tabular.schema()

Get a copy of the table configuration schema.

Examples:

Print a human-readable version of the schema:

$schema = $lib.tabular.schema()
$lib.print($lib.yaml.save($schema))

Returns:

The table configuration schema. The return type is dict.

$lib.tags

Storm utility functions for tags.

$lib.tags.prefix(names, prefix, ispart=$lib.false)

Normalize and prefix a list of syn:tag:part values so they can be applied.

Examples:

Add tag prefixes and then use them to tag nodes:

$tags = $lib.tags.prefix($result.tags, vtx.visi)
{ for $tag in $tags { [ +#$tag ] } }

Args:

names (list): A list of syn:tag:part values to normalize and prefix.

prefix (str): The string prefix to add to the syn:tag:part values.

ispart (boolean): Whether the names have already been normalized. Normalization will be skipped if set to true.

Returns:

A list of normalized and prefixed syn:tag values. The return type is list.

$lib.telepath

A Storm Library for making Telepath connections to remote services.

$lib.telepath.open(url)

Open and return a Telepath RPC proxy.

Args:: url (str): The Telepath URL to connect to.
Returns:: A object representing a Telepath Proxy. The return type is telepath:proxy.

$lib.time

A Storm Library for interacting with timestamps.

$lib.time.day(tick)

Returns the day part of a time value.

Args:: tick (time): A time value.
Returns:: The day part of the time expression. The return type is int.

$lib.time.dayofmonth(tick)

Returns the index (beginning with 0) of the day within the month.

Args:: tick (time): A time value.
Returns:: The index of the day within month. The return type is int.

$lib.time.dayofweek(tick)

Returns the index (beginning with monday as 0) of the day within the week.

Args:: tick (time): A time value.
Returns:: The index of the day within week. The return type is int.

$lib.time.dayofyear(tick)

Returns the index (beginning with 0) of the day within the year.

Args:: tick (time): A time value.
Returns:: The index of the day within year. The return type is int.

$lib.time.format(valu, format)

Format a Synapse timestamp into a string value using datetime.strftime().

Examples:

Format a timestamp into a string:

cli> storm $now=$lib.time.now() $str=$lib.time.format($now, '%A %d, %B %Y') $lib.print($str)

Tuesday 14, July 2020

Args:

valu (int): A timestamp in epoch milliseconds.

format (str): The strftime format string.

Returns:

The formatted time string. The return type is str.

$lib.time.fromunix(secs)

Normalize a timestamp from a unix epoch time in seconds to milliseconds.

Examples:

Convert a timestamp from seconds to millis and format it:

cli> storm $seconds=1594684800 $millis=$lib.time.fromunix($seconds)
     $str=$lib.time.format($millis, '%A %d, %B %Y') $lib.print($str)

Tuesday 14, July 2020

Args:

secs (int): Unix epoch time in seconds.

Returns:

The normalized time in milliseconds. The return type is int.

$lib.time.hour(tick)

Returns the hour part of a time value.

Args:: tick (time): A time value.
Returns:: The hour part of the time expression. The return type is int.

$lib.time.minute(tick)

Returns the minute part of a time value.

Args:: tick (time): A time value.
Returns:: The minute part of the time expression. The return type is int.

$lib.time.month(tick)

Returns the month part of a time value.

Args:: tick (time): A time value.
Returns:: The month part of the time expression. The return type is int.

$lib.time.monthofyear(tick)

Returns the index (beginning with 0) of the month within the year.

Args:: tick (time): A time value.
Returns:: The index of the month within year. The return type is int.

$lib.time.now()

Get the current epoch time in milliseconds.

Returns:: Epoch time in milliseconds. The return type is int.

$lib.time.parse(valu, format, errok=$lib.false)

Parse a timestamp string using datetime.strptime() into an epoch timestamp.

Examples:

Parse a string as for its month/day/year value into a timestamp:

cli> storm $s='06/01/2020' $ts=$lib.time.parse($s, '%m/%d/%Y') $lib.print($ts)

1590969600000

Args:

valu (str): The timestamp string to parse.

format (str): The format string to use for parsing.

errok (boolean): If set, parsing errors will return $lib.null instead of raising an exception.

Returns:

The epoch timestamp for the string. The return type is int.

$lib.time.second(tick)

Returns the second part of a time value.

Args:: tick (time): A time value.
Returns:: The second part of the time expression. The return type is int.

$lib.time.sleep(valu)

Pause the processing of data in the storm query.

Notes:: This has the effect of clearing the Snap’s cache, so any node lifts performed after the $lib.time.sleep(...) executes will be lifted directly from storage.
Args:: valu (int): The number of seconds to pause for.
Returns:: The return type is null.

$lib.time.ticker(tick, count=$lib.null)

Periodically pause the processing of data in the storm query.

Notes:

This has the effect of clearing the Snap’s cache, so any node lifts performed after each tick will be lifted directly from storage.

Args:

tick (int): The amount of time to wait between each tick, in seconds.

count (int): The number of times to pause the query before exiting the loop. This defaults to None and will yield forever if not set.

Yields:

This yields the current tick count after each time it wakes up. The return type is int.

$lib.time.toUTC(tick, timezone)

Adjust an epoch milliseconds timestamp to UTC from the given timezone.

Args:

tick (time): A time value.

timezone (str): A timezone name. See python pytz docs for options.

Returns:

An ($ok, $valu) tuple. The return type is list.

$lib.time.year(tick)

Returns the year part of a time value.

Args:: tick (time): A time value.
Returns:: The year part of the time expression. The return type is int.

$lib.trigger

A Storm Library for interacting with Triggers in the Cortex.

$lib.trigger.add(tdef)

Add a Trigger to the Cortex.

Args:: tdef (dict): A Trigger definition.
Returns:: The new trigger. The return type is trigger.

$lib.trigger.del(prefix)

Delete a Trigger from the Cortex.

Args:: prefix (str): A prefix to match in order to identify a trigger to delete. Only a single matching prefix will be deleted.
Returns:: The iden of the deleted trigger which matched the prefix. The return type is str.

$lib.trigger.disable(prefix)

Disable a Trigger in the Cortex.

Args:: prefix (str): A prefix to match in order to identify a trigger to disable. Only a single matching prefix will be disabled.
Returns:: The iden of the trigger that was disabled. The return type is str.

$lib.trigger.enable(prefix)

Enable a Trigger in the Cortex.

Args:: prefix (str): A prefix to match in order to identify a trigger to enable. Only a single matching prefix will be enabled.
Returns:: The iden of the trigger that was enabled. The return type is str.

$lib.trigger.get(iden)

Get a Trigger in the Cortex.

Args:: iden (str): The iden of the Trigger to get.
Returns:: The requested trigger object. The return type is trigger.

$lib.trigger.list(all=$lib.false)

Get a list of Triggers in the current view or every view.

Args:: all (boolean): Get a list of all the readable Triggers in every readable View.
Returns:: A list of trigger objects the user is allowed to access. The return type is list.

$lib.trigger.mod(prefix, query)

Modify an existing Trigger in the Cortex.

Args:

prefix (str): A prefix to match in order to identify a trigger to modify. Only a single matching prefix will be modified.

query: The new Storm query to set as the trigger query. The input type may be one of the following: str, storm:query.

Returns:

The iden of the modified Trigger The return type is str.

$lib.user

A Storm Library for interacting with data about the current user.

$lib.user.allowed(permname, gateiden=$lib.null, default=$lib.false)

Check if the current user has a given permission.

Args:

permname (str): The permission string to check.

gateiden (str): The authgate iden.

default (boolean): The default value.

Returns:

True if the user has the requested permission, false otherwise. The return type is boolean.

$lib.user.iden

The user GUID for the current storm user.

Returns:: The type is str.

$lib.user.name()

Get the name of the current runtime user.

Returns:: The username. The return type is str.

$lib.user.profile

Get a dictionary representing the current user’s profile information.

Returns:: The type is auth:user:profile.

$lib.user.vars

Get a dictionary representing the current user’s persistent variables.

Returns:: The type is auth:user:vars.

$lib.vars

A Storm Library for interacting with runtime variables.

$lib.vars.del(name)

Unset a variable in the current Runtime.

Args:: name (str): The variable name to remove.
Returns:: The return type is null.

$lib.vars.get(name, defv=$lib.null)

Get the value of a variable from the current Runtime.

Args:

name (str): Name of the variable to get.

defv (prim): The default value returned if the variable is not set in the runtime.

Returns:

The value of the variable. The return type is any.

$lib.vars.list()

Get a list of variables from the current Runtime.

Returns:: A list of variable names and their values for the current Runtime. The return type is list.

$lib.vars.set(name, valu)

Set the value of a variable in the current Runtime.

Args:

name (str): Name of the variable to set.

valu (prim): The value to set the variable too.

Returns:

The return type is null.

$lib.vars.type(valu)

Get the type of the argument value.

Args:: valu (any): Value to inspect.
Returns:: The type of the argument. The return type is str.

$lib.vault

A Storm Library for interacting with vaults.

$lib.vault.add(name, vtype, scope, owner, secrets, configs)

Create a new vault.

Args:

name (str): The name of the new vault.

vtype (str): The type of this vault.

scope (str): Scope for this vault. One of “user”, “role”, “global”, or $lib.null for unscoped vaults.

owner (str): User/role iden for this vault if scope is “user” or “role”. None for “global” scope vaults.

secrets (dict): The initial secret data to store in this vault.

configs (dict): The initial config data to store in this vault.

Returns:

Iden of the newly created vault. The return type is str.

$lib.vault.byname(name)

Get a vault by name.

Args:

name (str):: The name of the vault to retrieve. If user only has PERM_READ, the secrets data will not be returned. If the user has PERM_EDIT or higher, secrets data will be included in the vault.

Returns:

The requested vault. The return type is dict.

$lib.vault.bytype(vtype, scope=$lib.null)

Get a vault for a specified vault type.

Args:

vtype (str): The vault type to retrieved.

scope (str): The scope for the specified type. If $lib.null, then getByType will search.

Returns:

Vault or $lib.null if the vault could not be retrieved. The return type is vault.

$lib.vault.get(iden)

Get a vault by iden.

Args:

iden (str):: The iden of the vault to retrieve. If user only has PERM_READ, the secrets data will not be returned. If the user has PERM_EDIT or higher, secrets data will be included in the vault.

Returns:

The requested vault. The return type is dict.

$lib.vault.list()

List vaults accessible to the current user.

Returns:: Yields vaults. The return type is list.

$lib.vault.print(vault)

Print the details of the specified vault.

Args:: vault (dict): The vault to print.
Returns:: The return type is null.

$lib.version

A Storm Library for interacting with version information.

$lib.version.commit()

The synapse commit hash for the local Cortex.

Returns:: The commit hash. The return type is str.

$lib.version.matches(vertup, reqstr)

Check if the given version triple meets the requirements string.

Examples:

Check if the synapse version is in a range:

$synver = $lib.version.synapse()
if $lib.version.matches($synver, ">=2.9.0") {
    $dostuff()
}

Args:

vertup (list): Triple of major, minor, and patch version integers.

reqstr (str): The version string to compare against.

Returns:

True if the version meets the requirements, False otherwise. The return type is boolean.

$lib.version.synapse()

The synapse version tuple for the local Cortex.

Returns:: The version triple. The return type is list.

$lib.view

A Storm Library for interacting with Views in the Cortex.

$lib.view.add(layers, name=$lib.null, worldreadable=$lib.false)

Add a View to the Cortex.

Args:

layers (list): A list of layer idens which make up the view.

name (str): The name of the view.

worldreadable (boolean): Grant read access to the all role.

Returns:

A view object representing the new View. The return type is view.

$lib.view.del(iden)

Delete a View from the Cortex.

Args:: iden (str): The iden of the View to delete.
Returns:: The return type is null.

$lib.view.get(iden=$lib.null)

Get a View from the Cortex.

Args:: iden (str): The iden of the View to get. If not specified, returns the current View.
Returns:: The storm view object. The return type is view.

$lib.view.list(deporder=$lib.false)

List the Views in the Cortex.

Args:: deporder (boolean): Return the lists in bottom-up dependency order.
Returns:: List of view objects. The return type is list.

$lib.xml

A Storm library for parsing XML.

$lib.xml.parse(valu)

Parse an XML string into an xml:element tree.

Args:: valu (str): The XML string to parse into an xml:element tree.
Returns:: An xml:element for the root node of the XML tree. The return type is xml:element.

$lib.yaml

A Storm Library for saving/loading YAML data.

$lib.yaml.load(valu)

Decode a YAML string/bytes into an object.

Args:: valu (str): The string to decode.
Returns:: The decoded primitive object. The return type is prim.

$lib.yaml.save(valu, sort_keys=$lib.true)

Encode data as a YAML string.

Args:

valu (object): The object to encode.

sort_keys (boolean): Sort object keys.

Returns:

A YAML string. The return type is str.