Storm Package: synapse-search
The following Commands are available from this package. This documentation is generated for version 4.18.0 of the package.
Storm Commands
This package implements the following Storm Commands.
search
Search the full text indexed node properties for a token, yielding nodes in the current view.
The search can be executed on a specific property, or all indexed properties.
In either case results are returned as matching nodes in descending order of match quality.
The tokenization is best suited to English text searching, and therefore queries such as IP addresses
may not return quality results. Search terms must also be at least 5 characters in length.
Examples:
// Search all indexed properties for one token
search gibson
// Search all indexed properties for multiple tokens
search billy gibson books
// Search a specific property for multiple tokens
search --prop media:news:summary billy gibson books
// Search parsed files
search --prop file:bytes:_text billy gibson
// Search all properties in a given form (e.g. media:news:title, media:news:summary, etc.)
search --form media:news billy gibson
// Increase size to return more matching results
search --size 1000 billy gibson books
Usage: search [options] <terms>
Options:
--help : Display the command usage.
--prop <prop> : Search only the given fully qualified property name.
--form <form> : Search all properties within the given form.
--size <size> : Limit the number of results to the given size. (default: 100)
--debug : Enable debug logging message.
Arguments:
<terms> [<terms> ...] : Search terms to query for as ngrams.
search.files.add
Index text for a file extracted by the FileParser service.
This command requires the FileParser to be loaded in the Cortex.
Examples:
// Add files to a queue to extract text
file:bytes#my.files | search.files.add
Usage: search.files.add [options]
Options:
--help : Display the command usage.
--debug : Enable debug logging message.
search.index.add
Add an index for the given fully qualified property for all layers in the Cortex.
This command will index all existing and future nodes matching the property.
Examples:
// Index the ps:person:name property
search.index.add ps:person:name
Usage: search.index.add [options] <prop>
Options:
--help : Display the command usage.
--priority <priority> : Priority relative to other indices to process initial indexing. (default: 50)
--debug : Enable debug logging message.
Arguments:
<prop> : The fully qualified property name to index.
search.index.del
Delete an index for a specific field for all layers.
This command will delete all index data for the field and not restart indexing.
Examples:
// Basic usage
search.index.del ps:person:name
// Delete index for parsed files
search.index.del file:bytes:_text
Usage: search.index.del [options] <prop>
Options:
--help : Display the command usage.
--debug : Enable debug logging message.
Arguments:
<prop> : The fully qualified property name to delete the index for.
search.index.reload
Delete index data and start indexing again.
This command will delete all the index data and start over. If --fullreset
option is provided, the existing fields will also be cleared (including default properties).
Usage: search.index.reload [options]
Options:
--help : Display the command usage.
--fullreset : Delete all field information.
--debug : Enable debug logging message.
search.status
Print the current indexing status.
Usage: search.status [options]
Options:
--help : Display the command usage.
--debug : Display the status for individual properties.
Storm Modules
This package implements the following Storm Modules.
search
getViewLayers()
Get a list of layer idens in the current view.
- Returns:
List of layer idens. The return type is
list
.
getAllLayers(warn_noedits=$lib.true)
Get a list of all layer idens in the Cortex.
- Args:
warn_noedits (bool): Whether to warn if a layer does not have logedits enabled.
- Returns:
List of layer idens. The return type is
list
.
getNodeInfo(iden, prop=$lib.null, lyrs=$lib.null)
Get index metadata for a node iden.
- Args:
iden (str): The iden of the node.
prop (str): Optionally get the metadata for a fully qualified node property.
lyrs (list): List of layer idens; if not provided defaults to the current view layers.
- Returns:
An ok bool flag and an info dict when ok=$lib.true. The return type is
list
.
iterNodes(prop=$lib.null, lyrs=$lib.null)
Iterate over indexed nodes.
if $lib.debug=$lib.true, index metadata will also be printed for each node.
- Args:
prop (str): Only return nodes indexed for the fully qualified node property.
lyrs (list): List of layer idens; if not provided defaults to the current view layers.
- Yields:
The indexed nodes. The return type is
node
.
queryNodes(terms, prop=$lib.null, form=$lib.null, lyrs=$lib.null)
Yield nodes from a query.
- Args:
terms (list): A list of string terms to query.
prop (str): Only return nodes indexed for the fully qualified node property.
form (str): Only return nodes indexed for any property within a given form.
lyrs (list): List of layer idens; if not provided defaults to the current view layers.
- Yields:
Matching nodes. The return type is
node
.
multiQueryNodes(terms, props, lyrs=$lib.null)
Yield nodes from a query for multiple props. Results are merged based on match score.
- Args:
terms (list): A list of string terms to query.
props (str): Only return nodes indexed for the fully qualified node properties.
lyrs (list): List of layer idens; if not provided defaults to the current view layers.
- Yields:
Matching nodes. The return type is
node
.
search(tokens)
Emit matches in the current view for a given set of tokens.
- Args:
tokens (list): A list of string terms to query.
- Yields:
List of integer score for the match and the node buid. The return type is
list
.
addIndex(prop, priority=(50))
Add a new index.
- Args:
prop (str): Fully qualified property name to add.
priority (int): Priority to assign to the indexing operation.
- Returns:
An ok bool flag and a message string. The return type is
list
.
delIndex(prop)
Delete an index.
- Args:
prop (str): Fully qualified property name to delete.
- Returns:
An ok bool flag and a message string. The return type is
list
.
reindex(fullreset=$lib.false)
Reindex all fields or drop all indexes if fullreset is True.
- Args:
fullreset (bool): If False existing fields are reindexed from stored text, else they are dropped.
- Returns:
An ok bool flag and a message string. The return type is
list
.
getStatus(all_fields=$lib.false)
Get the current indexing status.
- Args:
all_fields (bool): If $lib.false, only return fields being actively indexed.
- Returns:
A dict with keys “progress” and “fields”. The return type is
dict
.
addToknRoot(tokn, root)
Add a custom root for a token.
- Args:
tokn (str): Token string.
root (str): Custom root string.
- Returns:
An ok bool flag and a message string. The return type is
list
.
delToknRoot(tokn)
Delete a custom root for a token.
- Args:
tokn (str): Token string.
- Returns:
An ok bool flag and a message string. The return type is
list
.
getToknRoot(tokn)
Get the custom root assigned for a token.
- Args:
tokn (str): Token string.
- Returns:
An ok bool flag and the assigned root or an error message string. The return type is
list
.
listToknRoots(size=$lib.null)
List custom token root assignments.
- Args:
size (int): List up to <size> assignments.
- Yields:
Token-root pairs. The return type is
list
.