Storm Reference - Storm Commands¶
Storm commands are built-in or custom commands that can be used with the Synapse storm command itself. As such, Storm commands must be executed following the storm
command:
storm <command>
or
storm <query> | <command> [ | <query> ]
The pipe symbol ( |
) is used to send (pipe) the output from a Storm query to any of the Storm commands, and to send the output from a Storm command back to a Storm query.
Built-in commands are native to the Storm library and loaded by default within a given Cortex. Built-in commands comprise a set of helper commands that perform a variety of specialized tasks that are useful regardless of the types of data stored in the Cortex or the types of analysis performed.
Custom commands are Storm commands that have been added to a Cortex to invoke the execution of dynamically loaded modules. Dynamically loaded modules are typically custom modules that have been added to Synapse to support domain-specific analysis. For example, a knowledge domain that requires tracking of IP addresses might have access to a third-party service such as Maxmind to obtain up-to-date data on the assigned Autonomous System (AS) or geographical location of a given IP address. A custom maxmind
module and associated Storm command could be added to Synapse to query the Maxmind database and update the appropriate secondary properties on the associated inet:ipv4
nodes directly from Storm.
The full list of storm commands (built-in and custom) available in a given Cortex can be displayed with storm help
.
Help for a specific Storm command can be displayed with storm <command> --help
.
This section details the usage and syntax for built-in Storm commands. Many of the commands below, such as count
, limit
, max
/ min
, or delnode
, directly support analyst tasks within Storm. Other commands, such as those used to manage daemons, queues, packages, or services, may be primarily of interest to Syanpse administrators or developers.
- help
- background
- count
- cron
- delnode
- dmon
- edges
- feed
- graph
- iden
- layer
- lift
- limit
- macro
- max
- min
- model
- movetag
- parallel
- pkg
- ps
- queue
- reindex
- scrape
- service
- sleep
- spin
- splice
- tee
- tree
- trigger
- uniq
- view
See Storm Reference - Document Syntax Conventions for an explanation of the syntax format used below.
The Storm query language is covered in detail starting with the Storm Reference - Introduction section of the Synapse User Guide.
Note
Storm commands, including custom commands, are added to the Cortex as runtime nodes (“runt nodes” - see Node, Runt) of the form syn:cmd
. These runt nodes can be lifted and filtered just like standard nodes in a Cortex.
Example
Lift the syn:cmd
node for the Storm movetag
command:
cli> storm syn:cmd=movetag
Executing query at 2021/01/22 01:09:18.537
syn:cmd=movetag
:doc = Rename an entire tag tree and preserve time intervals.
complete. 1 nodes in 10 ms (100/sec).
Note
While the documentation for built-in Storm commands is relatively basic, the syn:cmd
form within the data model includes additional secondary propeties that can be used to make commands (particularly custom commands) more “self-documenting” to users, because detail about the command can be introspected directly within the data model from within Storm. For example, the :input
and :output
properties can be used to specify a list (array) of forms that can be input (submitted) to the command and a list of forms that may be created by the command, respectively.
help¶
The help
command (storm help
) displays the list of available built-in commands and a brief message describing each command. Help on individual commands is available via <command> --help
.
Syntax:
cli> storm help
Executing query at 2021/01/22 01:09:18.558
package: synapse
background : Execute a query pipeline as a background task.
count : Iterate through query results, and print the resulting number of nodes
cron.add : Add a recurring cron job to a cortex.
cron.at : Adds a non-recurring cron job.
cron.cleanup : Delete all completed at jobs
cron.del : Delete a cron job from the cortex.
cron.disable : Disable a cron job in the cortex.
cron.enable : Enable a cron job in the cortex.
cron.list : List existing cron jobs in the cortex.
cron.mod : Modify an existing cron job's query.
cron.stat : Gives detailed information about a cron job.
delnode : Delete nodes produced by the previous query logic.
dmon.list : List the storm daemon queries running in the cortex.
edges.del : Bulk delete light edges from input nodes.
feed.list : List the feed functions available in the Cortex
graph : Generate a subgraph from the given input nodes and command line options.
help : List available commands and a brief description for each.
iden : Lift nodes by iden.
layer.add : Add a layer to the cortex.
layer.del : Delete a layer from the cortex.
layer.get : Get a layer from the cortex.
layer.list : List the layers in the cortex.
layer.set : Set a layer option.
lift.byverb : Lift nodes from the current view by an light edge verb.
limit : Limit the number of nodes generated by the query in the given position.
macro.del : Remove a macro definition from the cortex.
macro.exec : Execute a named macro.
macro.get : Display the storm query for a macro in the cortex.
macro.list : List the macros set on the cortex.
macro.set : Set a macro definition in the cortex.
max : Consume nodes and yield only the one node with the highest value for a property or variable.
merge : Merge edits from the incoming nodes down to the next layer.
min : Consume nodes and yield only the one node with the lowest value for a property.
model.deprecated.check: Check for lock status and the existance of deprecated model elements
model.deprecated.lock : Edit lock status of deprecated model elements.
model.deprecated.locks: Display lock status of deprecated model elements.
model.edge.del : Delete a global key-value pair for an edge verb in the current view.
model.edge.get : Retrieve key-value pairs an edge verb in the current view.
model.edge.list : List all edge verbs in the current view and their doc key (if set).
model.edge.set : Set an key-value for an edge verb that exists in the current view.
movetag : Rename an entire tag tree and preserve time intervals.
parallel : Execute part of a query pipeline in parallel.
pkg.del : Remove a storm package from the cortex.
pkg.list : List the storm packages loaded in the cortex.
pkg.load : Load a storm package from an HTTP URL.
ps.kill : Kill a running task/query within the cortex.
ps.list : List running tasks in the cortex.
queue.add : Add a queue to the cortex.
queue.del : Remove a queue from the cortex.
queue.list : List the queues in the cortex.
reindex : Use admin privileges to re index/normalize node properties.
scrape : Use textual properties of existing nodes to find other easily recognizable nodes.
service.add : Add a storm service to the cortex.
service.del : Remove a storm service from the cortex.
service.list : List the storm services configured in the cortex.
sleep : Introduce a delay between returning each result for the storm query.
spin : Iterate through all query results, but do not yield any.
splice.list : Retrieve a list of splices backwards from the end of the splicelog.
splice.undo : Reverse the actions of syn:splice runt nodes.
sudo : Deprecated sudo command.
tee : Execute multiple Storm queries on each node in the input stream, joining output streams together.
tree : Walk elements of a tree using a recursive pivot.
trigger.add : Add a trigger to the cortex.
trigger.del : Delete a trigger from the cortex.
trigger.disable : Disable a trigger in the cortex.
trigger.enable : Enable a trigger in the cortex.
trigger.list : List existing triggers in the cortex.
trigger.mod : Modify an existing trigger's query.
uniq : Filter nodes by their uniq iden values.
view.add : Add a view to the cortex.
view.del : Delete a view from the cortex.
view.exec : Execute a storm query in a different view.
view.fork : Fork a view in the cortex.
view.get : Get a view from the cortex.
view.list : List the views in the cortex.
view.merge : Merge a forked view into its parent view.
view.set : Set a view option.
wget : Retrieve bytes from a URL and store them in the axon. Yields inet:urlfile nodes.
For detailed help on any command, use <cmd> --help
complete. 0 nodes in 8 ms (0/sec).
background¶
The background
command allows you to execute a Storm query as a background task (e.g., to free up the cmdr CLI / Storm runtime for additional queries). Use of background
is a “fire-and-forget” process - any status messages (warnings or errors) are not returned to the console, and if the query is interrupted for any reason, it will not resume.
See also parallel.
Syntax:
cli> storm background --help
Executing query at 2021/01/22 01:09:18.585
Execute a query pipeline as a background task.
NOTE: Variables are passed through but nodes are not
Usage: background [options] <query>
Options:
--help : Display the command usage.
Arguments:
<query> : The query to execute in the background.
complete. 0 nodes in 8 ms (0/sec).
count¶
The count
command enumerates the number of nodes returned from a given Storm query and displays the resultant nodes and associated node count.
Syntax:
cli> storm count --help
Executing query at 2021/01/22 01:09:18.606
Iterate through query results, and print the resulting number of nodes
which were lifted. This does yield the nodes counted.
Example:
foo:bar:size=20 | count
Usage: count [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Count the number of email address nodes:
inet:email | count
- Count the number of DNS A records for the domain woot.com:
inet:dns:a:fqdn=woot.com | count
Usage Notes:
count
does not consume nodes, so Storm will stream the nodes being counted to the CLI output while the command executes. To count nodes without streaming the output,count
can be piped to the spin command (i.e., <query> | count | spin).Spin
consumes nodes and so will prevent nodes processed by thecount
command from streaming.
cron¶
Note
See the Storm Reference - Automation guide for additional background on cron jobs (as well as triggers and macros), including examples.
Storm includes cron.*
commands that allow you to create scheduled Cron jobs. Within Synapse, jobs are Storm queries that execute within a Cortex on a recurring or non-recurring (cron.at
) basis.
Help for individual cron.*
commands can be displayed using:
storm <command> --help | -h
Cron jobs (including jobs created with cron.at
) are added to the Cortex as runtime nodes (“runt nodes” - see Node, Runt) of the form syn:cron
. These runt nodes can be lifted and filtered just like standard nodes in a Cortex.
Note
The Storm cron.*
commands replace the Synapse cmdr-based cron command, which is being deprecated.
cron.add¶
The cron.add
command creates an individual cron job within a Cortex.
Syntax:
cli> storm cron.add --help
Executing query at 2021/01/22 01:09:18.759
Add a recurring cron job to a cortex.
Notes:
All times are interpreted as UTC.
All arguments are interpreted as the job period, unless the value ends in
an equals sign, in which case the argument is interpreted as the recurrence
period. Only one recurrence period parameter may be specified.
Currently, a fixed unit must not be larger than a specified recurrence
period. i.e. '--hour 7 --minute +15' (every 15 minutes from 7-8am?) is not
supported.
Value values for fixed hours are 0-23 on a 24-hour clock where midnight is 0.
If the --day parameter value does not start with a '+' and is an integer, it is
interpreted as a fixed day of the month. A negative integer may be
specified to count from the end of the month with -1 meaning the last day
of the month. All fixed day values are clamped to valid days, so for
example '-d 31' will run on February 28.
If the fixed day parameter is a value in ([Mon, Tue, Wed, Thu, Fri, Sat,
Sun] if locale is set to English) it is interpreted as a fixed day of the
week.
Otherwise, if the parameter value starts with a '+', then it is interpreted
as a recurrence interval of that many days.
If no plus-sign-starting parameter is specified, the recurrence period
defaults to the unit larger than all the fixed parameters. e.g. '--minute 5'
means every hour at 5 minutes past, and --hour 3, --minute 1 means 3:01 every day.
At least one optional parameter must be provided.
All parameters accept multiple comma-separated values. If multiple
parameters have multiple values, all combinations of those values are used.
All fixed units not specified lower than the recurrence period default to
the lowest valid value, e.g. --month +2 will be scheduled at 12:00am the first of
every other month. One exception is if the largest fixed value is day of the
week, then the default period is set to be a week.
A month period with a day of week fixed value is not currently supported.
Fixed-value year (i.e. --year 2019) is not supported. See the 'at'
command for one-time cron jobs.
As an alternative to the above options, one may use exactly one of
--hourly, --daily, --monthly, --yearly with a colon-separated list of
fixed parameters for the value. It is an error to use both the individual
options and these aliases at the same time.
Examples:
Run a query every last day of the month at 3 am
cron.add --hour 3 --day -1 {#foo}
Run a query every 8 hours
cron.add --hour +8 {#foo}
Run a query every Wednesday and Sunday at midnight and noon
cron.add --hour 0,12 --day Wed,Sun {#foo}
Run a query every other day at 3:57pm
cron.add --day +2 --minute 57 --hour 15 {#foo}
Usage: cron.add [options] <query>
Options:
--help : Display the command usage.
--minute <minute> : Minute value for job or recurrence period.
--name <name> : An optional name for the cron job.
--doc <doc> : An optional doc string for the cron job.
--hour <hour> : Hour value for job or recurrence period.
--day <day> : Day value for job or recurrence period.
--month <month> : Month value for job or recurrence period.
--year <year> : Year value for recurrence period.
--hourly <hourly> : Fixed parameters for an hourly job.
--daily <daily> : Fixed parameters for a daily job.
--monthly <monthly> : Fixed parameters for a monthly job.
--yearly <yearly> : Fixed parameters for a yearly job.
Arguments:
<query> : Query for the cron job to execute.
complete. 0 nodes in 9 ms (0/sec).
cron.at¶
The cron.at
command creates a non-recurring cron job within a Cortex. Note that just like standard cron jobs, jobs created with cron.at
will persist (remain in the list of cron jobs and as syn:cron
runt nodes) until they are explicitly removed using cron.del
.
Syntax:
cli> storm cron.at --help
Executing query at 2021/01/22 01:09:18.782
Adds a non-recurring cron job.
Notes:
This command accepts one or more time specifications followed by exactly
one storm query in curly braces. Each time specification may be in synapse
time delta format (e.g --day +1) or synapse time format (e.g.
20501217030432101). Seconds will be ignored, as cron jobs' granularity is
limited to minutes.
All times are interpreted as UTC.
The other option for time specification is a relative time from now. This
consists of a plus sign, a positive integer, then one of 'minutes, hours,
days'.
Note that the record for a cron job is stored until explicitly deleted via
"cron.del".
Examples:
# Run a storm query in 5 minutes
cron.at --minute +5 {[inet:ipv4=1]}
# Run a storm query tomorrow and in a week
cron.at --day +1,+7 {[inet:ipv4=1]}
# Run a query at the end of the year Zulu
cron.at --dt 20181231Z2359 {[inet:ipv4=1]}
Usage: cron.at [options] <query>
Options:
--help : Display the command usage.
--minute <minute> : Minute(s) to execute at.
--hour <hour> : Hour(s) to execute at.
--day <day> : Day(s) to execute at.
--dt <dt> : Datetime(s) to execute at.
--now : Execute immediately.
Arguments:
<query> : Query for the cron job to execute.
complete. 0 nodes in 9 ms (0/sec).
cron.cleanup¶
The cron.cleanup
command can be used to remove any one-time cron jobs (“at” jobs) that have completed.
Syntax:
cli> storm cron.cleanup --help
Executing query at 2021/01/22 01:09:18.805
Delete all completed at jobs
Usage: cron.cleanup [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
cron.list¶
The cron.list
command displays the set of cron jobs in the Cortex that the current user can view / modify based on their permissions.
Cron jobs are displayed in alphanumeric order by job Iden. Jobs are sorted upon Cortex initialization, so newly-created jobs will be displayed at the bottom of the list until the list is re-sorted the next time the Cortex is restarted.
Note
Jobs can also be viewed in runt node form as syn:cron
nodes.
Syntax:
cli> storm cron.list --help
Executing query at 2021/01/22 01:09:18.826
List existing cron jobs in the cortex.
Usage: cron.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
cron.stat¶
The cron.stat
command displays statistics for an individual cron job and provides more detail on an individual job vs. cron.list
, including any errors and the interval at which the job executes. To view the stats for a job, you must provide the first portion of the job’s iden (i.e., enough of the iden that the job can be uniquely identified), which can be obtained using cron.list
or by lifting the appropriate syn:cron
node.
Syntax:
cli> storm cron.stat --help
Executing query at 2021/01/22 01:09:18.846
Gives detailed information about a cron job.
Usage: cron.stat [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid cron job iden is accepted.
complete. 0 nodes in 9 ms (0/sec).
cron.mod¶
The cron.mod
command modifies the Storm query associated with a specific cron job. To modify a job, you must provide the first portion of the job’s iden (i.e., enough of the iden that the job can be uniquely identified), which can be obtained using cron.list
or by lifting the appropriate syn:cron
node.
Note
Other aspects of the cron job, such as its schedule for execution, cannot be modified once the job has been created. To change these aspects you must delete and re-add the job.
Syntax:
cli> storm cron.mod --help
Executing query at 2021/01/22 01:09:18.868
Modify an existing cron job's query.
Usage: cron.mod [options] <iden> <query>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid cron job iden is accepted.
<query> : New storm query for the cron job.
complete. 0 nodes in 85 ms (0/sec).
cron.disable¶
The cron.disable
command disables a job and prevents it from executing without removing it from the Cortex. To disable a job, you must provide the first portion of the job’s iden (i.e., enough of the iden that the job can be uniquely identified), which can be obtained using cron.list
or by lifting the appropriate syn:cron
node.
Syntax:
cli> storm cron.disable --help
Executing query at 2021/01/22 01:09:18.972
Disable a cron job in the cortex.
Usage: cron.disable [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid cron job iden is accepted.
complete. 0 nodes in 8 ms (0/sec).
cron.enable¶
The cron.enable
command enables a disabled cron job. To enable a job, you must provide the first portion of the job’s iden (i.e., enough of the iden that the job can be uniquely identified), which can be obtained using cron.list
or by lifting the appropriate syn:cron
node.
Note
Cron jobs, including non-recurring jobs added with cron.at
, are enabled by default upon creation.
Syntax:
cli> storm cron.enable --help
Executing query at 2021/01/22 01:09:18.993
Enable a cron job in the cortex.
Usage: cron.enable [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid cron job iden is accepted.
complete. 0 nodes in 9 ms (0/sec).
cron.del¶
The cron.del
command permanently removes a cron job from the Cortex. To delete a job, you must provide the first portion of the job’s iden (i.e., enough of the iden that the job can be uniquely identified), which can be obtained using cron.list
or by lifting the appropriate syn:cron
node.
Syntax:
cli> storm cron.del --help
Executing query at 2021/01/22 01:09:19.015
Delete a cron job from the cortex.
Usage: cron.del [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid cron job iden is accepted.
complete. 0 nodes in 8 ms (0/sec).
delnode¶
The delnode
command deletes a node or set of nodes from a Cortex.
Warning
The Storm delnode
command has the potential to be destructive if executed on an incorrect, badly formed, or mistyped query. Users are strongly encouraged to validate their query by first executing it on its own to confirm it returns the expected nodes before piping the query to the delnode
command.
Syntax:
cli> storm delnode --help
Executing query at 2021/01/22 01:09:19.036
Delete nodes produced by the previous query logic.
(no nodes are returned)
Example
inet:fqdn=vertex.link | delnode
Usage: delnode [options]
Options:
--help : Display the command usage.
--force : Force delete even if it causes broken references (requires admin).
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Delete the node for the domain woowoo.com:
inet:fqdn=woowoo.com | delnode
- Forcibly delete all nodes with the #testing tag:
#testing | delnode --force
Usage Notes:
delnode
operates on the output of a previous Storm query.delnode
will attempt to perform some basic sanity-checking to help prevent egregious mistakes. For example,delnode
will return an error if you attempt to delete a node that is still referenced by another node (such as aninet:fqdn
that is referenced by aninet:dns:a
node). Similarly, delnode will return an error if you attempt to delete asyn:tag
node if that tag is still applied to other nodes. However, delnode cannot prevent all mistakes.- Edge / digraph nodes (such as
edge:refs
oredge:has
) are exempt from these “reference checks”. It is possible to delete an edge node wheren1
and / orn2
still exist and without affectingn1
orn2
. This is generally desired behavior as the relationship between the nodes (the edge) may need to be removed without affecting the nodes themselves. - However, the opposite is also true - since
delnode
does not check for references to edge nodes when deleting nodes, it is possible leave “orphaned” edge nodes with a missingn1
orn2
.
- Edge / digraph nodes (such as
- The
--force
parameter will forcibly delete the nodes input to the command, regardless of any sanity-checking errors or other conditions. This parameter should be used with extreme caution as it may result in broken references within the Cortex.
dmon¶
Storm includes dmon.*
commands that allow you work with daemons (see Daemon).
Help for individual dmon.*
commands can be displayed using:
storm <command> --help | -h
dmon.list¶
The dmon.list
command displays the set of running dmon queries in the Cortex.
Syntax:
cli> storm dmon.list --help
Executing query at 2021/01/22 01:09:19.203
List the storm daemon queries running in the cortex.
Usage: dmon.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
edges¶
Storm includes edges.*
commands that allow you to work with lightweight (light) edges. Also see the lift.byverb
and model.edge.*
commands under lift and model below.
Help for individual edge.*
commands can be displayed using:
storm <command> --help | -h
edges.del¶
The edges.del
command is designed to delete multiple light edges to (or from) a set of nodes (contrast with using Storm edit syntax - see Delete Light Edges).
Syntax:
cli> storm edges.del --help
Executing query at 2021/01/22 01:09:19.224
Bulk delete light edges from input nodes.
Examples:
# Delete all "foo" light edges from an inet:ipv4
inet:ipv4=1.2.3.4 | edges.del foo
# Delete light edges with any verb from a node
inet:ipv4=1.2.3.4 | edges.del *
# Delete all "foo" light edges to an inet:ipv4
inet:ipv4=1.2.3.4 | edges.del foo --n2
Usage: edges.del [options] <verb>
Options:
--help : Display the command usage.
--n2 : Delete light edges where input node is N2 instead of N1.
Arguments:
<verb> : The verb of light edges to delete.
complete. 0 nodes in 9 ms (0/sec).
feed¶
Storm includes feed.*
commands that allow you work with feeds (see Feed).
Help for individual feed.*
commands can be displayed using:
storm <command> --help | -h
feed.list¶
The feed.list
command displays available feed functions in the Cortex.
Syntax:
cli> storm feed.list --help
Executing query at 2021/01/22 01:09:19.246
List the feed functions available in the Cortex
Usage: feed.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
graph¶
The graph
command generates a subgraph based on a specified set of nodes and parameters.
Syntax:
cli> storm graph --help
Executing query at 2021/01/22 01:09:19.266
Generate a subgraph from the given input nodes and command line options.
Example:
Using the graph command::
inet:fqdn | graph
--degrees 2
--filter { -#nope }
--pivot { <- meta:seen <- meta:source }
--form-pivot inet:fqdn {<- * | limit 20}
--form-pivot inet:fqdn {-> * | limit 20}
--form-filter inet:fqdn {-inet:fqdn:issuffix=1}
--form-pivot syn:tag {-> *}
--form-pivot * {-> #}
Usage: graph [options]
Options:
--help : Display the command usage.
--degrees <degrees> : How many degrees to graph out.
--pivot <pivot> : Specify a storm pivot for all nodes. (must quote)
--filter <filter> : Specify a storm filter for all nodes. (must quote)
--no-edges : Do not include light weight edges in the per-node output.
--form-pivot <form_pivot> : Specify a <form> <pivot> form specific pivot.
--form-filter <form_filter> : Specify a <form> <filter> form specific filter.
--refs : Do automatic in-model pivoting with node.getNodeRefs().
--yield-filtered : Yield nodes which would be filtered. This still performs pivots to collect edge data,but does not yield pivoted nodes.
--no-filter-input : Do not drop input nodes if they would match a filter.
complete. 0 nodes in 8 ms (0/sec).
iden¶
The iden
command lifts one or more nodes by their node identifier (node ID / iden).
Syntax:
cli> storm iden --help
Executing query at 2021/01/22 01:09:19.286
Lift nodes by iden.
Example:
iden b25bc9eec7e159dce879f9ec85fb791f83b505ac55b346fcb64c3c51e98d1175 | count
Usage: iden [options] <iden>
Options:
--help : Display the command usage.
Arguments:
[<iden> ...] : Iden to lift nodes by. May be specified multiple times.
complete. 0 nodes in 9 ms (0/sec).
Example:
- Lift the node with node ID d7fb3ae625e295c9279c034f5d91a7ad9132c79a9c2b16eecffc8d1609d75849:
iden d7fb3ae625e295c9279c034f5d91a7ad9132c79a9c2b16eecffc8d1609d75849
Usage Notes:
- The node ID (iden) for a given node can be obtained by lifting the node using the
--debug
or--raw
option to the storm command:storm --debug inet:fqdn=woot.com
layer¶
Storm includes layer.*
commands that allow you work with layers (see Layer).
Help for individual layer.*
commands can be displayed using:
storm <command> --help | -h
layer.add¶
The layer.add
command adds a layer to the Cortex.
Syntax
cli> storm layer.add --help
Executing query at 2021/01/22 01:09:19.326
Add a layer to the cortex.
Usage: layer.add [options]
Options:
--help : Display the command usage.
--lockmemory : Should the layer lock memory for performance.
--readonly : Should the layer be readonly.
--growsize <growsize> : Amount to grow the map size when necessary.
--upstream <upstream> : One or more telepath urls to receive updates from.
--name <name> : The name of the layer.
complete. 0 nodes in 9 ms (0/sec).
layer.set¶
The layer.set
command sets an option for the specified layer.
Syntax
cli> storm layer.set --help
Executing query at 2021/01/22 01:09:19.347
Set a layer option.
Usage: layer.set [options] <iden> <name> <valu>
Options:
--help : Display the command usage.
Arguments:
<iden> : Iden of the layer to modify.
<name> : The name of the layer property to set.
<valu> : The value to set the layer property to.
complete. 0 nodes in 8 ms (0/sec).
layer.get¶
The layer.get
command retrieves the specified layer from a Cortex.
Syntax
cli> storm layer.get --help
Executing query at 2021/01/22 01:09:19.368
Get a layer from the cortex.
Usage: layer.get [options] <iden>
Options:
--help : Display the command usage.
Arguments:
[iden] : Iden of the layer to get. If no iden is provided, the main layer will be returned.
complete. 0 nodes in 8 ms (0/sec).
layer.list¶
The layer.list
command lists the available layers in a Cortex.
Syntax
cli> storm layer.list --help
Executing query at 2021/01/22 01:09:19.391
List the layers in the cortex.
Usage: layer.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
layer.del¶
The layer.del
command deletes a layer from a Cortex.
Syntax
cli> storm layer.del --help
Executing query at 2021/01/22 01:09:19.410
Delete a layer from the cortex.
Usage: layer.del [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Iden of the layer to delete.
complete. 0 nodes in 8 ms (0/sec).
lift¶
Storm includes lift.*
commands that allow you to perform specialized lift operations.
Help for individual lift.*
commands can be displayed using:
storm <command> --help | -h
lift.byverb¶
The lift.byverb
command lifts nodes that are connected by the specified lightweight (light) edge. By default, the command lifts the N1 nodes (i.e., the nodes on the left side of the directional light edge relationship: n1 -(<verb>)> n2
)
Note
For other commands associated with light edges, see edges.del
and model.edge.*
under edges and model respectively.
Syntax:
cli> storm lift.byverb --help
Executing query at 2021/01/22 01:09:19.430
Lift nodes from the current view by an light edge verb.
Examples:
# Lift all the n1 nodes for the light edge "foo"
lift.byverb "foo"
# Lift all the n2 nodes for the light edge "foo"
lift.byverb --n2 "foo"
Notes:
Only a single instance of a node will be yielded from this command
when that node is lifted via the light edge membership.
Usage: lift.byverb [options] <verb>
Options:
--help : Display the command usage.
--n2 : Lift by the N2 value instead of N1 value.
Arguments:
<verb> : The edge verb to lift nodes by.
complete. 0 nodes in 9 ms (0/sec).
limit¶
The limit
command restricts the number of nodes returned from a given Storm query to the specified number of nodes.
Syntax:
cli> storm limit --help
Executing query at 2021/01/22 01:09:19.452
Limit the number of nodes generated by the query in the given position.
Example:
inet:ipv4 | limit 10
Usage: limit [options] <count>
Options:
--help : Display the command usage.
Arguments:
<count> : The maximum number of nodes to yield.
complete. 0 nodes in 8 ms (0/sec).
Example:
- Lift ten IP address nodes:
inet:ipv4 | limit 10
Usage Notes:
- If the limit number specified (i.e.,
limit 100
) is greater than the total number of nodes returned from the Storm query, no limit will be applied to the resultant nodes (i.e., all nodes will be returned). - By design,
limit
imposes an artificial limit on the nodes returned by a query, which may impair effective analysis of data by restricting results. As such,limit
is most useful for viewing a subset of a large result set or an exemplar node for a given form. - While
limit
returns a sampling of nodes, it is not statistically random for the purposes of population sampling for algorithmic use.
macro¶
Note
See the Storm Reference - Automation guide for additional background on macros (as well as triggers and cron jobs), including examples.
Storm includes macro.*
commands that allow you work with macros (see Macro).
Help for individual macro.*
commands can be displayed using:
storm <command> --help | -h
macro.list¶
The macro.list
command lists the macros in a Cortex.
Syntax:
cli> storm macro.list --help
Executing query at 2021/01/22 01:09:19.653
List the macros set on the cortex.
Usage: macro.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
macro.set¶
The macro.set
command creates (or modifies) a macro in a Cortex.
Syntax:
cli> storm macro.set --help
Executing query at 2021/01/22 01:09:19.674
Set a macro definition in the cortex.
Variables can also be used that are defined outside the definition.
Examples:
macro.set foobar ${ [+#foo] }
# Use variable from parent scope
macro.set bam ${ [ inet:ipv4=$val ] }
$val=1.2.3.4 macro.exec bam
Usage: macro.set [options] <name> <storm>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the macro to set.
<storm> : The storm command string or embedded query to set.
complete. 0 nodes in 8 ms (0/sec).
macro.get¶
The macro.get
command retrieves and displays the specified macro.
Syntax:
cli> storm macro.get --help
Executing query at 2021/01/22 01:09:19.694
Display the storm query for a macro in the cortex.
Usage: macro.get [options] <name>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the macro to display.
complete. 0 nodes in 8 ms (0/sec).
macro.exec¶
The macro.exec
command executes the specified macro.
Syntax:
cli> storm macro.exec --help
Executing query at 2021/01/22 01:09:19.714
Execute a named macro.
Example:
inet:ipv4#cno.threat.t80 | macro.exec enrich_foo
Usage: macro.exec [options] <name>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the macro to execute
complete. 0 nodes in 8 ms (0/sec).
macro.del¶
The macro.del
command deletes the specified macro from a Cortex.
Syntax:
cli> storm macro.del --help
Executing query at 2021/01/22 01:09:19.734
Remove a macro definition from the cortex.
Usage: macro.del [options] <name>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the macro to delete.
complete. 0 nodes in 8 ms (0/sec).
max¶
The max
command returns the node from a given set that contains the highest value for a specified secondary property, tag interval, or variable.
Syntax:
cli> storm max --help
Executing query at 2021/01/22 01:09:19.754
Consume nodes and yield only the one node with the highest value for a property or variable.
Examples:
file:bytes +#foo.bar | max :size
file:bytes +#foo.bar +.seen ($tick, $tock) = .seen | max $tick
Usage: max [options] <valu>
Options:
--help : Display the command usage.
Arguments:
<valu> : No help available
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Return the DNS A record for woot.com with the most recent
.seen
value:
inet:dns:a:fqdn=woot.com | max .seen
- Return the most recent WHOIS record for domain woot.com:
inet:whois:rec:fqdn=woot.com | max :asof
min¶
The min
command returns the node from a given set that contains the lowest value for a specified secondary property, tag interval, or variable.
Syntax:
cli> storm min --help
Executing query at 2021/01/22 01:09:20.038
Consume nodes and yield only the one node with the lowest value for a property.
Examples:
file:bytes +#foo.bar | min :size
file:bytes +#foo.bar | min .seen
Usage: min [options] <valu>
Options:
--help : Display the command usage.
Arguments:
<valu> : No help available
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Return the DNS A record for woot.com with the oldest
.seen
value:
inet:dns:a:fqdn=woot.com | min .seen
- Return the oldest WHOIS record for domain woot.com:
inet:whois:rec:fqdn=woot.com | min :asof
model¶
Storm includes model.*
commands that allow you to work with model elements.
model.deprecated.*
commands allow you to view model elements (forms or properties) that have been marked as “deprecated”, determine whether your Cortex contains deprecated nodes / nodes with deprecated properties, and optionally lock / unlock those properties to prevent (or allow) continued creation of deprecated model elements.
model.edge.*
commands allow you to work with lightweight (light) edges. (See also the edges.del
and lift.byverb
commands under edges and lift, respectively.)
- model.deprecated.check
- model.deprecated.lock
- model.deprecated.locks
- model.edge.list
- model.edge.set
- model.edge.get
- model.edge.del
Help for individual model.*
commands can be displayed using:
storm <command> --help | -h
model.deprecated.check¶
The model.deprecated.check
command lists deprecated elements, their lock status, and whether deprecated elements exist in the Cortex.
Syntax:
cli> storm model.deprecated.check --help
Executing query at 2021/01/22 01:09:20.122
Check for lock status and the existance of deprecated model elements
Usage: model.deprecated.check [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
model.deprecated.lock¶
The model.deprecated.lock
command allows you to lock or unlock (e.g., disallow or allow the use of) deprecated model elements in a Cortex.
Syntax:
cli> storm model.deprecated.lock --help
Executing query at 2021/01/22 01:09:20.142
Edit lock status of deprecated model elements.
Usage: model.deprecated.lock [options] <name>
Options:
--help : Display the command usage.
--unlock : Unlock rather than lock the deprecated property.
Arguments:
<name> : The deprecated form or property name to lock or * to lock all.
complete. 0 nodes in 9 ms (0/sec).
model.deprecated.locks¶
The model.deprecated.locks
command displays the lock status of all deprecated model elements.
Syntax:
cli> storm model.deprecated.locks --help
Executing query at 2021/01/22 01:09:20.162
Display lock status of deprecated model elements.
Usage: model.deprecated.locks [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
model.edge.list¶
The model.edge.list
command displays the set of light edges currently defined in the Cortex and any doc
values they have set on them.
Syntax:
cli> storm model.edge.list --help
Executing query at 2021/01/22 01:09:20.182
List all edge verbs in the current view and their doc key (if set).
Usage: model.edge.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
model.edge.set¶
The model.edge.set
command allows you to set the value of an given key on a light edge (such as a doc
value to specify a definition for the light edge). The current list of valid keys include the following:
doc
Syntax:
cli> storm model.edge.set --help
Executing query at 2021/01/22 01:09:20.202
Set an key-value for an edge verb that exists in the current view.
Usage: model.edge.set [options] <verb> <key> <valu>
Options:
--help : Display the command usage.
Arguments:
<verb> : The edge verb to add a key to.
<key> : The key name (e.g. doc).
<valu> : The string value to set.
complete. 0 nodes in 8 ms (0/sec).
model.edge.get¶
The model.edge.get
command allows you to retrieve all of the keys that have been set on a light edge.
Syntax:
cli> storm model.edge.get --help
Executing query at 2021/01/22 01:09:20.222
Retrieve key-value pairs an edge verb in the current view.
Usage: model.edge.get [options] <verb>
Options:
--help : Display the command usage.
Arguments:
<verb> : The edge verb to retrieve.
complete. 0 nodes in 11 ms (0/sec).
model.edge.del¶
The model.edge.del
command allows you to delete the key from a light edge (such as a doc
property to specify a definition for the light edge). Deleting an key from a specific light edge does not delete the key from Synapse (e.g., the property can be re-added to the light edge or to other light edges).
Syntax:
cli> storm model.edge.del --help
Executing query at 2021/01/22 01:09:20.245
Delete a global key-value pair for an edge verb in the current view.
Usage: model.edge.del [options] <verb> <key>
Options:
--help : Display the command usage.
Arguments:
<verb> : The edge verb to delete documentation for.
<key> : The key name (e.g. doc).
complete. 0 nodes in 8 ms (0/sec).
movetag¶
The movetag
command moves a Synapse tag and its associated tag tree from one location in a tag hierarchy to another location. It is equivalent to “renaming” a given tag and all of its subtags. Moving a tag consists of:
- Creating the new
syn:tag
node(s). - Copying the definitions (
:title
and:doc
properties) from the oldsyn:tag
node to the newsyn:tag
node. - Applying the new tag(s) to the nodes with the old tag(s).
- If the old tag(s) have associated timestamps / time intervals, they will be applied to the new tag(s).
- Deleting the old tag(s) from the nodes.
- Setting the
:isnow
property of the oldsyn:tag
node(s) to reference the newsyn:tag
node.- The old
syn:tag
nodes are not deleted. - Once the
:isnow
property is set, attempts to apply the old tag will automatically result in the new tag being applied.
- The old
Syntax:
cli> storm movetag --help
Executing query at 2021/01/22 01:09:20.264
Rename an entire tag tree and preserve time intervals.
Example:
movetag foo.bar baz.faz.bar
Usage: movetag [options] <oldtag> <newtag>
Options:
--help : Display the command usage.
Arguments:
<oldtag> : The tag tree to rename.
<newtag> : The new tag tree name.
complete. 0 nodes in 9 ms (0/sec).
Examples:
- Move the tag named #research to #internal.research:
movetag research internal.research
- Move the tag tree #aka.fireeye.malware to #aka.feye.mal:
movetag aka.fireeye.malware aka.feye.mal
Usage Notes:
Warning
movetag
should be used with caution as when used incorrectly it can result in “deleted” (inadvertently moved / removed) or orphaned (inadvertently retained) tags. For example, in the second example query above, all aka.fireeye.malware
tags are renamed aka.feye.mal
, but the tag aka.fireeye
still exists and is still applied to all of the original nodes. In other words, the result of the above command will be that nodes previously tagged aka.fireeye.malware
will now be tagged both aka.feye.mal
and aka.fireeye
. Users may wish to test the command on sample data first to understand its effects before applying it in a live Cortex.
pkg¶
Storm includes pkg.*
commands that allow you work with Storm packages (see Package).
Help for individual pkg.*
commands can be displayed using:
storm <command> --help | -h
Packages typically contain extended Storm commands and Storm library code used to implement a Storm Service.
pkg.list¶
The pkg.list
command lists each Storm package loaded in the Cortex. Output is displayed in tabluar form and includes the package name and version information.
Syntax:
cli> storm pkg.list --help
Executing query at 2021/01/22 01:09:20.487
List the storm packages loaded in the cortex.
Usage: pkg.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
pkg.del¶
The pkg.del
command removes a Storm package from the Cortex.
Syntax:
cli> storm pkg.del --help
Executing query at 2021/01/22 01:09:20.507
Remove a storm package from the cortex.
Usage: pkg.del [options] <name>
Options:
--help : Display the command usage.
Arguments:
<name> : The name (or name prefix) of the package to remove.
complete. 0 nodes in 8 ms (0/sec).
ps¶
Storm includes ps.*
commands that allow you work with Storm tasks/queries.
Help for individual ps.*
commands can be displayed using:
storm <command> --help | -h
ps.list¶
The ps.list
command lists the currently executing tasks/queries. By default, the command displays the first 120 characters of the executing query. The --verbose
option can be used to display the full query regardless of length.
Syntax:
cli> storm ps.list --help
Executing query at 2021/01/22 01:09:20.527
List running tasks in the cortex.
Usage: ps.list [options]
Options:
--help : Display the command usage.
--verbose : Enable verbose output.
complete. 0 nodes in 8 ms (0/sec).
ps.kill¶
The ps.kill
command can be used to terminate an executing task/query. The command requires the Iden of the task to be terminated, which can be obtained with ps.list.
Syntax:
cli> storm ps.kill --help
Executing query at 2021/01/22 01:09:20.547
Kill a running task/query within the cortex.
Usage: ps.kill [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid process iden is accepted.
complete. 0 nodes in 8 ms (0/sec).
parallel¶
The Storm parallel
command allows you to execute a Storm query using a specified number of query pipelines. This can improve performance for some queries.
See also background.
Syntax:
cli> storm parallel --help
Executing query at 2021/01/22 01:09:20.567
Execute part of a query pipeline in parallel.
This can be useful to minimize round-trip delay during enrichments.
Examples:
inet:ipv4#foo | parallel { $place = $lib.import(foobar).lookup(:latlong) [ :place=$place ] }
NOTE: Storm variables set within the parallel query pipelines do not interact.
Usage: parallel [options] <query>
Options:
--help : Display the command usage.
--size <size> : The number of parallel Storm pipelines to execute.
Arguments:
<query> : The query to execute in parallel.
complete. 0 nodes in 9 ms (0/sec).
queue¶
Storm includes queue.*
commands that allow you work with queues (see Queue).
Help for individual queue.*
commands can be displayed using:
storm <command> --help | -h
queue.add¶
The queue.add
command adds a queue to the Cortex.
Syntax:
cli> storm queue.add --help
Executing query at 2021/01/22 01:09:20.588
Add a queue to the cortex.
Usage: queue.add [options] <name>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the new queue.
complete. 0 nodes in 8 ms (0/sec).
queue.list¶
The queue.list
command lists each queue in the Cortex.
Syntax:
cli> storm queue.list --help
Executing query at 2021/01/22 01:09:20.608
List the queues in the cortex.
Usage: queue.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
queue.del¶
The queue.del
command removes a queue from the Cortex.
Syntax:
cli> storm queue.del --help
Executing query at 2021/01/22 01:09:20.628
Remove a queue from the cortex.
Usage: queue.del [options] <name>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the queue to remove.
complete. 0 nodes in 8 ms (0/sec).
reindex¶
The reindex
command reindexes a given node property. This is an administrative command that is typically used when data model updates have been pushed to a Cortex and existing node properties must be migrated to the new model.
Syntax:
cli> storm reindex --help
Executing query at 2021/01/22 01:09:20.648
Use admin privileges to re index/normalize node properties.
NOTE: Currently does nothing but is reserved for future use.
Usage: reindex [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
scrape¶
The scrape
command parses one or more secondary properties or variables of the inbound node(s), attempts to identify (“scrape”) common forms from the content, and returns nodes for the identified forms (creating the nodes if they do not already exist). This is useful (for example) to extract forms such as email addresses, domains, URLs, hashes, etc. from unstructured text.
scrape
can optionally link the source nodes(s) to the scraped forms via refs
light edges.
By default, the scrape command will yield the nodes that it received as input (i.e., the command will not consume nodes by default) to allow command chaining. However if executed with the option --yield
the scrape
command will yield the scraped nodes rather than the input nodes.
cli> storm scrape --help
Executing query at 2021/01/22 01:09:20.668
Use textual properties of existing nodes to find other easily recognizable nodes.
Examples:
# Scrape properties from inbound nodes and create standalone nodes.
inet:search:query | scrape
# Scrape properties from inbound nodes and make refs light edges to the scraped nodes.
inet:search:query | scrape --refs
# Scrape only the :engine and :text props from the inbound nodes.
inet:search:query | scrape :text :engine
# Scrape properties inbound nodes and yield newly scraped nodes.
inet:search:query | scrape --yield
# Skip re-fanging text before scraping.
inet:search:query | scrape --skiprefang
Usage: scrape [options] <values>
Options:
--help : Display the command usage.
--refs : Create refs light edges to any scraped nodes from the input node
--yield : Include newly scraped nodes in the output
--skiprefang : Do not remove de-fanging from text before scraping
Arguments:
[<values> ...] : Specific relative properties or variables to scrape
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Scrape common forms and create nodes from the body of domain WHOIS record(s) for domain woot.com:
inet:whois:rec:fqdn=woot.com | scrape :text
- Scrape common forms from the content of a set of Internet posts from a given account, link the created nodes to the original posts, and uniq the results:
inet:web:post:acct=(twitter.com,finley1589) | scrape :text --refs --yield | uniq
Usage Notes:
- If no properties or variables are specified,
scrape
will attempt to scrape all properties by default. - If any scraped secondary properties are already recognizable / common properties, those properties will also be scraped (and optionally linked, if
--refs
is also specified). That is, scraping the:md5
secondary property of a node will return ahash:md5
node with that property’s value. scrape
will only scrape node properties; it will not scrape the content of any files that may be referenced by those properties. For example, attempting to scrape the:file
property of amedia:news
node will scrape and extract the value of the property (typically a SHA256 hash) if present; it will not extract indicators from the referenced file itself.- With respect to cyber threat data in particular,
scrape
cannot identify and extract “defanged” values such as “hxxp://” for URLs or “woot[.]com” for domains.
service¶
Storm includes service.*
commands that allow you work with Storm services (see Service).
Help for individual service.*
commands can be displayed using:
storm <command> --help | -h
service.add¶
The service.add
command adds a Storm service to the Cortex.
Syntax:
cli> storm service.add --help
Executing query at 2021/01/22 01:09:20.879
Add a storm service to the cortex.
Usage: service.add [options] <name> <url>
Options:
--help : Display the command usage.
Arguments:
<name> : The name of the service.
<url> : The telepath URL for the remote service.
complete. 0 nodes in 8 ms (0/sec).
service.list¶
The service.list
command lists each Storm service in the Cortex.
Syntax:
cli> storm service.list --help
Executing query at 2021/01/22 01:09:20.900
List the storm services configured in the cortex.
Usage: service.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
service.del¶
The service.del
command removes a Storm service from the Cortex.
Syntax:
cli> storm service.del --help
Executing query at 2021/01/22 01:09:20.919
Remove a storm service from the cortex.
Usage: service.del [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : The service identifier or prefix.
complete. 0 nodes in 8 ms (0/sec).
sleep¶
The sleep
command adds a delay in returning each result for a given Storm query. By default, query results are streamed back and displayed as soon as they arrive for optimal performance. A sleep
delay effectively slows the display of results.
Syntax:
cli> storm sleep --help
Executing query at 2021/01/22 01:09:20.939
Introduce a delay between returning each result for the storm query.
NOTE: This is mostly used for testing / debugging.
Example:
#foo.bar | sleep 0.5
Usage: sleep [options] <delay>
Options:
--help : Display the command usage.
Arguments:
<delay> : Delay in floating point seconds.
complete. 0 nodes in 9 ms (0/sec).
Example:
- Retrieve domain nodes from a Cortex every second:
inet:email | sleep 1.0
spin¶
The spin
command is used to suppress the output of a Storm query. Spin
simply consumes all nodes sent to the command, so no nodes are output to the CLI. This allows you to execute a Storm query and view messages and results without displaying the associated nodes.
Syntax:
cli> storm spin --help
Executing query at 2021/01/22 01:09:26.038
Iterate through all query results, but do not yield any.
This can be used to operate on many nodes without returning any.
Example:
foo:bar:size=20 [ +#hehe ] | spin
Usage: spin [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Count the number of email addresses without displaying the inet:email nodes:
inet:email | count | spin
- Add the tag #int.research to any domain containing the string “firefox” but do not display the nodes.
inet:fqdn~=firefox [+#int.research] | spin
splice¶
Storm includes splice.*
commands that allow you work with splices (see Splice).
Splices are represented as runtime nodes (“runt nodes” - see Node, Runt) of the form syn:splice
. These runt nodes can be lifted and filtered just like standard nodes in a Cortex.
Help for individual splice.*
commands can be displayed using:
storm <command> --help | -h
splice.list¶
The splice.list
command allows you to list (view) splices in the splice log. By default, splices are displayed starting with the most recent and working backwards through the log.
Syntax:
cli> storm splice.list --help
Executing query at 2021/01/22 01:09:26.191
Retrieve a list of splices backwards from the end of the splicelog.
Examples:
# Show the last 10 splices.
splice.list | limit 10
# Show splices after a specific time.
splice.list --mintime "2020/01/06 15:38:10.991"
# Show splices from a specific timeframe.
splice.list --mintimestamp 1578422719360 --maxtimestamp 1578422719367
Notes:
If both a time string and timestamp value are provided for a min or max,
the timestamp will take precedence over the time string value.
Usage: splice.list [options]
Options:
--help : Display the command usage.
--maxtimestamp <maxtimestamp>: Only yield splices which occurred on or before this timestamp.
--mintimestamp <mintimestamp>: Only yield splices which occurred on or after this timestamp.
--maxtime <maxtime> : Only yield splices which occurred on or before this time.
--mintime <mintime> : Only yield splices which occurred on or after this time.
complete. 0 nodes in 9 ms (0/sec).
splice.undo¶
The splice.undo
command allows a user with appropriate permissions to roll back or undo the specified set of splices (changes).
Syntax:
cli> storm splice.undo --help
Executing query at 2021/01/22 01:09:26.213
Reverse the actions of syn:splice runt nodes.
Examples:
# Undo the last 5 splices.
splice.list | limit 5 | splice.undo
# Undo splices after a specific time.
splice.list --mintime "2020/01/06 15:38:10.991" | splice.undo
# Undo splices from a specific timeframe.
splice.list --mintimestamp 1578422719360 --maxtimestamp 1578422719367 | splice.undo
Usage: splice.undo [options]
Options:
--help : Display the command usage.
--force : Force delete nodes even if it causes broken references (requires admin).
complete. 0 nodes in 8 ms (0/sec).
tee¶
The tee
command executes multiple Storm queries on the inbound nodes and returns the combined result set.
Syntax:
cli> storm tee --help
Executing query at 2021/01/22 01:09:26.233
Execute multiple Storm queries on each node in the input stream, joining output streams together.
Commands are executed in order they are given.
Examples:
# Perform a pivot out and pivot in on a inet:ivp4 node
inet:ipv4=1.2.3.4 | tee { -> * } { <- * }
# Also emit the inbound node
inet:ipv4=1.2.3.4 | tee --join { -> * } { <- * }
Usage: tee [options] <query>
Options:
--help : Display the command usage.
--join : Emit inbound nodes after processing storm queries.
Arguments:
[<query> ...] : Specify a query to execute on the input nodes.
complete. 0 nodes in 9 ms (0/sec).
Examples:
- Return the set of domains and IP addresses associated with a set of DNS A records.
inet:fqdn:zone=mydomain.com -> inet:dns:a | tee { -> inet:fqdn } { -> inet:ipv4 }
- Return the set of domains and IP addresses associated with a set of DNS A records along with the original DNS A records.
inet:fqdn:zone=mydomain.com -> inet:dns:a | tee --join { -> inet:fqdn } { -> inet:ipv4 }
Usage Notes:
tee
can take an arbitrary number of Storm queries (i.e., 1 to n queries) as arguments.
tree¶
The tree
command recursively performs the specified pivot until no additional nodes are returned.
Syntax:
cli> storm tree --help
Executing query at 2021/01/22 01:09:26.465
Walk elements of a tree using a recursive pivot.
Examples:
# pivot upward yielding each FQDN
inet:fqdn=www.vertex.link | tree { :domain -> inet:fqdn }
Usage: tree [options] <query>
Options:
--help : Display the command usage.
Arguments:
<query> : The pivot query
complete. 0 nodes in 9 ms (0/sec).
Example:
- List the full set of tags in the “TTP” tag hierarchy.
syn:tag=ttp | tree { $node.value() -> syn:tag:up }
Usage Notes:
tree
is useful for “walking” a set of properties with a single command vs. performing an arbitrary nubmer of pivots until the end of the data is reached.
trigger¶
Note
See the Storm Reference - Automation guide for additional background on triggers (as well as cron jobs and macros), including examples.
Storm includes trigger.*
commands that allow you to create automated event-driven triggers (see Trigger) using the Storm query syntax.
Help for individual trigger.*
commands can be displayed using:
storm <command> --help | -h
Triggers are added to the Cortex as runtime nodes (“runt nodes” - see Node, Runt) of the form syn:trigger
. These runt nodes can be lifted and filtered just like standard nodes in a Cortex.
Note
The Storm trigger.*
commands replace the Synapse cmdr-based trigger command, which is being deprecated.
trigger.add¶
The trigger.add
command adds a trigger to a Cortex.
Syntax:
cli> storm trigger.add --help
Executing query at 2021/01/22 01:09:26.591
Add a trigger to the cortex.
Notes:
Valid values for condition are:
* tag:add
* tag:del
* node:add
* node:del
* prop:set
When condition is tag:add or tag:del, you may optionally provide a form name
to restrict the trigger to fire only on tags added or deleted from nodes of
those forms.
The added tag is provided to the query as an embedded variable '$tag'.
Simple one level tag globbing is supported, only at the end after a period,
that is aka.* matches aka.foo and aka.bar but not aka.foo.bar. aka* is not
supported.
Examples:
# Adds a tag to every inet:ipv4 added
trigger.add node:add --form inet:ipv4 --query {[ +#mytag ]}
# Adds a tag #todo to every node as it is tagged #aka
trigger.add tag:add --tag aka --query {[ +#todo ]}
# Adds a tag #todo to every inet:ipv4 as it is tagged #aka
trigger.add tag:add --form inet:ipv4 --tag aka --query {[ +#todo ]}
Usage: trigger.add [options] <condition>
Options:
--help : Display the command usage.
--form <form> : Form to fire on.
--tag <tag> : Tag to fire on.
--prop <prop> : Property to fire on.
--query <query> : Query for the trigger to execute.
--disabled : Create the trigger in disabled state.
Arguments:
<condition> : Condition for the trigger.
complete. 0 nodes in 9 ms (0/sec).
trigger.list¶
The trigger-list
command displays the set of triggers in the Cortex that the current user can view / modify based on their permissions. Triggers are displayed at the cmdr CLI in tabular format, with columns including the user who created the trigger, the Iden of the trigger, the condition that fires the trigger (i.e., node:add
), and the Storm query associated with the trigger.
Triggers are displayed in alphanumeric order by iden. Triggers are sorted upon Cortex initialization, so newly-created triggers will be displayed at the bottom of the list until the list is re-sorted the next time the Cortex is restarted.
Note
Triggers can also be viewed in runt node form as syn:trigger
nodes.
Syntax:
cli> storm trigger.list --help
Executing query at 2021/01/22 01:09:26.614
List existing triggers in the cortex.
Usage: trigger.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
trigger.mod¶
The trigger.mod
command modifies the Storm query associated with a specific trigger. To modify a trigger, you must provide the first portion of the trigger’s iden (i.e., enough of the iden that the trigger can be uniquely identified), which can be obtained using trigger.list
or by lifting the appropriate syn:trigger
node.
Note
Other aspects of the trigger, such as the condition used to fire the trigger or the tag or property associated with the trigger, cannot be modified once the trigger has been created. To change these aspects, you must delete and re-add the trigger.
Syntax:
cli> storm trigger.mod --help
Executing query at 2021/01/22 01:09:26.634
Modify an existing trigger's query.
Usage: trigger.mod [options] <iden> <query>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid trigger iden is accepted.
<query> : New storm query for the trigger.
complete. 0 nodes in 8 ms (0/sec).
trigger.disable¶
The trigger.disable
command disables a trigger and prevents it from firing without removing it from the Cortex. To disable a trigger, you must provide the first portion of the trigger’s iden (i.e., enough of the iden that the trigger can be uniquely identified), which can be obtained using trigger.list
or by lifting the appropriate syn:trigger
node.
Syntax:
cli> storm trigger.disable --help
Executing query at 2021/01/22 01:09:26.655
Disable a trigger in the cortex.
Usage: trigger.disable [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid trigger iden is accepted.
complete. 0 nodes in 8 ms (0/sec).
trigger.enable¶
The trigger-enable
command enables a disabled trigger. To enable a trigger, you must provide the first portion of the trigger’s iden (i.e., enough of the iden that the trigger can be uniquely identified), which can be obtained using trigger.list
or by lifting the appropriate syn:trigger
node.
Note
Triggers are enabled by default upon creation.
Syntax:
cli> storm trigger.enable --help
Executing query at 2021/01/22 01:09:26.676
Enable a trigger in the cortex.
Usage: trigger.enable [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid trigger iden is accepted.
complete. 0 nodes in 8 ms (0/sec).
trigger.del¶
The trigger.del
command permanently removes a trigger from the Cortex. To delete a trigger, you must provide the first portion of the trigger’s iden (i.e., enough of the iden that the trigger can be uniquely identified), which can be obtained using trigger.list
or by lifting the appropriate syn:trigger
node.
Syntax:
cli> storm trigger.del --help
Executing query at 2021/01/22 01:09:26.696
Delete a trigger from the cortex.
Usage: trigger.del [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Any prefix that matches exactly one valid trigger iden is accepted.
complete. 0 nodes in 8 ms (0/sec).
uniq¶
The uniq
command removes duplicate results from a Storm query. Results are uniqued based on each node’s node identifier (node ID / iden) so that only the first node with a given node ID is returned.
Syntax:
cli> storm uniq --help
Executing query at 2021/01/22 01:09:26.719
Filter nodes by their uniq iden values.
When this is used a Storm pipeline, only the first instance of a
given node is allowed through the pipeline.
Examples:
#badstuff +inet:ipv4 ->* | uniq
Usage: uniq [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
Examples:
- Lift all of the unique IP addresses that domains associated with the Fancy Bear threat group have resolved to:
inet:fqdn#aka.threatconnect.thr.fancybear -> inet:dns:a -> inet:ipv4 | uniq
view¶
Storm includes view.*
commands that allow you work with views (see View).
Help for individual view.*
commands can be displayed using:
storm <command> --help | -h
view.add¶
The view.add
command adds a view to the Cortex.
Syntax:
cli> storm view.add --help
Executing query at 2021/01/22 01:09:26.923
Add a view to the cortex.
Usage: view.add [options]
Options:
--help : Display the command usage.
--name <name> : The name of the new view.
--layers [<layers> ...] : Layers for the view.
complete. 0 nodes in 8 ms (0/sec).
view.fork¶
The view.fork
command forks an existing view from the Cortex. Forking a view creates a new view with a new writeable layer on top of the set of layers from the previous (forked) view.
Syntax:
cli> storm view.fork --help
Executing query at 2021/01/22 01:09:26.944
Fork a view in the cortex.
Usage: view.fork [options] <iden>
Options:
--help : Display the command usage.
--name <name> : Name for the newly forked view.
Arguments:
<iden> : Iden of the view to fork.
complete. 0 nodes in 9 ms (0/sec).
view.set¶
The view.set
command sets a property on the specified view.
Syntax:
cli> storm view.set --help
Executing query at 2021/01/22 01:09:26.966
Set a view option.
Usage: view.set [options] <iden> <name> <valu>
Options:
--help : Display the command usage.
Arguments:
<iden> : Iden of the view to modify.
<name> : The name of the view property to set.
<valu> : The value to set the view property to.
complete. 0 nodes in 8 ms (0/sec).
view.get¶
The view.get
command retrieves an existing view from the Cortex.
Syntax:
cli> storm view.get --help
Executing query at 2021/01/22 01:09:26.986
Get a view from the cortex.
Usage: view.get [options] <iden>
Options:
--help : Display the command usage.
Arguments:
[iden] : Iden of the view to get. If no iden is provided, the main view will be returned.
complete. 0 nodes in 8 ms (0/sec).
view.list¶
The view.list
command lists the views in the Cortex.
Syntax:
cli> storm view.list --help
Executing query at 2021/01/22 01:09:27.007
List the views in the cortex.
Usage: view.list [options]
Options:
--help : Display the command usage.
complete. 0 nodes in 8 ms (0/sec).
view.exec¶
The view.exec
command executes a Storm query in the specified view.
Syntax:
cli> storm view.exec --help
Executing query at 2021/01/22 01:09:27.028
Execute a storm query in a different view.
NOTE: Variables are passed through but nodes are not
Examples:
// Move some tagged nodes to another view
inet:fqdn#foo.bar $fqdn=$node.value() | view.exec 95d5f31f0fb414d2b00069d3b1ee64c6 { [ inet:fqdn=$fqdn ] }
Usage: view.exec [options] <view> <storm>
Options:
--help : Display the command usage.
Arguments:
<view> : The GUID of the view in which the query will execute.
<storm> : The storm query to execute on the view.
complete. 0 nodes in 8 ms (0/sec).
view.merge¶
The view.merge
command merges data from a forked view into its parent view.
Syntax:
cli> storm view.merge --help
Executing query at 2021/01/22 01:09:27.048
Merge a forked view into its parent view.
Usage: view.merge [options] <iden>
Options:
--help : Display the command usage.
--delete : Once the merge is complete, delete the layer and view.
Arguments:
<iden> : Iden of the view to merge.
complete. 0 nodes in 9 ms (0/sec).
view.del¶
The view.del
command permanently deletes a view from the Cortex.
Syntax:
cli> storm view.del --help
Executing query at 2021/01/22 01:09:27.070
Delete a view from the cortex.
Usage: view.del [options] <iden>
Options:
--help : Display the command usage.
Arguments:
<iden> : Iden of the view to delete.
complete. 0 nodes in 8 ms (0/sec).