Storm Reference - Pivoting

Pivot operations are performed on the output of a previous Storm operation such as a lift or filter. A pivot operation navigates from one set of nodes to another based on some relationship between the nodes. This relationship is commonly one of the following:

The nodes have properties that share the same value.
The nodes are joined by a light edge.

While all node-to-node navigation is referred to generically as “pivoting”, for clarity we make a distinction with respect to Storm:

a pivot operation navigates between nodes that share a property value; and
a traversal operation navigates between nodes that are connected by a light edge.

Tip

Traversal is also known as a “walk” operation (i.e., you can traverse a light edge or walk a light edge).

Pivots and traversals each have their own operator (symbol) used to represent the operation. The operations can also be combined (“pivot and traverse”) in various ways using additional operators.

Generically, all variations of pivots and traversals (including combined operations) require:

a set of source nodes;
an operator (symbol) for the kind of operation to be performed; and
the target node(s) for the operation.

Pivot and traversal operations both consume nodes - each operation navigates “away” from the source nodes to the target nodes, which become your new result set / working set. Join Operations can be used to retain the source nodes and combine (join) them with the target nodes in your result set.

See Storm Reference - Document Syntax Conventions for an explanation of the syntax format used below.

See Storm Reference - Type-Specific Storm Behavior for details on special syntax or handling for specific data types.

Navigating the Data Model

To navigate the data in Synapse (pivot between properties or traverse light edges), it helps to understand how forms, properties, and types are related, and to be familiar with common light edge conventions and how they are used to connect various forms.

The Synapse Data Model Forms documentation lists:
- All forms in Synapse, with their associated properties and types.
- The edges that can be used to link a given form (whether as the source or target of the edge).
For Optic users, Optic’s Data Model Explorer allows you to search, cross-reference, and view data model objects. In addition to the information above, Data Model Explorer also lists all of the forms that a form is referenced by (i.e., where that form’s type is a secondary property on another form).

Using the Wildcard as a Target

In many cases you can use the wildcard or asterisk ( * ) as the target of a pivot or traversal to represent “all the things” that can be reached by your operation. One use for the wildcard is to “explore” a subset of connected data - for example, if you are not sure what relationships exist among the data in your instance of Synapse, and you want to see “what’s connected”.

Keep in mind that individual wildcard pivots or traversals (as described below) only show you nodes that are connected to your source nodes by the specific operation performed; this is generally not “all the nodes” that are connected. (See the examples below for details on individual operations.)

The following Storm expression can be used to show all nodes that are connected to your source nodes by any type of property pivot or edge traversal relationship:

<source_nodes> tee { --> * } { <-- * }

The expression uses the Storm tee command to perform two operations on the set of source nodes (a Pivot Out and Traverse and a Pivot In and Traverse, with the wildcard as the target in each case) and return the combined results.

Tip

This query is equivalent to using the Explore button in the Optic UI to navigate.

There are two minor exceptions to this “show me all the connections” query:

The query will not return connections to edge nodes where the source node is an Ndef of an associated edge node. Edge nodes are not commonly used (in many cases, they have been deprecated in favor of light edges). Pivot operations involving edge nodes are described below.
The query will not return property connections where nodes may have have a common property value, but the properties are of different types (use of the wildcard to “find” relationships depends on Type Awareness).

Pivot Operations

Pivot operations navigate between sets of nodes that have properties that share a common value. Each pivot operation requires:

the source node(s) for the pivot;
a pivot operator (such as ->); and
the target of the pivot.

Unless otherwise specified, the target ( <target> ) of a pivot can be:

a form name (e.g., hash:md5 );
a partial form name (wildcard match, e.g., hash:*);
a form and property name (e.g., file:bytes:md5);
an Interface name (e.g., it:host:activity);
an interface and property name (e.g., file:mime:msoffice:application);
a list of form names (e.g., ( inet:dns:request, inet:flow )); or
a wildcard / asterisk ( * ).

For the specialized use case of Raw Pivot Syntax, the target of the pivot is a Storm expression.

Note

You cannot specify property values in pivot operations. For example, the following is invalid:

inet:fqdn=vertex.link -> inet:dns:a:ipv4=127.0.0.1

If you want to pivot to a specific node or subset of nodes, you must navigate to the target forms, and then filter your results based on the property value(s) you are interested in:

inet:fqdn=vertex.link -> inet:dns:a +:ipv4=127.0.0.1

Depending on the kind of pivot operation, you may need to specify a source property for the pivot as well; see the discussion of Implicit Pivot Syntax below.

While there are a few specialized use cases (such as pivoting to or from tags), most pivots involve navigating between the following kinds of properties:

primary to secondary;
secondary to primary;
secondary to secondary; or
primary to primary.

Primary to primary property pivots are a specialized use case that is commonly handled using Raw Pivot Syntax.

Tip

In Synapse, these property-to-property relationships are implicit relationships. You do not need to explicitly “define” the relationship or “link” the nodes. In fact, if the source and target properties have the same value and the same Type, Storm can use Synapse’s Type Awareness to simplify pivot operations and identify relationships among nodes.

Implicit Pivot Syntax

Pivot operations in Storm can always be executed by explicitly specifying the source and target properties for the pivot. This is referred to as explicit pivot syntax or “explicit syntax” for short.

When researching network infrastructure, a common set of pivots is to navigate from a set of FQDNs (inet:fqdn nodes) to their DNS A records (inet:dns:a nodes) and then to the IPv4 addresses (inet:ipv4 nodes) that the A records point to. The following Storm query performs those pivots using explicit syntax:

inet:fqdn = vertex.link -> inet:dns:a:fqdn :ipv4 -> inet:ipv4

The query:

lifts the FQDN vertex.link;
pivots from the FQDN to any DNS A node with the same FQDN property value (-> inet:dns:a:fqdn); and
pivots from the :ipv4 property of the inet:dns:a nodes to any inet:ipv4 nodes with the same value ( :ipv4 -> inet:ipv4 ).

We explicitly specify inet:dns:a:fqdn as the target property of our first pivot; and we explicitly specify the :ipv4 property of the inet:dns:a nodes as the source property of our second pivot. Explicit syntax tells Storm exactly what you want to do; there is no ambiguity in the query or in “how” you want to navigate the data.

Note

When specifying a secondary property as the source of a pivot (such as :ipv4 above), you must specify the property using relative property syntax (i.e., using the property name alone). If you were to use full property syntax (inet:dns:a:ipv4) Synapse would interpret that as a lift operation - i.e., “after you pivot to the DNS A records with an FQDN of vertex.link, then lift all DNS A records that have an IPv4 property, and pivot to ALL of the associated IPv4 nodes”.

Explicit syntax is precise, but there is extra work (“more typing”) involved to create the query, especially when there is an “obvious” source and / or target for the pivot. In other words, if you are pivoting from an FQDN (inet:fqdn) to a DNS A record (inet:dns:a), the :fqdn property is the only target property that makes sense.

In these cases, you can use implicit pivot syntax (implicit syntax) for your Storm query. Implicit syntax takes advantage of Synapse’s Type Awareness to “know” which properties can be pivoted to (or from), given the source and target forms for the pivot operation. With implicit syntax, you do not need to specify the source or target property in cases where it is self-evident for the forms used.

Using implicit syntax, we can rewrite the above query as follows:

inet:fqdn = vertex.link -> inet:dns:a -> inet:ipv4

With implicit syntax, we can simply specify the source and target forms, and allow Synapse to identify the source and target properties using types and type awareness.

Implicit syntax can be used in the following cases where the source and target properties have the same type and the same value:

Primary to secondary property pivots.
Secondary to primary property pivots.

Tip

This includes cases where the secondary property value is the node definition (Ndef) of the corresponding primary property (these cases are uncommon, but includes forms such as risk:technique:masquerade). Note that this does not extend to legacy edge nodes that are composite forms, which have their own optimizations and pivot syntax (see Edge Node Pivots, below).

Implicit pivot syntax cannot be used in the following cases:

Primary to primary property pivots (see Raw Pivot Syntax).
Secondary to secondary property pivots.
Pivots between primary and secondary properties involving tags (i.e., where the type is syn:tag). See Tag Pivots below.
Pivots between primary and secondary properties with the same value but of different types.
Pivots between primary and secondary properties where the source or target is ambiguous (e.g., where a form has more than one property of the same type, and you only want to pivot to (or from) one of them).

Tip

The examples below use implicit syntax where possible and may include explicit syntax for completeness. Where implicit syntax cannot be used, only explicit syntax is shown.

Pivot Out

Most pivots are “pivot out” operations (or variations). “Pivot out” refers to the “direction” of the pivot operator symbol: an “arrow” ( -> ) that points “out” from left to right. “Pivot out” operations are so common that they are generally just referred to as “pivots” or “pivot operations”.

Pivot out operations require:

the source node(s) for the pivot;
the pivot operator ( -> ); and
the target of the pivot.

The target(s) that are appropriate for a particular pivot out operation depend on the source and target properties and the specific navigation you want to perform.

Primary to Secondary Property Pivot

When pivoting from the primary property of a set of source nodes to target nodes with the same secondary property value (e.g., from a set of FQDNs to their associated DNS A nodes), the target can be:

a form name or interface name;
a form or interface name with a property name; or
a partial form name (wildcard form match).

Implicit syntax can be used for these pivots if the target property is self-evident / unambiguous.

Syntax:

<query> -> <form> [ : | . | ._ <prop> ]

<query> -> <interface> [ : <prop> ]

<query> -> <partial_form_name> *

Examples:

Pivot from a set of FQDNs (inet:fqdn nodes) to their associated DNS requests (inet:dns:request nodes):

<inet:fqdn nodes> -> inet:dns:request

The query above uses implicit syntax; Synapse recognizes that the logical target property (given source nodes that are FQDNs) is :query:name:fqdn. You could optionally use explicit syntax instead:

<inet:fqdn nodes> -> inet:dns:request:query:name:fqdn

Pivot from a set of IPv4 addresses (inet:ipv4 nodes) to any network flows (inet:flow nodes) associated with the IPs:

<inet:ipv4 nodes> -> inet:flow

The query above uses implicit syntax. Note that because inet:flow nodes have two target properties of type inet:ipv4 (:src:ipv4 and :dst:ipv4), the result of this query will be all inet:flow nodes where the inbound IPv4s are either the source or destination IP. If you only want to see flows where the inbound IPv4s are the destination IP (for example), you must use explicit syntax to clarify this:

<inet:ipv4 nodes> -> inet:flow:dst:ipv4

Pivot from a set of tags (syn:tag nodes) to the threat clusters (risk:threat nodes) represented by those tags:

<syn:tag nodes> -> risk:threat:tag

The query above uses explicit syntax. Even though the :tag property is of type syn:tag, syn:tag nodes have specialized handling with respect to pivoting in Storm (see Pivot from Tags and Pivot to Tags below). Because pivots involving syn:tag nodes are handled differently by default, you must use explicit syntax when pivoting between tags (syn:tag nodes) and properties whose type is syn:tag.

Pivot from a set of FQDNs (inet:fqdn nodes) to any associated DNS records (e.g., inet:dns:a, inet:dns:cname, inet:dns:ns, inet:dns:request, etc.):

<inet:fqdn nodes> -> inet:dns:*

The query above uses the wildcard ( * ) as a partial match for any form name that starts with inet:dns:.

Pivot from a set of FQDNs (inet:fqdn nodes) to any associated DNS A or DNS AAAA records:

<inet:fqdn nodes> -> inet:dns:a*

The query above uses the wildcard ( * ) as a partial match for any form name that starts with inet:dns:a followed by zero or more characters.

Pivot from a set of files to all host execution nodes (all nodes of all forms that inherit the it:host:activity interface - e.g., it:exec:file:add, it:exec:url, etc.) associated with those files:

<file:bytes nodes> -> it:host:activity

The query above uses implicit syntax. The it:host:activity interface defines two properties of type file:bytes for host activity nodes - :exe and :sandbox:file. The query above will return all activity nodes where the inbound files appear in either property. If you only want to see activity nodes whose :exe property (for example) matches the inbound files, you need to use explicit syntax:

<file:bytes nodes> -> it:host:activity:exe

Pivot from a set of usernames (inet:user nodes) to any file paths (file:path nodes) where the username is an element of the path (a :base value - for example, where a username might appear in a PDB path):

<inet:user nodes> -> file:path:base

The query above uses explicit syntax. The inet:user form has a type of inet:user, but the :base property of the file:path form has a type of file:base. Because the properties are two different types, you must use explicit syntax.

Tip

The query above will return the (partial) file:path whose final element is the user name (e.g., if your source node is inet:user=evilhacker, the query will return c:\users\evilhacker). You can use the Storm tree command to recursively pivot through the remaining file:path elements to obtain the full path containing the username:

<inet:user nodes> -> file:path:base tree { -> file:path:dir }

Pivot from a set of FQDNs (inet:fqdn nodes) to the “masquerade” technique nodes (risk:technique:masquerade nodes) associated with those FQDNs:

<inet:fqdn nodes> -> risk:technique:masquerade

The above query uses implicit pivot syntax.

A risk:technique:masquerade node represents an object that is purposely crafted to imitate (masquerade as) another object, typically for malicious purposes. The node records the relationship between the “original” object being imitated (risk:technique:masquerade:target) and the object that is imitating it (risk:technique:masquerade:node). (Both properties are of type “ndef” (node definition), and consist of ( <form>, <valu> ) pairs.)

Because the query above uses implicit syntax, it will return any risk:technique:masquerade nodes where the inbound FQDNs are either the :target or :node value. If (for example) the inbound FQDNs were a set of suspicious FQDNs and you wanted to return only those “masquerade” nodes where the inbound FQDNs were the :node masquerading as a valid FQDN, you would need to use explicit syntax:

<inet:fqdn nodes> -> risk:technique:masquerade:node

Secondary to Primary Property Pivot

When pivoting from a secondary property of a set of source nodes to target nodes with the same primary property (e.g., from a set of DNS A nodes to their associated FQDNs), the target can be:

a form name;
a list of form names;
a wildcard.

Implicit syntax can be used for these pivots if the source property is self-evident / unambiguous.

Tip

Use of the the wildcard (asterisk) character ( * ) is known as a “wildcard pivot out”. This pivot navigates from any/all secondary properties on the source nodes to the nodes corresponding to those property values. Contrast this operation with the “wildcard pivot in”, described under Pivot In.

Syntax:

<query> [ : | . | :_ <prop> ] -> <form>

<query> [ : | . | :_ <prop> ] -> ( <form_1> , <form_2> … )

<query> -> *

Note

If you specify a source property for the pivot, you must use the relative property name (i.e., the property name alone, preceded by its separator character).

Examples:

Pivot from a set of DNS A records (inet:dns:a nodes) to their associated FQDNs (inet:fqdn nodes):

<inet:dns:a nodes> -> inet:fqdn

The query above uses implicit syntax. Synapse recognizes that, given a target form of inet:fqdn, the logical source property is the :fqdn property of the inet:dns:a nodes. You could optionally use explicit syntax:

<inet:dns:a nodes> :fqdn -> inet:fqdn

Pivot from a set of DNS NS records (inet:dns:ns nodes) to their associated FQDNs (inet:fqdn nodes):

<inet:dns:ns nodes> -> inet:fqdn

The query above uses implicit syntax. Because inet:dns:ns nodes have two properties of type inet:fqdn (:zone and :ns), Synapse will pivot to the FQDNs associated with both values. If you only want to pivot to the FQDNs associated with the name server (NS) FQDNs (for example), you must use explicit syntax:

<inet:dns:ns nodes> :ns -> inet:fqdn

Pivot from a set of X509 certificate metadata nodes (crypto:x509:cert nodes) to the associated SHA1 fingerprints (hash:sha1 nodes) and to any FQDNs (inet:fqdn nodes) associated with the certificates:

<crypto:x509:cert nodes> -> ( hash:sha1, inet:fqdn )

Tip

Specifying a list of target forms allows you to perform a more focused pivot (in contrast to pivoting to any / all target forms using a wildcard). The elements of the list must be enclosed in parentheses and separated by commas.

Pivot from a set of X509 certificate metadata nodes (crypto:x509:cert nodes) to any/all nodes associated with any of the certificates’ secondary properties:

<crypto:x509:cert nodes> -> *

The query above is an example of a wildcard pivot out. For any secondary properties on the source nodes, the query will return the associated nodes. For example, if the crypto:x509:cert:identities:ipv4s property is set, the query will return the associated inet:ipv4 nodes. A wildcard pivot out is also known as a “refs out” pivot (for “references”) because it pivots to the nodes “referenced by” the source nodes’ secondary properties.

Secondary to Secondary Property Pivot

When pivoting from a secondary property of a set of source nodes to target nodes with the same secondary property (e.g., from the :ipv4 property of a set of DNS A nodes to a set of network flow nodes with the same IPv4 as a :dst:ipv4 property), the target can be:

a form name and property name;
a list of form and property names; or
an interface name and property name.

You must use explicit syntax to specify both the source and target properties.

Syntax:

<query> : | . | :_ <prop> -> <form> : | . | :_ <prop>

<query> : | . | :_ <prop> -> <interface> : <prop>

Examples:

Pivot from the WHOIS records (inet:whois:rec nodes) for a set of domains to the DNS A records (inet:dns:a nodes) for the same domains:

<inet:whois:rec nodes> :fqdn -> inet:dns:a:fqdn

Tip

In many cases, secondary to secondary property pivots are a “shortcut” for two pivot operations (secondary to primary and primary to secondary) that perform the equivalent navigation. For example:

<inet:whois:rec nodes> -> inet:fqdn -> inet:dns:a

Pivot from a set of DNS requests (inet:dns:request nodes) to all host activity nodes (all nodes of all forms that inherit the it:host:activity interface) that share the same file as their :exe property:

<inet:dns:request nodes> :exe -> it:host:activity:exe

Pivot from a set of DNS A records (inet:dns:a nodes) to any network flows (inet:flow) or service banners (inet:banner) associated with the IPs:

<inet:dns:a nodes> -> ( inet:flow:dst:ipv4, inet:banner:server:ipv4 )

Pivot In

The pivot in operator is an “arrow” ( <- ) that points “in” from right to left.

Pivot operations navigate between sets of nodes that share a common property value. There is no “direction” to this relationship; logically, pivot operations are functionally equivalent whether the pivot “arrow” points out (left to right) or in (right to left):

Pivot from a set of FQDNs to their associated DNS A records:

<inet:fqdn nodes> -> inet:dns:a
Pivot from a set of FQDNs to the DNS A records that reference them:

<inet:fqdn nodes> <- inet:dns:a

Because Storm evaluates operations from left to right, the “pivot out” arrow is generally more intuitive and has been used to implement nearly all pivot operations in Storm (the second example, above, is not supported and will generate a StormRuntimeError).

The pivot in operation is a specialized operation that can only be used with the wildcard ( * ) as a target. This wildcard pivot in operation navigates from the primary property of the source node(s) to any nodes where that value is a secondary property. A wildcard pivot in is also known as a “refs in” pivot (for “references”) because it pivots out to the nodes that “reference” the source nodes’ primary property. Contrast this operation with the “wildcard pivot out”, described under Secondary to Primary Property Pivot.

Syntax:

<query> <- *

Example:

Pivot from a set of FQDNs (inet:fqdn nodes) to all nodes with a secondary property that references a source FQDN:

<inet:fqdn nodes> <- *

A wildcard pivot in will return any node with a secondary property value that matches any of the source FQDNs. For example, the above query could return various DNS records (inet:dns:a, inet:dns:mx), URLs (inet:url), email addresses (inet:email), articles (media:news), and so on.

Raw Pivot Syntax

Raw pivot syntax is a pivot operation where the target of the pivot is expressed as a Storm query. The use of raw pivot syntax is uncommon for interactive Storm queries, but is useful when you need greater flexibility in specifying the pivot target vs. using standard property-to-property pivot syntax.

Raw pivot syntax requires:

the source node(s) for the pivot;
a pivot operator; and
the target of the pivot (specified as a Storm query enclosed in curly braces).

Both pivot ( -> ) and pivot and join ( -+> ) operators can be used with raw pivot syntax. Pivot in ( <- ) and pivot in and join ( <+- ) are not supported.

Use cases for raw pivot syntax include primary-to-primary property pivots and pivots where the value of the target property (primary or secondary) is computed from the input node(s) (e.g., using a variable derived from the inbound nodes. See Storm Reference - Advanced - Variables for a discussion of using variables in Storm).

Syntax:

<query> <pivot operator> { <query> }

Examples:

The following is a simple example to illustrate the concept.

Pivot from a set of strings (it:dev:str nodes) representing domains to the associated FQDNs (inet:fqdn nodes):

<it:dev:str nodes> $fqdn = $node.value() -> { inet:fqdn ?= $fqdn }

The query above is a primary-to-primary property pivot between forms of different types. The query lifts the it:dev:str nodes, defines the variable $fqdn as the value of the node, then pivots to the inet:fqdn nodes with the same value. The FQDN nodes are lifted using the Storm query; the “Try” Operator is used in the event that any of the inbound it:dev:str nodes are invalid as FQDNs.

Note that you could create an equivalent Storm query using only lift and filter operations:

<it:dev:str nodes> $fqdn = $node.value() inet:fqdn ?= $fqdn -it:dev:str

This second query still lifts the it:dev:str nodes and sets the $fqdn variable, but then lifts the inet:fqdn nodes directly and drops (filters out) the original it:dev:str nodes.

While both queries return the same result (the inet:fqdn nodes), using raw pivot syntax is slightly more efficient because the pivot operation drops the it:dev:str nodes for you (pivots consume nodes); there is no need to filter them out at the end, as in the second example. As always, these efficiencies may be trivial for smaller queries but can be significant for larger queries.

Pivot from a set of HTTP referer headers (inet:http:request:header nodes) to any URLs (inet:url nodes) specified as the header value:

inet:http:request:header:name = referer $url = :value -> { inet:url ?= $url }

The query above illustrates using raw pivot syntax to pivot from a secondary to a primary property where the properties are of different types. The query lifts all inet:http:request:header nodes where the :name value is referer, sets the associated :value property (a str type) as the variable $url, then pivots to any inet:url nodes (an inet:url type) that have the same value. The “Try” Operator operator is used in the event any of the referer values contain bad or malformed data.

Tag Pivots

Tags in Synapse are unique in that they are both nodes and labels that are applied to other nodes. (See the Analytical Model document for additional discussion of tags as both nodes and labels.)

Because of tags’ unique role, pivot syntax used with tags navigates between tags as nodes (syn:tag nodes) and tags as labels (i.e., nodes that have the tags applied), as opposed to performing standard property-to-property pivots.

Both the pivot out ( -> ) and pivot out and join ( -+> ) operator are supported for tag pivots. Pivot in ( <- ) and pivot in and join ( <+- ) are not supported.

Tip

The custom behavior used with tag pivots may lead to counterintuitive results when attempting to pivot between tags (syn:tag nodes) and properties that are syn:tag types (such as risk:threat:tag or ou:technique:tag).

For example, if you attempt to pivot from a syn:tag node used to associate nodes with a threat cluster to the risk:threat node representing the cluster, the following Storm query will fail to return the expected results:

syn:tag=rep.talos.lazarus -> risk:threat

The query looks as though it should return the appropriate risk:threat node using implicit syntax to navigate to the risk:threat:tag property, using type awareness. However, because the default Storm behavior when pivoting from a syn:tag node is to pivot to nodes that have the tag, the above query is actualy attempting to pivot to any risk:threat nodes that are tagged #rep.talos.lazarus. The query is syntactically correct, so will not generate an error; but it will not return the expected nodes (and likely will not return any nodes at all).

Explicitly specifying the :tag property will return the risk:threat node as expected:

syn:tag=rep.talos.lazarus -> risk:threat:tag

Pivot to Tags

Pivot to tags syntax allows you to pivot from a set of nodes with tags to the set of syn:tag nodes representing those tags. Pivot to tag operations require:

the source node(s) for the pivot;
a pivot operator; and
the set of tags (syn:tag nodes) that is the target of the pivot.

Pivot to tags uses the “hashtag” symbol ( # ) to indicate that the target of the pivot is a set of syn:tag nodes (by default, the set of all leaf tags on the source nodes). The target expression can be modified to refer to:

all tag nodes,
all tag nodes matching a specified pattern, or
the tag node matching a specific tag.

Pivot to Leaf Tags

The hashtag symbol ( # ) used by itself as a target refers to the syn:tag nodes for all leaf tags applied to the source nodes. A leaf tag is the longest / final tag in a tag tree. For example, if a node has the tag #rep.eset.sednit (which is comprised of the three tags #rep, #rep.eset, and #rep.eset.sednit), the leaf tag is #rep.eset.sednit.

Syntax:

<query> <pivot_operator> #

Examples:

Pivot from a set of nodes to the tags (syn:tag nodes) for all leaf tags applied to those nodes:

<query> -> #

Pivot to All Tags

The hashtag symbol can be used with the wildcard ( #* ) to pivot to all syn:tag nodes applied to the source nodes, not just the leaf tags. For example, if a node has the tag #rep.eset.sednit (which is comprised of the three tags #rep, #rep.eset, and #rep.eset.sednit), all three tags will be returned.

Syntax:

<query> <pivot_operator> #*

Pivot from a set of nodes to the tags (syn:tag nodes) for all tags applied to those nodes:

<query> -> #*

Pivot to Tags Matching a Pattern

The single ( * ) or double ( ** ) asterisk can be used to specify a set of target tags that match the specified tag glob expression. The single asterisk is used to match within a single tag element. The double asterisk is used to match across tag elements.

Syntax:

<query> <pivot_operator> # <string> | * | ** [ . <string> | * | ** … ]

Pivot from a set of nodes to the tags (syn:tag nodes) associated with any third-party reporting where the third tag element is “bisonal”:

<query> -> #rep.*.bisonal

Vertex uses the rep tag tree to indicate third-party reporting / assertions. The query above will match syn:tag nodes from the source nodes where the tag has rep as the first tag element, has any value as the second element, and has bisonal as the third element. This would include tags such as:

rep.alienvault.bisonal
rep.malwarebazaar.bisonal

Pivot from a set of nodes to the tags (syn:tag nodes) representing any third party reporting where the tag ends in the string “bisonal”:

<query> -> #rep.**bisonal

The query above will match any tag that has rep as the first element, followed by any number of elements that end in the string “bisonal”. This would include the tags noted above, as well as tags such as:

rep.malwarebazaar.3p.intezer.bisonal

Pivot to Specific Tag

Pivot to the syn:tag node for a specific tag by specifying the exact tag as the target.

Syntax:

<query> <pivot operator> # <tag>

Pivot from a set of nodes to the syn:tag node for the tag “cno.ttp.phish.attach”:

<query> -> #cno.ttp.phish.attach

Pivot from Tags

Pivot from tags syntax allows you to pivot from a set of syn:tag nodes to the nodes that have those tags.

Pivot from tag operations require:

the source node(s) for the pivot (syn:tag nodes);
a pivot operator; and
the target forms for the pivot.

The target for a pivot from tags operation can be:

a form name;
a partial form name (wildcard match);
an interface name;
a list of form names; or
a wildcard.

Pivot from tags pivots to nodes that have the exact source tag(s) applied. For example, pivoting from the tag syn:tag=rep.eset.sednit will not return nodes with only #rep or #rep.eset. Conversely, pivoting from syn:tag=rep will return nodes with any tag in the rep tag tree (because the rep tag is included in all of those trees).

Tip

A pivot from tags operation is similar to a Lift by Tag or Lift Form by Tag operation. The following Storm queries are equivalent:

syn:tag=rep.microsoft.forest_blizzard -> *

#rep.microsoft.forest_blizzard

As are these:

syn:tag=rep.microsoft.forest_blizzard -> ( inet:fqdn, inet:ipv4 )

inet:fqdn#rep.microsoft.forest_blizzard inet:ipv4#rep.microsoft.forest_blizzard

Pivot from tags can be useful when used with Pivot to Tags. For example, you can take a set of inbound tagged nodes, use pivot to tags to navigate to some or all of the associated syn:tag nodes, and then use pivot from tags to navigate to other nodes that have the same tags.

Syntax:

<syn:tag node(s)> <pivot operator> <form>

<syn:tag node(s)> <pivot operator> <partial_form_name> *

<syn:tag node(s)> <pivot operator> <interface>

<syn:tag node(s)> <pivot operator> ( <form_1> , <form_2> … )

<syn:tag node(s)> <pivot operator> *

Examples:

Pivot from a set of syn:tag nodes to any files (file:bytes nodes) with those tags applied:

<syn:tag nodes> -> file:bytes

Pivot from a set of syn:tag nodes to any DNS nodes with those tags applied:

<syn:tag nodes> -> inet:dns:*

The query above would return (for example) inet:dns:a nodes, inet:dns:request nodes, etc.

Pivot from a set of syn:tag nodes to any host activity nodes (all nodes of all forms that inherit the it:host:activity interface) with those tags applied:

<syn:tag nodes> -> it:host:activity

Pivot from a set of syn:tag nodes to any IPv4 (inet:ipv4), IPv6 (inet:ipv6), server (inet:server), or network flow (inet:flow) nodes with those tags applied and retain the syn:tag nodes in the results (pivot and join):

<syn:tag nodes> -+> ( inet:ipv4, inet:ipv6, inet:server, inet:flow )

Pivot from a set of syn:tag nodes to all nodes that have any of the tags applied:

<syn:tag nodes> -> *

Edge Node Pivots

Edge nodes (also called digraph nodes) are specialized nodes whose purpose is to link two arbitrary nodes in a specific relationship. Edge nodes are composite forms but are unique in that, because the node(s) in the relationship may be arbitrary, the edge node’s primary property consists of at least one value that is a Node Definition or ndef - that is, a (<form>, <valu>) pair vs. a standard <valu>. (Some edge nodes include a time value as a third element of the primary property if the arbitrary relationship occurred at a specific time.)

This means that pivots to or from edge nodes must account for having a form and property value in common between the source and target, not just a property value.

Both the pivot ( -> ) and pivot and join ( -+> ) operators are supported for edge pivots. The pivot in operator ( <- ) is supported for specialized cases. The pivot in and join operator ( <+- ) is not supported.

Note

Composite edge nodes are largely legacy elements in the Synapse data model. In many cases, the use of lightweight (light) edges is now preferred over edge nodes. For example, edge:has nodes have largely been replaced by -(has)> light edges; and the use of meta:seen nodes is discouraged in favor of -(seen)> light edges.

Edge nodes may be preferred when you need to record additional information about the relationship (edge nodes have properties, and you can apply tags to edge nodes). We recommend choosing one option (edge nodes or light edges) for a given relationship for consistency (i.e., so analysts do not have to query for the same type of relationship in two different ways).

In addition, where there is a need to:

link two arbitrary nodes in some type of relationship, and
record additional information about the relationships (i.e., where use of a light edge is not appropriate)

…the current preference is to create a guid form whose secondary properties include one or more ndefs instead of a composite edge node (an example of this kind of form is the risk:technique:masquerade form). In particular, use of a guid form with ndef secondary properties greatly simplifies pivoting to or from nodes that use ndefs, eliminating the need for analysts to use or be aware of specialized Edge Node Pivots.

By using guid forms, the ndef value(s) are no longer tied to the node’s primary property the way they are in a legacy composite edge node. This allows Synapse and Storm to treat pivots between nodes and ndef properties as “standard” primary-to-secondary or secondary-to-primary property pivots (using an optimization similar to that used for type awareness and implicit pivot syntax).

Pivot to Edge Nodes

Pivoting to edge nodes requires:

the source node(s) for the pivot;
the pivot operator;
the target of the pivot.

When pivoting to edge nodes, the target can be:

a form name (edge form);
a partial edge form name (wildcard match) - supported for pivot out / pivot out and join only;
an interface (in cases where an edge form inherits an interface); or
a form (edge form) and property name.

Tip

Storm uses some optimizations to simplify working with edge nodes.

When pivoting out ( -> or -+> ) to a set of edge nodes, the target of the pivot is assumed to be the edge nodes whose n1 value matches the (<form>, <valu>) of the inbound nodes. This means that you do not need to specify the target property for an edge pivot unless the target is the edge nodes’ n2 property.

As an alternative, the pivot in operator ( <- ) can be used specifically to pivot from the source nodes to the n2 property of the target edge nodes.

Note that the use of the pivot in operator ( <- ) to pivot to edge nodes’ n1 property is not supported, even if you specify it as the target property. In addition, the target of a pivot in operation cannot be a partial edge form name (wildcard match).

Syntax:

<query> <pivot operator> <target> [ :n2 ]

<query> <- <edge_form>

Examples:

Note

The examples below are for illustrative purposes to show the Storm syntax used to navigate edge nodes. Based on current modeling best practices, the relationships shown here using edge nodes would typically be represented using light edges or guid forms.

Pivot from a set of articles (media:news nodes) to the edge nodes (edge:refs nodes) showing things that the articles reference (e.g., such as indicators like hashes or FQDNs):

<media:news nodes> -> edge:refs

Pivot from a set of vulnerabilities (risk:vuln nodes) to the edge nodes (edge:has nodes) showing which nodes have those vulnerabilities (e.g., such as an it:host):

<risk:vuln nodes> -> edge:has:n2

Because the risk:vuln ndefs are the n2 value of the edge nodes, we must specify :n2 as the target property. The following pivot in operation will return the same results:

<risk:vuln nodes> <- edge:has

Pivot from a set of nodes to any edge nodes (e.g., edge:has, edge:refs) where the inbound nodes are the n1 of any of the edge nodes:

<query> -> edge:*

Pivot from Edge Nodes

Pivoting from edge nodes requires:

the source node(s) (edge nodes) for the pivot;
the pivot operator;
the target of the pivot.

When pivoting from edge nodes, the target can be:

a form name;
a partial form name (wildcard match) - supported for pivot out / pivot out and join only;
an interface; or
a wildcard.

Tip

Storm uses some optimizations to simplify working with edge nodes.

When pivoting out ( -> or -+> ) from a set of edge nodes, the source of the pivot is assumed to be the n2 of the edge nodes. If you want to pivot out from edge nodes’ n1 property, you must specify :n1 as the source property.

As an alternative, the pivot in operator ( <- ) can be used to specifically to pivot from the edge nodes’ to the n1 property. The use of the pivot in operator to pivot from edge nodes’ n2 property is not supported, even if you specify it as the source property. In addition, the target of a pivot in operation cannot be a partial form name (wildcard match).

Syntax:

<edge_form> [ :n1 ] <pivot operator> <target>

<edge_form> <- <form> | <interface> | *

Examples:

Note

The examples below are for illustrative purposes to show the Storm syntax used to navigate edge nodes. Based on current modeling best practices, the relationships shown here using edge nodes would typically be represented using light edges.

Pivot from a set of “has” edge nodes (edge:has nodes) to all of the objects the nodes “have”:

<edge:has nodes> -> *

Pivot from a set of “has” edge nodes (edge:has nodes) to all of the objects that “have” things:

<edge:has nodes> :n1 -> *

You can also use the pivot in operator to pivot from the :n1 property by default:

<edge:has nodes> <- *

Pivot from a set of “has” edge nodes to any vulnerabilities that the objects “have”:

<edge:has nodes> -> risk:vuln

Pivot Across Edge Nodes

Because edge nodes represent relationships, analytically we are often more interested in the nodes on “either side” of the edge node than we are in the edge node itself. For this reason, the pivot operators have been optimized to allow for easily navigating “across” these edge nodes.

Pivoting across edge nodes still entails two pivot operations (pivot to edges and pivot from edges, as described above). Like all Storm operations, each type of pivot can be performed independently and combined with other operations (e.g., lift, pivot to edges, filter, etc.)

When you pivot to edge nodes and immediately pivot from edge nodes (navigating “across” the edges), Storm optimizes the pivot syntax to simplify this process.

Specifically:

Two pivot out operators ( -> or -+> ) can be combined to easily pivot from:

source nodes to the edge nodes’ :n1 property, and
the edge nodes; :n2 property to the target nodes.

<source_nodes> -> <edge_nodes> -> <target_nodes>

By optimizing which proprty of the edge node is assumed to be the source or target of the pivot operation, Synapse makes it easy to navigate across the edge relationship intuitively from left (:n1) to right (:n2) without the need to explicitly specify source and target properties.

Similarly, two pivot in operators ( <- or <+- ) can be used to pivot from:

source nodes to the edge nodes’ :n2 property, and
the edge nodes’ :n1 property to the target nodes.

<source_nodes> <- <edge_nodes> <- <target_nodes>

This allows you to navigate intuitively across the edge relationship “backwards” from right (:n2) to left (:n1).

Traversal Operations

Traversal operations are used to navigate between sets of nodes that are linked using a lightweight (light) edge. Traversal is still considered a “pivot” between sets of nodes, but is named differently to distinguish light edge navigation from property-to-property pivot navigation.

Tip

There are no light edges (i.e., specific light edge verbs) defined within Synapse by default. Users can create and define their own according to their needs. However, both the Synapse Data Model Forms documentation and the Optic UI’s Data Model Explorer list edges used by The Vertex Project by convention which are recommended for use (note that Power-Ups may create these edges).

The Storm model, edges, and lift.byverb commands can be used to work with light edges in Synapse.

Each traversal operation requires:

the source node(s) for the operation;
the traversal operator; and
the target forms for the traversal.

Traversal is also referred to as “walking” - that is, you can traverse a light edge or walk a light edge.

The traversal operator is an “arrow” with embedded parentheses. The parentheses enclose a string (commonly a verb) for the relationship(s) represented by the edge:

-(<verb>)>

<verb> can be a single edge, a list of edges, or a wildcard.

Unlike property-to-property relationships, edge relationships have a direction. There is a “source” node (n1) and a “target” node (n2) for the relationship itself; an article (media:news node, the n1) can reference an indicator such as a hash (hash:md5 node, the n2) but it does not make sense for a hash to “reference” an article.

The pivot operator ( -> ) and its variations “point” from left to right (other than a few specialized cases) by convention. In contrast, the traversal operator can “point” in either direction, depending on which nodes (which “side” of the edge relationship) are inbound. Both of the syntaxes below are equally valid:

<media:news nodes> -(refs)> <hash:md5 nodes>

<hash:md5 nodes> <(refs)- <media:news nodes>

Tip

In Synapse, lightweight edge relationships are explicit relationships. The light edges used to connect nodes must be defined within Synapse, and nodes joined by light edges must be explicitly linked.

Unless otherwise specified, the target ( <target> ) of an edge traversal can be:

a form name (e.g., hash:md5 );
a form name, comparison operator, and value (e.g., a primary property value, such as inet:fqdn~=news);
a form and property name (e.g., inet:url:fqdn);
a form and property name, comparison operator, and value (e.g., a secondary property value, such as risk:tool:software:used@=(2018, 2022))
a partial form name (wildcard match, e.g., hash:*);
an interface name (e.g., it:host:activity);
a list of form names (e.g., ( hash:sha256, file:bytes )); or
a wildcard / asterisk ( * ).

Note

The ability to specify a secondary property and value is supported in Synapse 2.x.x, but will be removed in 3.x.x. You can obtain the same result by traversing to a form and property name, and then filtering on the value. For example, the following Storm queries are equivalent:

media:news -(refs)> inet:url:fqdn~=justice.gov

and

media:news -(refs)> inet:url:fqdn +:fqdn~=justice.gov

Traverse a Specific Edge

Specify the name (verb) of the edge you want to traverse to navigate a single edge.

Syntax:

<query> -( <verb> )> <target>

<query> <( <verb> )- <target>

Examples:

Traverse the “uses” light edge from a threat cluster (risk:threat node) to the tools or malware (risk:tool:software nodes) used by the cluster:

<risk:threat> -(uses)> risk:tool:software

Traverse the “references” (refs) light edge from an article (media:news node) to all of the nodes “referenced” by the article:

<media:news> -(refs)> *

Traverse the “ipwhois” light edge from a set of IPv4 addresses (inet:ipv4 nodes) to the network registration / network WHOIS records (inet:whois:iprec nodes) the IPs are associated with:

<inet:ipv4 nodes> <(ipwhois)- inet:whois:iprec

Traverse the “seen” light edges from a set of DNS A records (inet:dns:a nodes) to the sources (meta:source nodes) that “saw” (observed or provided data for) the record:

<inet:dns:a nodes> <(seen)- meta:source

Traverse Multiple Edges

Specify a list of edge names (verbs) to traverse multiple edges to their targets.

Syntax:

<query> -( ( <verb1> , <verb2> [ , <verb3> ] ) )> <target>

<query> <( ( <verb1> , <verb2> [ , <verb3> ] ) )- <target>

Tip

Because each light edge represents a different relationship whose targets may vary widely, the wildcard ( * ) is frequently used as the target when traversing multiple edges (though this is not a requirement).

Example:

Traverse the “references” (refs) and “seen” light edges from an FQDN to any nodes linked via those light edges (i.e., articles (media:news nodes) that reference the FQDN and data sources (meta:source nodes) that “saw” the FQDN):

<inet:fqdn> <( ( refs, seen ) )- *

Traverse All Edges

Use the wildcard (asterisk) character ( * ) to traverse any edges present in the specified direction to their targets.

Tip

Using the wildcard to specify any edge name is useful when:

you want to traverse any / all edges;
you want to navigate to a specific set of targets (regardless of the particular edges); or
if you are not familiar with the edges that may be used with your source nodes and simply want to explore any connections that may be present.

Syntax:

<query> -( * )> <target>

<query> <( * )- <target>

Examples:

For a threat cluster (risk:threat node), traverse any light edges linking the cluster to any vulnerabilities (risk:vuln nodes) or victim organizations (ou:org nodes):

<risk:threat> -(*)> ( risk:vuln, ou:org )

For a vulnerability (risk:vuln node), navigate to any forms that are connected to the vulnerability by any edge:

risk:vuln <(*)- *

Vertex Project conventions can link risk:vuln nodes to things that have the vulnerability ( via <(has)- edges) or things that use the vulnerability ( via <(uses)- edges - e.g., a threat cluster (risk:threat) or a file (file:bytes) that makes use of an exploit for the vulnerability, etc.).

Pivot and Traverse Operations

Property pivots and edge traversals can be combined into a single “pivot and traverse” operation to perform “both types” of navigation simultaneously from a set of source nodes. “Pivot and traverse” combines a pivot operation with a traverse all edges operation in the specified direction.

Each pivot and traverse operation requires:

the source node(s) for the operation;
the pivot and traverse operator; and
the target of the operation.

The pivot and traverse operator is a “double arrow” ( --> ) with two dashes (vs. one for a standard pivot). The operator can point “out” or “in”, depending on the specific pivot and traverse operation you want to perform.

Because pivot and traverse operations perform “all available” navigation in a given direction, the only valid target for this operation is the wildcard ( * ).

Tip

The combined “pivot and traverse” operators are commonly used to “explore” a subset of connected nodes. Note that the Storm tee command can be used to perform concurrent pivot in and traverse / pivot out and traverse operations on an inbound set of nodes:

<query> | tee { --> * } { <-- * }

This combined operation shows you all nodes connected to your source nodes by any property or edge. It is equivalent to using the Explore button in the Optic UI.

Pivot Out and Traverse

The pivot out and traverse (walk) light edges operator ( --> ) combines a wildcard pivot out (“refs out”) operation ( -> * ) with a wildcard (any / all edges) edge traversal operation ( -(*)> * ).

Syntax:

<query> --> *

Examples:

Pivot from a set of IP netblock registration records (inet:whois:iprec nodes) to all nodes associated with the records’ secondary properties and all nodes linked to the records by light edges:

<inet:whois:iprec nodes> --> *

Pivot In and Traverse

The pivot in and traverse (walk) light edges operator ( <-- * ) combines a wildcard pivot in (“refs in”) operation ( <- * ) with a wildcard (any / all edges) edge traversal operation ( <(*)- * ).

Syntax:

<query> <-- *

Examples:

Pivot from a set of IPv4 addresses (inet:ipv4 nodes) to all nodes that reference the IPs and all nodes linked to the IPs by light edges:

<inet:ipv4> <-- *

Join Operations

Like most Storm operations, pivots and traversals consume nodes. With both types of operations, you navigate “away” from your source nodes; the nodes that are the target of your operation become your working set / result set.

With join operations, the inbound nodes are retained and combined (joined) with the target nodes in a single result set.

Each join operation requires:

the source node(s) for the operation;
the appropriate join operator; and
the target of the operation.

Join operators are variations on the standard pivot and traversal operators, but include a “plus” sign ( + ) in the operator to indicate the join. For example, the pivot and join operator is the pivot arrow ( -> ) combined with the plus sign to represent “pivot and join” ( -+> ).

Pivot and Join

The pivot and join operator ( -+> ) is the pivot “arrow” with an embedded plus sign ( + ) to represent joining the source and target nodes. The operator can be used anywhere the standard pivot operator ( -> ) is used, including the wildcard pivot out. Refer to the syntax examples under Pivot Out for the various types of pivot operations (e.g., primary to secondary property, secondary to secondary property, etc.)

Examples:

Pivot from a set of organizations (ou:org nodes) to any associated contacts (ps:contact nodes), retaining the organizations in the results:

<ou:org nodes> -+> ps:contact

Pivot from a set of DNS A records (inet:dns:a nodes) to their associated IPv4 addresses (inet:ipv4 nodes), retaining the DNS A records in the results:

<inet:dns:a nodes> -+> inet:ipv4

Pivot from a set of domain WHOIS records (inet:whois:rec nodes) to the DNS A records (inet:dns:a nodes) associated with the FQDNs, retaining the WHOIS records in the results:

<inet:whois:rec nodes> :fqdn -+> inet:dns:a:fqdn

Pivot from all secondary properties of a set of files (file:bytes nodes) to the associated nodes, retaining the files in the results:

<file:bytes nodes> -+> *

Pivot In and Join

The pivot in and join operator ( <+- ) can be used to perform a wildcard pivot in operation and join the results with the source nodes. Just as with a standard pivot in, the wildcard is the only valid target for this operation.

Syntax:

<query> <+- *

Example:

Pivot from a set of files (file:bytes nodes) to any nodes that reference the files, retaining the original files in the results:

<file:bytes nodes> <+- *

Traverse and Join

The traverse and join operator ( -(<verb>)+> ) is a traversal “arrow” with a plus sign ( + ) to represent joining the source and target nodes. The operator can be used anywhere the standard ( -(<verb>)> ) traversal operator is used. The operator can be used with a named edge or edges (e.g., -(refs)+> or <+((uses, targets))-) or with a wildcard to represent any / all edges (e.g., -(*)+> ).

Syntax:

<query> -( <verb> )+> <target>

<query> <+( <verb> )- <target>

<query> -( ( <verb1> , <verb2> [ , <verb3> …] ) )+> <target>

<query> <+( ( <verb1> , <verb2> [ , <verb3> …] ) )- <target>

<query> -( * )+> <target>

<query> <+( * )- <target>

Examples:

Traverse the “refs” light edge from an article (media:news node) and join the article with the FQDNs (inet:fqdn nodes) “referenced” by the article:

<media:news> -(refs)+> inet:fqdn

Join an article (media:news node) with any/all nodes referenced by the article:

<media:news> -(refs)+> *

Join a threat cluster (risk:threat node) with any nodes used or targeted by the cluster:

<risk:threat> -( (uses, targets) )+> *

Traverse and join any/all light edges from a vulnerability (risk:vuln node) to all nodes linked by any light edge:

<risk:vuln> <+(*)- *

Common edges used with vulnerabilites (risk:vuln nodes) include “uses” and “has”.

Pivot, Traverse, and Join

The pivot, traverse, and join operator ( --+> ) combines the “pivot and traverse” double arrow with a plus sign ( + ) to represent joining the source and target nodes. The operator can be used anywhere the standard pivot and traverse operator ( --> ) is used. Because combined pivot and traverse operations perform “all available” navigation, the only valid target for this operation is the wildcard ( * ).

Syntax:

<query> --+> *

<query> <+-- *

Examples:

Join a set of articles (media:news nodes) with all nodes representing the articles’ secondary properties (pivot out) and all nodes linked by any “right-facing” light edge:

<media:news> --+> *

Join a set of IPv4 addresses (inet:ipv4 nodes) with all nodes that reference the IPs (pivot in) and all nodes linked to the IPs by “left-facing” light edges:

<inet:ipv4> <+-- *