Storm Reference - Lifting

Lift operations retrieve a set of nodes based on the specified criteria. While all lift operations are retrieval operations, they can be broken down into “types” of lifts based on the criteria, comparison operator, or special handler used:

• Lift by Form

• Lift by Property

• Lift by Property Value - Standard Comparison Operators

• Lift by Property Value - Extended Comparison Operators

• Tag Lifts

In addition, the specialized “reverse” keyword and the “try” operator can each be used with lift operations to modify their behavior:

• “reverse” Keyword

• “Try” Operator

Tip

When performing lift operations, you can specify the name of an Interface to represent all forms that inherit that interface. See the sections below for details and examples.

See Storm Reference - Document Syntax Conventions for an explanation of the syntax format used below.

See Storm Reference - Type-Specific Storm Behavior for details on special syntax or handling for specific data types.

Lift by Form

“Lift by form” operations return all nodes of that form in Synapse. The wildcard (asterisk) character ( * ) can be used to lift all nodes of all forms that match a partial form name / namespace.

If a form inherits an Interface, you can specify the interface name to lift all nodes of all forms that inherit the interface.

Tip

In a production instance of Synapse, lifting all nodes of a commonly used form (e.g., inet:fqdn or inet:ipv4) or lifting by an interface that is inherited by numerous forms (e.g. it:host:activity) may return thousands or tens of thousands of nodes. Lifting by form or interface can be used with the Storm limit command to return only a specified number of nodes (for example, to view a sample of available data).

Lift by Form Name

A “lift by form name” operation returns all nodes for the specified Form. This type of lift requires the name of the form whose nodes you want to lift.

Syntax:

<form>

Examples:

Lift all FQDN (inet:fqdn) nodes:

inet:fqdn

Lift all nodes representing articles (media:news nodes):

media:news

Lift by Form Name - Wildcard

You can use the wildcard (asterisk) character ( * ) to specify all forms that match a partial form name. Use of the wildcard is not limited to form namespace boundaries.

Note

The wildcard can only be used at the end of the partial match. It cannot be used at the beginning or in the middle of the form name. For example, both of the following are invalid:

*:header

it:exec:*:del

In addition, use of the wildcard does not extetnd to partial matching of property names. For example, the following is invalid:

file:bytes:mime:pe:*

Syntax:

<partial_form_name> *

Examples:

Lift all nodes in the MITRE ATT&CK form namespace (e.g., it:mitre:attack:group, it:mitre:attack:technique, etc.):

it:mitre:attack:*

Lift all DNS A (inet:dns:a) and DNS AAAA (inet:dns:aaaa) nodes:

inet:dns:a*

Lift Form by Interface

You can use the name of an Interface to lift all forms that inherit that interface.

Note

When lifting by interface, you cannot use the wildcard ( * ) character to match multiple interface names. Synapse will interpret use of the wildcard as an attempt to match multiple form names.

Syntax:

<interface>

Examples:

Lift all host activity nodes (all nodes of all forms that inherit it:host:activity interface):

it:host:activity

Lift all taxonomy nodes (all nodes of all forms that inherit the meta:taxonomy interface):

meta:taxonomy

Lift by Property

A “lift by property” operation returns all nodes that have the specified Property set, regardless of the property value. In most cases, this type of lift requires the full name (i.e., the combined form and property name) of the property you want to use to lift the nodes. When lifting by universal property, the form name is only needed if you want to lift nodes of a specific form that have the universal property.

Lift by Secondary Property

Syntax:

<form> : <prop>

Examples:

Lift all IPv4 (inet:ipv4) nodes that have a location (:loc property):

inet:ipv4:loc

Lift all file (file:bytes) nodes that have a PDB path (:mime:pe:pdbpath property):

file:bytes:mime:pe:pdbpath

Lift by Interface Property

If a form inherits an Interface, you can lift all nodes of all forms that have an interface-derived property by specifying the full name of the interface and its property.

Syntax:

<interface> : <prop>

Examples:

Lift all host activity nodes (all nodes of all forms that inherit the it:host:activity interface) that have a :time property:

it:host:activity:time

Lift all protocol request nodes (all nodes of all forms that inherit the inet:proto:request interface) that have an associated network flow (i.e., that have a :flow property):

inet:proto:request:flow

Lift by Universal Property

Syntax:

[ <form> ] . <prop>

A Universal Property applies to and can be used by any node. Synapse uses two universal properties:

• .created, a time (date/time) value representing when a node was created in Synapse.

• .seen, an interval (pair of date/time values) that can be used optionally to represent “when” the object represented by a node existed or was observed.

Examples:

Lift all email (inet:email) nodes with a .seen property:

inet:email.seen

Lift all nodes with a .seen property:

.seen

Lift all nodes in Synapse:

.created

Tip

Because the .created property is automatically set for every node when it is first added to Synapse, lifting by the .created property effectively lifts every node in Synapse (technically, every node in the current View).

Lift by Extended Property

Syntax:

<form> :_ <prop>

An Extended Property is a custom property that has been added to the Synapse data model to capture specialized data for a given form. To avoid potential namespace collisions with standard properties, extended property names must begin with an underscore. In addition, we recommend using the name of the source or vendor of the property data as the first property namespace element.

Examples:

Lift all of the files (file:bytes) nodes that have a VirusTotal “reputation” extended property (:_virustotal:reputation property):

file:bytes:_virustotal:reputation

Note

The :_virustotal:reputation extended property is added to the Synapse data model by the Synapse-VirusTotal Power-Up.

Lift by Property Value - Standard Comparison Operators

A “lift by property value” operation returns the node(s) whose property matches the specified value. This type of lift requires:

• the form name or full property name (i.e., the combined form and property name) that you will use to lift the node(s);

• a Comparison Operator to specify how the property value should be evaluated; and

• the property value.

A “lift by property value” can be performed using primary, secondary, universal, or extended properties.

Tip

When lifting by a secondary property value, you can specify either a form name or an Interface name.

In Synapse, we define standard comparison operators as the following set of operators:

• equal to ( = )

• less than ( < )

• greater than ( > )

• less than or equal to ( <= )

• greater than or equal to ( >= )

The “Try” Operator ( ?= ) can optionally be used in place of the standard equal to operator ( = ). Use of the try operator is generally not required for interactive Storm queries, but may be useful for more complex Storm queries (such as automation or Storm-based ingest queries).

The most commonly used standard comparison operator is the equal to ( = ) operator. Comparison operators that expect a quantity (i.e., the inequality symbools <, >, <=, and >=) can only be used with properties whose type supports the comparison (e.g., integers, dates/times, etc.)

Tip

IPv4 addresses (inet:ipv4 nodes) are stored as their decimal integer equivalents (even though they are displayed in familiar “dotted decimal” format), and can be used with the various inequality operators:

inet:ipv4 < 192.168.0.0

IPv6 addresses (inet:ipv6 nodes) are stored as strings due to limitations with msgpack, but can be used with the various inequality operators if enclosed in single or double quotes:

inet:ipv6 >= '::0'

Lift by Primary Property Value

Syntax:

<form> <operator> <valu>

Examples:

Lift the FQDN vertex.link:

inet:fqdn = vertex.link

Lift the DNS A record showing that domain woot.com resolved to IP 1.2.3.4:

inet:dns:a = (woot.com, 1.2.3.4)

Lift the organization node whose primary property is the specified guid (globally unique identifier):

ou:org = 4b0c2c5671874922ce001d69215d032f

Lift the Autonomous System (inet:asn) nodes whose AS number is less than 50:

inet:asn < 50

Lift by Secondary Property Value

Syntax:

<form> : <prop> <operator> <pval>

Examples:

Lift the organization node with the alias “vertex”:

ou:org:alias = vertex

Lift all DNS A records for the FQDN “hugesoft.org”:

inet:dns:a:fqdn = hugesoft.org

Lift all the files with a PE compiled time of 1992-06-19 22:22:17:

file:bytes:mime:pe:compiled = '1992/06/19 22:22:17'

Lift all the files with a PE compiled time that falls within the year 2019:

file:bytes:mime:pe:compiled = 2019*

Lift files whose size is less than 1MB:

file:bytes:size < 1000000

Lift domain WHOIS records for FQDNs registered (created) after January 1, 2023:

inet:whois:rec:created > 2023/01/01

Lift PE files that were compiled during the year 2012 or earlier:

file:bytes:mime:pe:compiled <= 2012*

Lift contact data (ps:contact nodes) for individuals where the date of birth (:dob property) is January 1, 1990 or later:

ps:contact:dob >= 1990/01/01

Usage Notes:

• When lifting nodes by secondary property value where the value is a time (date / time), you do not need to use full YYYY/MM/DD hh:mm:ss.mmm syntax. Synapse allows the use of both lower resolution values (e.g., YYYY/MM/DD) and wildcard values (e.g., YYYY/MM*). In particular, wildcard syntax can be used to specify any values that match the wildcard expression. See the type-specific documentation for time types for a detailed discussion of these behaviors.

Lift by Interface Property Value

If a form inherits an Interface, you can lift all nodes of all forms with a specific interface-derived property value by using the name of the interface.

Tip

Synapse returns results in lexical order (sorted, ascending to descending) based on the way the queried property is indexed. When using an interface to lift by secondary property, Synapse performs the lifts for each form in parallel, and yields the results in order. See the “reverse” Keyword section for additional discussion of this concept.

Syntax:

<interface> : <prop> <operator> <pval>

Examples:

Lift all host activity nodes (all nodes of all forms that inherit the it:host:activity interface) associated with the host name “ron-pc”:

it:host:activity:host = { it:host:name=ron-pc }

Tip

The :host property is a guid value (the guid of an it:host node). The example above uses a Subquery to specify the host guid value using another Storm query (i.e., “The guid of the it:host node whose :name is ron-pc”) insetad of specifying the guid value directly. Subqueries are a useful way to work with guid forms by referencing nodes using more human-friendly secondary property values. See Using Subqueries to Reference Nodes for a more detailed discussion.

Lift all host activity nodes (all nodes of all forms that inherit the it:host:activity interface) that were observed on or after February 1, 2024:

it:host:activity:time >= 2024/02/01

Lift by Universal Property Value

Synapse has two built-in universal properties:

• .created (a time) which represents the date and time a node was created in Synapse; and

• .seen (an interval), a pair of date / time values that can optionally be used to represent when a node existed or was observed.

Times (date / time values) are stored as integers (epoch milliseconds) in Synapse and can be lifted using any standard comparison operator.

Because intervals are a pair of date / time values, they can only be lifted using the equal to ( = ) standard comparison operator to specify an exact match for the interval values.

The Lift by Time or Interval (@=) and Lift by Range (*range=) extended comparison operators provide additional flexibility when lifting by times and intervals.

See also the time and ival sections of the Storm Reference - Type-Specific Storm Behavior guide for additional details on working with times and intervals in Synapse.

Syntax:

[ <form> ] . <prop> <operator> <pval>

Examples:

Lift all nodes created after January 1, 2024:

.created >= 2024/01/01

Lift all DNS A records (inet:dns:a nodes) whose .seen property exactly matches the specified interval:

inet:dns:a.seen = ('2023/11/19 12:11:42.041', '2023/12/27 08:05:47.776')

Lift by Extended Property Value

When lifting by extended property value, you can use any standard comparison operator supported by the property’s type. For example, if the extended property is a string, only the equal to ( = ) standard operator is supported. If the extended property is an integer, any of the standard operators can be used.

Syntax:

<form> :_ <prop> <operator> <pval>

Example:

Lift all files with a VirusTotal “reputation” score less than -50:

file:bytes:_virustotal:reputation < -50

Lift by Property Value - Extended Comparison Operators

Storm supports a set of extended comparison operators (comparators) for specialized lift operations.

• Lift by Regular Expression (~=)

• Lift by Prefix (^=)

• Lift by Time or Interval (@=)

• Lift by Range (*range=)

• Lift by Set Membership (*in=)

• Lift by Proximity (*near=)

• Lift by (Arrays) (*[ ])

Just as with standard comparison operators, lifting by property value with extended comparison operators requires:

• the form name or full property name (i.e., the combined form and property name) that you will use to lift the node(s);

• a Comparison Operator to specify how the property value should be evaluated; and

• the property value.

Each extended comparison operator can be used with any kind of property (primary, secondary, universal, or extended) whose Type is appropriate for the comparison used.

Tip

When lifting by a secondary property value, you can specify either a form name or an Interface name.

Synapse returns results in lexical order (sorted, ascending to descending) based on the way the queried property is indexed. When using an interface to lift by secondary property, Synapse performs the lifts for each form in parallel, and yields the results in order. See the “reverse” Keyword section for additional discussion of this concept.

Lift by Regular Expression (~=)

The extended comparator ~= is used to lift nodes based on PCRE-compatible regular expressions.

Tip

Lift by Prefix (^=) can be used to match the beginning of string-based properties. It is much faster than lifting by regex for start-of-string matches and should be used instead of a regular expression lift where possible.

Syntax:

<form> [ : | . | :_ <prop> ] ~= <regex>

<interface> : <prop> ~= <regex>

Examples:

Lift all files with PDB paths containing the string “rouji”:

file:bytes:mime:pe:pdbpath ~= rouji

Lift all organizations whose name contains a string that starts with “v”, followed by 0 or more characters, followed by “x”:

ou:org:name ~= '^v.*x'

Lift by Prefix (^=)

Synapse performs prefix indexing on string and string-derived types, which optimizes lifting nodes whose <valu> or <pval> starts with a given prefix (substring). The extended comparator ^= is used to lift nodes by prefix.

Note

Extended string types that support dotted notation (such as the loc or syn:tag types) have custom behaviors with respect to lifting and filtering by prefix.

inet:fqdn nodes are indexed in reverse string order so cannot be lifted using the prefix extended operator. However, reverse indexing allows wildcard ( * ) matching of the beginning of any FQDN string.

See the relevant sections in the Storm Reference - Type-Specific Storm Behavior guide for details.

Syntax:

<form> [ : | . | :_ <prop> ] ^= <prefix>

<interface> : <prop> ^= <prefix>

Examples:

Lift all email addresses that start with “abuse”:

inet:email ^= abuse

Lift all organizations whose name starts with “ministry”:

ou:org:name ^= ministry

Lift all Microsoft Office metadata nodes (all nodes of all forms that inherit the file:mime:msoffice interface) whose :author name starts with “DESKTOP”:

file:mime:msoffice:author ^= DESKTOP

Lift all tags (syn:tag) nodes in the “rep.alienvault” tree where the third tag element starts with the numeral “0”:

syn:tag ^= rep.alienvault.0

Tip

Even though tag elements are dot-separated, when lifting syn:tag nodes by prefix the prefix string is not constrained to dot boundaries. In other words, a prefix lift used with tags can match a partial tag element. The query above will match all of the following tags:

• syn:tag=rep.alienvault.0_day

• syn:tag=rep.alienvault.0_days

• syn:tag=rep.alienvault.0day

• syn:tag=rep.alienvault.0days

• syn:tag=rep.alienvault.0ktapus

Lift by Time or Interval (@=)

Many forms include properties that are date / time values (<ptype> = <time>) or time windows / intervals (<ptype> = <ival>). The time/interval extended comparator @= is used to lift nodes based on comparisons among various combinations of times and intervals.

Tip

See Storm Reference - Type-Specific Storm Behavior for additional detail on the use and behavior of time and ival data types.

Syntax:

<form> : | . | :_ <prop> @=( <ival_min> , <ival_max> )

<form> : | . | :_ <prop> @= <time>

<interface> : <prop> @=( <ival_min> , <ival_max> )

<interface> : <prop> @= <time>

Examples:

Lift all DNS A records whose .seen values fall between July 1, 2022 and August 1, 2022:

inet:dns:a.seen @= ( 2022/07/01, 2022/08/01 )

Lift all DNS requests that occurred on May 3, 2023 (between 05/03/2023 00:00:00 and 05/03/2023 23:59:59):

inet:dns:request:time @= ( '2023/05/03 00:00:00', '2023/05/04 00:00:00' )

Lift the WHOIS email nodes that were observed between July 1, 2023 and the present:

inet:whois:email.seen @= ( 2023/07/01, now )

Lift the network flows that occurred within the past day:

inet:flow:time @= ( now, '-1 day' )

Lift all host activity nodes (all nodes of all forms that inherit the it:host:activity interface) that occurred within the past three hours:

it:host:activity:time @= (now, '-3 hours')

Usage Notes:

• When specifying an interval with the @= operator, the minimum value is included in the interval for comparison purposes but the maximum value is not. This is equivalent to “greater than or equal to <min> and less than <max>”. This behavior differs from that of the *range= operator, which includes both the minimum and maximum.

• Comparing intervals to intervals: when using an interval with the @= operator to lift nodes based on an interval property, Synapse returns nodes whose interval value has any overlap with the specified interval. For example:

  • A lift interval of September 1, 2018 to October 1, 2018 ( 2018/09/01, 2018/10/01 ) will match nodes with any of the following intervals:

    • August 12, 2018 to September 6, 2018 ( 2018/08/12, 2018/09/06 ).

    • September 13, 2018 to September 17, 2018 ( 2018/09/13, 2018/09/17 ).

    • September 30, 2018 to November 5, 2018 ( 2018/09/30, 2018/11/05 ).

• Comparing intervals to times: When using an interval with the @= operator to lift nodes based on a time property, Synapse returns nodes whose time value falls within the specified interval.

• Comparing times to times: When using a time with the @= operator to lift nodes based on a time property, Synapse returns nodes whose timestamp is an exact match of the specified time. In other words, in this case the interval comparator ( @= ) behaves like the equal to comparator ( = ).

• When specifying date / time and interval values, Synapse allows the use of both lower resolution values (e.g., YYYY/MM/DD), and wildcard values (e.g., YYYY/MM*). Wildcard time syntax may provide a simpler and more intuitive means to specify some intervals. For example inet:whois:rec:asof=2018* is equivalent to inet:whois:rec:asof@=('2018/01/01', '2019/01/01').

• Time-based keywords (such as now) and relative time syntax (expressions such as +-1 hour or -7 days) can be used for interval values.

  See the type-specific documentation for time and ival types for a detailed discussion of these behaviors.

Lift by Range (*range=)

The range extended comparator (*range=) supports lifting nodes whose <form> = <valu> or <prop> = <pval> fall within a specified range of values. The comparator can be used with types such as integers and times.

Note

The *range= operator can be used to lift both inet:ipv4 and inet:ipv6 values (which are stored as decimal integers and strings, respectively). However, ranges of inet:ipv4 and inet:ipv6 nodes can also be lifted directly by specifying the lower and upper addresses in the range using <min>-<max> format. For example:

inet:ipv4 = 192.168.0.0-192.168.0.10

Because IPv6 nodes are stored as strings, the range must be enclosed in quotes:

inet:ipv6 = "::0-ff::ff"

The *range= operator cannot be used to compare a time range with a property value that is an interval (ival type). The interval ( @= ) operator should be used instead.

Syntax:

<form> [ : | . | :_ <prop> ] *range = ( <range_min> , <range_max> )

<interface> : <prop> *range = ( <range_min> , <range_max> )

Examples:

Lift all files whose size is between 1000 and 100000 bytes:

file:bytes:size *range= ( 1000, 100000 )

Lift all files whose VirusTotal “reputation” score is between -20 and 20:

file:bytes:_virustotal:reputation *range= ( -20, 20 )

Lift all domain WHOIS records that were captured / retrieved between November 29, 2013 and June 14, 2016:

inet:whois:rec:asof *range= ( 2013/11/29, 2016/06/14 )

Lift all DNS requests made within one day of December 1, 2021:

inet:dns:request:time *range= ( 2021/12/01, '+-1 day' )

Usage Notes:

• When specifying a range, both the minimum and maximum values are included in the range. This is the equivalent of “greater than or equal to <min> and less than or equal to <max>”).

• When specifying a range of time values, Synapse allows the use of both lower resolution values (e.g., YYYY/MM/DD) and wildcard values (e.g., YYYY/MM*) for the minimum and/or maximum range values. In some cases, wildcard time syntax may provide a simpler and more intuitive means to specify some time ranges. For example inet:whois:rec:asof=2018* is equivalent to inet:whois:rec:asof*range=('2018/01/01', '2018/12/31 23:59:59.999'). See the type-specific documentation for time types for a detailed discussion of these behaviors.

• When using keywords (such as now) or relative values (such as -1 hour) to specify a range of times, the first value in the range is calculated relative to the current time and the second value is calculated relative to the first value.

• If you specify a range value that is nonsencical or exclusionary (such as ( 47, 16 )), Synapse will not generate an error and will simply fail to return results. (The expression is syntacticaly correct, but no value is both greater than 47 and less than 16).

Lift by Set Membership (*in=)

The set membership extended comparator (*in=) supports lifting nodes whose <form> = <valu> or <prop> = <pval> matches any of a set of specified values. The comparator can be used with any type.

Syntax:

<form> [ : | . | :_ <prop> ] *in = ( <set_1> , <set_2> , … )

<interface> : <prop> *in = ( <set_1> , <set_2> , … )

Examples:

Lift organization names matching any of the specified values:

ou:name *in= ( fsb, gru, svr )

Lift all IPv4 addresses associated with any of the specified Autonomous System (AS) numbers:

inet:ipv4:asn *in= ( 9009, 20473, 44477 )

Lift all tags (syn:tag nodes) whose final tag element matches any of the specified string values:

syn:tag:base *in= ( plugx, korplug, sogu, kaba )

Lift by Proximity (*near=)

The proximity extended comparator (*near=) supports lifting nodes by “nearness” to another node. Currently, *near= supports proximity based on geospatial location (i.e., nodes within a given radius of a specified latitude / longitude).

Syntax:

<form> : | . | :_ <prop> *near = (( <lat> , <long> ), <radius> )

Examples:

Lift all locations (geo:place nodes) within 500 meters of the Russian Cryptographic Museum (where the coordinates 55.83069,37.59781 represent the Museum’s location):

geo:place:latlong *near= ( (55.83069, 37.59781), 500m )

Usage Notes:

• Radius can be specified in the following units. The terms recognized by Storm are listed in parentheses. For example, radius can be specified as 2km or ‘2 km’ or ‘0.5 meters’ or ‘6 feet’ (distance expressions that contain spaces need to be enclosed in single or double quotes):

  • Kilometers (km / kilometer / kilometers)

  • Meters (m / meter / meters)

  • Centimeters (cm / centimeter / centimeters)

  • Millimeters (mm / millimeter / millimeters)

  • Miles (mile / miles)

  • Yards (yard / yards)

  • Feet (foot / feet)

• Radius values of less than 1 must be specified with a leading zero (e.g., 0.5km).

• The *near= comparator works for geospatial data by lifting nodes within a square bounding box centered at <lat>,<long>, then filters the nodes returned by ensuring that they are within the great-circle distance given by the <radius> argument.

Lift by (Arrays) (*[ ])

Storm uses a special syntax to lift (or filter) by comparison with one or more elements of an array type. The syntax consists of an asterisk ( * ) preceding a set of square brackets ( [ ] ), where the square brackets contain a comparison operator and a value that can match one or more elements in the array. This allows users to match values in the array list without needing to know the exact order or values of the array itself.

Syntax:

<form> : | . | :_ <prop> [ <operator> <pval> ]

Examples:

Lift the x509 certificates (crypto:x509:cert nodes) that reference FQDNs ending with “.xyz”:

crypto:x509:cert:identities:fqdns *[ = '*.xyz' ]

Lift the MITRE ATT&CK groups whose names include the string ‘bear’:

it:mitre:attack:group:names *[ ~= bear ]

Usage Notes:

• The comparison operator used must be valid for lift operations for the type used in the array. For example, inet:fqdn suffix matching (i.e., crypto:x509:cert:identities:fqdns *[ = '*.com' ]), can be used when lifting arrays consisting of domains, but the prefix operator ( ^= ), which is only valid when filtering inet:fqdns, cannot.

• The standard equals ( = ) operator can be used to filter nodes based on array properties, but the value specified must exactly match the full property value in question. For example:

  ou:org:names=( "the vertex project", "the vertex project llc", vertex )

• See the array section of the Storm Reference - Type-Specific Storm Behavior document for additional details.

Tag Lifts

Tags in Synapse can represent observations or assessments. They are used to provide context to nodes (in the form of “labels” applied to nodes) and to group related nodes.

Storm supports lifting nodes based on the tag(s) applied to the node, as well lifting based on tag timestamps, tag properties, or tag property values.

The “hashtag” symbol ( # ) is used to specify a tag name when lifting by tag.

Lift by Tag

A “lift by tag” operation lifts all nodes that have the specified tag.

Syntax:

# <tag>

Examples:

Lift all nodes that ESET associates with Sednit:

#rep.eset.sednit

Lift all nodes associated with anonymized infrastructure:

#cno.infra.anon

Tip

Tags are hierarchical, and each tag element is its own tag; the tag #cno.infra.anon consists of the tags #cno, #cno.infra, and #cno.infra.anon. Lifting nodes using a tag “higher up” in the tag hierarchy will lift all nodes with specified tag or any tag “lower down” in the hierarchy. In other words, lifting by #cno.infra.anon will lift “anonymized” infrastructure, whether the infrastructure is a VPN (#cno.infra.anon.vpn), a TOR node (#cno.infra.anon.tor), or an anonymous proxy (#cno.infra.anon.proxy).

Lift Form by Tag

Lift form by tag lifts only those nodes of the specified form that have a particular tag.

Syntax:

<form> # <tag>

Examples:

Lift the FQDN nodes that ESET associates with Sednit:

inet:fqdn#rep.eset.sednit

Lift the inet:ipv4 nodes associated with DNS sinkhole infrastructure:

inet:ipv4#cno.infra.dns.sink.hole

Tip

A “lift form by tag” operation is equivalent to a “lift and filter” operation that first lifts all nodes of the specified form, and then filters the results to only those nodes with the specified tag. This set of operations is potentially resource-intensive and inefficient (why lift all nodes only to discard most of them?). Instead, Synapse specifically optimizes “lift form by tag” operations to only lift nodes that have the tag.

In fact, if you specify a Storm query such as inet:fqdn +#rep.mandiant.apt1, Synapse will execute the query as if you had specified the “lift form by tag” query inet:fqdn#rep.mandiant.apt1. In other words, in some cases Synapse knows to “do what you mean” in order to process your queries more efficiently.

Lift Using Tag Timestamps

A tag timestamp can be thought of as a specialized “property” of a tag that happens to be a date / time range (interval). You can lift nodes based on tag timestamp values using any comparison operator supported by interval (ival types). The time / interval extended operator ( @= ) is used most often, but equal to ( = ) can also be used to exact match the values in the interval.

See Lift by Time or Interval (@=) for additional detail on the use of the @= operator.

Syntax:

[ <form> ] # <tag> @= <time> | ( <min_time> , <max_time> )

Lift any nodes that were associated with anonymous VPN infrastructure between December 1, 2023 and January 1, 2024:

#cno.infra.anon.vpn @= ( 2023/12/01, 2024/01/01 )

Lift the FQDNs that were owned / controlled by Threat Cluster 15 as of October 30, 2021:

inet:fqdn#cno.threat.t15.own @= 2021/10/30

Lift the IP addresses that were TOR exit nodes between April 1, 2023 and July 1, 2023:

inet:ipv4#cno.infra.anon.tor.exit @= ( 2023/04/01, 2023/07/01 )

Lift Using Tag Properties

Tag Properties can be used to provide additional context to tags. Storm supports lifting nodes whose tags have a specific tag property (regardless of the value of the property).

Note

In many cases, information previously recorded using a tag property is better suited to the use of an Extended Property.

Syntax:

[ <form> ] # <tag> : <tagprop>

Lift any nodes with a “:risk” property reported by Symantec:

#rep.symantec:risk

Lift all FQDNs with a “:risk” property reported by Symantec:

inet:fqdn#rep.symantec:risk

Note

You must specify a tag associated with the tag property. It is not possible to lift nodes based on a particular tag property being present on any tag (e.g., Storm queries such as #:risk or inet:fqdn#:risk will generate BadSyntax errors).

Lift Using Tag Property Values

Storm supports lifting nodes based on the value of a tag property (similar to lifting by the value of a node property).

You can lift nodes based on tag property values using any comparison operator supported by the property’s Type. For example, if the tag property is defined as an integer (int) type, you can use any comparison operator supported by integers.

The “Try” Operator ( ?= ) can optionally be used in place of the standard equal to operator ( = ) for tag property values. Use of the try operator is generally not required for interactive Storm queries, but may be useful for more complex Storm queries (such as automation or Storm-based ingest queries).

Syntax:

[ <form> ] # <tag> : <tagprop> <operator> <pval>

Lift any nodes with a “:risk” property value of 100 as reported by ESET:

#rep.eset:risk = 100

Lift all FQDNs with a “:risk” property value greater than 90 as reported by domaintools:

inet:fqdn#rep.domaintools:risk > 90

Lift all FQDNs with a “:risk” property with a value between 45 and 70 as reported by Symantec:

inet:fqdn#rep.symantec:risk *range= ( 45, 70 )

Recursive Tag Lift (##)

Tags can be applied to syn:tag nodes to record additional information about the the observation represented by the syn:tag node itself. In other words, tags (labels) can be used to provide additional context about tags (syn:tag nodes).

The ability to “tag the tags” can be used to represent certain types of analytical relationships. For example:

• syn:tag nodes representing threat groups can be tagged to indicate their assessed country of origin.

• syn:tag nodes representing malware or tools can be tagged with their assessed availability (e.g., public, private, private but shared, etc.)

A recursive tag lift retrieves all nodes with the specified tag. If the results include any syn:tag nodes, the recursive lift will also lift any nodes with those tags. The process continues until no more syn:tag nodes are returned.

The final result set returned by a recursive tag lift includes all of the nodes that were lifted recursively, but will not include any lifted syn:tag nodes themselves.

The “double hashtag” symbol ( ## ) is used to specify a recursive tag lift.

Syntax:

## <tag>

Example:

You are using “availability” tags to show the general availability of malware or tools reported by Mandiant. You add the appropriate “availability” tag to the syn:tag node that represents the associated malware. For example, you apply the tag #rep.mandiant.avail.public to the node syn:tag=rep.mandiant.gh0st because Mandiant reported that the source code for the Gh0st backdoor is publicly available.

Lift all the nodes (e.g., indicators of compromise) associated with any malware family or tool that Mandiant reports is publicly available:

##rep.mandiant.avail.public

The query above will:

• Lift all nodes tagged #rep.mandiant.avail.public, such as syn:tag nodes for tools or malware families that Mandiant assesses are publicly available (e.g., syn:tag=rep.mandiant.gh0st or syn:tag=rep.mandiant.beacon).

• Lift any nodes tagged with those tags (e.g., #rep.mandiant.gh0st or #rep.mandiant.beacon). This would typically include IOCs such as hashes, FQDNs, IPv4s, URLs, etc.

• If any nodes tagged with the additional tags (#rep.mandiant.gh0st, etc.) are syn:tag nodes, repeat the process, continuing until no more syn:tag nodes are lifted.

• Return the recursively lifted set of nodes (excluding any syn:tag nodes).

Note

“Tag the tags” is one approach to provide context to things that tags represent and may be suitable for some use cases. However, the Synapse data model now includes forms to represent objects or concepts that are commonly associated with tags, and that can be linked to their associated tag via a :tag secondary property. For example, a risk:tool:software node can represent Mandiant’s reporting on Gh0st malware, with a :tag property that could be set to rep.mandiant.gh0st. The node can be used to record additional context about Mandiant’s Gh0st, including things like availability, alternate names used in reporting, and so on. In short, using a form with secondary properties to provide “context” (that is still linked to an associated tag) gives you greater flexibility to record that context and simplifies lifting, filtering, and pivoting across similar nodes.

See Tags Associated with Nodes for a brief discussion of this concept, or the User Guide for the Vertex-Threat-Intel Power-Up (in particular, the Threat Intel Model section) for additional examples.

“reverse” Keyword

Synapse indexes property values so that data (nodes) can be lifted (retrieved) and returned quickly. By default, lift results are returned in lexical order (i.e., sorted in ascending order), based on the property specified in the lift (primary, secondary, universal, or extended) and the way the property is indexed.

The reverse keyword can be used to return the specified nodes in reverse lexical order (i.e., sorted in descending order). To perform a “reverse” lift, specify the reverse keyword and enclose the lift operation in parentheses.

A “reverse” lift can be followed by additional Storm operations (pivots, filters, commands) just like a “normal” lift.

Tip

When using the reverse keyword to lift by secondary property value using an Interface name, Synapse performs the lifts for each form in parallel, and yields the results in descending order. For example, the following query will return all nodes of all forms that inherit the it:host:activity interface that have a :time value greater than or equal to 2024/02/01, sorted in descending order (most recent first):

reverse (it:host:activity:time >= 2024/02/01)

Syntax:

reverse ( <lift> )

Examples:

Lift inet:ipv4 nodes with a :loc property (sorted descending based on the :loc property value):

storm> reverse ( inet:ipv4:loc )
inet:ipv4=197.155.229.194
        :loc = zw.ha.harare
        :type = unicast
        .created = 2024/04/18 23:53:15.254
inet:ipv4=41.221.147.14
        :loc = zw
        :type = unicast
        .created = 2024/04/18 23:53:15.260
inet:ipv4=41.164.23.42
        :loc = za.wc.worcester
        :type = unicast
        .created = 2024/04/18 23:53:12.697
inet:ipv4=155.254.9.3
        :loc = us.mt.three forks
        :type = unicast
        .created = 2024/04/18 23:53:12.703
inet:ipv4=102.64.66.222
        :loc = tz.02.dar es salaam
        :type = unicast
        .created = 2024/04/18 23:53:12.708

Lift five inet:ipv4 nodes (sorted descending based on the integer value of the inet:ipv4 primary property):

storm> reverse ( inet:ipv4 ) | limit 5
inet:ipv4=255.255.255.255
        :type = private
        .created = 2024/04/18 23:53:15.322
inet:ipv4=223.159.33.195
        :type = unicast
        .created = 2024/04/18 23:53:15.327
inet:ipv4=198.42.76.23
        :type = unicast
        .created = 2024/04/18 23:53:15.333
inet:ipv4=197.155.229.194
        :loc = zw.ha.harare
        :type = unicast
        .created = 2024/04/18 23:53:15.254
inet:ipv4=185.29.8.215
        :type = unicast
        .created = 2024/04/18 23:53:14.904
        #cno.infra.anon.tor.exit = (2023/05/08 14:30:51.000, 2024/01/04 22:05:03.000)

Lift the five most recently-created inet:email nodes (sorted descending by the .created property value):

storm> reverse ( inet:email.created ) | limit 5
inet:[email protected]
        :fqdn = gmail.com
        :user = illia.volochii
        .created = 2024/04/18 23:53:15.452
inet:[email protected]
        :fqdn = fastmail.fm
        :user = dholth
        .created = 2024/04/18 23:53:15.445
inet:[email protected]
        :fqdn = nextday.fi
        :user = alex.gronholm
        .created = 2024/04/18 23:53:15.438
inet:[email protected]
        :fqdn = hammer-software.com
        :user = support
        .created = 2024/04/18 23:53:15.432
inet:[email protected]
        :fqdn = adnoc.ae
        :user = 20231128124623.11d85d83ed11a341
        .created = 2024/04/18 23:53:15.426

Note

In some cases, Synapse uses specialized indexing to optimize specific Storm operations (such as the ability to lift forms by tag) or to make it easier to work with certain types of data (type-specific behavior). For example, FQDN strings (inet:fqdn types) are reversed before being indexed.

Where specialized indexing is used, both “normal” and “reverse” lifts still return nodes in lexical or reverse lexical order, respectively. However, the “sort order” of the results may not be apparent, based on the custom criteria used to index the nodes.

See the Storm Reference - Type-Specific Storm Behavior section for details on some type-specific behaviors, including any custom indexing for the listed types.

“Try” Operator

The Storm “try” operator ( ?= ) can be used in lift operations as an alternative to the equal to ( = ) comparison operator.

Properties in Synapse are subject to Type Enforcement. Type enforcement makes a reasonable attempt to ensure that a value “makes sense” for the property in question - that the value you specify for an inet:ipv4 node looks reasonably like an IPv4 address (and not an FQDN or URL). If you try to lift a set of nodes using a property value that does not pass Synapse’s type enforcement validation, Synapse will generate an error. The error will cause the currently executing Storm query to halt and stop processing. For example, the following query halts based on the bad value (evil.com) provided for an inet:ipv4 node:

storm> inet:ipv4 = evil.com inet:ipv4 = 8.8.8.8
ERROR: illegal IP address string passed to inet_aton

When using the try operator ( ?= ), Synapse will to attempt (try) to lift the node(s) using the specified property value. However, instead of halting in the event of an error, Synapse will ignore the error (silently fail on that specific lift operation) but continue processing the rest of the Storm query. Using the try operator below, Synapse ignores the bad value for the first IPv4 address but returns the second one:

storm> inet:ipv4 ?= evil.com inet:ipv4 ?= 8.8.8.8
inet:ipv4=8.8.8.8
        :type = unicast
        .created = 2024/04/18 23:53:14.956
        #rep.domaintools:risk = 42

The try operator is generally not necessary for interactive Storm queries. However, it can be very useful for more complex Storm queries or Storm-based automation (see Storm Reference - Automation), where a single badly-formatted lift operation (potentially relying on input or data from a third-party data source) could cause the query to fail during execution.

Tip

The try operator can also be used when lifting using an Interface.

Syntax:

<form> ?= <valu>

<form> : <prop> ?= <pval>

<interface> : <prop> ?= <pval>

Tip

See the array section of the Storm Reference - Type-Specific Storm Behavior for specialized “try” syntax when working with array properties.

Examples:

Try to lift the MD5 node 174cc541c8d9e1accef73025293923a6:

storm> hash:md5 ?= 174cc541c8d9e1accef73025293923a6
hash:md5=174cc541c8d9e1accef73025293923a6
        .created = 2024/04/18 23:53:15.562

Try to lift the DNS A nodes whose :ipv4 property is 192.168.0.100:

storm> inet:dns:a:ipv4 ?= 192.168.0.100
inet:dns:a=('woot.com', '192.168.0.100')
        :fqdn = woot.com
        :ipv4 = 192.168.0.100
        .created = 2024/04/18 23:53:15.605

Try to lift the email address nodes for ron@vertex.link and ozzie@vertex.link:

Notice that despite the first email address being entered incorrectly, the error message is suppressed, and the query executes to completion.

storm> inet:email ?= 'ron[at]vertex.link' inet:email ?= '[email protected]'
inet:[email protected]
        :fqdn = vertex.link
        :user = ozzie
        .created = 2024/04/18 23:53:15.653

Try to lift any Microsoft Office document metadata nodes (all nodes of all forms that inherit the file:mime:msoffice interface) whose author (:author property) is ‘Rafael Moon’:

file:mime:msoffice:author ?= 'Rafael Moon'