Analytical Model - Tag Concepts

Recall from Data Model - Terminology that two of the key components within Synapse are nodes and tags. Broadly speaking:

  • Nodes represent “things”: observables that can be verified and are unlikely to change over time.

  • Tags represent assessments: conclusions that may change in light of new data.

The types, forms, and properties that define nodes make up the Synapse data model. The tags applied to nodes can be thought of as the analytical model used to record assessments about Synapse data. This section provides additional background on tags before a more in-depth discussion on their use:

Tags as Nodes

Tags are used to record analytical observations, but tags are also nodes within Synapse. Every tag is a syn:tag node.

A tag node’s primary property (<form> = <valu>) is the name of the tag; so the tag foo.bar has the primary property syn:tag = foo.bar. The dotted notation can be used to construct tag hierarchies / tag trees to organize tags and represent varying levels of specificity (see below).

This example shows the node for the tag syn:tag = rep.feye.apt1:

storm> syn:tag=rep.feye.apt1
syn:tag=rep.feye.apt1
        :base = apt1
        :depth = 2
        :doc = Indicator or activity FireEye calls (or associates with) APT1.
        :title = APT1 (FireEye)
        :up = rep.feye
        .created = 2022/12/05 20:25:18.269

The syn:tag node has the following properties:

  • .created, which is a universal property showing when the node was added to a Cortex.

  • :title and :doc, which store concise and more detailed definitions for the tag. Having definitions on tag nodes helps ensure tags are applied (and interpreted) correctly by Synapse analysts and other users.

The :depth, :up, and :base secondary properties help to lift and pivot across tag nodes:

  • :depth is the “location” of the tag in a given tag tree, with the count starting from zero. A single-element tag (syn:tag = rep) has :depth = 0, while a three-element tag (syn:tag = rep.feye.apt1) has :depth = 2.

  • :base is the final (rightmost) element in the tag tree.

  • :up is the tag one “level” up in the tag tree.

Additional information on viewing and pivoting across tags can be found in Storm Reference - Model Introspection. For details on the Storm query language, see Storm Reference - Introduction.

Tags (syn:tag forms) have a number of type-specific behaviors within Synapse with respect to how they are indexed, created, and manipulated via Storm. Most important for practical purposes is that syn:tag nodes are created “on the fly” when a tag is applied to another node. You do not need to create the syn:tag node before the tag can be used; applying the tag will cause the creation of the appropriate syn:tag node (or nodes).

See the syn:tag section within Storm Reference - Type-Specific Storm Behavior for additional detail on tags and tag behavior in Synapse and Storm.

Tags as Labels

Synapse does not include any pre-populated tags. A good set of tags (that is, a good analytical model) should be structured to best answer relevant questions for the analysis being performed. Organizations using Synapse have the flexibility to create a tag structure that is most useful to them.

A tag’s value (syn:tag = <valu>) is simply a string and can be set to any user-defined alphanumeric value. The strings are designed to use a dotted naming convention, with the period ( . ) used as a separator to delimit individual elements of a tag if necessary. This dotted notation means it is possible to create tag hierarchies or tag trees. These trees can be used to “categorize” different types of tags (with each top-level or root tag representing a particular category). The structure can also support increasingly detailed or specific observations. For example, the top level tag foo can represent a broad set of observations, while foo.bar and foo.baz could represent subsets of foo or more specific observations related to foo.

Within a tag tree, specific terms are used for the tags and their components:

  • Leaf tag: The full tag.

  • Root tag: The top / leftmost element in a given tag.

  • Base tag: The bottom / rightmost element in a given tag.

For the tag foo.bar.baz:

  • foo.bar.baz is the leaf tag (leaf).

  • foo is the root tag (root).

  • baz is the base tag (base).

When you apply a tag to a node, all of the tags above that tag in the tag tree are automatically applied as well (and the appropriate syn:tag nodes are created if they do not exist). That is, when you apply the tag foo.bar.baz to a node, Synapse automatically applies the tags foo.bar and foo as well.

When you delete (remove) a tag from a node, the tag and all tags below it in the tag tree are deleted. If you delete the tag foo.bar.baz from a node, the tags foo.bar and foo will remain. However, if you delete the tag foo from a node with the tag foo.bar.baz, then all three tags (foo, foo.bar, and foo.bar.baz) are deleted.

Deleting a tag from a node does not delete the syn:tag node for the tag itself.

See the syn:tag section within Storm Reference - Type-Specific Storm Behavior for additional detail on tags and tag behavior within Synapse and Storm.

See Analytical Model - Tags as Analysis and Design Concepts - Analytical Model for additional considerations for tag use and creating tag trees.

Tag Timestamps

Applying a tag to a node has a particular meaning; it is an assessment about that node with respect to the current data in Synapse. Many assessments are binary in the sense that they are either always true or always false; in these cases, the presence or absence of a tag can accurately reflect the current assessment, based on available data.

There are other cases where an assessment may be true only for a period of time or within a specified time frame. Internet infrastructure is one example; you can annotate whether an IP address is part of an anonymization service such as TOR using tags such as cno.infra.anon.tor. But this information can change over time if the TOR service is removed or the IP address is reallocated to a different customer. The relevant tag can be applied while the IP is a TOR node and removed when that is no longer true; but completely removing the tag causes us to lose the historical knowledge that the IP was a TOR node at one time.

Synapse supports the optional use of timestamps (technically, time intervals) with any tag applied to a node. The timestamps can represent “when” (first known / last known times) the assessment represented by the tag was relevant for the node to which the tag is applied. (These timestamps are analogous to the .seen universal property used to represent the first and last known times the data represented by a node was true / real / in existence.)

Applying a timestamp to a tag affects that specific tag only. The timestamps are not automatically propagated to tags higher up (or lower down) in the tag tree. This is because the specific tag to which the timestamps are applied is the most relevant with respect to those timestamps; tags elsewhere in the tree may have different shades of meaning and the timestamps may not apply to those tags in the same way (or at all).

Like .seen properties, tag timestamps represent a time range and not necessarily specific instances (other than the “first known” and “last known” observations). This means that the assessment represented by the tag is not guaranteed to have been true throughout the entire date range (though depending on the meaning of the tag, that may in fact be the case). That said, the use of timestamps allows much greater granularity in recording observations in cases where the timing of an assessment (“when” something was true or applicable) is relevant.

Example - Tor Exit Nodes

Many web sites provide lists of TOR nodes or allow users to query IP addresses to determine whether they are TOR nodes. These sites may provide “first seen” and “last seen” dates for when the IP was identified as part of the TOR network. These dates can be used as timestamps for “when” the tag #cno.infra.anon.tor was applicable to that IP address.

If we have a data source that verifies that IP address 197.231.221.211 was a TOR node between December 19, 2017 and February 15, 2019, we can apply the tag #cno.infra.anon.tor with the appropriate time range as follows:

storm> inet:ipv4 = 197.231.221.211 [ +#cno.infra.anon.tor = (2017/12/19, 2019/02/15) ]
inet:ipv4=197.231.221.211
        :asn = 37560
        :dns:rev = exit1.ipredator.se
        :latlong = 8.4219,-9.7478
        :loc = lr.lo.voinjama
        :type = unicast
        .created = 2022/12/05 20:25:18.345
        #cno.infra.anon.tor = (2017/12/19 00:00:00.000, 2019/02/15 00:00:00.000)

Tag Display

When a tag is used as a label applied to a node, the data is displayed differently than it is for a syn:tag node. This example shows a node with multiple tags applied:

storm> inet:fqdn = aunewsonline.com
inet:fqdn=aunewsonline.com
        :domain = com
        :host = aunewsonline
        :issuffix = False
        :iszone = True
        :zone = aunewsonline.com
        .created = 2022/12/05 20:25:18.367
        #cno.threat.t15.own = (2009/09/08 00:00:00.000, 2013/09/08 00:00:00.000)
        #rep.feye.apt1
        #rep.symantec.commentcrew

Tags on a node are listed alphabetically following the node’s properties. Tags are prefixed with the pound / hashtag ( # ) symbol to indicate they are tags.

By default, Storm displays only the leaf tags applied to a node (e.g., #rep.feye.apt1 but not #rep.feye or #rep) and any tags with Tag Timestamps or Tag Properties (even if they are not leaf tags).

Any timestamp values are displayed following an equals sign after the tag. In the example above, the tag #cno.threat.t15.own indicates the domain is associated with (“owned” by) internally-tracked Threat Cluster 15 (T15). The dates reflect our assessment that T15 “owned” / controlled the FQDN between September 8, 2009 and September 8, 2013.

Tag Properties

Synapse supports the creation and use of custom tag properties that can provide additional context to a given tag or set of tags. Tag properties must be created programmatically before they can be used. Once a tag property is created, it can be applied (appended) to any tag.

Note

Synapse still supports the use of tag properties, but their use is now discouraged in most cases in favor of extended model properties. A discussion of extended model elements (forms, properties, etc.) is beyond the scope of this document. Storm libraries for working with extended model elements can be found here: $lib.model.ext.