Synapse Glossary¶
This Glossary provides a quick reference for common terms related to Synapse technical and analytical concepts.
A¶
Analytical Model¶
See Model, Analytical.
Autoadd¶
Short for “automatic addition”. Within Synapse, a feature of node creation where any secondary properties that are derived from a node’s primary property are automatically set when the node is created. Because these secondary properties are based on the node’s primary property (which cannot be changed once set), the secondary properties are read-only.
For example, creating the node inet:email=alice@mail.somecompany.org will result in the autoadd of the secondary
properties inet:email:user=alice and inet:email:domain=mail.somecompany.org.
See also the related concept Depadd.
B¶
C¶
Cell¶
The Cell is a basic building block of Synapse services, including the Cortex. See Synapse Architecture for more information about what a Cell provides.
Comparator¶
Short for Comparison Operator.
Comparison Operator¶
A symbol or set of symbols used in the Storm language to evaluate Node property values against one or more specified values. Comparison operators can be grouped into standard and extended operators.
Comparison Operator, Standard¶
The set of common operator symbols used to evaluate (compare) values in Storm. Standard comparison operators include
equal to (=), greater than (>), less than (<), greater than or equal to (>=), and less than or equal
to (<=).
Comparison Operator, Extended¶
The set of Storm-specific operator symbols or expressions used to evaluate (compare) values in Storm based on custom or
Storm-specific criteria. Extended comparison operators include regular expression (~=), time/interval (@=), set
membership (*in=), tag (#), and so on.
Composite Form¶
See Form, Composite.
Constant¶
In Storm, a constant is a value that cannot be altered during normal execution, i.e., the value is constant.
Contrast with Variable. See also Runtsafe and Non-Runtsafe.
Constructor¶
Within Synapse, a constructor is code that defines how a Property value of a given Type can be constructed to ensure that the value is well-formed for its type. Also known as a Ctor for short. Constructors support Type Normalization and Type Enforcement.
Cortex¶
A Cortex is Synapse’s implementation of an individual Hypergraph. Cortex features include scalability, key/value-based node properties, and a Data Model which facilitates normalization.
Cron¶
Within Synapse cron jobs are used to create scheduled tasks, similar to the Linux/Unix “cron” utility. The task to be executed by the cron job is specified using the Storm query language.
See the Storm command reference for the cron command and the Storm Reference - Automation document for additional detail.
Ctor¶
Pronounced “see-tore”. Short for Constructor.
D¶
Daemon¶
Similar to a traditional Linux or Unix daemon, a Synapse daemon (“dmon”) is a long-running or recurring query or process that runs continuously in the background. A dmon is typically implemented by a Storm Service and may be used for tasks such as processing elements from a Queue. A dmon allows for non-blocking background processing of non-critical tasks. Dmons are persistent and will restart if they exit.
Data Model¶
See Model, Data.
Deconflictable¶
Within Synapse, a term typically used with respect to Node creation. A node is deconflictable if, upon node
creation, Synapse can determine whether the node already exists within a Cortex (i.e., the node creation attempt is
deconflicted against existing nodes). For example, on attempting to create the node inet:fqdn=woot.com Synapse can
deconflict the node by checking whether a node of the same form with the same primary property already exists.
Whether a node is deconflictable is often an issue with GUID forms. A GUID Form whose primary property is an arbitrary GUID is not deconflictable. A GUID form whose primary property is generated from a defined or predictable set of strings (such as a subset of the form’s secondary property values) may be deconflictable. See the guid section of the Storm Reference - Type-Specific Storm Behavior document for additional detail.
Depadd¶
Short for “dependent addition”. Within Synapse, when a node’s secondary property is set, if that secondary property is of a type that is also a form, Synapse will automatically create the node with the corresponding primary property value if it does not already exist. (You can look at this as the secondary property value being “dependent on” the existence of the node with the corresponding primary property value.)
For example, creating the node inet:email=alice@mail.somecompany.org will set (via Autoadd) the
secondary property inet:email:domain=mail.somecompany.org. Synapse will automatically create the node
inet:fqdn=mail.somecompany.org as a dependent addition if it does not exist.
(Note that limited recursion will occur between dependent additions (depadds) and automatic additions (autoadds).
When inet:fqdn=mail.somecompany.org is created via depadd, Synapse will set (via autoadd)
inet:fqdn:domain=somecompany.org, which will result in the creation (via depadd) of the node
inet:fqdn=somecompany.org if it does not exist, etc.)
See also the related concept Autoadd.
Derived Property¶
See Property, Derived.
Directed Edge¶
See Edge, Directed.
Directed Graph¶
See Graph, Directed.
E¶
Edge¶
In a traditional Graph, an edge is used to connect exactly two nodes (vertexes). Compare with Hyperedge.
Edge, Directed¶
In a Directed Graph, a directed edge is used to connect exactly two nodes (vertexes) in a one-way (directional) relationship. Compare with Hyperedge.
Edge, Lightweight (Light)¶
In Synapse, a lightweight (light) edge is a mechanism that links two arbitrary forms via a user-defined verb that describes the linking relationship. Light edges are not forms and so do not support secondary properties or tags. They are meant to simplify performance, representation of data, and Synapse hypergraph navigation for many use cases. Contrast with Form, Edge.
Extended Comparison Operator¶
F¶
Feed¶
A feed is an ingest API consisting of a set of ingest formats (e.g., file formats, record formats) used to parse records directly into nodes. Feeds are typically used for bulk node creation, such as ingesting data from an external source or system.
Filter¶
Within Synapse, one of the three primary methods for interacting with data in a Cortex. A filter operation downselects a subset of nodes following a lift operation. Compare with Lift and Pivot.
See Storm Reference - Filtering for additional detail.
Filter, Subquery¶
Within Synapse, a subquery filter is a filter that consists of a Storm expression.
See Subquery Filters for additional detail.
Fork¶
Within Synapse, forking refers to the process of creating a new, empty, writable Layer on top of the fork’s original View. The writable layer from the original view becomes read-only with respect to the fork. Any changes made within a forked view are made within the new writable layer. These changes can optionally be merged back into the original view, or discarded. (Note that any view-specific automation, such as triggers, dmons, or cron jobs, are not copied to the forked view. However, depending on the automation, it may be activated if / when data is merged down into the original view.
Form¶
Within Synapse, a form is the definition of an object in the Synapse data model. A form acts as a “template” that specifies how to create an object (Node) within a Cortex. A form consists of (at minimum) a Primary Property and its associated Type. Depending on the form, it may also have various secondary properties with associated types.
See the Form section in the Data Model - Terminology document for additional detail.
Form, Composite¶
In the Synpase Data Model, a category of form whose primary property is an ordered set of two or more
comma-separated typed values. Examples include DNS A records (inet:dns:a) and web-based
accounts (inet:web:acct).
See Form, Edge.
Form, Edge¶
In the Synapse Data Model, a specialized composite form (Form, Composite) whose primary property consists of two Ndef values. Edge forms can be used to link two arbitrary forms via a generic relationship where additional information needs to be captured about that relationship (i.e., via secondary properpties and/or tags). Contrast with Edge, Lightweight (Light).
Form, GUID¶
In the Synpase Data Model, a specialized case of a Simple Form whose primary property is a
GUID. The GUID can be either arbitrary (in which case it is not considered
Deconflictable) or constructed from a specified set of values (with the goal of being
Deconflictable). Examples include file execution data (e.g., inet:file:exec:read) or
articles (media:news).
Form, Simple¶
In the Synapse Data Model, a category of form whose primary property is a single typed value. Examples
include domains (inet:fqdn) or hashes (e.g., hash:md5).
Fused Knowledge¶
See Knowledge, Fused.
G¶
Graph¶
A graph is a mathematical structure used to model pairwise relations between objects. Graphs consist of vertices (or nodes) that represent objects and edges that connect exactly two vertices in some type of relationship. Nodes and edges in a graph are typically represented by dots or circles conneted by lines.
See Background - Graphs and Hypergraphs for additional detail on graphs and hypergraphs.
Graph, Directed¶
A directed graph is a Graph where the edges representing relationships between nodes have a “direction”. Given node X and node Y connected by edge E, the relationship is valid for X -> E -> Y but not Y -> E -> X. For example, the relationship “Fred owns bank account #01234567” is valid, but “bank account #01234567 owns Fred” is not. Nodes and edges in a directed graph are typically represented by dots or circles connected by arrows.
See Background - Graphs and Hypergraphs for additional detail on graphs and hypergraphs.
GUID¶
Short for Globally Unique Identifier. Within Synapse, a GUID is a Type specified as a 128-bit value that is unique within a given Cortex. GUIDs are used as primary properties for forms that cannot be uniquely represented by a specific value or set of values. Not to be confused with the Microsoft-specific definition of GUID, which is a 128-bit value with a specific format (see https://msdn.microsoft.com/en-us/library/aa373931.aspx).
GUID Form¶
See Form, GUID.
H¶
Hive¶
The Hive is a key/value storage mechanism which is used to persist various data structures required for operating a Synapse Cell.
Hyperedge¶
A hyperedge is an edge within a Hypergraph that can join any number of nodes (vs. a Graph or Directed Graph where an edge joins exactly two nodes). A hyperedge joining an arbitrary number of nodes can be difficult to visualize in flat, two-dimensional space; for this reason hyperedges are often represented as a line or “boundary” encircling a set of nodes, thus “joining” those nodes into a related group.
See Background - Graphs and Hypergraphs for additional detail on graphs and hypergraphs.
Hypergraph¶
A hypergraph is a generalization of a Graph in which an edge can join any number of nodes. If a Directed Graph where edges join exactly two nodes is two-dimensional, then a hypergraph where a Hyperedge can join any number (n-number) of nodes is n-dimensional.
See Background - Graphs and Hypergraphs for additional detail on graphs and hypergraphs.
I¶
Iden¶
Short for Identifier. Within Synapse, the hexadecimal representation of a unique identifier (e.g., for a node, a task, a trigger, etc.) The term “identifier” / “iden” is used regardless of how the specific identifier is generated.
Instance Knowledge¶
See Knowledge, Instance.
K¶
Knowledge, Fused¶
If a form within the Synapse data model has a “range” of time elements (i.e., an interval such as “first seen”/”last
seen”), the form typically represents fused knowledge – a period of time during which an object, relationship, or
event was known to exist. Forms representing fused knowledge can be thought of as combining n number of instance
knowledge observations. inet:dns:query, inet:dns:a, and inet:whois:email forms are examples of fused
knowledge.
See Instance Knowledge vs. Fused Knowledge for a more detailed discussion.
Knowledge, Instance¶
If a form within the Synapse data model has a specific time element (i.e., a single date/time value), the form
typically represents instance knowledge – a single instance or occurrence of an object, relationship, or event.
inet:dns:request and inet:whois:rec forms are examples of instance knowledge.
See Instance Knowledge vs. Fused Knowledge for a more detailed discussion.
L¶
Layer¶
Within Synapse, a layer is the substrate that contains node data and where permissions enforcement occurs. Viewed another way, a layer is a storage and write permission boundary. By default, a Cortex has a single layer and a single View, meaning that by default all nodes are stored in one layer and all changes are written to that layer. However, multiple layers can be created for various purposes such as: separating data from different data sources (e.g., a read-only layer consisting of third-party data and associated tags can be created underneath a “working” layer, so that the third-party data is visible but cannot be modified); providing users with a personal “scratch space” where they can make changes in their layer without affecting the underlying main Cortex layer; or segregating data sets that should be visible/accessible to some users but not others.
Layers are closely related to views (see View). The order in which layers are instantiated within a view matters; in a multi-layer view, typically only the topmost layer is writeable by that view’s users, with subsequent (lower) layers read-only. Explicit actions can push upper-layer writes downward (merge) into lower layers.
Lift¶
Within Synapse, one of the three primary methods for interacting with data in a Cortex. A lift is a read operation that selects a set of nodes from the Cortex. Compare with Filter and Pivot.
See Storm Reference - Lifting for additional detail.
Lightweight (Light) Edge¶
M¶
Macro¶
A macro is a stored Storm query. Macros support the full range of Storm syntax and features.
See the Storm command reference for the macro command and the Storm Reference - Automation for additional detail.
Merge¶
Within Synapse, merge refers to the process of copying changes made within a forked (see Fork) View into the original view.
Model¶
Within Synapse, a system or systems used to represent data and/or assertions in a structured manner. A well-designed model allows efficient and meaningful exploration of the data to identify both known and potentially arbitrary or discoverable relationships.
N¶
Ndef¶
Pronounced “en-deff”. Short for node definition. A node’s Form and associated value
(i.e., <form> = <valu> ) represented as comma-separated elements enclosed in parentheses: (<form>,<valu>).
Node¶
A node is a unique object within a Cortex. Where a Form is a template that defines the
charateristics of a given object, a node is a specific instance of that type of object. For example, inet:fqdn
is a form; inet:fqdn=woot.com is a node.
See Node in the Data Model - Terminology document for additional detail.
Node, Runt¶
Short for “runtime node”. A runt node is a node that does not persist within a Cortex but is created at runtime when
a Cortex is initiated. Runt nodes are commonly used to represent metadata associated with Synapse, such as data model
elements like forms (syn:form) and properties (syn:prop) or automation elements like triggers (syn:trigger)
or cron jobs (syn:cron).
Non-Runtime Safe¶
See Non-Runtsafe.
Non-Runtsafe¶
Short for “non-runtime safe”. Non-runtsafe refers to the use of variables within Storm. A variable that is
non-runtsafe has a value that may change based on the specific node passing through the Storm pipeline. A variable
whose value is set to a node property, such as $fqdn = :fqdn is an example of a non-runtsafe variable (i.e., the
value of the secondary property :fqdn may be different for different nodes, so the value of the variable will be
different based on the specific node being operated on).
Contrast with Runtsafe.
P¶
Package¶
A package is a set of commands and library code used to implement a Storm Service. When a new Storm service is loaded into a Cortex, the Cortex verifes that the service is legitimate and then requests the service’s packages in order to load any extended Storm commands associated with the service and any library code used to implement the service.
Pivot¶
Within Synapse, one of the three primary methods for interacting with data in a Cortex. A pivot operation allows navigation of the hypergraph following a lift operation. A pivot moves from a set of nodes with one or more properties with specified value(s) to a set of nodes with a property having the same value(s). Compare with Lift and Filter.
See Storm Reference - Pivoting for additional detail.
Primary Property¶
See Property, Primary.
Property¶
Within Synapse, properties are individual elements that define a Form or (along with their specific values) that comprise a Node. Every property in Synapse must have a defined Type.
See the Property section in the Data Model - Terminology document for additional detail.
Property, Derived¶
Within Synapse, a derived property is a secondary property that can be extracted (derived) from a node’s primary
property. For example, the domain inet:fqdn=www.google.com can be used to derive inet:fqdn:domain=google.com
and inet:fqdn:host=www; the DNS A record inet:dns:a=(woot.com, 1.2.3.4) can be used to derive
inet:dns:a:fqdn=woot.com and inet:dns:a:ipv4=1.2.3.4.
Synapse will automatically set (Autoadd) any secondary properties that can be derived from a node’s primary property. Because derived properties are based on primary property values, derived secondary properties are always read-only (i.e., cannot be modified once set).
Property, Primary¶
Within Synapse, a primary property is the property that defines a given Form in the data model. The primary property of a form must be defined such that the value of that property is unique across all possible instances of that form. Primary properties are always read-only (i.e., cannot be modified once set).
Property, Relative¶
Within Synapse, a relative property is a Secondary Property referenced using only the portion of the
property’s namespace that is relative to the form’s Primary Property. For example, inet:dns:a:fqdn is
the full name of the “domain” secondary property of a DNS A record form (inet:dns:a). :fqdn is the relative
property / relative property name for that same property.
Property, Secondary¶
Within Synapse, secondary properties are optional properties that provide additional detail about a Form. Within the data model, secondary properties may be defined with optional constraints, such as:
- Whether the property is read-only once set.
- Any normalization (outside of type-specific normalization) that should occur for the property (such as converting a string to all lowercase).
Property, Universal¶
Within Synapse, a universal property is a Secondary Property that is applicable to all forms and may
optionally be set for any form where the property is applicable. For example, .created is a universal property
whose value is the date/time when the associated node was created in a Cortex.
Q¶
Queue¶
Within Synapse, a queue is a basic first-in, first-out (FIFO) data structure used to store and serve objects in a classic pub/sub (publish/subscribe) manner. Any primitive (such as a node iden) can be placed into a queue and then consumed from it. Queues can be used (for example) to support out-of-band processing by allowing non-critical tasks to be executed in the background. Queues are persistent; i.e., if a Cortex is restarted, the queue and any objects in the queue are retained.
R¶
Relative Property¶
See Property, Relative.
Repr¶
Short for “representation”. The repr of a Property defines how the property should be displayed in cases where the display format differs from the storage format. For example, date/time values in Synapse are stored in epoch milliseconds but are displayed in human-friendly “yyyy/mm/dd hh:mm:ss.mmm” format.
Runt Node¶
See Node, Runt.
Runtsafe¶
Short for “runtime safe”. Runtsafe refers to the use of variables within Storm. A variable that is runtsafe has a
value that will not change based on the specific node passing through the Storm pipeline. A variable whose value is
explcitly set, such as $fqdn = woot.com is an example of a runtsafe varaible.
Contrast with Non-Runtsafe.
S¶
Secondary Property¶
See Property, Secondary.
Service¶
A Storm service is a registerable remote component that can provide packages (Package) and additional APIs to Storm and Storm commands. A service resides on a Telepath API endpoint outside of the Cortex. When a service is loaded into a Cortex, the Cortex queries the endpoint to determine if the service is legitimate and, if so, loads the associated Package to implement the service. An advantage of Storm services (over, say, additional Python modules) is that services can be restarted to reload their service definitions and packages while a Cortex is still running – thus allowing a service to be updated without having to restart the entire Cortex.
Simple Form¶
See Form, Simple.
Slab¶
A Slab is a core Synapse component which is used for persisting data on disk into a LMDB backed database. The Slab interface offers an asyncio friendly interface to LMDB objects, while allowing users to largely avoid having to handle native transactions themselves.
Splice¶
A splice is an atomic change made to data within a Cortex, such as node creation or deletion, adding or removing a tag, or setting, modifying, or removing a property. All changes within a Cortex may be retrieved as individual splices within the Cortex’s splice log.
Standard Comparison Operator¶
Storm¶
Storm is the custom, domain-specific language used to interact with data in a Synapse Cortex.
See Storm Reference - Introduction for additional detail.
Subquery¶
Within Synapse, a subquery is a Storm query that is executed inside of another Storm query.
See Storm Reference - Subqueries for additional detail.
Subquery Filter¶
See Filter, Subquery.
T¶
Tag¶
Within Synapse, a tag is a label applied to a node that provides additional context about the node. Tags typically represent assessments or judgements about the data represented by the node.
See the Tag section in the Data Model - Terminology document for additional detail.
Tag, Base¶
Within Synapse, the lowest (rightmost) tag element in a tag hierarchy. For example, for the tag #foo.bar.baz,
baz is the base tag.
Tag, Leaf¶
The full tag path / longest tag in a given tag hierarchy. For example, for the tag #foo.bar.baz, foo.bar.baz
is the leaf tag.
Tag, Root¶
Within Synapse, the highest (leftmost) tag element in a tag hierarchy. For example, for the tag #foo.bar.baz,
foo is the root tag.
Telepath¶
Telepath is a lightweight remote procedure call (RPC) protocol used in Synapse. See Telepath RPC in the Synapse Architecture guide for additional detail.
Traverse¶
In a Graph or Directed Graph, traversal refers to navigating the data in the graph by pathing along the edges between nodes. In a Hypergraph, because there are no edges, navigation between nodes is commonly performed using a Pivot.
Trigger¶
Within Synapse, a trigger is a Storm query that is executed automatically upon the occurrence of a specified event within a Cortex (such as adding a node or applying a tag). “Trigger” refers collectively to the event and the query fired (“triggered”) by the event.
See the Storm command reference for the trigger command and the Storm Reference - Automation for additional detail.
Type¶
Within Synapse, a type is the definition of a data element within the data model. A type describes what the element is and enforces how it should look, including how it should be normalized.
See the Type section in the Data Model - Terminology document for additional detail.
Type, Base¶
Within Synapse, base types include standard types such as integers and strings, as well as common types defined within
or specific to Synapse, including globally unique identifiers (guid), date/time values (time), time intervals
(ival), and tags (syn:tag). Many forms within the Synapse data model are built upon (extensions of) a subset
of common types.
Type, Model-Specific¶
Within Synapse, knowledge-domain-specific forms may themselves be specialized types. For example, an IPv4 address
(inet:ipv4) is its own specialized type. While an IPv4 address is ultimately stored as an integer, the type has
additional constraints, e.g., IPv4 values must fall within the allowable IPv4 address space.
Type Awareness¶
Type awareness is the feature of the Storm query language that facilitates and simplifies navigation through the Hypergraph when pivoting across nodes. Storm leverages knowledge of the Synapse Data Model (specifically knowledge of the type of each node property) to allow pivoting between primary and secondary properties of the same type across different nodes without the need to explicitly specify the properties involved in the pivot.
Type Enforcement¶
Within Synapse, the process by which property values are required to conform to value and format constraints defined
for that Type within the data model before they can be set. Type enforcement helps to limit bad data being
entered in to a Cortex by ensuring values entered make sense for the specified data type (e.g., that an IP address
cannot be set as the value of a property defined as a domain (inet:fqdn) type, and that the integer value of the
IP falls within the allowable set of values for IP address space).
Type Normalization¶
Within Synapse, the process by which properties of a particular type are standardized and formatted in order to ensure consistency in the data model. Normalization may include processes such as converting user-friendly input into a different format for storage (e.g., converting an IP address entered in dotted-decimal notation to an integer), converting certain string-based values to all lowercase, and so on.
U¶
Universal Property¶
See Property, Universal.
V¶
Variable¶
In Storm, a variable is an identifier with a value that can be defined and/or changed during normal execution, i.e., the value is variable.
Contrast with Constant. See also Runtsafe and Non-Runtsafe.
See Storm Reference - Advanced - Variables for a more detailed discussion of variables.
View¶
Within Synapse, a view is a ordered set of layers (see Layer) and associated permissions that are used to synthesize nodes from the Cortex, determining both the nodes that are visible to users via that view and where (i.e., in what layer) any changes made by a view’s users are recorded. A default Cortex consists of a single layer and a single view, meaning that by default all nodes are stored in one layer and all changes are written to that layer.
In multi-layer systems, a view consists of the set of layers that should be visible to users of that view, and the order in which the layers should be instantiated for that view. Order matters because typically only the topmost layer is writeable by that view’s users, with subsequent (lower) layers read-only. Explicit actions can push upper-layer writes downward (merge) into lower layers.