Package Documentation

Storm Package: synapse-urlscan

The following Commands are available from this package. This documentation is generated for version 1.5.0 of the package.

Storm Commands

This package implements the following Storm Commands.

urlscan.enrich

Enrich nodes by issuing URLScan queries for inbound nodes.

This command constructs a URLScan query using inbound nodes and submits it
to the search endpoint to retrieve archived scans. The results are used to create
inet:url nodes. In cases where the URL queried does not match the final URL
returned by URLScan, inet:urlredir nodes will be created to represent the redirect
chain of URLs.

If IP address information is available, the corresponding inet:dns:* nodes will
be created to represent it.

To download the full scan output of each search result, the --detail option can
be used. This will query the results API and poll until the scan is complete.
Once results are available, they will be downloaded and ingested into the Cortex.
Using this option will cause command execution to take significantly longer and
consume additional quota.

When the --detail option is used, ``requests`` light edges will be created from
the inet:url node to each inet:http:request node that is created. Additionally,
the final DOM tree will be downloaded to the Axon and have the file:bytes node
linked to the result inet:url with a ``dom`` light edge.

Examples:

  // Enrich an inet:url node
  inet:url=https://support.vertex.link | urlscan.enrich

  // Enrich an inet:fqdn node and yield the inet:url nodes created
  inet:fqdn=vertex.link | urlscan.enrich --yield


Usage: urlscan.enrich [options]

Options:

  --help                      : Display the command usage.
  --time <time>               : Filter by results within a specific time interval.
  --detail                    : Download and ingest the full scan results from the results endpoint.
  --dom                       : Download the DOM tree for the result.
  --screenshot                : Download the screenshot for the result.
  --yield                     : Yield the newly created nodes.
  --debug                     : Print debugging messages.
  --asof <asof>               : Specify the maximum age for a cached result. To disable caching, use --asof now. (default: -30days)
  --size <size>               : Limit the number of results ingested to the given size (per-node).

urlscan.quota

Print out usage and usage limits

Usage: urlscan.quota [options]

Options:

  --help                      : Display the command usage.

urlscan.scan

Submit URLs to be scanned to URLScan.

This command takes an inet:url node as input and submits the url URLScan to
be scanned.

After submitting the URL to the scan endpoint, this command will query the results
API and poll until the scan is complete. Once results are available, they will be
downloaded and ingested into the Cortex.

When ingesting the results, ``requests`` light edges will be created from
the inet:url node to each inet:http:request node that is created. Additionally,
the final DOM tree will be downloaded to the Axon and have the file:bytes node
linked to the resulting inet:url with a ``dom`` light edge.

In cases where the URL queried does not match the final URL returned by URLScan,
inet:urlredir nodes will be created to represent the redirect chain of URLs.

NOTE - This command will also create an it:exec:query node to represent the
       query syntax and link resulting nodes to it via -(found)> edges.

Examples:

  // Scan an inet:url node
  inet:url=https://vertex.link | urlscan.scan

  // Scan an inet:url node with a custom HTTP referer string
  inet:url=https://vertex.link | urlscan.scan --referer "https://example.com/"


Usage: urlscan.scan [options]

Options:

  --help                      : Display the command usage.
  --customagent <customagent> : Override User-Agent for this scan.
  --referer <referer>         : Override HTTP referer for this scan.
  --visibility <visibility>   : One of public, unlisted, private. Defaults to your API key's configured default visibility.
  --country <country>         : Specify which country the scan should be performed from (2-Letter ISO-3166-1 alpha-2 country). (choices: au, ca, de, dk, es, fi, fr, gb, is, it, jp, nl, no, nz, pl, se, sg, us)
  --debug                     : Print debugging messages.

urlscan.setup.apikey

Set the urlscan.io API key.

Usage: urlscan.setup.apikey [options] <apikey>

Options:

  --help                      : Display the command usage.
  --self                      : Set the key as a user variable. If not used, the key is set globally.

Arguments:

  <apikey>                    : The urlscan.io API key string.