Package Documentation

Storm Package: synapse-urlscan

The following Commands are available from this package. This documentation is generated for version 2.0.0 of the package.

Storm Commands

This package implements the following Storm Commands.

urlscan.config.add

Add a Synapse-URLScan configuration.

The proxy argument can be set to one of the following values:
  true: Use the Cortex configured proxy if set.
  false: Do not use the Cortex configured proxy if set.
  <str>: A proxy URL to use.

Examples:

  // Add a global configuration
  urlscan.config.add default myapikey1234

  // Add a configuration for the current user
  urlscan.config.add --scope self myconfig myapikey5678


Usage: urlscan.config.add [options] <name> <apikey>

Options:

  --help                      : Display the command usage.
  --ssl-noverify              : Do not perform SSL/TLS verification.
  --proxy <proxy>             : Configure the proxy usage. (default: True)
  --tag-prefix <tag_prefix>   : The tag prefix to use when recording data from urlscan.io. (default: rep.urlscan)
  --scope <scope>             : Set the scope on the configuration. (default: global, choices: global, self, unscoped)

Arguments:

  <name>                      : A unique name for the configuration.
  <apikey>                    : The urlscan.io API key string.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.config.del

Delete a Synapse-URLScan configuration.

Examples:

  // Delete the "myconfig" configuration
  urlscan.config.del myconfig


Usage: urlscan.config.del [options] <name>

Options:

  --help                      : Display the command usage.

Arguments:

  <name>                      : The name of the configuration.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.config.list

Display the list of Synapse-URLScan configurations you have access to.


Usage: urlscan.config.list [options]

Options:

  --help                      : Display the command usage.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.config.migrate

Migrate options for all Synapse-URLScan configurations.

The proxy argument can be set to one of the following values:
  true: Use the Cortex configured proxy if set.
  false: Do not use the Cortex configured proxy if set.
  <str>: A proxy URL to use.

Examples:

  // Migrate the tag prefix for all configurations
  urlscan.config.migrate --tag-prefix my.tagpref


Usage: urlscan.config.migrate [options]

Options:

  --help                      : Display the command usage.
  --ssl-verify <ssl_verify>   : Set whether to verify the SSL certificate of the server. (default: None)
  --proxy <proxy>             : Configure the proxy usage. (default: None)
  --tag-prefix <tag_prefix>   : The tag prefix to use when recording data from urlscan.io. (default: None)

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

urlscan.config.show

Show the details of a configuration.

Examples:

  // Show the in-use configuration for the current user
  urlscan.config.show

  // Show a configuration by name
  urlscan.config.show myconfig


Usage: urlscan.config.show [options] <name>

Options:

  --help                      : Display the command usage.

Arguments:

  [name]                      : The name of the configuration.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.config.update

Update the configuration of a defined Synapse-URLScan configuration.

The proxy argument can be set to one of the following values:
  true: Use the Cortex configured proxy if set.
  false: Do not use the Cortex configured proxy if set.
  <str>: A proxy URL to use.

Examples:

  // Set the permission level for user "myuser" to "admin" on the "myconfig"
  // configuration
  urlscan.config.update myconfig --perm user myuser admin

  // Set the permission level for the "all" role to "deny" on the
  // "myconfig" configuration
  urlscan.config.update myconfig --perm role all deny

  // Do not verify the SSL certificate when connecting to "myconfig"
  urlscan.config.update myconfig --ssl-verify (false)

  // Change the tag prefix used by the "myconfig" configuration
  urlscan.config.update myconfig --tag-prefix my.prefix

  // Change the name of the "myconfig" configuration to "otherconfig"
  urlscan.config.update myconfig --name otherconfig


Usage: urlscan.config.update [options] <config>

Options:

  --help                      : Display the command usage.
  --apikey <apikey>           : The urlscan.io API key string. (default: None)
  --perm <perm>               : Set the permission level for a user or role on this configuration.
                                Arguments to this option are ``scope``, ``name``, ``level``:
                                  <scope>: The scope for the permission, either "user" or "role".
                                  <name>: The user/role name depending on scope.
                                  <level>: The $lib.auth.easyperm.level, or None to remove the permission.
                                 (default: None)
  --ssl-verify <ssl_verify>   : Set whether to verify the SSL certificate of the server. (default: None)
  --proxy <proxy>             : Configure the proxy usage. (default: None)
  --tag-prefix <tag_prefix>   : The tag prefix to use when recording data from urlscan.io. (default: None)
  --name <name>               : Rename the configuration. (default: None)

Arguments:

  <config>                    : The name of the configuration to modify.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.enrich

Enrich nodes by issuing URLScan queries for inbound nodes.

This command constructs a URLScan query using inbound nodes and submits it
to the search endpoint to retrieve archived scans. The results are used to create
inet:url nodes. In cases where the URL queried does not match the final URL
returned by URLScan, inet:urlredir nodes will be created to represent the redirect
chain of URLs.

If IP address information is available, the corresponding inet:dns:* nodes will
be created to represent it.

To download the full scan output of each search result, the --detail option can
be used. This will query the results API and poll until the scan is complete.
Once results are available, they will be downloaded and ingested into the Cortex.
Using this option will cause command execution to take significantly longer and
consume additional quota.

When the --detail option is used, ``requests`` light edges will be created from
the inet:url node to each inet:http:request node that is created. Additionally,
the final DOM tree will be downloaded to the Axon and have the file:bytes node
linked to the result inet:url with a ``dom`` light edge.

When --detail is specified, the --download-responses option may be specified with
a value of "primary" or "all" to download HTTP response data available in the
results. If "primary" is specified, only the primary response or file download
will be retrieved. If "all" is specified, all available HTTP responses will be
downloaded.

Note:

  By default, only the overall URLScan maliciousness score will be populated.
  The URLScan specific score (not including community verdicts), is only available
  --detail is specified.

Examples:

  // Enrich an inet:url node
  inet:url=https://support.vertex.link | urlscan.enrich

  // Enrich an inet:fqdn node and yield the inet:url nodes created
  inet:fqdn=vertex.link | urlscan.enrich --yield


Endpoints:

  (user-configured base URL)
    /api/v1/search/           : Search for archived scans.
    /api/v1/result/{uuid}/    : Retrieve the result of a scan.

Usage: urlscan.enrich [options]

Options:

  --help                      : Display the command usage.
  --time <time>               : Filter by results within a specific time interval.
  --detail                    : Download and ingest the full scan results from the results endpoint.
  --download-responses <download_responses>: Download HTTP responses bodies and file downloads in the result (must be specified with
                                --detail). (choices: primary, all)
  --dom                       : Download the DOM tree for the result.
  --screenshot                : Download the screenshot for the result.
  --yield                     : Yield the newly created nodes.
  --debug                     : Print debugging messages.
  --asof <asof>               : This argument is deprecated and no longer has any effect.
  --size <size>               : Limit the number of results ingested to the given size (per-node).
  --config <config>           : Override the default config with the provided name.

Inputs:

  inet:fqdn                   : Search for scans of urls with this domain.
  inet:url                    : Search for scans of this url.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.quota

Print out usage and usage limits

Endpoints:

  (user-configured base URL)
    /user/quotas/             : Get API quota information.

Usage: urlscan.quota [options]

Options:

  --help                      : Display the command usage.
  --config <config>           : Override the default config with the provided name.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.scan

Submit URLs to be scanned to URLScan.

This command takes an inet:url node as input and submits the url URLScan to
be scanned.

After submitting the URL to the scan endpoint, this command will query the results
API and poll until the scan is complete. Once results are available, they will be
downloaded and ingested into the Cortex.

When ingesting the results, ``requests`` light edges will be created from
the inet:url node to each inet:http:request node that is created. Additionally,
the final DOM tree will be downloaded to the Axon and have the file:bytes node
linked to the resulting inet:url with a ``dom`` light edge.

The --download-responses option may be specified with a value of "primary" or
"all" to download HTTP response data available in the results. If "primary" is
specified, only the primary response or file download will be retrieved. If "all"
is specified, all available HTTP responses will be downloaded.

In cases where the URL queried does not match the final URL returned by URLScan,
inet:urlredir nodes will be created to represent the redirect chain of URLs.

NOTE - This command will also create an it:exec:query node to represent the
       query syntax and link resulting nodes to it via -(found)> edges.

Examples:

  // Scan an inet:url node
  inet:url=https://vertex.link | urlscan.scan

  // Scan an inet:url node with a custom HTTP referer string
  inet:url=https://vertex.link | urlscan.scan --referer "https://example.com/"


Endpoints:

  (user-configured base URL)
    /api/v1/scan/             : Submit a URL for scanning.
    /api/v1/result/{uuid}/    : Retrieve the result of a scan.

Usage: urlscan.scan [options]

Options:

  --help                      : Display the command usage.
  --customagent <customagent> : Override User-Agent for this scan.
  --referer <referer>         : Override HTTP referer for this scan.
  --visibility <visibility>   : One of public, unlisted, private. Defaults to your API key's configured default
                                visibility.
  --country <country>         : Specify which country the scan should be performed from (2-Letter ISO-3166-1 alpha-2
                                country). (choices: au, ca, de, dk, es, fi, fr, gb, is, it, jp, nl, no, nz, pl, se, sg,
                                us)
  --download-responses <download_responses>: Download HTTP responses bodies and file downloads in the result. (choices: primary,
                                all)
  --debug                     : Print debugging messages.
  --config <config>           : Override the default config with the provided name.

Inputs:

  inet:url                    : URL to be scanned.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.search.certificates

Search for certificates with the URLScan search API endpoint.

This command takes inbound inet:fqdn nodes and searches the
URLScan certificates datasource for SSL certificates. The --query
argument may instead be used to provide a query string in Lucene/ES
syntax. The results are used to create crypto:x509:cert nodes.

This command will also create an it:exec:query node to represent the
query syntax and link resulting nodes to it via -(found)> edges.

Note: This command requires an API key with urlscan Pro access.

For more information on the query syntax for this command, please visit
the URLScan documentation at https://urlscan.io/docs/search/

Examples:

    // Search for certificates using an inet:fqdn node
    inet:fqdn=vertex.link | urlscan.search.certificates

    // Search using --query and yield the created crypto:x509:cert nodes
    urlscan.search.certificates --query vertex.link --yield


Endpoints:

  (user-configured base URL)
    /api/v1/search/           : Search for archived scans.

Usage: urlscan.search.certificates [options]

Options:

  --help                      : Display the command usage.
  --query <query>             : Lucene/ES query syntax. See URLScan docs for details.
  --yield                     : Yield the newly created nodes.
  --debug                     : Print debugging messages.
  --asof <asof>               : This argument is deprecated and no longer has any effect.
  --size <size>               : Limit the number of results ingested to the given size (per-node).
  --config <config>           : Override the default config with the provided name.

Inputs:

  inet:fqdn                   : Search for certificates related to this domain.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.search.hostnames

Search for hostnames with the URLScan search API endpoint.

This command takes inbound inet:fqdn nodes and searches the
URLScan hostnames datasource for matches. The --query argument
may instead be used to provide a query string in Lucene/ES syntax.
The results are used to create inet:fqdn nodes.

This command will also create an it:exec:query node to represent the
query syntax and link resulting nodes to it via -(found)> edges.

Note: This command requires an API key with urlscan Pro access.

For more information on the query syntax for this command, please visit
the URLScan documentation at https://urlscan.io/docs/search/

Examples:

    // Search for hostnames using an inet:fqdn node
    inet:fqdn=vertex.link | urlscan.search.hostnames

    // Search using --query and yield the created inet:fqdn nodes
    urlscan.search.hostnames --query vertex.link --yield


Endpoints:

  (user-configured base URL)
    /api/v1/search/           : Search for archived scans.

Usage: urlscan.search.hostnames [options]

Options:

  --help                      : Display the command usage.
  --query <query>             : Lucene/ES query syntax. See URLScan docs for details.
  --yield                     : Yield the newly created nodes.
  --debug                     : Print debugging messages.
  --asof <asof>               : This argument is deprecated and no longer has any effect.
  --size <size>               : Limit the number of results ingested to the given size (per-node).
  --config <config>           : Override the default config with the provided name.

Inputs:

  inet:fqdn                   : Search for hostnames related to this domain.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

urlscan.search.incidents

Search for incidents with the URLScan search API endpoint.

This command takes inbound inet:fqdn, inet:ipv4, inet:ipv6, or inet:url
nodes and searches the URLScan incidents datasource for existing
incidents. The --query argument may instead be used to provide a query
string in Lucene/ES syntax. The results are used to create meta:rule nodes.

This command will also create an it:exec:query node to represent the
query syntax and link resulting nodes to it via -(found)> edges.

Note: This command requires an API key with urlscan Pro access.

For more information on the query syntax for this command, please visit
the URLScan documentation at https://urlscan.io/docs/search/

Examples:

    // Search for incidents using an inet:fqdn node
    inet:fqdn=vertex.link | urlscan.search.incidents

    // Search for incidents flagged as malicious and yield the
    // created meta:rule nodes
    urlscan.search.incidents --query "allTags:ismalicious" --yield


Endpoints:

  (user-configured base URL)
    /api/v1/search/           : Search for archived scans.

Usage: urlscan.search.incidents [options]

Options:

  --help                      : Display the command usage.
  --query <query>             : Lucene/ES query syntax. See URLScan docs for details.
  --yield                     : Yield the newly created nodes.
  --debug                     : Print debugging messages.
  --size <size>               : Limit the number of results ingested to the given size (per-node).
  --config <config>           : Override the default config with the provided name.

Inputs:

  inet:fqdn                   : Search for incidents related to this domain.
  inet:ipv4                   : Search for incidents related to this IP.
  inet:ipv6                   : Search for incidents related to this IP.
  inet:url                    : Search for incidents related to this URL.

The command is accessible to users with one or more of the following permissions:

  • power-ups.urlscan.admin

  • power-ups.urlscan.user

Storm Modules

This package does not export any Storm APIs.