User Guide

Synapse-Splunk User Guide

Synapse-Splunk adds new Storm commands to allow you to query the Splunk API using your existing API key.

Getting Started

Check with your Admin to enable permissions and find out if you need a personal API key.

Examples

Setting your personal API key

To set-up a personal use API key:

> splunk.setup.apikey --self myapikey
Setting Splunk API key for the current user.

Query Splunk log events and ingest the results

Results can first be viewed without creating any nodes by using the --dryrun option.

> splunk.search --dryrun 'index="_internal" log_level="ERROR" | head 2'
{'_bkt': 'test~0~C61E9364-9D2D-42B7-AA95-213229664105',
 '_cd': '0:154',
 '_indextime': '1664496211',
 '_kv': '1',
 '_raw': '09-30-2022 00:03:31.526 +0000 WARN  SearchMessages - '
         'orig_component="SearchStatusEnforcer" app="search" '
         'sid="rt_md_1664496118.586" message_key="" message=Search '
         'cancellation requested.',
 '_serial': '0',
 '_si': ('737ef4afe3c4', 'test'),
 '_sourcetype': 'logs',
 '_subsecond': '.526',
 '_time': '2022-09-30T00:03:31.526+00:00',
 'host': '737ef4afe3c4',
 'log_level': 'WARN',
 'message': 'Search cancellation requested.'}
{'_bkt': 'test~0~C61E9364-9D2D-42B7-AA95-213229664105',
 '_cd': '0:147',
 '_indextime': '1664496035',
 '_kv': '1',
 '_raw': '09-29-2022 23:29:01.532 +0000 ERROR SearchMessages - '
         'orig_component="SearchStatusEnforcer" app="search" '
         'sid="1664494056.516" message_key="" message=Search auto-canceled',
 '_serial': '1',
 '_si': ('737ef4afe3c4', 'test'),
 '_sourcetype': 'logs',
 '_subsecond': '.532',
 '_time': '2022-09-29T23:29:01.532+00:00',
 'host': '737ef4afe3c4',
 'log_level': 'ERROR',
 'message': 'Search auto-canceled'}

Running the command without --dryrun will create it:log:event nodes, and also scrape any indicators from the _raw field.

> splunk.search --yield 'index="_internal" log_level="ERROR" | head 2'
it:log:event=958b9f192509c9cbc5eb0e50abceb59b
        .created = 2024/11/19 21:29:23.959
        :data = {'host': '737ef4afe3c4', 'message': 'Search cancellation requested.', 'log_level': 'WARN', '_bkt': 'test~0~C61E9364-9D2D-42B7-AA95-213229664105', '_cd': '0:154', '_indextime': '1664496211', '_kv': '1', '_raw': '09-30-2022 00:03:31.526 +0000 WARN  SearchMessages - orig_component="SearchStatusEnforcer" app="search" sid="rt_md_1664496118.586" message_key="" message=Search cancellation requested.', '_serial': '0', '_si': ('737ef4afe3c4', 'test'), '_sourcetype': 'logs', '_subsecond': '.526', '_time': '2022-09-30T00:03:31.526+00:00'}
        :host = 47d005f7456d0ae4083c30459b877249
        :mesg = 09-30-2022 00:03:31.526 +0000 WARN  SearchMessages - orig_component="SearchStatusEnforcer" app="search" sid="rt_md_1664496118.586" message_key="" message=Search cancellation requested.
        :severity = warning
        :time = 2022/09/30 00:03:31.526
it:log:event=cc89a74e03c0ba5c9eee6bd45ae4b6ec
        .created = 2024/11/19 21:29:24.076
        :data = {'host': '737ef4afe3c4', 'message': 'Search auto-canceled', 'log_level': 'ERROR', '_bkt': 'test~0~C61E9364-9D2D-42B7-AA95-213229664105', '_cd': '0:147', '_indextime': '1664496035', '_kv': '1', '_raw': '09-29-2022 23:29:01.532 +0000 ERROR SearchMessages - orig_component="SearchStatusEnforcer" app="search" sid="1664494056.516" message_key="" message=Search auto-canceled', '_serial': '1', '_si': ('737ef4afe3c4', 'test'), '_sourcetype': 'logs', '_subsecond': '.532', '_time': '2022-09-29T23:29:01.532+00:00'}
        :host = 47d005f7456d0ae4083c30459b877249
        :mesg = 09-29-2022 23:29:01.532 +0000 ERROR SearchMessages - orig_component="SearchStatusEnforcer" app="search" sid="1664494056.516" message_key="" message=Search auto-canceled
        :severity = err
        :time = 2022/09/29 23:29:01.532

Use the Storm API to create a custom ingest

$splunk = $lib.import(splunk)
$query = '| from inputlookup:"geo_attr_countries" | search country=Domin*'
for $item in $splunk.search(query=$query) {
    [ geo:name=$item.country ]
}

geo:name=dominica
        .created = 2024/11/19 21:29:24.429
geo:name=dominican republic
        .created = 2024/11/19 21:29:24.436

Use of `meta:source` nodes

Synapse-Splunk uses a meta:source node and -(seen)> light weight edges to track nodes observed from the Splunk API.

> meta:source=3918e28ef98e21b7d5ed6791ee9f3394
meta:source=3918e28ef98e21b7d5ed6791ee9f3394
        .created = 2024/11/19 21:29:23.749
        :name = splunk api

Storm can be used to filter nodes to include/exclude nodes which have been observed by Synapse-Splunk. The following example shows how to filter the results of a query to include only results observed by Synapse-Splunk:

> it:log:event:severity=err +{ <(seen)- meta:source=3918e28ef98e21b7d5ed6791ee9f3394 }
it:log:event=cc89a74e03c0ba5c9eee6bd45ae4b6ec
        .created = 2024/11/19 21:29:24.076
        :data = {'host': '737ef4afe3c4', 'message': 'Search auto-canceled', 'log_level': 'ERROR', '_bkt': 'test~0~C61E9364-9D2D-42B7-AA95-213229664105', '_cd': '0:147', '_indextime': '1664496035', '_kv': '1', '_raw': '09-29-2022 23:29:01.532 +0000 ERROR SearchMessages - orig_component="SearchStatusEnforcer" app="search" sid="1664494056.516" message_key="" message=Search auto-canceled', '_serial': '1', '_si': ('737ef4afe3c4', 'test'), '_sourcetype': 'logs', '_subsecond': '.532', '_time': '2022-09-29T23:29:01.532+00:00'}
        :host = 47d005f7456d0ae4083c30459b877249
        :mesg = 09-29-2022 23:29:01.532 +0000 ERROR SearchMessages - orig_component="SearchStatusEnforcer" app="search" sid="1664494056.516" message_key="" message=Search auto-canceled
        :severity = err
        :time = 2022/09/29 23:29:01.532