User Guide

Synapse FileParser provides additional Storm commands to parse files and ingest extracted data. For details on supported formats see Formats.

Getting Started

Check with your Global Admin to enable permissions.

Examples

Scrape indicators using the text parser

> file:bytes#myfile.txt | fileparser.parse | -(refs)> *
fileparser parsing sha256: f0c6fc65a4e95a58529be2806b0e218eab9752385a02d39a2b7932e17b8589a0
inet:ipv4=1.2.3.4
        .created = 2024/04/12 19:15:54.915
        :type = unicast
inet:[email protected]
        .created = 2024/04/12 19:15:54.997
        :fqdn = email.com
        :user = me
inet:fqdn=email.com
        .created = 2024/04/12 19:15:54.997
        :domain = com
        :host = email
        :issuffix = false
        :iszone = true
        :zone = email.com
inet:fqdn=foo.com
        .created = 2024/04/12 19:15:54.844
        :domain = com
        :host = foo
        :issuffix = false
        :iszone = true
        :zone = foo.com

Parse an X.509 certificate

> file:bytes#myfile.x509 | fileparser.parse | :sha256 -> crypto:x509:cert:sha256
fileparser parsing sha256: c0cf8de4947844d1105b1cf6d374ea3c7b298aae98218f44229aee422beca9f4
crypto:x509:cert=ed2487a9affdaed4a4590009ef278b6f
        .created = 2024/04/12 19:15:58.780
        :algo = 1.2.840.113549.1.1.11
        :file = sha256:c0cf8de4947844d1105b1cf6d374ea3c7b298aae98218f44229aee422beca9f4
        :identities:emails = ['[email protected]']
        :identities:fqdns = ['testcert.com']
        :identities:ipv4s = ['1.2.3.4']
        :identities:ipv6s = ['2001:db8::1000']
        :identities:urls = ['https://testcert.com']
        :issuer = CN=testcert.com
        :md5 = f813d229e19ba45f6e5ba349b5f05d2a
        :rsa:key = ('b447826d6013b3adc7c93c101b3e1d55919d8babfba9a8ff2c242150564d17eb6756fd4dfeb9c0d9c31da84401d961a49920fbdf3e0bfd0a07f9c04203c86ff158654476f68265e1e83a63a5a1eb029670351dbd08c1eda2f8baaf5523ca7543149220e1215e04727524fd52d0b2859dac73c619d9cfb3b302e85b7798e35233', '65537')
        :selfsigned = true
        :serial = 00000000000000000000000000000000000003e8
        :sha1 = 9556bc50fc8703710fb5bc2a1c2c0f01456389c2
        :sha256 = c0cf8de4947844d1105b1cf6d374ea3c7b298aae98218f44229aee422beca9f4
        :signature = 00f497f5826e7483a23cf5c0b063e966fbce697de273e5f12bd24dfb479686ce18e9b612c3453981f87c8c1d291c08a564a891ca41fbe9b7425d395b2e82ab5fe777e8afb4e8d7973dfa5d4e16746209ad004db5c539a6ed52ca697f9ed64cf918b6043c65a8a2661985a1a6040447fc82fee07d27bda39d6c5981ca062fae64
        :subject = CN=testcert.com
        :validity:notafter = 2029/09/30 21:25:49.000
        :validity:notbefore = 2019/10/03 21:25:49.000
        :version = v3

Parse a Zip archive and pivot to the archive entries

> file:bytes#myfile.zip | fileparser.parse | -> file:archive:entry
fileparser parsing sha256: 58802d62b47d9b551a4eada5215c7155a9a9d16bddc8980cdededfbadfb8f1b9
file:archive:entry=1dbd4aba8f47f4aafdf8152c83018051
        .created = 2024/04/12 19:16:06.391
        :archived:size = 1341
        :file = sha256:2da752c0ef01dd5687fe3e13364523f7165f6f7d707fd00abf7bb181dd5bd01c
        :parent = sha256:58802d62b47d9b551a4eada5215c7155a9a9d16bddc8980cdededfbadfb8f1b9
        :path = ziptestsub/foobar.message
file:archive:entry=6aa66b1f25bf28e26c406230abfd0454
        .created = 2024/04/12 19:16:02.661
        :archived:size = 89
        :file = sha256:f0c6fc65a4e95a58529be2806b0e218eab9752385a02d39a2b7932e17b8589a0
        :parent = sha256:58802d62b47d9b551a4eada5215c7155a9a9d16bddc8980cdededfbadfb8f1b9
        :path = test.txt

Pivot to indicators scraped from the text file in the Zip archive.

> file:bytes#myfile.zip -> file:archive:entry:parent +:path=test.txt :file -> file:bytes -(refs)> *
inet:ipv4=1.2.3.4
        .created = 2024/04/12 19:15:54.915
        :type = unicast
inet:[email protected]
        .created = 2024/04/12 19:15:54.997
        :fqdn = email.com
        :user = me
inet:fqdn=email.com
        .created = 2024/04/12 19:15:54.997
        :domain = com
        :host = email
        :issuffix = false
        :iszone = true
        :zone = email.com
inet:fqdn=foo.com
        .created = 2024/04/12 19:15:54.844
        :domain = com
        :host = foo
        :issuffix = false
        :iszone = true
        :zone = foo.com

Use of meta:source nodes

Synapse-FileParser uses a meta:source node and -(seen)> light weight edges to track nodes extracted from parsed files.

> meta:source=0be9660ae19b7113ed6c62f05c824f7c
meta:source=0be9660ae19b7113ed6c62f05c824f7c
        .created = 2024/04/12 19:15:48.227
        :name = synapse file parser v3

Storm can be used to filter nodes to include/exclude nodes which have been observed by Synapse-FileParser. The following example shows how to filter the results of a query to include only results observed by Synapse-FileParser:

> inet:email:fqdn=email.com +{ <(seen)- meta:source=0be9660ae19b7113ed6c62f05c824f7c }
inet:[email protected]
        .created = 2024/04/12 19:15:54.997
        :fqdn = email.com
        :user = me