synapse package¶
The synapse intelligence analysis framework.
Subpackages¶
- synapse.cmds package
- synapse.data package
- synapse.lib package
- Subpackages
- synapse.lib.crypto package
- synapse.lib.platforms package
- synapse.lib.stormlib package
- Submodules
- synapse.lib.stormlib.backup module
- synapse.lib.stormlib.infosec module
- synapse.lib.stormlib.json module
- synapse.lib.stormlib.macro module
- synapse.lib.stormlib.model module
- synapse.lib.stormlib.modelext module
- synapse.lib.stormlib.project module
- synapse.lib.stormlib.stix module
- synapse.lib.stormlib.version module
- Submodules
- synapse.lib.agenda module
- synapse.lib.aha module
- synapse.lib.ast module
- synapse.lib.autodoc module
- synapse.lib.base module
- synapse.lib.boss module
- synapse.lib.cache module
- synapse.lib.cell module
- synapse.lib.certdir module
- synapse.lib.chop module
- synapse.lib.cli module
- synapse.lib.cmd module
- synapse.lib.cmdr module
- synapse.lib.config module
- synapse.lib.const module
- synapse.lib.coro module
- synapse.lib.datfile module
- synapse.lib.dyndeps module
- synapse.lib.encoding module
- synapse.lib.gis module
- synapse.lib.grammar module
- synapse.lib.hashitem module
- synapse.lib.hashset module
- synapse.lib.health module
- synapse.lib.hive module
- synapse.lib.hiveauth module
- synapse.lib.httpapi module
- synapse.lib.ingest module
- synapse.lib.interval module
- synapse.lib.jsonstor module
- synapse.lib.jupyter module
- synapse.lib.layer module
- synapse.lib.link module
- synapse.lib.lmdbslab module
- synapse.lib.modelrev module
- synapse.lib.module module
- synapse.lib.modules module
- synapse.lib.msgpack module
- synapse.lib.nexus module
- synapse.lib.node module
- synapse.lib.output module
- synapse.lib.parser module
- synapse.lib.provenance module
- synapse.lib.queue module
- synapse.lib.ratelimit module
- synapse.lib.reflect module
- synapse.lib.scope module
- synapse.lib.scrape module
- synapse.lib.share module
- synapse.lib.slaboffs module
- synapse.lib.slabseqn module
- synapse.lib.snap module
- synapse.lib.spooled module
- synapse.lib.storm module
- synapse.lib.storm_format module
- synapse.lib.stormctrl module
- synapse.lib.stormhttp module
- synapse.lib.stormsvc module
- synapse.lib.stormtypes module
- synapse.lib.stormwhois module
- synapse.lib.task module
- synapse.lib.thishost module
- synapse.lib.thisplat module
- synapse.lib.threads module
- synapse.lib.time module
- synapse.lib.trigger module
- synapse.lib.types module
- synapse.lib.urlhelp module
- synapse.lib.version module
- synapse.lib.view module
- Subpackages
- synapse.lookup package
- synapse.models package
- Subpackages
- Submodules
- synapse.models.auth module
- synapse.models.base module
- synapse.models.crypto module
- synapse.models.dns module
- synapse.models.economic module
- synapse.models.files module
- synapse.models.geopol module
- synapse.models.geospace module
- synapse.models.inet module
- synapse.models.infotech module
- synapse.models.language module
- synapse.models.material module
- synapse.models.media module
- synapse.models.orgs module
- synapse.models.person module
- synapse.models.proj module
- synapse.models.risk module
- synapse.models.syn module
- synapse.models.telco module
- synapse.models.transport module
- synapse.servers package
- synapse.tools package
- Subpackages
- Submodules
- synapse.tools.autodoc module
- synapse.tools.backup module
- synapse.tools.cellauth module
- synapse.tools.cmdr module
- synapse.tools.csvtool module
- synapse.tools.easycert module
- synapse.tools.feed module
- synapse.tools.genpkg module
- synapse.tools.guid module
- synapse.tools.healthcheck module
- synapse.tools.json2mpk module
- synapse.tools.pullfile module
- synapse.tools.pushfile module
- synapse.tools.rstorm module
Submodules¶
synapse.axon module¶
-
class
synapse.axon.
Axon
[source]¶ Bases:
synapse.lib.cell.Cell
-
confdefs
= {'http:proxy': {'description': 'An aiohttp-socks compatible proxy URL to use in the wget API.', 'type': 'string'}, 'max:bytes': {'description': 'The maximum number of bytes that can be stored in the Axon.', 'hidecmdl': True, 'minimum': 1, 'type': 'integer'}, 'max:count': {'description': 'The maximum number of files that can be stored in the Axon.', 'hidecmdl': True, 'minimum': 1, 'type': 'integer'}}¶
-
del_
(sha256)[source]¶ Remove the given bytes from the Axon by sha256.
Parameters: sha256 (bytes) – The sha256, in bytes, to remove from the Axon. Returns: True if the file is removed; false if the file is not present. Return type: boolean
-
dels
(sha256s)[source]¶ Given a list of sha256 hashes, delete the files from the Axon.
Parameters: sha256s (list) – A list of sha256 hashes in bytes form. Returns: A list of booleans, indicating if the file was deleted or not. Return type: list
-
get
(sha256)[source]¶ Get bytes of a file.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Examples
Get the bytes from an Axon and process them:
buf = b'' async for bytz in axon.get(sha256): buf =+ bytz await dostuff(buf)
Yields: bytes – Chunks of the file bytes. Raises: synapse.exc.NoSuchFile
– If the file does not exist.
-
has
(sha256)[source]¶ Check if the Axon has a file.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Returns: True if the Axon has the file; false otherwise. Return type: boolean
-
hashes
(offs)[source]¶ Yield hash rows for files that exist in the Axon in added order starting at an offset.
Parameters: offs (int) – The index offset. Yields: (int, (bytes, int)) – An index offset and the file SHA-256 and size.
-
history
(tick, tock=None)[source]¶ Yield hash rows for files that existing in the Axon after a given point in time.
Parameters: - tick (int) – The starting time (in epoch milliseconds).
- tock (int) – The ending time to stop iterating at (in epoch milliseconds).
Yields: (int, (bytes, int)) – A tuple containing time of the hash was added and the file SHA-256 and size.
-
holdHashLock
(hashbyts)[source]¶ A context manager that synchronizes edit access to a blob.
Parameters: hashbyts (bytes) – The blob to hold the lock for.
-
iterMpkFile
(sha256)[source]¶ Yield items from a MsgPack (.mpk) file in the Axon.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Yields: Unpacked items from the bytes.
-
metrics
()[source]¶ Get the runtime metrics of the Axon.
Returns: A dictionary of runtime data about the Axon. Return type: dict
-
put
(byts)[source]¶ Store bytes in the Axon.
Parameters: byts (bytes) – The bytes to store in the Axon. Notes
This API should not be used for files greater than 128 MiB in size.
Returns: A tuple with the file size and sha256 hash of the bytes. Return type: tuple(int, bytes)
-
puts
(files)[source]¶ Store a set of bytes in the Axon.
Parameters: files (list) – A list of bytes to store in the Axon. Notes
This API should not be used for storing more than 128 MiB of bytes at once.
Returns: A list containing tuples of file size and sha256 hash of the saved bytes. Return type: list(tuple(int, bytes))
-
save
(sha256, genr)[source]¶ Save a generator of bytes to the Axon.
Parameters: - sha256 (bytes) – The sha256 hash of the file in bytes.
- genr – The bytes generator function.
Returns: The size of the bytes saved.
Return type: int
-
size
(sha256)[source]¶ Get the size of a file in the Axon.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Returns: The size of the file, in bytes. If not present, None is returned. Return type: int
-
upload
()[source]¶ Get an Upload object.
Notes
The UpLoad object should be used to manage uploads greater than 128 MiB in size.
Examples
Use an UpLoad object to upload a file to the Axon:
async with axon.upload() as upfd: # Assumes bytesGenerator yields bytes async for byts in bytsgenerator(): upfd.write(byts) upfd.save()
Use a single UpLoad object to save multiple files:
async with axon.upload() as upfd: for fp in file_paths: # Assumes bytesGenerator yields bytes async for byts in bytsgenerator(fp): upfd.write(byts) upfd.save()
Returns: An Upload manager object. Return type: UpLoad
-
wants
(sha256s)[source]¶ Get a list of sha256 values the axon does not have from a input list.
Parameters: sha256s (list) – A list of sha256 values as bytes. Returns: A list of bytes containing the sha256 hashes the Axon does not have. Return type: list
-
wget
(url, params=None, headers=None, json=None, body=None, method='GET', ssl=True, timeout=None)[source]¶ Stream a file download directly into the Axon.
Parameters: - url (str) – The URL to retrieve.
- params (dict) – Additional parameters to add to the URL.
- headers (dict) – Additional HTTP headers to add in the request.
- json – A JSON body which is included with the request.
- body – The body to be included in the request.
- method (str) – The HTTP method to use.
- ssl (bool) – Perform SSL verification.
- timeout (int) – The timeout of the request, in seconds.
Notes
The response body will be stored, regardless of the response code. The
ok
value in the reponse does not reflect that a status code, such as a 404, was encountered when retrieving the URL.The dictionary returned by this may contain the following values:
{ 'ok': <boolean> - False if there were exceptions retrieving the URL. 'url': <str> - The URL retrieved (which could have been redirected) 'code': <int> - The response code. 'mesg': <str> - An error message if there was an exception when retrieving the URL. 'headers': <dict> - The response headers as a dictionary. 'size': <int> - The size in bytes of the response body. 'hashes': { 'md5': <str> - The MD5 hash of the response body. 'sha1': <str> - The SHA1 hash of the response body. 'sha256': <str> - The SHA256 hash of the response body. 'sha512': <str> - The SHA512 hash of the response body. } }
Returns: A information dictionary containing the results of the request. Return type: dict
-
-
class
synapse.axon.
AxonApi
[source]¶ Bases:
synapse.lib.cell.CellApi
,synapse.lib.share.Share
-
del_
(sha256)[source]¶ Remove the given bytes from the Axon by sha256.
Parameters: sha256 (bytes) – The sha256, in bytes, to remove from the Axon. Returns: True if the file is removed; false if the file is not present. Return type: boolean
-
dels
(sha256s)[source]¶ Given a list of sha256 hashes, delete the files from the Axon.
Parameters: sha256s (list) – A list of sha256 hashes in bytes form. Returns: A list of booleans, indicating if the file was deleted or not. Return type: list
-
get
(sha256)[source]¶ Get bytes of a file.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Examples
Get the bytes from an Axon and process them:
buf = b'' async for bytz in axon.get(sha256): buf =+ bytz await dostuff(buf)
Yields: bytes – Chunks of the file bytes. Raises: synapse.exc.NoSuchFile
– If the file does not exist.
-
has
(sha256)[source]¶ Check if the Axon has a file.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Returns: True if the Axon has the file; false otherwise. Return type: boolean
-
hashes
(offs)[source]¶ Yield hash rows for files that exist in the Axon in added order starting at an offset.
Parameters: offs (int) – The index offset. Yields: (int, (bytes, int)) – An index offset and the file SHA-256 and size.
-
history
(tick, tock=None)[source]¶ Yield hash rows for files that existing in the Axon after a given point in time.
Parameters: - tick (int) – The starting time (in epoch milliseconds).
- tock (int) – The ending time to stop iterating at (in epoch milliseconds).
Yields: (int, (bytes, int)) – A tuple containing time of the hash was added and the file SHA-256 and size.
-
iterMpkFile
(sha256)[source]¶ Yield items from a MsgPack (.mpk) file in the Axon.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Yields: Unpacked items from the bytes.
-
metrics
()[source]¶ Get the runtime metrics of the Axon.
Returns: A dictionary of runtime data about the Axon. Return type: dict
-
put
(byts)[source]¶ Store bytes in the Axon.
Parameters: byts (bytes) – The bytes to store in the Axon. Notes
This API should not be used for files greater than 128 MiB in size.
Returns: A tuple with the file size and sha256 hash of the bytes. Return type: tuple(int, bytes)
-
puts
(files)[source]¶ Store a set of bytes in the Axon.
Parameters: files (list) – A list of bytes to store in the Axon. Notes
This API should not be used for storing more than 128 MiB of bytes at once.
Returns: A list containing tuples of file size and sha256 hash of the saved bytes. Return type: list(tuple(int, bytes))
-
size
(sha256)[source]¶ Get the size of a file in the Axon.
Parameters: sha256 (bytes) – The sha256 hash of the file in bytes. Returns: The size of the file, in bytes. If not present, None is returned. Return type: int
-
upload
()[source]¶ Get an Upload object.
Notes
The UpLoad object should be used to manage uploads greater than 128 MiB in size.
Examples
Use an UpLoad object to upload a file to the Axon:
async with axonProxy.upload() as upfd: # Assumes bytesGenerator yields bytes async for byts in bytsgenerator(): upfd.write(byts) upfd.save()
Use a single UpLoad object to save multiple files:
async with axonProxy.upload() as upfd: for fp in file_paths: # Assumes bytesGenerator yields bytes async for byts in bytsgenerator(fp): upfd.write(byts) upfd.save()
Returns: An Upload manager object. Return type: UpLoadShare
-
wants
(sha256s)[source]¶ Get a list of sha256 values the axon does not have from a input list.
Parameters: sha256s (list) – A list of sha256 values as bytes. Returns: A list of bytes containing the sha256 hashes the Axon does not have. Return type: list
-
wget
(url, params=None, headers=None, json=None, body=None, method='GET', ssl=True, timeout=None)[source]¶ Stream a file download directly into the Axon.
Parameters: - url (str) – The URL to retrieve.
- params (dict) – Additional parameters to add to the URL.
- headers (dict) – Additional HTTP headers to add in the request.
- json – A JSON body which is included with the request.
- body – The body to be included in the request.
- method (str) – The HTTP method to use.
- ssl (bool) – Perform SSL verification.
- timeout (int) – The timeout of the request, in seconds.
Notes
The response body will be stored, regardless of the response code. The
ok
value in the reponse does not reflect that a status code, such as a 404, was encountered when retrieving the URL.The dictionary returned by this may contain the following values:
{ 'ok': <boolean> - False if there were exceptions retrieving the URL. 'url': <str> - The URL retrieved (which could have been redirected) 'code': <int> - The response code. 'mesg': <str> - An error message if there was an exception when retrieving the URL. 'headers': <dict> - The response headers as a dictionary. 'size': <int> - The size in bytes of the response body. 'hashes': { 'md5': <str> - The MD5 hash of the response body. 'sha1': <str> - The SHA1 hash of the response body. 'sha256': <str> - The SHA256 hash of the response body. 'sha512': <str> - The SHA512 hash of the response body. } }
Returns: A information dictionary containing the results of the request. Return type: dict
-
-
class
synapse.axon.
AxonHttpBySha256V1
(application: tornado.web.Application, request: tornado.httputil.HTTPServerRequest, **kwargs)[source]¶ Bases:
synapse.lib.httpapi.Handler
-
class
synapse.axon.
AxonHttpDelV1
(application: tornado.web.Application, request: tornado.httputil.HTTPServerRequest, **kwargs)[source]¶ Bases:
synapse.lib.httpapi.Handler
-
class
synapse.axon.
AxonHttpHasV1
(application: tornado.web.Application, request: tornado.httputil.HTTPServerRequest, **kwargs)[source]¶ Bases:
synapse.lib.httpapi.Handler
-
class
synapse.axon.
AxonHttpUploadV1
(application: tornado.web.Application, request: tornado.httputil.HTTPServerRequest, **kwargs)[source]¶ Bases:
synapse.lib.httpapi.StreamHandler
-
data_received
(chunk)[source]¶ Implement this method to handle streamed request data.
Requires the .stream_request_body decorator.
May be a coroutine for flow control.
-
on_connection_close
()[source]¶ Called in async handlers if the client closed the connection.
Override this to clean up resources associated with long-lived connections. Note that this method is called only if the connection was closed during asynchronous processing; if you need to do cleanup after every request override on_finish instead.
Proxies may keep a connection open for a time (perhaps indefinitely) after the client has gone away, so this method may not be called promptly after the end user closes their connection.
-
on_finish
()[source]¶ Called after the end of a request.
Override this method to perform cleanup, logging, etc. This method is a counterpart to prepare.
on_finish
may not produce any output, as it is called after the response has been sent to the client.
-
prepare
()[source]¶ Called at the beginning of a request before get/post/etc.
Override this method to perform common initialization regardless of the request method.
Asynchronous support: Use
async def
or decorate this method with .gen.coroutine to make it asynchronous. If this method returns anAwaitable
execution will not proceed until theAwaitable
is done.New in version 3.1: Asynchronous support.
-
-
class
synapse.axon.
UpLoad
[source]¶ Bases:
synapse.lib.base.Base
An object used to manage uploads to the Axon.
synapse.common module¶
-
synapse.common.
buid
(valu=None)[source]¶ A binary GUID like sequence of 32 bytes.
Parameters: - valu (object) – Optional, if provided, the hash of the msgpack
- form of the object is returned. This can be used to (encoded) –
- stable buids. (create) –
Notes
By default, this returns a random 32 byte value.
Returns: A 32 byte value. Return type: bytes
-
synapse.common.
chunks
(item, size)[source]¶ Divide an iterable into chunks.
Parameters: - item – Item to slice
- size (int) – Maximum chunk size.
Notes
This supports Generator objects and objects which support calling the __getitem__() method with a slice object.
Yields: Slices of the item containing up to “size” number of items.
-
synapse.common.
config
(conf, confdefs)[source]¶ Initialize a config dict using the given confdef tuples.
-
synapse.common.
ehex
(byts)[source]¶ Encode a bytes variable to a string using binascii.hexlify.
Parameters: byts (bytes) – Bytes to encode. Returns: A string representing the bytes. Return type: str
-
synapse.common.
envbool
(name, defval='false')[source]¶ Resolve an environment variable to a boolean value.
Parameters: - name (str) – Environment variable to resolve.
- defval (str) – Default string value to resolve as.
Notes
False values will be consider strings “0” or “false” after lower casing.
Returns: True if the envar is set, false if it is set to a false value. Return type: boolean
-
synapse.common.
gendir
(*paths, **opts)[source]¶ Return the absolute path of the joining of the arguments, creating a directory at the resulting path if one does not exist.
Performs home directory(~) and environment variable expansion.
Parameters: - *paths ([str,..]) – A list of path elements
- **opts – arguments as kwargs to os.makedirs
-
synapse.common.
genfile
(*paths)[source]¶ Create or open (for read/write) a file path join.
Parameters: *paths – A list of paths to join together to make the file. Notes
If the file already exists, the fd returned is opened in
r+b
mode. Otherwise, the fd is opened inw+b
mode.The file position is set to the start of the file. The user is responsible for truncating (
fd.truncate()
) if the existing file contents are not desired, or seeking to the end (fd.seek(0, 2)
) to append.Returns: A file-object which can be read/written too. Return type: io.BufferedRandom
-
synapse.common.
genpath
(*paths)[source]¶ Return an absolute path of the joining of the arguments as path elements
Performs home directory(
~
) and environment variable expansion on the joined pathParameters: *paths ([str,..]) – A list of path elements Note
All paths used by Synapse operations (i.e. everything but the data) shall use this function or one of its callers before storing as object properties.
-
synapse.common.
getexcfo
(e)[source]¶ Get an err tufo from an exception.
Parameters: e (Exception) – An Exception (or Exception subclass). Notes
This can be called outside of the context of an exception handler, however details such as file, line, function name and source may be missing.
Returns: Return type: ((str, dict))
-
synapse.common.
getfile
(*paths, **opts)[source]¶ Return a file at the path resulting from joining of the arguments, or None if the file does not exist.
Parameters: - *paths ([str,..]) – A list of path elements
- **opts – arguments as kwargs to io.open
Returns: A file-object which can be read/written too.
Return type: io.BufferedRandom
-
synapse.common.
guid
(valu=None)[source]¶ Get a 16 byte guid value.
By default, this is a random guid value.
Parameters: valu – Object used to construct the guid valu from. This must be able to be msgpack’d. Returns: 32 character, lowercase ascii string. Return type: str
-
synapse.common.
hugenum
(valu)[source]¶ Return a decimal.Decimal with proper precision for use as a synapse hugenum.
-
synapse.common.
intify
(x)[source]¶ Ensure ( or coerce ) a value into being an integer or None.
Parameters: x (obj) – An object to intify Returns: The int value ( or None ) Return type: (int)
-
synapse.common.
iterfd
(fd, size=10000000)[source]¶ Generator which yields bytes from a file descriptor.
Parameters: - fd (file) – A file-like object to read bytes from.
- size (int) – Size, in bytes, of the number of bytes to read from the
- at a given time. (fd) –
Notes
If the first read call on the file descriptor is a empty bytestring, that zero length bytestring will be yielded and the generator will then be exhuasted. This behavior is intended to allow the yielding of contents of a zero byte file.
Yields: bytes – Bytes from the file descriptor.
-
synapse.common.
listdir
(*paths, glob=None)[source]¶ List the (optionally glob filtered) full paths from a dir.
Parameters: - *paths ([str,..]) – A list of path elements
- glob (str) – An optional fnmatch glob str
-
synapse.common.
lockfile
(path)[source]¶ A file lock with-block helper.
Parameters: path (str) – A path to a lock file. Examples
Get the lock on a file and dostuff while having the lock:
path = '/hehe/haha.lock' with lockfile(path): dostuff()
Notes
This is curently based on fcntl.lockf(), and as such, it is purely advisory locking. If multiple processes are attempting to obtain a lock on the same file, this will block until the process which has the current lock releases it.
Yields: None
-
synapse.common.
merggenr
(genrs, cmprkey)[source]¶ Iterate multiple sorted async generators and yield their results in order.
Parameters: - genrs (Sequence[AsyncGenerator[T]]) – a sequence of async generator that each yield sorted items
- cmprkey (Callable[T, T, bool]) – a comparison function over the items yielded
Note
If the genrs yield increasing items, cmprkey should return True if the first parameter is less than the second parameter, e.g lambda x, y: x < y.
-
synapse.common.
merggenr2
(genrs, cmprkey=None, reverse=False)[source]¶ Optimized version of merggenr based on heapq.merge
-
synapse.common.
now
()[source]¶ Get the current epoch time in milliseconds.
This relies on time.time(), which is system-dependent in terms of resolution.
Examples
Get the current time and make a row for a Cortex:
tick = now() row = (someiden, 'foo:prop', 1, tick) core.addRows([row])
Returns: Epoch time in milliseconds. Return type: int
-
synapse.common.
reqdir
(*paths)[source]¶ Return the absolute path of the joining of the arguments, raising an exception if a directory does not exist at the resulting path.
Performs home directory(~) and environment variable expansion.
Parameters: - *paths ([str,..]) – A list of path elements
- **opts – arguments as kwargs to os.makedirs
-
synapse.common.
reqfile
(*paths, **opts)[source]¶ Return a file at the path resulting from joining of the arguments, raising an exception if the file does not exist.
Parameters: - *paths ([str,..]) – A list of path elements
- **opts – arguments as kwargs to io.open
Returns: A file-object which can be read/written too.
Return type: io.BufferedRandom
-
synapse.common.
reqjsonsafe
(item)[source]¶ Returns None if item is json serializable, otherwise raises an exception
-
synapse.common.
reqpath
(*paths)[source]¶ Return the absolute path of the joining of the arguments, raising an exception if a file doesn’t exist at resulting path
Parameters: *paths ([str,..]) – A list of path elements
-
synapse.common.
setlogging
(mlogger, defval=None)[source]¶ Configure synapse logging.
Parameters: - mlogger (logging.Logger) – Reference to a logging.Logger()
- defval (str) – Default log level. May be an integer.
Notes
This calls logging.basicConfig and should only be called once per process.
Returns: None
-
synapse.common.
spin
(genr)[source]¶ Crank through a generator but discard the yielded values.
Parameters: genr – Any generator or iterable valu. Notes
This generator is exhausted via the
collections.dequeue()
constructor with amaxlen=0
, which will quickly exhaust an iterator staying in C code as much as possible.Returns: None
-
synapse.common.
switchext
(*paths, ext)[source]¶ Return an absolute path of the joining of the arguments with the extension replaced.
If an extension does not exist, it will be added.
Parameters: - *paths ([str,..]) – A list of path elements
- ext (str) – A file extension (e.g. ‘.txt’). It should begin with a period.
-
synapse.common.
todo
(_todoname, *args, **kwargs)[source]¶ Construct and return a todo tuple of (name, args, kwargs).
Note: the odd name for the first parameter is to avoid collision with keys in kwargs.
-
synapse.common.
tuplify
(obj)[source]¶ Convert a nested set of python primitives into tupleized forms via msgpack.
-
synapse.common.
uhex
(text)[source]¶ Decode bytes to a string using binascii.unhexlify. :param text: Text to decode. :type text: str
Returns: The decoded bytes. Return type: bytes
-
synapse.common.
vertup
(vstr)[source]¶ Convert a version string to a tuple.
Example
ver = vertup(‘1.3.30’)
synapse.cortex module¶
-
class
synapse.cortex.
CoreApi
[source]¶ Bases:
synapse.lib.cell.CellApi
The CoreApi is exposed when connecting to a Cortex over Telepath.
Many CoreApi methods operate on packed nodes consisting of primitive data structures which can be serialized with msgpack/json.
An example of a packaged Node:
( (<form>, <valu>), { "props": { <name>: <valu>, ... }, "tags": { "foo": <time>, "foo.bar": <time>, }, })
-
addCronJob
(cdef)[source]¶ Add a cron job to the cortex
A cron job is a persistently-stored item that causes storm queries to be run in the future. The specification for the times that the queries run can be one-shot or recurring.
Parameters: - query (str) – The storm query to execute in the future
- reqs (Union[Dict[str, Union[int, List[int]]], List[Dict[..]]]) – Either a dict of the fixed time fields or a list of such dicts. The keys are in the set (‘year’, ‘month’, ‘dayofmonth’, ‘dayofweek’, ‘hour’, ‘minute’. The values must be positive integers, except for the key of ‘dayofmonth’ in which it may also be a negative integer which represents the number of days from the end of the month with -1 representing the last day of the month. All values may also be lists of valid values.
- incunit (Optional[str]) – A member of the same set as above, with an additional member ‘day’. If is None (default), then the appointment is one-shot and will not recur.
- incvals (Union[int, List[int]) – A integer or a list of integers of the number of units
- Returns (bytes):
- An iden that can be used to later modify, query, and delete the job.
Notes
reqs must have fields present or incunit must not be None (or both) The incunit if not None it must be larger in unit size than all the keys in all reqs elements.
-
addForm
(formname, basetype, typeopts, typeinfo)[source]¶ Add an extended form to the data model.
Extended forms must begin with _
-
addFormProp
(form, prop, tdef, info)[source]¶ Add an extended property to the given form.
Extended properties must begin with _
-
addNodeTag
(iden, tag, valu=(None, None))[source]¶ Add a tag to a node specified by iden.
Parameters: - iden (str) – A hex encoded node BUID.
- tag (str) – A tag string.
- valu (tuple) – A time interval tuple or (None, None).
-
addNodes
(nodes)[source]¶ Add a list of packed nodes to the cortex.
Parameters: nodes (list) – [ ( (form, valu), {‘props’:{}, ‘tags’:{}}), … ] Yields: (tuple) – Packed node tuples ((form,valu), {‘props’: {}, ‘tags’:{}}) Deprecated in 2.0.0
-
addUnivProp
(name, tdef, info)[source]¶ Add an extended universal property.
Extended properties must begin with _
-
callStorm
(text, opts=None)[source]¶ Return the value expressed in a return() statement within storm.
-
count
(text, opts=None)[source]¶ Count the number of nodes which result from a storm query.
Parameters: - text (str) – Storm query text.
- opts (dict) – Storm query options.
Returns: The number of nodes resulting from the query.
Return type: (int)
-
delCronJob
(iden)[source]¶ Delete a cron job
Parameters: iden (bytes) – The iden of the cron job to be deleted
-
delNodeTag
(iden, tag)[source]¶ Delete a tag from the node specified by iden. Deprecated in 2.0.0.
Parameters: - iden (str) – A hex encoded node BUID.
- tag (str) – A tag string.
-
disableCronJob
(iden)[source]¶ Enable a cron job
Parameters: iden (bytes) – The iden of the cron job to be changed
-
enableCronJob
(iden)[source]¶ Enable a cron job
Parameters: iden (bytes) – The iden of the cron job to be changed
-
eval
(text, opts=None)[source]¶ Evaluate a storm query and yield packed nodes.
NOTE: This API is deprecated as of 2.0.0 and will be removed in 3.0.0
-
exportStorm
(text, opts=None)[source]¶ Execute a storm query and package nodes for export/import.
- NOTE: This API yields nodes after an initial complete lift
- in order to limit exported edges.
-
getCoreInfo
()[source]¶ Return static generic information about the cortex including model definition
-
getCoreInfoV2
()[source]¶ Return static generic information about the cortex including model definition
-
getFeedFuncs
()[source]¶ Get a list of Cortex feed functions.
Notes
Each feed dictinonary has the name of the feed function, the full docstring for the feed function, and the first line of the docstring broken out in their own keys for easy use.
Returns: A tuple of dictionaries. Return type: tuple
-
getModelDict
()[source]¶ Return a dictionary which describes the data model.
Returns: A model description dictionary. Return type: (dict)
-
getPropNorm
(prop, valu)[source]¶ Get the normalized property value based on the Cortex data model.
Parameters: - prop (str) – The property to normalize.
- valu – The value to normalize.
Returns: A two item tuple, containing the normed value and the info dictionary.
Return type: (tuple)
Raises: s_exc.NoSuchProp
– If the prop does not exist.s_exc.BadTypeValu
– If the value fails to normalize.
-
getProvStack
(iden: str)[source]¶ Return the provenance stack associated with the given iden.
Parameters: iden (str) – the iden of the provenance stack Note: the iden appears on each splice entry as the ‘prov’ property
-
getTypeNorm
(name, valu)[source]¶ Get the normalized type value based on the Cortex data model.
Parameters: - name (str) – The type to normalize.
- valu – The value to normalize.
Returns: A two item tuple, containing the normed value and the info dictionary.
Return type: (tuple)
Raises: s_exc.NoSuchType
– If the type does not exist.s_exc.BadTypeValu
– If the value fails to normalize.
-
iterFormRows
(layriden, form, stortype=None, startvalu=None)[source]¶ Yields buid, valu tuples of nodes of a single form, optionally (re)starting at startvalue
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- form (str) – A form name
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
iterPropRows
(layriden, form, prop, stortype=None, startvalu=None)[source]¶ Yields buid, valu tuples of nodes with a particular secondary property, optionally (re)starting at startvalue
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- form (str) – A form name.
- prop (str) – A secondary property name.
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
iterTagPropRows
(layriden, tag, prop, form=None, stortype=None, startvalu=None)[source]¶ Yields (buid, valu) that match a tag:prop, optionally (re)starting at startvalu.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- tag (str) – tag name
- prop (str) – prop name
- form (Optional[str]) – optional form name
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
iterTagRows
(layriden, tag, form=None, starttupl=None)[source]¶ Yields (buid, (valu, form)) values that match a tag and optional form, optionally (re)starting at starttupl.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- tag (str) – the tag to match
- form (Optional[str]) – if present, only yields buids of nodes that match the form.
- starttupl (Optional[Tuple[buid, form]]) – if present, (re)starts the stream of values there.
Returns: AsyncIterator[Tuple(buid, (valu, form))]
Note
This yields (buid, (tagvalu, form)) instead of just buid, valu in order to allow resuming an interrupted call by feeding the last value retrieved into starttupl
-
iterUnivRows
(layriden, prop, stortype=None, startvalu=None)[source]¶ Yields buid, valu tuples of nodes with a particular universal property, optionally (re)starting at startvalue
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- prop (str) – A universal property name.
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
provStacks
(offs, size)[source]¶ Return stream of (iden, provenance stack) tuples at the given offset.
-
reqValidStorm
(text, opts=None)[source]¶ Parse a Storm query to validate it.
Parameters: - text (str) – The text of the Storm query to parse.
- opts (dict) – A Storm options dictionary.
Returns: If the query is valid.
Return type: True
Raises: BadSyntaxError
– If the query is invalid.
-
spliceHistory
()[source]¶ Yield splices backwards from the end of the splice log.
Will only return the user’s own splices unless they are an admin.
-
splices
(offs=None, size=None, layriden=None)[source]¶ Return the list of splices at the given offset.
-
splicesBack
(offs=None, size=None)[source]¶ Return the list of splices backwards from the given offset.
-
storm
(text, opts=None)[source]¶ Evaluate a storm query and yield result messages.
Yields: ((str,dict)) – Storm messages.
-
syncLayerNodeEdits
(offs, layriden=None, wait=True)[source]¶ Yield (indx, mesg) nodeedit sets for the given layer beginning at offset.
Once caught up, this API will begin yielding nodeedits in real-time. The generator will only terminate on network disconnect or if the consumer falls behind the max window size of 10,000 nodeedit messages.
-
-
class
synapse.cortex.
Cortex
[source]¶ Bases:
synapse.lib.cell.Cell
A Cortex implements the synapse hypergraph.
The bulk of the Cortex API lives on the Snap() object which can be obtained by calling Cortex.snap() in a with block. This allows callers to manage transaction boundaries explicitly and dramatically increases performance.
-
addCronJob
(cdef)[source]¶ Add a cron job to the cortex. Convenience wrapper around agenda.add
A cron job is a persistently-stored item that causes storm queries to be run in the future. The specification for the times that the queries run can be one-shot or recurring.
Parameters: - query (str) – The storm query to execute in the future
- reqs (Union[Dict[str, Union[int, List[int]]], List[Dict[..]]]) – Either a dict of the fixed time fields or a list of such dicts. The keys are in the set (‘year’, ‘month’, ‘dayofmonth’, ‘dayofweek’, ‘hour’, ‘minute’. The values must be positive integers, except for the key of ‘dayofmonth’ in which it may also be a negative integer which represents the number of days from the end of the month with -1 representing the last day of the month. All values may also be lists of valid values.
- incunit (Optional[str]) – A member of the same set as above, with an additional member ‘day’. If is None (default), then the appointment is one-shot and will not recur.
- incvals (Union[int, List[int]) – A integer or a list of integers of the number of units
- Returns (bytes):
- An iden that can be used to later modify, query, and delete the job.
Notes
reqs must have fields present or incunit must not be None (or both) The incunit if not None it must be larger in unit size than all the keys in all reqs elements. Non-recurring jobs may also have a req of ‘now’ which will cause the job to also execute immediately.
-
addFeedData
(name, items, *, viewiden=None)[source]¶ Add data using a feed/parser function.
Parameters: - name (str) – The name of the feed record format.
- items (list) – A list of items to ingest.
- iden (str) – The iden of a view to use. If a view is not specified, the default view is used.
-
addLayer
(ldef=None, nexs=True)[source]¶ Add a Layer to the cortex.
Parameters: - ldef (Optional[Dict]) – layer configuration
- nexs (bool) – whether to record a nexus transaction (internal use only)
-
addNodeTag
(user, iden, tag, valu=(None, None))[source]¶ Add a tag to a node specified by iden.
Parameters: - iden (str) – A hex encoded node BUID.
- tag (str) – A tag string.
- valu (tuple) – A time interval tuple or (None, None).
-
addNodes
(nodedefs, view=None)[source]¶ Quickly add/modify a list of nodes from node definition tuples. This API is the simplest/fastest way to add nodes, set node props, and add tags to nodes remotely.
Parameters: nodedefs (list) – A list of node definition tuples. See below. A node definition tuple is defined as:
( (form, valu), {‘props’:{}, ‘tags’:{})The “props” or “tags” keys may be omitted.
-
addRuntLift
(prop, func)[source]¶ Register a runt lift helper for a given prop.
Parameters: - prop (str) – Full property name for the prop to register the helper for.
- func –
Returns: None.
Return type: None
-
addStormPkg
(pkgdef)[source]¶ Add the given storm package to the cortex.
This will store the package for future use.
-
cloneLayer
(iden, ldef=None)[source]¶ Make a copy of a Layer in the cortex.
Parameters: - iden (str) – Layer iden to clone
- ldef (Optional[Dict]) – Layer configuration overrides
Note
This should only be called with a reasonably static Cortex due to possible races.
-
confbase
= {'aha:admin': {'description': 'An AHA client certificate CN to register as a local admin user.', 'type': 'string'}, 'aha:leader': {'description': 'The AHA service name to claim as the active instance of a storm service.', 'type': 'string'}, 'aha:name': {'description': 'The name of the cell service in the aha service registry.', 'type': 'string'}, 'aha:network': {'description': 'The AHA service network. This makes aha:name/aha:leader relative names.', 'type': 'string'}, 'aha:registry': {'description': 'The telepath URL of the aha service registry.', 'items': {'type': 'string'}, 'type': ['string', 'array']}, 'aha:svcinfo': {'description': 'An AHA svcinfo object. If set, this overrides self discovered Aha service information.', 'properties': {'urlinfo': {'properties': {'host': {'type': 'string'}, 'port': {'type': 'integer'}, 'schema': {'type': 'string'}}, 'required': ('host', 'port', 'scheme'), 'type': 'object'}}, 'required': ('urlinfo',), 'type': 'object'}, 'auth:anon': {'description': 'Allow anonymous telepath access by mapping to the given user name.', 'type': 'string'}, 'auth:conf': {'description': 'Extended configuration to be used by an alternate auth constructor.', 'hideconf': True, 'type': 'object'}, 'auth:ctor': {'description': 'Allow the construction of the cell auth object to be hooked at runtime.', 'hideconf': True, 'type': 'string'}, 'auth:passwd': {'description': 'Set to <passwd> (local only) to bootstrap the root user password.', 'type': 'string'}, 'backup:dir': {'description': 'A directory outside the service directory where backups will be saved.', 'type': 'string'}, 'cell:guid': {'description': 'An optional hard-coded GUID to store as the permanent GUID for the cell.', 'type': 'string'}, 'dmon:listen': {'description': 'A config-driven way to specify the telepath bind URL.', 'type': ['string', 'null']}, 'https:port': {'description': 'A config-driven way to specify the HTTPS port.', 'type': ['integer', 'null']}, 'mirror': {'description': 'A telepath URL for our upstream mirror (we must be a backup!).', 'type': 'string'}, 'nexslog:async': {'default': False, 'description': '(Experimental) Map the nexus log LMDB instance with map_async=True.', 'type': 'boolean'}, 'nexslog:en': {'default': True, 'description': 'Record all changes to the cell. Required for mirroring (on both sides).', 'type': 'boolean'}}¶
-
confdefs
= {'axon': {'description': 'A telepath URL for a remote axon.', 'type': 'string'}, 'cron:enable': {'default': True, 'description': 'Enable cron jobs running.', 'type': 'boolean'}, 'http:proxy': {'description': 'An aiohttp-socks compatible proxy URL to use storm HTTP API.', 'type': 'string'}, 'layer:lmdb:map_async': {'default': True, 'description': 'Set the default lmdb:map_async value in LMDB layers.', 'type': 'boolean'}, 'layers:lockmemory': {'default': False, 'description': 'Should new layers lock memory for performance by default.', 'type': 'boolean'}, 'layers:logedits': {'default': True, 'description': 'Whether nodeedits are logged in each layer.', 'type': 'boolean'}, 'max:nodes': {'description': 'Maximum number of nodes which are allowed to be stored in a Cortex.', 'hidecmdl': True, 'minimum': 1, 'type': 'integer'}, 'modules': {'default': [], 'description': 'A list of module classes to load.', 'type': 'array'}, 'provenance:en': {'default': False, 'description': 'Enable provenance tracking for all writes.', 'type': 'boolean'}, 'storm:log': {'default': False, 'description': 'Log storm queries via system logger.', 'type': 'boolean'}, 'storm:log:level': {'default': 30, 'description': 'Logging log level to emit storm logs at.', 'type': 'integer'}, 'trigger:enable': {'default': True, 'description': 'Enable triggers running.', 'type': 'boolean'}}¶
-
delCronJob
(iden)[source]¶ Delete a cron job
Parameters: iden (bytes) – The iden of the cron job to be deleted
-
delNodeTag
(user, iden, tag)[source]¶ Delete a tag from the node specified by iden.
Parameters: - iden (str) – A hex encoded node BUID.
- tag (str) – A tag string.
-
delTagModel
(tagname)[source]¶ Delete all the model specification properties for a tag.
Parameters: tagname (str) – The name of the tag.
-
disableCronJob
(iden)[source]¶ Enable a cron job
Parameters: iden (bytes) – The iden of the cron job to be changed
-
enableCronJob
(iden)[source]¶ Enable a cron job
Parameters: iden (bytes) – The iden of the cron job to be changed
-
eval
(text, opts=None)[source]¶ Evaluate a storm query and yield packed nodes.
NOTE: This API is deprecated as of 2.0.0 and will be removed in 3.0.0
-
getLayer
(iden=None)[source]¶ Get a Layer object.
Parameters: iden (str) – The layer iden to retrieve. Returns: A Layer object. Return type: Layer
-
getPropNorm
(prop, valu)[source]¶ Get the normalized property value based on the Cortex data model.
Parameters: - prop (str) – The property to normalize.
- valu – The value to normalize.
Returns: A two item tuple, containing the normed value and the info dictionary.
Return type: (tuple)
Raises: s_exc.NoSuchProp
– If the prop does not exist.s_exc.BadTypeValu
– If the value fails to normalize.
-
getStormDocs
()[source]¶ Get a struct containing the Storm Types documentation.
Returns: A Dictionary of storm documentation information. Return type: dict
-
getTagModel
(tagname)[source]¶ Retrieve the tag model specification for a tag.
Returns: The tag model specification or None. Return type: (dict)
-
getTypeNorm
(name, valu)[source]¶ Get the normalized type value based on the Cortex data model.
Parameters: - name (str) – The type to normalize.
- valu – The value to normalize.
Returns: A two item tuple, containing the normed value and the info dictionary.
Return type: (tuple)
Raises: s_exc.NoSuchType
– If the type does not exist.s_exc.BadTypeValu
– If the value fails to normalize.
-
getView
(iden=None, user=None)[source]¶ Get a View object.
Parameters: iden (str) – The View iden to retrieve. Returns: A View object. Return type: View
-
hiveapi
¶ alias of
synapse.lib.hive.HiveApi
-
isTagValid
(tagname)[source]¶ Check if a tag name is valid according to tag model regular expressions.
Returns: True if the tag is valid. Return type: (bool)
-
iterFormRows
(layriden, form, stortype=None, startvalu=None)[source]¶ Yields buid, valu tuples of nodes of a single form, optionally (re)starting at startvalu.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- form (str) – A form name.
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
iterPropRows
(layriden, form, prop, stortype=None, startvalu=None)[source]¶ Yields buid, valu tuples of nodes with a particular secondary property, optionally (re)starting at startvalu.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- form (str) – A form name.
- prop (str) – A universal property name.
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
iterTagPropRows
(layriden, tag, prop, form=None, stortype=None, startvalu=None)[source]¶ Yields (buid, valu) that match a tag:prop, optionally (re)starting at startvalu.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- tag (str) – tag name
- prop (str) – prop name
- form (Optional[str]) – optional form name
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
iterTagRows
(layriden, tag, form=None, starttupl=None)[source]¶ Yields (buid, (valu, form)) values that match a tag and optional form, optionally (re)starting at starttupl.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- tag (str) – the tag to match
- form (Optional[str]) – if present, only yields buids of nodes that match the form.
- starttupl (Optional[Tuple[buid, form]]) – if present, (re)starts the stream of values there.
Returns: AsyncIterator[Tuple(buid, (valu, form))]
Note
This yields (buid, (tagvalu, form)) instead of just buid, valu in order to allow resuming an interrupted call by feeding the last value retrieved into starttupl
-
iterUnivRows
(layriden, prop, stortype=None, startvalu=None)[source]¶ Yields buid, valu tuples of nodes with a particular universal property, optionally (re)starting at startvalu.
Parameters: - layriden (str) – Iden of the layer to retrieve the nodes
- prop (str) – A universal property name.
- stortype (Optional[int]) – a STOR_TYPE_* integer representing the type of form:prop
- startvalu (Any) – The value to start at. May only be not None if stortype is not None.
Returns: AsyncIterator[Tuple(buid, valu)]
-
layerapi
¶ alias of
synapse.lib.layer.LayerApi
-
classmethod
layrctor
(*args, **kwargs)¶
-
listTagModel
()[source]¶ Retrieve a list of the tag model specifications.
Returns: A list of tag model specification tuples. Return type: ([(str, dict), ..])
-
loadCoreModule
(ctor, conf=None)[source]¶ Load a single cortex module with the given ctor and conf.
Parameters: - ctor (str) – The python module class path
- conf (dict) – Config dictionary for the module
-
loadStormPkg
(pkgdef)[source]¶ Load a storm package into the storm library for this cortex.
NOTE: This will not persist the package (allowing service dynamism).
-
offTagAdd
(name, func)[source]¶ Unregister a callback for tag addition.
Parameters: - name (str) – The name of the tag or tag glob.
- func (function) – The callback func(node, tagname, tagval).
-
offTagDel
(name, func)[source]¶ Unregister a callback for tag deletion.
Parameters: - name (str) – The name of the tag or tag glob.
- func (function) – The callback func(node, tagname, tagval).
-
onTagAdd
(name, func)[source]¶ Register a callback for tag addition.
Parameters: - name (str) – The name of the tag or tag glob.
- func (function) – The callback func(node, tagname, tagval).
-
onTagDel
(name, func)[source]¶ Register a callback for tag deletion.
Parameters: - name (str) – The name of the tag or tag glob.
- func (function) – The callback func(node, tagname, tagval).
-
popTagModel
(tagname, name)[source]¶ Pop a property from the model specification of a tag.
Parameters: - tagname (str) – The name of the tag.
- name (str) – The name of the specification property.
Returns: The current value of the property.
Return type: (object)
-
reqValidStorm
(text, opts=None)[source]¶ Parse a storm query to validate it.
Parameters: - text (str) – The text of the Storm query to parse.
- opts (dict) – A Storm options dictionary.
Returns: If the query is valid.
Return type: True
Raises: BadSyntaxError
– If the query is invalid.
-
runRuntLift
(full, valu=None, cmpr=None)[source]¶ Execute a runt lift function.
Parameters: - full (str) – Property to lift by.
- valu –
- cmpr –
Returns: - Yields bytes, list tuples where the list contains a series of
key/value pairs which are used to construct a Node object.
Return type: bytes, list
-
setStormCmd
(cdef)[source]¶ Set pure storm command definition.
Parameters: cdef (dict) – A Pure Stormcmd definition dictionary. Notes
The definition dictionary is formatted like the following:
{ 'name': <name>, 'cmdargs': [ (<name>, <opts>), ] 'cmdconf': { <str>: <valu> }, 'storm': <text>, }
-
setStormSvcEvents
(iden, edef)[source]¶ Set the event callbacks for a storm service. Extends the sdef dict.
Parameters: - iden (str) – The service iden.
- edef (dict) – The events definition.
Notes
The edef is formatted like the following:
{ <name> : { 'storm': <storm> } }
where
name
is one of the following items:add
Run the given storm ‘before the service is first added (a la service.add), but not on a reconnect.del
Run the given storm after the service is removed (a la service.del), but not on a disconnect.Returns: An updated storm service definition dictionary. Return type: dict
-
setTagModel
(tagname, name, valu)[source]¶ Set a model specification property for a tag.
Parameters: - tagname (str) – The name of the tag.
- name (str) – The name of the property.
- valu (object) – The value of the property.
- Tag Model Properties:
- regex - A list of None or regular expression strings to match each tag level. prune - A number that determines how many levels of pruning are desired.
Examples
await core.setTagModel(“cno.cve”, “regex”, (None, None, “[0-9]{4}”, “[0-9]{5}”))
-
setViewLayers
(layers, iden=None)[source]¶ Parameters: - layers ([str]) – A top-down list of of layer guids
- iden (str) – The view iden (defaults to default view).
-
snap
(user=None, view=None)[source]¶ Return a transaction object for the default view.
Parameters: - user (str) – The user to get the snap for.
- view (View) – View object to use when making the snap.
Notes
This must be used as an asynchronous context manager.
Returns: A Snap object for the view. Return type: s_snap.Snap
-
spliceHistory
(user)[source]¶ Yield splices backwards from the end of the nodeedit log.
Will only return user’s own splices unless they are an admin.
-
syncIndexEvents
(matchdef, offsdict=None, wait=True)[source]¶ Yield (offs, layriden, <STYPE>, <item>) tuples from the nodeedit logs of all layers starting from the given nexus/layer offset (they are synchronized). Only edits that match the filter in matchdef will be yielded, plus EDIT_PROGRESS (see layer.syncIndexEvents) messages.
The format of the 4th element of the tuple depends on STYPE. STYPE is one of the following constants:
SYNC_LAYR_ADD: item is an empty tuple () SYNC_LAYR_DEL: item is an empty tuple () SYNC_NODEEDIT: item is (buid, form, ETYPE, VALS, META)) or (None, None, s_layer.EDIT_PROGRESS, (), ())For edits in the past, events are yielded in offset order across all layers. For current data (wait=True), events across different layers may be emitted slightly out of offset order.
Note
Will not yield any values from layers created with logedits disabled
Parameters: - matchdef (Dict[str, Sequence[str]]) – a dict describing which events are yielded. See layer.syncIndexEvents for matchdef specification.
- offsdict (Optional(Dict[str,int])) – starting nexus/editlog offset by layer iden. Defaults to 0 for unspecified layers or if offsdict is None.
- wait (bool) – whether to pend and stream value until this layer is fini’d
-
syncLayerNodeEdits
(iden, offs, wait=True)[source]¶ Yield (offs, mesg) tuples for nodeedits in a layer.
-
syncLayersEvents
(offsdict=None, wait=True)[source]¶ Yield (offs, layriden, STYP, item, meta) tuples for nodeedits for all layers, interspersed with add/del layer messages.
- STYP is one of the following constants:
- SYNC_NODEEDITS: item is a nodeedits (buid, form, edits) SYNC_LAYR_ADD: A layer was added (item and meta are empty) SYNC_LAYR_DEL: A layer was deleted (item and meta are empty)
Parameters: - offsdict (Optional(Dict[str,int])) – starting nexus/editlog offset by layer iden. Defaults to 0 for unspecified layers or if offsdict is None.
- wait (bool) – whether to pend and stream value until this layer is fini’d
-
updateCronJob
(iden, query)[source]¶ Change an existing cron job’s query
Parameters: iden (bytes) – The iden of the cron job to be changed
-
viewapi
¶ alias of
synapse.lib.view.ViewApi
-
classmethod
viewctor
(*args, **kwargs)¶
-
-
synapse.cortex.
getTempCortex
(mods=None)[source]¶ Get a proxy to a cortex backed by a temporary directory.
Parameters: mods (list) – A list of modules which are loaded into the cortex. Notes
The cortex and temporary directory are town down on exit. This should only be called from synchronous code.
Returns: Proxy to the cortex.
-
synapse.cortex.
logger
= <Logger synapse.cortex (WARNING)>¶ A Cortex implements the synapse hypergraph object.
synapse.cryotank module¶
-
class
synapse.cryotank.
CryoApi
[source]¶ Bases:
synapse.lib.cell.CellApi
The CryoCell API as seen by a telepath proxy.
This is the API to reference for remote CryoCell use.
-
class
synapse.cryotank.
CryoCell
[source]¶ Bases:
synapse.lib.cell.Cell
-
init
(name, conf=None)[source]¶ Generate a new CryoTank with a given name or get an reference to an existing CryoTank.
Parameters: name (str) – Name of the CryoTank. Returns: A CryoTank instance. Return type: CryoTank
-
-
class
synapse.cryotank.
CryoTank
[source]¶ Bases:
synapse.lib.base.Base
A CryoTank implements a stream of structured data.
-
info
()[source]¶ Returns information about the CryoTank instance.
Returns: A dict containing items and metrics indexes. Return type: dict
-
metrics
(offs, size=None)[source]¶ Yield metrics rows starting at offset.
Parameters: - offs (int) – The index offset.
- size (int) – The maximum number of records to yield.
Yields: ((int, dict)) – An index offset, info tuple for metrics.
-
puts
(items, seqn=None)[source]¶ Add the structured data from items to the CryoTank.
Parameters: - items (list) – A list of objects to store in the CryoTank.
- seqn (iden, offs) – An iden / offset pair to record.
Returns: The ending offset of the items or seqn.
Return type: int
-
synapse.daemon module¶
-
class
synapse.daemon.
AsyncGenr
[source]¶ Bases:
synapse.lib.share.Share
-
typename
= 'genr'¶
-
-
class
synapse.daemon.
Daemon
[source]¶ Bases:
synapse.lib.base.Base
-
listen
(url, **opts)[source]¶ Bind and listen on the given host/port with possible SSL.
Parameters: - host (str) – A hostname or IP address.
- port (int) – The TCP port to bind.
Share an object via the telepath protocol.
Parameters: - name (str) – Name of the shared object
- item (object) – The object to share over telepath.
-
-
class
synapse.daemon.
Genr
[source]¶ Bases:
synapse.lib.share.Share
-
typename
= 'genr'¶
-
-
class
synapse.daemon.
Sess
[source]¶ Bases:
synapse.lib.base.Base
synapse.datamodel module¶
An API to assist with the creation and enforcement of cortex data models.
-
class
synapse.datamodel.
Form
(modl, name, info)[source]¶ Bases:
object
The Form class implements data model logic for a node form.
-
offAdd
(func)[source]¶ Unregister a callback for tag addition.
Parameters: - name (str) – The name of the tag.
- func (function) – The callback func(node)
-
onAdd
(func)[source]¶ Add a callback for adding this type of node.
The callback is executed after node construction.
Parameters: func (function) – A callback func(node) - def func(xact, node):
- dostuff()
-
prop
(name: str)[source]¶ Return a secondary property for this form by relative prop name.
Parameters: name (str) – The relative property name. Returns: The property or None. Return type: (synapse.datamodel.Prop)
-
-
class
synapse.datamodel.
Model
[source]¶ Bases:
object
The data model used by a Cortex hypergraph.
-
addDataModels
(mods)[source]¶ Add a list of (name, mdef) tuples.
A model definition (mdef) is structured as follows:
{ "ctors":( ('name', 'class.path.ctor', {}, {'doc': 'The foo thing.'}), ), "types":( ('name', ('basetype', {typeopts}), {info}), ), "forms":( (formname, (typename, typeopts), {info}, ( (propname, (typename, typeopts), {info}), )), ), "univs":( (propname, (typename, typeopts), {info}), ) "tagprops":( (tagpropname, (typename, typeopts), {info}), ) "interfaces":( (ifacename, { 'props': ((propname, (typename, typeopts), {info}),), 'doc': docstr, 'interfaces': (ifacename,) }), ) }
Parameters: mods (list) – The list of tuples. Returns: None
-
-
class
synapse.datamodel.
Prop
(modl, form, name, typedef, info)[source]¶ Bases:
object
The Prop class represents a property defined within the data model.
-
onDel
(func)[source]¶ Add a callback for deleting this property.
The callback is executed after the property is deleted.
Parameters: func (function) – A prop del callback. The callback is called within the current transaction, with the node, and the old property value (or None).
- def func(node, oldv):
- dostuff()
-
onSet
(func)[source]¶ Add a callback for setting this property.
The callback is executed after the property is set.
Parameters: func (function) – A prop set callback. The callback is called within the current transaction, with the node, and the old property value (or None).
- def func(node, oldv):
- dostuff()
-
wasSet
(node, oldv)[source]¶ Fire the onset() handlers for this property.
Parameters: - node (synapse.lib.node.Node) – The node whose property was set.
- oldv (obj) – The previous value of the property.
-
synapse.exc module¶
Exceptions used by synapse, all inheriting from SynErr
-
exception
synapse.exc.
AuthDeny
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BackupAlreadyRunning
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Only one backup may be running at a time
-
exception
synapse.exc.
BadArg
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Improper function arguments
-
exception
synapse.exc.
BadCast
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadCertHost
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadCmdName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadCmprType
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Attempt to compare two incomparable values
-
exception
synapse.exc.
BadCmprValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadConfValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
The configuration value provided is not valid.
This should contain the config name, valu and mesg.
-
exception
synapse.exc.
BadCoreStore
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
The storage layer has encountered an error
-
exception
synapse.exc.
BadCtorType
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadEccExchange
(*args, **info)[source]¶ Bases:
synapse.exc.CryptoErr
Raised when there is an issue doing a ECC Key Exchange
-
exception
synapse.exc.
BadFileExt
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadFormDef
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadHivePath
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadIndxValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadJsonText
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadLiftValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadMesgFormat
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadMesgVers
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadOperArg
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Improper storm function arguments
-
exception
synapse.exc.
BadOptValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadPkgDef
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadPropDef
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadStorageVersion
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Stored persistent data is incompatible with running software
-
exception
synapse.exc.
BadSyntax
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadTag
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadTime
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadTypeDef
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadTypeValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadUrl
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
BadVersion
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Generic Bad Version exception.
-
exception
synapse.exc.
CantDelCmd
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantDelForm
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantDelNode
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantDelProp
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantDelType
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantDelUniv
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantMergeView
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CantRevLayer
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
CliFini
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Raised when the CLI is to exit.
-
exception
synapse.exc.
CryptoErr
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Raised when there is a synapse.lib.crypto error.
-
exception
synapse.exc.
DataAlreadyExists
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Cannot copy data to a location that already contains data
-
exception
synapse.exc.
DbOutOfSpace
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DmonSpawn
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Raised by a dispatched telepath method that has answered the call using a spawned process. ( control flow that is compatible with aborting standard calls, generators, and async generators ).
-
exception
synapse.exc.
DupFileName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupFormName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupIden
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupIndx
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupPropName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupRoleName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupStormSvc
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupTagPropName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
DupUserName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
FeatureNotSupported
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
FileExists
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
HitLimit
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
InconsistentStorage
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Stored persistent data is inconsistent
-
exception
synapse.exc.
IsDeprLocked
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
IsFini
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
IsReadOnly
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
IsRuntForm
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
LayerInUse
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
LinkErr
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
LinkShutDown
(*args, **info)[source]¶ Bases:
synapse.exc.LinkErr
-
exception
synapse.exc.
ModAlreadyLoaded
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
MustBeJsonSafe
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NeedConfValu
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoCertKey
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Raised when a Cert object requires a RSA Private Key to perform an operation and the key is not present.
-
exception
synapse.exc.
NoSuchAbrv
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchAct
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchAuthGate
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchCert
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchCmd
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchCmpr
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchCond
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchCtor
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchDecoder
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchDir
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchDyn
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchEncoder
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchFile
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchForm
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchFunc
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchIden
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchImpl
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchIndx
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchLayer
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchLift
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchMeth
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchName
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchObj
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchOpt
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchPath
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchPivot
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchPkg
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchProp
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchRole
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchStor
(name)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchStormSvc
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchTagProp
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchType
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchUniv
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchUser
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchVar
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NoSuchView
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NotANumberCompared
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NotMsgpackSafe
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
NotReady
(*args, **info)[source]¶ Bases:
synapse.exc.Retry
-
exception
synapse.exc.
ParserExit
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Raised by synapse.lib.cmd.Parser on Parser exit()
-
exception
synapse.exc.
PathExists
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
ReadOnlyLayer
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
ReadOnlyProp
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
RecursionLimitHit
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
Retry
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
SchemaViolation
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
SlabAlreadyOpen
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
SpawnExit
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
StepTimeout
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
Raised when a TestStep.wait() call times out.
-
exception
synapse.exc.
StormRuntimeError
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
SynErr
(*args, **info)[source]¶ Bases:
Exception
-
exception
synapse.exc.
TeleRedir
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
-
exception
synapse.exc.
TimeOut
(*args, **info)[source]¶ Bases:
synapse.exc.SynErr
synapse.glob module¶
-
synapse.glob.
sync
(coro, timeout=None)[source]¶ Schedule a coroutine to run on the global loop and return it’s result.
Parameters: coro (coroutine) – The coroutine instance. Notes
This API is thread safe and should only be called by non-loop threads.
-
synapse.glob.
synchelp
(f)[source]¶ The synchelp decorator allows the transparent execution of a coroutine using the global loop from a thread other than the event loop. In both use cases, teh actual work is done by the global event loop.
Examples
Use as a decorator:
@s_glob.synchelp async def stuff(x, y): await dostuff()
Calling the stuff function as regular async code using the standard await syntax:
valu = await stuff(x, y)
Calling the stuff function as regular sync code outside of the event loop thread:
valu = stuff(x, y)
synapse.mindmeld module¶
synapse.telepath module¶
An RMI framework for synapse.
-
class
synapse.telepath.
Aware
[source]¶ Bases:
object
The telepath.Aware mixin allows shared objects to handle individual links managed by the Daemon.
-
getTeleApi
(link, mesg, path)[source]¶ Return a shared object for this link. :param link: A network link. :type link: synapse.lib.link.Link :param mesg: The tele:syn handshake message. :type mesg: (str,dict)
-
-
class
synapse.telepath.
Client
[source]¶ Bases:
synapse.lib.base.Base
A Telepath client object which reconnects and allows waiting for link up.
Notes
The conf data allows changing parameters such as timeouts, retry period, and link pool size. The default conf data can be seen below:
conf = { 'timeout': 10, 'retrysleep': 0.2, 'link_poolsize': 4, }
-
class
synapse.telepath.
Genr
[source]¶ Bases:
synapse.telepath.Share
-
class
synapse.telepath.
GenrIter
(proxy, todo, share)[source]¶ Bases:
object
An object to help delay a telepath call until iteration.
-
class
synapse.telepath.
GenrMethod
(proxy, name, share=None)[source]¶ Bases:
synapse.telepath.Method
-
class
synapse.telepath.
Method
(proxy, name, share=None)[source]¶ Bases:
object
The telepath Method is used to provide proxy method calls.
-
class
synapse.telepath.
Pipeline
[source]¶ Bases:
synapse.lib.base.Base
-
class
synapse.telepath.
Proxy
[source]¶ Bases:
synapse.lib.base.Base
A telepath Proxy is used to call remote APIs on a shared object.
Example
import synapse.telepath as s_telepath
# open the “foo” object shared in a dmon on localhost:3344
async def doFooThing():
proxy = await s_telepath.openurl(‘tcp://127.0.0.1:3344/foo’)
valu = await proxy.getFooValu(x, y)
The proxy (and openurl function) may also be used from sync code:
proxy = s_telepath.openurl(‘tcp://127.0.0.1:3344/foo’)
valu = proxy.getFooValu(x, y)
-
call
(methname, *args, **kwargs)[source]¶ Call a remote method by name.
Parameters: - methname (str) – The name of the remote method.
- *args – Arguments to the method call.
- **kwargs – Keyword arguments to the method call.
Most use cases will likely use the proxy methods directly:
The following two are effectively the same:
valu = proxy.getFooBar(x, y) valu = proxy.call(‘getFooBar’, x, y)
-
getPipeline
(genr, name=None)[source]¶ Construct a proxy API call pipeline in order to make multiple telepath API calls while minimizing round trips.
Parameters: - genr (async generator) – An async generator that yields todo tuples.
- name (str) – The name of the shared object on the daemon.
Example
- def genr():
- yield s_common.todo(‘getFooByBar’, 10) yield s_common.todo(‘getFooByBar’, 20)
- for retn in proxy.getPipeline(genr()):
- valu = s_common.result(retn)
-
Bases:
synapse.lib.base.Base
The telepath client side of a dynamically shared object.
-
class
synapse.telepath.
Task
[source]¶ Bases:
object
A telepath Task is used to internally track calls/responses.
-
synapse.telepath.
addAhaUrl
(url)[source]¶ Add (incref) an aha registry URL.
NOTE: You may also add a list of redundant URLs.
-
synapse.telepath.
alias
(name)[source]¶ Resolve a telepath alias via ~/.syn/aliases.yaml
Parameters: name (str) – Name of the alias to resolve. Notes
An exact match against the aliases will always be returned first. If no exact match is found and the name contains a ‘/’ in it, the value before the slash is looked up and the remainder of the path is joined to any result. This is done to support dynamic Telepath share names.
Returns: The url string, if present in the alias. None will be returned if there are no matches. Return type: str
-
synapse.telepath.
delAhaUrl
(url)[source]¶ Remove (decref) an aha registry URL.
NOTE: You may also remove a list of redundant URLs.
-
synapse.telepath.
disc_consul
(info)[source]¶ Support for updating a URL info dictionary which came from a protocol+consul:// URL.
Notes
This updates the info-dictionary in place, placing the
host
value into anoriginal_host
key, and updatinghost
andport
.By default we pull the
host
value from the catalogAddress
value, and theport
from theServicePort
value.The following HTTP parameters are supported:
- consul: This is the consul host (schema, fqdn and port) to connect to.
- consul_tag: If set, iterate through the catalog results until a result is found which matches the tag value. This is a case sensitive match.
- consul_tag_address: If set, prefer the
TaggedAddresses
from the catalog. - consul_service_tag_address: If set, prefer the associated value from the
ServiceTaggedAddresses
field. - consul_nosslverify: If set, disables SSL verification.