When viewed as a library, not just an application, Synapse is made of up a few core components and concepts.
The Synapse library is broken out in a hierarchical fashion. The root of the library contains application level code, such as the implementations of the Cortex, Axon, Cryotank, as well as the Telepath client and server components. There are also a set of common helper functions (common.py) and exceptions (exc.py). There are several submodules available as well:
- Command implementations for the Cmdr CLI tool
- Data files stored in the library.
- The lib module contains many of the primitives used by applications in order to implement them.
- The lookup module contains various lookup definitions.
- The models directory contains the core Synapse data model definitions.
- The servers module contains servers use to start and run Synapse applications.
- This is test code. It also contains a useful helper
synapse.tests.utilswhich defines our base test class.
- The tools module contains various tools used to interact with the Synapse ecosystem.
There is one base class that many objects inherit from, the
Base (base.py) class. The
Base class provides a
few useful components (including, but not limited too):
- A way to do asynchronous object construction by override the
__anit__method. This method is executed inside the python ioloop, allowing the object construction to do async function calls. An implementer still needs to call
await s_base.Base.__anit__(self)first in order to ensure that the
Baseis setup properly.
- A way to register object teardown methods and perform object teardowns via the
fini(). These allow us to keep more granular control over how things are shut down and resources are released, versus relying solely on the garbage collector to handle teardowns properly. Often times, order matters, so we need to be sure that things are torn down cleanly. These routines can be registered during
Baseobjects are made via await the call to the
Base.anit()function. If the
__anit__function completed then the
anittedattribute on the object will be True, otherwise it will be False.
- Context manager support. The
Baseobject has native async context manager support, and upon exiting the context it will call
fini()to do teardown. This pattern is convenient since it allows us to freely create
Baseclasses without having to remember to always have to tear them down.
Basecontains helpers for implementing an observable design pattern, where functions can be registered as event handlers, and events can be fired on the object at will. This can be very powerful for signaling across disparate components which would be otherwise too heavy to have explicit callbacks for.
Basecontains helpers for executing asyncio coroutines on the ioloop. This is most commonly done via the
schedCoroTaskroutine. This will schedule the coroutine to run on the ioloop, register the task with the
Baseand return the asyncio future. During
Basefini, any coroutines still executing will be cancelled. This makes it very easy to schedule free-running coroutines from any
There are a few very important classes which use the
- The Synapse
Cell. This is a batteries included primitive for running an application.
- The Telepath
Daemon. This serves as a RPC server component.
- The Telepath
Proxy. This serves as a RPC client component.
Cell (cell.py) is a
Base implementation which has several components available to it:
- It is a
Base, so it benefits from all the components a
- It contains support for configuration directives at start time, so a cell can have well defined configuration options availble to it.
- It has persistent storage available via two different mechanisms, a LMDB slab for arbitrary data that is local to the
cell, and a
Hivefor key-value data storage that can be remotely read and written.
- It handles user authentication and authorization via user data stored in the
Cellis Telepath aware, and will start his own
Daemonthat allows remote access. By default, the
Cellhas a PF Unix socket available for access, so local telepath access is trivial.
- Since the
Cellis Telepath aware, there is a base
CellApithat implements his RPC routines.
Cellimplementers can easily sublcass the
CellApiclass to add additional RPC routines.
Cellalso contains hooks for easily starting a Tornado webserver. This allows us to trivially add web API routes to an object.
Bosswhich can be used to remotely enumerate and cancel managed coroutines.
Since the cell contains so much core management functionality, adding functionality to the Synapse
all applications using a Cell to be immediately extended to take advantage of that functionality without having to
revisit multiple different implementations to update them. For this reason, our core application components (the
CryoCell) all implement the
Cell class. For example, if we add a new user management
capability, that is now available to all those applications, as well as any others
The application level components themselves have servers in the
synapse.servers module, but there is also a generic
server for starting any cell,
synapse.servers.cell. These servers will create the
Cell, and also add any
additional RPC or HTTP API listening servers as necessary. Those are the preferred ways to run an application
implemented via a
The Telepath RPC protocol is a lightweight RPC protocol used in Synapse. The server component, the previously mentioned
Daemon, is used to share objects. An object may or may not be Telepath aware. In the case that it is not aware, all
of its methods are exposed via Telepath. Objects which are Telepath aware, such as the
Cell, implement an API
interface that allows much more fine grained control over the the methods which are remotely available.
The base Telepath client is the
Proxy class, this is used to connect to the Daemon. The
attribute lookups to make and set remote method helpers at runtime, and sends those requests to the Daemon to be
serviced. A very brief example of this is the following:
import synapse.telepath as s_telepath url = 'tcp://user:email@example.com:27492/someObject' async with await s_telepath.openurl(url) as proxy: # Make attribute called "someMethod" on the proxy # then send a task to the server called "someMethod" # with the argument of somearg=1234 resp = proxy.someMethod(somearg=1234) # The resp is the result of calling the someMethod argument on # the object named someObject on the daemon. print(resp)
A few notes about Telepath:
- Telepath remote call arguments and server responses must be able to be serialized using the msgpack protocol.
- Telepath supports generator protocols; so a server API may be a synchronous or asynchronous generator. From the proxy perspective, these are both considered asynchronous generators.
- The Telepath
Proxycontains some helpers that allow is to be used from non-async code. These helpers run their API calls through the currently running ioloop, and will cause the client to make an ioloop if one is not currently running.
- Remote calls that raise exceptions on the server will have that exception serialized and sent back to the
Proxywill then raise an exception to the caller.
- Methods calls prefixed with a underscore (
_somePrivatMethod()for example) will be rejected by the
Daemon. This does allow us to protect private methods on shared objects.