DevOps Guide
Overview
For a general overview of common devops tasks for Synapse services see Synapse Devops Guide - Overview.
Common DevOps Tasks
Specify a Bucket
The gcs:bucket
option can be used to set the bucket name, which must be a globally
unique name. If the bucket does not exist when Synapse GCS Axon starts up, the gcs:project
option must be set to specify the project to create the bucket under.
For example, to set using an environment variable in the docker-compose.yaml
file:
environment:
# ...
- SYN_GCSAXON_GCS_BUCKET=uniquebucket-123456
- SYN_GCSAXON_GCS_PROJECT=myproject-123456
Provide GCS Credentials
By default the GCS Axon will use the service-accounts information in Google Compute Engine metadata for generating authorization tokens. For more information about GCE metadata, see the Google Cloud docs.
If you are not running the GCS Axon in GCE, you may provide credentials as a JSON string containing
service account or authorized user credentials in the gcs:credentials
configuration value, or
a path to a file containing service account or authorized user credentials with the
gcs:credentials:path
configuration value.
An example docker-compose.yaml
file can be seen below. This example omits the gcs:credentials
configuration.
This would cause GCS to resolve the service credentials from GCE metadata:
version: "3.3"
services:
00.axon:
user: "999"
image: vertexproject/synapse-gcs:v1.x.x
network_mode: host
restart: unless-stopped
volumes:
- ./storage:/vertex/storage
environment:
- SYN_GCSAXON_BUCKET: uniquebucket-123456
- SYN_GCSAXON_PROJECT: myproject-123456
- SYN_GCSAXON_HTTPS_PORT: null
- SYN_GCSAXON_AHA_PROVISION: "ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>"
Deploy a Mirror
A Synapse GCS Axon mirror can independently read and write to the GCS storage bucket, with all changes to the local index data written back to the upstream.
Inside the AHA container
Generate a one-time use URL for provisioning from inside the AHA container:
python -m synapse.tools.aha.provision.service 01.axon --mirror axon
You should see output that looks similar to this:
one-time use URL: ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>
On the Host
Create the container storage directory:
mkdir -p /srv/syn/01.axon/storage
chown -R 999 /srv/syn/01.axon/storage
Create the /srv/syn/01.axon/docker-compose.yaml
file with contents:
version: "3.3"
services:
01.axon:
user: "999"
image: vertexproject/synapse-gcs:v1.x.x
network_mode: host
restart: unless-stopped
volumes:
- ./storage:/vertex/storage
environment:
# disable HTTPS API for now to prevent port collisions
- SYN_GCSAXON_BUCKET: uniquebucket-123456
- SYN_GCSAXON_PROJECT: myproject-123456
- SYN_GCSAXON_HTTPS_PORT=null
- SYN_GCSAXON_AHA_PROVISION=ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>
Note
Don’t forget to replace your one-time use provisioning URL!
Start the container:
docker-compose --file /srv/syn/01.axon/docker-compose.yaml pull
docker-compose --file /srv/syn/01.axon/docker-compose.yaml up -d
Devops Details
Docker Images
The Synapse-GCS service is available as a Docker container from Docker Hub. The repository can be found at:
Note
There are tagged images available on Docker Hub which correspond to software releases seen in the changelog. The
docker tag master
is the latest development release. A generic major version tag is available, representing the
latest release on a given major version. For example, the v2.x.x
tag represents the most current release for
the v2.x.x
release line. You can utilize specific tagged versions, or a major version specifier, depending on
your chosen deployment strategy.
Required Permissions
The provided credentials must have the following permissions:
storage.buckets.get
This is used to ensure that the GCS bucket used to store files in exists.
storage.objects.get
This is used to retrieve files uploaded to the GCS bucket.
storage.objects.list
This is used to list the files in the GCS bucket for index recovery operations.
storage.objects.create
This is used to put files in the GCS bucket.
storage.objects.delete
This is used to delete files uploaded to the GCS bucket. In the case where a bulk delete is requested the delete operation is run on each object. If a GCS delete operation fails (e.g. a role permission error), the SHA-256 will still be removed from the Axon index and the object would need to be manually deleted in the bucket.
storage.buckets.create
This permission is only needed to make the bucket if the bucket does not exist. It is best practice when integrating with GCS to create the bucket separately, since the Synapse GCS service does not specify any sort of access policies when it creates the bucket.
Since Synapse GCS Axon implements the Synapse Axon API, permissions are also inherited.
For example, the axon.get
rule is required to retrieve a file and the axon.upload
rule is
required to upload a file.
Configuration Options
The following is a list of available configuration options.
aha:admin
An AHA client certificate CN to register as a local admin user.
- Type
string
- Environment Variable
SYN_GCSAXON_AHA_ADMIN
aha:leader
The AHA service name to claim as the active instance of a storm service.
- Type
string
- Environment Variable
SYN_GCSAXON_AHA_LEADER
aha:name
The name of the cell service in the aha service registry.
- Type
string
- Environment Variable
SYN_GCSAXON_AHA_NAME
aha:network
The AHA service network. This makes aha:name/aha:leader relative names.
- Type
string
- Environment Variable
SYN_GCSAXON_AHA_NETWORK
aha:provision
The telepath URL of the aha provisioning service.
- Type
['string', 'array']
- Environment Variable
SYN_GCSAXON_AHA_PROVISION
aha:registry
The telepath URL of the aha service registry.
- Type
['string', 'array']
- Environment Variable
SYN_GCSAXON_AHA_REGISTRY
aha:user
The username of this service when connecting to others.
- Type
string
- Environment Variable
SYN_GCSAXON_AHA_USER
auth:anon
Allow anonymous telepath access by mapping to the given user name.
- Type
string
- Environment Variable
SYN_GCSAXON_AUTH_ANON
auth:passwd
Set to <passwd> (local only) to bootstrap the root user password.
- Type
string
- Environment Variable
SYN_GCSAXON_AUTH_PASSWD
backup:dir
A directory outside the service directory where backups will be saved. Defaults to ./backups in the service storage directory.
- Type
string
- Environment Variable
SYN_GCSAXON_BACKUP_DIR
dmon:listen
A config-driven way to specify the telepath bind URL.
- Type
['string', 'null']
- Environment Variable
SYN_GCSAXON_DMON_LISTEN
gcs:bucket
The name of the bucket to use (must be a globally unique name).
- Type
string
- Environment Variable
SYN_GCSAXON_GCS_BUCKET
gcs:credentials
Service account or authorized user credentials blob.
- Type
object
- Environment Variable
SYN_GCSAXON_GCS_CREDENTIALS
gcs:credentials:path
Path to a file containing service account or authorized user credentials.
- Type
string
- Environment Variable
SYN_GCSAXON_GCS_CREDENTIALS_PATH
gcs:nocredentials
Disable authorization tokens (used for connecting to an emulated GCS server).
- Type
boolean
- Default Value
False
- Environment Variable
SYN_GCSAXON_GCS_NOCREDENTIALS
gcs:project
The name of the project to create the bucket in if it does not exist.
- Type
string
- Environment Variable
SYN_GCSAXON_GCS_PROJECT
gcs:retry
Maximum number of retries for failed connection attempts.
- Type
integer
- Default Value
5
- Environment Variable
SYN_GCSAXON_GCS_RETRY
gcs:ssl
Set to false to disable ssl verification when connecting to Google Cloud Storage.
- Type
boolean
- Default Value
True
- Environment Variable
SYN_GCSAXON_GCS_SSL
gcs:url
The base url to use when connecting to Google Cloud Storage.
- Type
string
- Default Value
'https://www.googleapis.com'
- Environment Variable
SYN_GCSAXON_GCS_URL
http:proxy
An aiohttp-socks compatible proxy URL to use in the wget API.
- Type
string
- Environment Variable
SYN_GCSAXON_HTTP_PROXY
https:headers
Headers to add to all HTTPS server responses.
- Type
object
- Environment Variable
SYN_GCSAXON_HTTPS_HEADERS
https:parse:proxy:remoteip
Enable the HTTPS server to parse X-Forwarded-For and X-Real-IP headers to determine requester IP addresses.
- Type
boolean
- Default Value
False
- Environment Variable
SYN_GCSAXON_HTTPS_PARSE_PROXY_REMOTEIP
https:port
A config-driven way to specify the HTTPS port.
- Type
['integer', 'null']
- Environment Variable
SYN_GCSAXON_HTTPS_PORT
limit:disk:free
Minimum disk free space percentage before setting the cell read-only.
- Type
['integer', 'null']
- Default Value
5
- Environment Variable
SYN_GCSAXON_LIMIT_DISK_FREE
max:bytes
The maximum number of bytes that can be stored in the Axon.
- Type
integer
- Environment Variable
SYN_GCSAXON_MAX_BYTES
max:count
The maximum number of files that can be stored in the Axon.
- Type
integer
- Environment Variable
SYN_GCSAXON_MAX_COUNT
max:users
Maximum number of users allowed on system, not including root or locked/archived users (0 is no limit).
- Type
integer
- Default Value
0
- Environment Variable
SYN_GCSAXON_MAX_USERS
nexslog:en
Record all changes to a stream file on disk. Required for mirroring (on both sides).
- Type
boolean
- Default Value
False
- Environment Variable
SYN_GCSAXON_NEXSLOG_EN
onboot:optimize
Delay startup to optimize LMDB databases during boot to recover free space and increase performance. This may take a while.
- Type
boolean
- Default Value
False
- Environment Variable
SYN_GCSAXON_ONBOOT_OPTIMIZE
tls:ca:dir
An optional directory of CAs which are added to the TLS CA chain for wget and wput APIs.
- Type
string
- Environment Variable
SYN_GCSAXON_TLS_CA_DIR