DevOps Guide

Overview

For a general overview of common devops tasks for Synapse services see Synapse Devops Guide - Overview.

Common DevOps Tasks

Specify a Container

The azure:container option can be used to set the container name to be used for storage. If the container does not exist when Synapse Azure Axon starts up, the container will be created.

For example, to set using an environment variable in the docker-compose.yaml file:

environment:
    # ...
    - SYN_AZUREAXON_AZURE_CONTAINER=mycontainer

Provide Azure Credentials

The Azure Axon will first check for an Azure Storage connection string to use for authentication. If no connection string is provided, the Axon will attempt to resolve credentials from the environment in the order listed in the DefaultAzureCredential documentation. Those credentials will be used with the URL provided in the azure:url configuration value to connect to Azure Storage.

An example docker-compose.yaml file can be seen below. This example omits the azure:connstr configuration which will cause the Azure Axon to attempt to resolve the service credentials from credentials in the environment:

version: "3.3"
services:
  00.axon:
    user: "999"
    image: vertexproject/synapse-axon-azure:v0.x.x
    network_mode: host
    restart: unless-stopped
    volumes:
        - ./storage:/vertex/storage
    environment:
        - SYN_AZUREAXON_AZURE_CONTAINER=mycontainer
        - SYN_AZUREAXON_AZURE_URL=https://<my-storage-account-name>.blob.core.windows.net/
        - SYN_AZUREAXON_HTTPS_PORT=null
        - SYN_AZUREAXON_AHA_PROVISION=ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>

Deploy a Mirror

A Synapse Azure Axon mirror can independently read and write to the Azure container, with all changes to the local index data written back to the upstream.

Inside the AHA container

Generate a one-time use URL for provisioning from inside the AHA container:

python -m synapse.tools.aha.provision.service 01.axon --mirror axon

You should see output that looks similar to this:

one-time use URL: ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>

On the Host

Create the container storage directory:

mkdir -p /srv/syn/01.axon/storage
chown -R 999 /srv/syn/01.axon/storage

Create the /srv/syn/01.axon/docker-compose.yaml file with contents:

version: "3.3"
services:
  01.axon:
    user: "999"
    image: vertexproject/synapse-axon-azure:v0.x.x
    network_mode: host
    restart: unless-stopped
    volumes:
        - ./storage:/vertex/storage
    environment:
        # disable HTTPS API for now to prevent port collisions
        - SYN_AZUREAXON_AZURE_CONTAINER=mycontainer
        - SYN_AZUREAXON_AZURE_CONNSTR=<Azure Storage account connection string>
        - SYN_AZUREAXON_HTTPS_PORT=null
        - SYN_AZUREAXON_AHA_PROVISION=ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>

Note

Don’t forget to replace your one-time use provisioning URL!

Start the container:

docker-compose --file /srv/syn/01.axon/docker-compose.yaml pull
docker-compose --file /srv/syn/01.axon/docker-compose.yaml up -d

Devops Details

Docker Images

The Synapse-Axon-Azure service is available as a Docker container from Docker Hub. The repository can be found at:

Note

There are tagged images available on Docker Hub which correspond to software releases seen in the changelog. The docker tag master is the latest development release. A generic major version tag is available, representing the latest release on a given major version. For example, the v2.x.x tag represents the most current release for the v2.x.x release line. You can utilize specific tagged versions, or a major version specifier, depending on your chosen deployment strategy.

Required Permissions

The provided credentials must have the following permissions:

Microsoft.Storage/storageAccounts/blobServices/containers/read

This is used to ensure that the Azure container used to store files in exists.

Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read

This is used to retrieve files uploaded to the Azure container, and list the files in the container for index recovery operations.

Microsoft.Storage/storageAccounts/blobServices/containers/blobs/add/action

These are use used to add files to the Azure container.

Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write

These are use used to write files to the Azure container.

Microsoft.Storage/storageAccounts/blobServices/containers/blobs/delete

This is used to delete files uploaded to the Azure container. In the case where a bulk delete is requested the delete operation is run on each object. If an Azure delete operation fails (e.g. a role permission error), the SHA-256 will still be removed from the Axon index and the object would need to be manually deleted in the bucket.

Microsoft.Storage/storageAccounts/blobServices/containers/write

This permission is only needed to make the container if the container does not exist. It is best practice when integrating with Azure to create the container separately, since the Synapse Azure service does not specify any sort of access policies when it creates the container.

Since Synapse Azure Axon implements the Synapse Axon API, permissions are also inherited. For example, the axon.get rule is required to retrieve a file and the axon.upload rule is required to upload a file.

Configuration Options

The following is a list of available configuration options.

aha:admin

An AHA client certificate CN to register as a local admin user.

Type

string

Environment Variable

SYN_AZUREAXON_AHA_ADMIN

aha:leader

The AHA service name to claim as the active instance of a storm service.

Type

string

Environment Variable

SYN_AZUREAXON_AHA_LEADER

aha:name

The name of the cell service in the aha service registry.

Type

string

Environment Variable

SYN_AZUREAXON_AHA_NAME

aha:network

The AHA service network.

Type

string

Environment Variable

SYN_AZUREAXON_AHA_NETWORK

aha:provision

The telepath URL of the aha provisioning service.

Type

['string', 'array']

Environment Variable

SYN_AZUREAXON_AHA_PROVISION

aha:registry

The telepath URL of the aha service registry.

Type

['string', 'array']

Environment Variable

SYN_AZUREAXON_AHA_REGISTRY

aha:user

The username of this service when connecting to others.

Type

string

Environment Variable

SYN_AZUREAXON_AHA_USER

auth:anon

Allow anonymous telepath access by mapping to the given user name.

Type

string

Environment Variable

SYN_AZUREAXON_AUTH_ANON

auth:passwd

Set to <passwd> (local only) to bootstrap the root user password.

Type

string

Environment Variable

SYN_AZUREAXON_AUTH_PASSWD

auth:passwd:policy

Specify password policy/complexity requirements.

Type

object

Environment Variable

SYN_AZUREAXON_AUTH_PASSWD_POLICY

azure:connstr

A connection string for accessing Azure storage.

Type

string

Environment Variable

SYN_AZUREAXON_AZURE_CONNSTR

azure:container

The name of the container to use.

Type

string

Environment Variable

SYN_AZUREAXON_AZURE_CONTAINER

azure:url

The storage account’s blob service account URL (https://<my-storage-account-name>.blob.core.windows.net/).

Type

string

Environment Variable

SYN_AZUREAXON_AZURE_URL

backup:dir

A directory outside the service directory where backups will be saved. Defaults to ./backups in the service storage directory.

Type

string

Environment Variable

SYN_AZUREAXON_BACKUP_DIR

dmon:listen

A config-driven way to specify the telepath bind URL.

Type

['string', 'null']

Environment Variable

SYN_AZUREAXON_DMON_LISTEN

health:sysctl:checks

Enable sysctl parameter checks and warn if values are not optimal.

Type

boolean

Default Value

True

Environment Variable

SYN_AZUREAXON_HEALTH_SYSCTL_CHECKS

http:proxy

An aiohttp-socks compatible proxy URL to use in the wget API.

Type

string

Environment Variable

SYN_AZUREAXON_HTTP_PROXY

https:headers

Headers to add to all HTTPS server responses.

Type

object

Environment Variable

SYN_AZUREAXON_HTTPS_HEADERS

https:parse:proxy:remoteip

Enable the HTTPS server to parse X-Forwarded-For and X-Real-IP headers to determine requester IP addresses.

Type

boolean

Default Value

False

Environment Variable

SYN_AZUREAXON_HTTPS_PARSE_PROXY_REMOTEIP

https:port

A config-driven way to specify the HTTPS port.

Type

['integer', 'null']

Environment Variable

SYN_AZUREAXON_HTTPS_PORT

limit:disk:free

Minimum disk free space percentage before setting the cell read-only.

Type

['integer', 'null']

Default Value

5

Environment Variable

SYN_AZUREAXON_LIMIT_DISK_FREE

max:bytes

The maximum number of bytes that can be stored in the Axon.

Type

integer

Environment Variable

SYN_AZUREAXON_MAX_BYTES

max:count

The maximum number of files that can be stored in the Axon.

Type

integer

Environment Variable

SYN_AZUREAXON_MAX_COUNT

max:users

Maximum number of users allowed on system, not including root or locked/archived users (0 is no limit).

Type

integer

Default Value

0

Environment Variable

SYN_AZUREAXON_MAX_USERS

nexslog:en

Record all changes to a stream file on disk. Required for mirroring (on both sides).

Type

boolean

Default Value

False

Environment Variable

SYN_AZUREAXON_NEXSLOG_EN

onboot:optimize

Delay startup to optimize LMDB databases during boot to recover free space and increase performance. This may take a while.

Type

boolean

Default Value

False

Environment Variable

SYN_AZUREAXON_ONBOOT_OPTIMIZE

tls:ca:dir

An optional directory of CAs which are added to the TLS CA chain for wget and wput APIs.

Type

string

Environment Variable

SYN_AZUREAXON_TLS_CA_DIR