DevOps Guide

Overview

For a general overview of common devops tasks for Synapse services see Synapse Devops Guide - Overview.

Common DevOps Tasks

Run Chromium in a Sandbox

By default the container will not run Chromium in a sandbox. To enable this setting an appropriate seccomp profile must be provided (see the Playwright Docker Documentation for a link to the recommended profile).

Examples of modifications to the docker-compose.yaml file are shown below:

#...
security_opt:
    - seccomp:/path/to/seccomp_profile.json
environment:
    # ...
    - SYN_PLAYWRIGHT_CHROMIUM_SANDBOX=true

It is recommended to verify that the security profile is set correctly via the following command on the host:

docker inspect --format="{{ .HostConfig.SecurityOpt }}" $(docker-compose --file /srv/syn/00.playwright/docker-compose.yaml ps -q 00.playwright)

The container logs should also be checked to verify there were no errors in applying the sandbox option:

docker-compose --file /srv/syn/00.playwright/docker-compose.yaml logs -f

For additional details see the Playwright Docker Documentation.

Deploy a Mirror

Inside the AHA container

Generate a one-time use URL for provisioning from inside the AHA container:

python -m synapse.tools.aha.provision.service 01.playwright --mirror playwright

You should see output that looks similar to this:

one-time use URL: ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>

On the Host

Create the container storage directory:

mkdir -p /srv/syn/01.playwright/storage
chown -R 999 /srv/syn/01.playwright/storage

Create the /srv/syn/01.playwright/docker-compose.yaml file with contents:

version: "3.3"
services:
  01.playwright:
    user: "999"
    image: vertexproject/synapse-playwright:v1.x.x
    network_mode: host
    restart: unless-stopped
    volumes:
        - ./storage:/vertex/storage
    environment:
        # disable HTTPS API for now to prevent port collisions
        - SYN_PLAYWRIGHT_HTTPS_PORT=null
        - SYN_PLAYWRIGHT_AHA_PROVISION=ssl://aha.<yournetwork>:27272/<guid>?certhash=<sha256>

Note

Don’t forget to replace your one-time use provisioning URL!

Start the container:

docker-compose --file /srv/syn/01.playwright/docker-compose.yaml pull
docker-compose --file /srv/syn/01.playwright/docker-compose.yaml up -d

Devops Details

Managing Resource Utilization

When using Storm to interact with Synapse-Playwright each request for a page will be created in a new dedicated Chromium browser. If the maximum browser count has already been reached, as defined in the workers configuration option, the request will wait until a browser worker is freed.

When pages are created, or spawned, they will continue to hold their resources until they are closed or the browser is closed. Synapse-Playwright APIs such as wget2pdf and wget2html will close the browser and page when they are finished, and any custom Storm should also follow this best practice (see the User Guide for examples).

By default the maximum number of workers will be set by the number of CPUs available. However, this can also be set directly. For example, the following additions to the docker-compose.yaml would set the maximum number of concurrent browsers to 4:

environment:
    # ...
    - SYN_PLAYWRIGHT_WORKERS=4

Microsoft recommends a minimum of 2GB of memory for their Playwright Docker container. To avoid memory contention it is recommended to start with at least 1GB for each concurrent browser.

Synapse-Playwright also always sets the disable-dev-shm-usage flag when launching the browser. This forces Chromium to use /tmp instead of shared memory, and this directory should have free disk space equal to at least the recommended memory detailed above.

It is recommended to monitor Synapse-Playwright resource utilization and adjust as needed. An INFO log message when a browser is created will indicate the number of concurrent browsers in use. The getCellInfo API also has a browser_count field.

Docker Images

The Synapse-Playwright service is available as a Docker container from Docker Hub. The repository can be found at:

Note

There are tagged images available on Docker Hub which correspond to software releases seen in the changelog. The docker tag master is the latest development release. A generic major version tag is available, representing the latest release on a given major version. For example, the v1.x.x tag represents the most current release for the v1.x.x release line. You can utilize specific tagged versions, or a major version specifier, depending on your chosen deployment strategy.

Configuration Options

The following is a list of available configuration options.

aha:admin

An AHA client certificate CN to register as a local admin user.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AHA_ADMIN

aha:leader

The AHA service name to claim as the active instance of a storm service.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AHA_LEADER

aha:name

The name of the cell service in the aha service registry.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AHA_NAME

aha:network

The AHA service network. This makes aha:name/aha:leader relative names.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AHA_NETWORK

aha:provision

The telepath URL of the aha provisioning service.

Type

['string', 'array']

Environment Variable

SYN_PLAYWRIGHT_AHA_PROVISION

aha:registry

The telepath URL of the aha service registry.

Type

['string', 'array']

Environment Variable

SYN_PLAYWRIGHT_AHA_REGISTRY

aha:user

The username of this service when connecting to others.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AHA_USER

auth:anon

Allow anonymous telepath access by mapping to the given user name.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AUTH_ANON

auth:passwd

Set to <passwd> (local only) to bootstrap the root user password.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AUTH_PASSWD

axon

Telepath URL to the Axon.

Type

string

Environment Variable

SYN_PLAYWRIGHT_AXON

backup:dir

A directory outside the service directory where backups will be saved. Defaults to ./backups in the service storage directory.

Type

string

Environment Variable

SYN_PLAYWRIGHT_BACKUP_DIR

chromium:sandbox

Should Chromium run in a sandbox (requires service to be run as non-root user).

Type

boolean

Default Value

False

Environment Variable

SYN_PLAYWRIGHT_CHROMIUM_SANDBOX

dmon:listen

A config-driven way to specify the telepath bind URL.

Type

['string', 'null']

Environment Variable

SYN_PLAYWRIGHT_DMON_LISTEN

https:headers

Headers to add to all HTTPS server responses.

Type

object

Environment Variable

SYN_PLAYWRIGHT_HTTPS_HEADERS

https:parse:proxy:remoteip

Enable the HTTPS server to parse X-Forwarded-For and X-Real-IP headers to determine requester IP addresses.

Type

boolean

Default Value

False

Environment Variable

SYN_PLAYWRIGHT_HTTPS_PARSE_PROXY_REMOTEIP

https:port

A config-driven way to specify the HTTPS port.

Type

['integer', 'null']

Environment Variable

SYN_PLAYWRIGHT_HTTPS_PORT

limit:disk:free

Minimum disk free space percentage before setting the cell read-only.

Type

['integer', 'null']

Default Value

5

Environment Variable

SYN_PLAYWRIGHT_LIMIT_DISK_FREE

max:users

Maximum number of users allowed on system, not including root or locked/archived users (0 is no limit).

Type

integer

Default Value

0

Environment Variable

SYN_PLAYWRIGHT_MAX_USERS

nexslog:en

Record all changes to a stream file on disk. Required for mirroring (on both sides).

Type

boolean

Default Value

False

Environment Variable

SYN_PLAYWRIGHT_NEXSLOG_EN

onboot:optimize

Delay startup to optimize LMDB databases during boot to recover free space and increase performance. This may take a while.

Type

boolean

Default Value

False

Environment Variable

SYN_PLAYWRIGHT_ONBOOT_OPTIMIZE

workers

Maximum parallel browser workers (defaults to number of CPUs in the system).

Type

integer

Environment Variable

SYN_PLAYWRIGHT_WORKERS