Deployment¶

Partly out of date

Rendered from ProjectInfo/server/deployment.md on docs/encryption-modes-spec. Its protocol content is still useful, but parts of it describe an older client or an earlier API. Where it disagrees with the code, the code wins — see the spec errata.

docker-compose.yml runs Vault in -dev mode and is not a deploy target. The push variables listed here are out of date.

Development¶

make dev          # docker compose up --build (full stack)
make dev-up       # docker compose up -d (no rebuild)
make dev-down     # docker compose down
make dev-clean    # docker compose down -v (destroys DB volumes)
make dev-logs     # docker compose logs -f

Dev compose file: Server/dev.docker-compose.yml

Dev characteristics: - All five app services are built from source (no pre-built images) - migrate container runs automatically and exits before app services start - Vault runs in dev mode (in-memory, auto-unsealed) — data is ephemeral - vault-init seeds zero placeholder keys on every up - All services log at debug level with pretty output - Database ports are exposed on localhost for direct psql access - NATS management port exposed at :8222

Cloudflare Tunnel (dev)¶

The dev compose has a cloudflared quick-tunnel service gated behind profiles: [external]:

docker compose -f dev.docker-compose.yml --profile external up -d

Prints a public *.trycloudflare.com URL. Useful for testing on mobile or sharing with others temporarily.

Production Compose (`docker-compose.yml`)¶

Images are expected to be pre-built and pushed to a registry. Set image names in the compose override or environment.

Container startup order¶

All app services declare depends_on with x-app-deps:

x-app-deps: &app-deps
  depends_on:
    postgres: { condition: service_healthy }
    valkey:   { condition: service_healthy }
    nats:     { condition: service_healthy }
    garage-init: { condition: service_completed_successfully }
    vault-init:  { condition: service_completed_successfully }
    migrate:     { condition: service_completed_successfully }

This means migrations, Vault initialization, and Garage bucket initialization must all succeed before any app service starts.

Environment Variables¶

Shared across all services¶

Variable	Description
`JWT_SECRET`	HS256 signing secret — must be identical across api, keyserver, voice, media
`DATABASE_URL`	PostgreSQL DSN: `postgres://user:pass@host:5432/dbname`
`REDIS_URL`	Redis/Valkey: `redis://host:6379`
`ENVIRONMENT`	`production` (enables JSON logging, disables dev KEK provider)

API service¶

Variable	Description
`PORT`	`8080`
`NATS_URL`	`nats://host:4222`
`MINIO_ENDPOINT`	Garage host:port
`MINIO_ACCESS_KEY`	Garage/S3 access key
`MINIO_SECRET_KEY`	Garage/S3 secret key
`MINIO_BUCKET`	Bucket for CSAM evidence (default `zipp-chat`)
`MINIO_USE_SSL`	`true` in production
`VAULT_ADDR`	`http://vault:8200`
`VAULT_TOKEN`	Vault root or policy-scoped token
`LIVEKIT_HOST`	LiveKit server URL
`LIVEKIT_API_KEY`	LiveKit API key
`LIVEKIT_API_SECRET`	LiveKit API secret

Keyserver service¶

Variable	Description
`PORT`	`8084`

(Shares JWT_SECRET, DATABASE_URL, REDIS_URL)

Media service¶

Variable	Description
`PORT`	`8082`
`MINIO_ENDPOINT`	Same as API (or different if using separate bucket)
`MINIO_ACCESS_KEY` / `MINIO_SECRET_KEY`	Same as API
`MINIO_BUCKET`	Bucket for file uploads (default `zipp-media`)
`VAULT_ADDR` / `VAULT_TOKEN`	Same as API

Voice service¶

Variable	Description
`PORT`	`8081`
`LIVEKIT_HOST` / `LIVEKIT_API_KEY` / `LIVEKIT_API_SECRET`	Same as API

Worker service¶

Variable	Description
`PORT`	`8083`
`NATS_URL`	Same as API
`FCM_PROJECT_ID`	Firebase project ID
`FCM_CREDENTIALS_FILE`	Path to service account JSON (mount as Docker secret)
`APNS_KEY_FILE`	Path to APNs `.p8` key file
`APNS_KEY_ID`	APNs key ID
`APNS_TEAM_ID`	Apple Team ID
`APNS_BUNDLE_ID`	App bundle identifier

Example: Server/docker-compose.env.example

Migrations¶

# In production (one-shot, then exit)
docker compose run --rm migrate

# Or directly with golang-migrate CLI
migrate -path migrations -database "$DATABASE_URL" up

# Create a new migration
migrate create -ext sql -dir migrations -seq your_feature_name
# → creates: migrations/000016_your_feature_name.up.sql
#             migrations/000016_your_feature_name.down.sql

IMPORTANT: Always write a correct .down.sql. CI runs up + down on every PR to catch migration bugs.

Vault Initialization¶

In production, Vault must be initialized separately (outside Docker Compose):

# Initialize Vault (first time only)
vault operator init

# Unseal (required after every restart)
vault operator unseal <key1>
vault operator unseal <key2>
vault operator unseal <key3>

# Enable KV v2
vault secrets enable -path=secret kv-v2

# Seed required secrets
vault kv put secret/media/kek key="$(openssl rand -base64 32)"
vault kv put secret/zipp-chat/media/evidence-kek key="$(openssl rand -base64 32)"
vault kv put secret/zipp-chat/media/csam-key key="$(openssl rand -base64 32)"

# Channel KEKs are auto-created by the api service on first message send

In development, infra/vault/dev-init.sh runs automatically and seeds zeroed keys. Do not use dev keys in production.

Nginx Routing (`infra/nginx/nginx.conf`)¶

Server name	Location	Upstream
`api-dev.zigz.uk`	`/api/keys/`	`keyserver:8084`
	`/api/voice/`	`voice:8081`
	`/api/`	`api:8080`
	`/gateway`	`api:8080` (WebSocket upgrade)
`media-dev.zigz.uk`	`/`	`media:8082`
`voice-dev.zigz.uk`	`/`	`voice:8081`
`app-dev.zigz.uk`	`/`	`web:80`
`livekit-dev.zigz.uk`	`/`	`livekit:7880` (WebSocket)

Change hostnames for production. Add TLS termination at nginx (with certbot or Cloudflare-proxied). In the current setup, Cloudflare Tunnel handles TLS — nginx sees plain HTTP.

WebSocket upgrade headers¶

location /gateway {
    proxy_pass http://api:8080;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_set_header Host $host;
    proxy_read_timeout 3600s;
}

The 3600s read timeout prevents nginx from killing idle WebSocket connections.

Garage (S3) Initialization (`infra/garage/`)¶

Garage uses a volume for data. On first start, garage-init creates two buckets: - zipp-chat — CSAM evidence - zipp-media — user file uploads

And creates an access key pair, outputting them to the logs. Copy the access key and secret key into MINIO_ACCESS_KEY / MINIO_SECRET_KEY in your env file.

NATS Configuration¶

NATS starts with JetStream enabled (-js) and monitoring on port 8222 (-m 8222).

No NATS streams or consumers are configured in v1 — all subscriptions are ephemeral core NATS. If you need durable consumers (e.g. for reliable push delivery), create a JetStream stream on channel.> and a consumer per worker instance.

Makefile Reference¶

make dev           # Build + start dev stack
make dev-up        # Start without rebuilding
make dev-down      # Stop
make dev-clean     # Stop + destroy volumes
make dev-logs      # Tail logs

make migrate-up    # Apply all pending migrations
make migrate-down  # Roll back one migration
make migrate-drop  # Drop all tables (dangerous)
make migrate-new   # Create new migration files (prompts for name)
make migrate-status # Show migration status

make test          # Run all tests
make build         # Build all service binaries
make tidy          # go mod tidy all modules in workspace