Encryption Modes — Canonical Specification¶

Canonical specification

This page is ProjectInfo/encryption-modes.md on docs/encryption-modes-spec, rendered at build time. That file is the source of truth; the site holds no copy of it.

Status: Draft for review. Supersedes the mode descriptions scattered across e2ee.md, server/encryption.md, data-types.md and client/*.md where they conflict. Nothing here is implemented yet beyond what §9 marks as built.

This document exists because the intent, the specs, and the running code have diverged three ways. It states one model for channel encryption, media encryption, and abuse reporting, and says explicitly what each mode gives up.

1. The requirement¶

It must be possible for a user's files and messages to be unreadable by the server, permanently, if they choose that — while still allowing a recipient who received CSAM to report it in a way the server can verify came from the accused.

Those two goals look contradictory. §7 shows they are not.

A second, softer requirement: a mode that keeps server-side search, history on join, and thumbnails, where the server can read content — disclosed to the user, and subject to subpoena. That is a legitimate product tier, not a compromise of the first.

2. Three-way divergence (read this before trusting any other doc)¶

Concept	Intent	`ProjectInfo/` specs	Implemented code
Mode names	"rolling" / "persistent"	`rolling_key` / `server_encrypted` / `peer_provided`	`secure` / `persistent`
Key unit	per member	per device (`crypto_box_seal` to device IK)	per user (`channel_keys.encrypted_ck`)
Rotation counter	on join/leave	`epoch_num` + `member_hash`, server-validated	`ck_version` INT, no validation
Key tables	—	`channel_key_epochs`, `channel_epoch_key_grants`	`channel_keys` only
Key endpoints	—	`POST /api/channels/{id}/epoch`, `.../epoch/{id}/grants`	`GET`/`PUT /api/channels/{id}/keys`
Message cipher	—	XChaCha20-Poly1305 (`crypto_secretbox`)	AES-256-GCM envelope
Media	client-encrypted, optional	`key_type` = `public`/`dm`/`server_encrypted` (legacy)	server-side only — client uploads plaintext
CSAM report body	plaintext + proof	`{attachment_id, evidence_note}`	`{reported_user_id, content_hash, decrypted_blob, ...}`
Report verification	required	unspecified anywhere	none — client is trusted

The specs describe a more advanced system than exists. The code implements a simpler, differently-named one. Neither matches the intent for media.

Naming decision¶

Decision: rolling_key, server_escrowed, peer_provided.

Two changes from every existing name.

Behaviour, not ranking. "secure" vs "persistent" is a false pairing — the escrowed mode is also persisted, and rolling-key data is also durable.

server_escrowed, not server_encrypted. Both other names imply the wrong thing. "persistent" says nothing about who can read it, and "server_encrypted" suggests the server is the one doing the encrypting — which, after §3.1, it no longer is. Escrow is the precise word: the client encrypts, and the server holds a copy of the key.

This mode is not E2EE, and must never be described as such. See §3.1.

What renaming costs¶

Surface	Currently says	Action
`channel_encryption_mode` Postgres enum	`persistent` / `secure`	Migration
`server/model.go` `EncryptionMode`	`persistent` / `secure`	Constant rename
`data-types.md`, `gateway.md`, `media-files.md`, `moderation.md`, `push-notifications.md`	`server_encrypted`	Find/replace

Cheapest done now, before any client depends on either name.

3. The modes¶

	`server_escrowed`	`rolling_key`	`peer_provided`
Message key held by	Server (Vault KEK)	Members' devices	Members' devices
Server can read	Yes — disclosed	No	No
Server holds any key material	Yes	No	No
Rotates	No	On join/leave	Out of band
History on join	Full	None	None
Server-side search	Yes	No	No
Subpoena exposure	Full content	Metadata only	Metadata only

peer_provided is specified in e2ee.md but has no delivery mechanism (keys are meant to arrive via DM). Open decision (§10): keep it or drop it — rolling_key covers the stated requirement on its own.

3.1 `server_escrowed`¶

Per-message random DEK, wrapped by a channel KEK in Vault. The change from today: the KEK is also delivered to members, so clients do the encryption and decryption. The server keeps a copy of the key so it can decrypt on demand — but does not do so in the normal path.

That distinction is the whole point. Today plaintext exists in every API response body, every nginx buffer, and every MESSAGE_CREATE NATS payload — all places that get logged, buffered and dumped on crash. Afterwards, plaintext exists server-side only when someone deliberately asks for it.

This is not E2EE. Do not call it E2EE.¶

End-to-end encryption means the endpoints hold keys the server does not. Here the server holds the key, so the guarantee is a policy one ("we don't decrypt") rather than a cryptographic one ("we can't"). Anyone with the key — an attacker inside Vault, or a court order — gets the plaintext.

Zoom shipped "end-to-end encryption" while holding keys and it cost them an FTC settlement and a great deal of credibility. For a product positioned as "Signal security", that is the most attackable claim available. It also contradicts this project's own stated principle of honest disclosure.

Accurate user-facing wording:

Encrypted on your device. We keep a copy of the key so search works, and we record every time it is used.

rolling_key is E2EE. server_escrowed is not. Say so in the channel-creation UI, where the choice is actually made.

Making "we don't decrypt" enforceable¶

Intent is not a control — today the decrypt path is the read path, so nothing prevents drift back. Three mechanisms, in increasing order of strength:

Separate the call sites. Nothing in the read path calls a decrypt function. Moderation and search call an explicit decryptForModeration(ctx, actor, reason). Different function, different review attention.
Log every use. audit_log already exists with actor_id, action, changes and reason, and is currently written from exactly one place. Add message.decrypt / media.decrypt actions. This turns the policy into something auditable, and optionally user-visible.
Remove the capability. Strongest: the API process does not hold the channel KEK at all. Search runs as a separate worker with its own Vault token and its own path, so "the API does not decrypt on read" is enforced by capability rather than by intent.

Aim for at least 1 and 2; 3 is the version worth advertising.

3.2 `rolling_key`¶

Epoch keys per e2ee.md. Key facts that must not be lost in implementation:

Grants are per device, sealed with crypto_box_seal to the device's identity key converted to X25519. Not per user — a user with three devices gets three grants.
The server independently recomputes member_hash and rejects a rotation whose grants do not cover every current member. This is what stops a malicious client from silently excluding someone.
ck_version is an optimistic concurrency counter; a losing racer gets 409 and retries.

4. Key distribution¶

Everything below depends on every user having published a key bundle. The client does not do this today (no /api/keys/bundle call, no Argon2, no PIN backup) — which is why both non-server modes are currently unreachable and everything silently falls back to server-side crypto.

This is the hard prerequisite for the entire document.

register / first login
  └─ generate IK (Ed25519), SPK, OTPKs
  └─ POST /api/keys/bundle          (public keys only)
  └─ PIN backup: Argon2id(pin, salt, 64MB, 3, p4) → masterKey
                 encrypted_blob = AES-256-GCM(masterKey, private keys)
                 PUT /api/keys/backup

The PIN restores private keys, not messages. Messages then become readable because those keys unwrap channel/epoch keys. Anyone expecting "my PIN decrypts my messages" has the wrong model — the PIN is one link in a chain.

5. Media¶

Media inherits its channel's encryption mode. One concept, not two.

Channel mode	File DEK wrapped to	Server can decrypt	Blocklist check at upload
`server_escrowed`	Media KEK (Vault)	Yes, on demand — never in the normal path	Client-side, advisory
`rolling_key` / `peer_provided`	Member devices	Never	Client-side, advisory

Thumbnails are generated client-side in every mode. Under §3.1 the server does not decrypt on the normal path, so it cannot resize an image for an escrowed channel any more than for a rolling-key one. This retires the planned services/media/internal/processor/ package — it is not needed and should not be written.

Avatars, icons and other public assets stay key_type = public — a plaintext public_dek that protects the storage layer only, not a confidentiality claim.

What changes from today¶

The current flow uploads plaintext (client-media.md: "The file field must contain the plaintext bytes. The server handles all encryption"), and the server generates thumbnails with imaging.Resize and matches sha256_hash against csam_blocklist. For non-server_escrowed channels all three become impossible:

Thumbnails move to the client in all modes (above). The existing server-side imaging.Resize path is removed along with the plan for processor/.
csam_blocklist matching at upload becomes advisory in all modes: the hash is client-supplied and therefore untrusted. Consistent with the stated client-side-only scanning philosophy, but say so plainly rather than letting it weaken quietly.
Push previews cannot include content.
Search moves to the client. See §6 — the index is built locally and shared between members as an opaque encrypted blob, so members piggyback on each other's indexing without the server being able to read any of it.

6. Search¶

Scope: this section is about user-facing search — a member finding a message in a channel they are already in. It is a product feature, and the shared index exists so members do not each repeat the same indexing work.

It is not a moderation mechanism. The index is never consulted for moderation, and §6.4 states that boundary explicitly. Search is where "the server does not read your messages" is easiest to lose by accident, so it gets its own rules.

6.1 The client builds the index¶

Indexing happens on the client, over plaintext it already has. This is what Signal does (a local full-text index) and it is the only approach that survives §3.1 — the server cannot index what it will not decrypt.

6.2 Why the index is NOT stored queryably¶

The tempting design is a server-side table of token → message_ids, with the token hashed or PRF'd so the server cannot read the term. Do not do this. A queryable index leaks by construction:

Deterministic tokens allow frequency analysis. The token for a given word is always the same value, so the server observes query frequency, result-set sizes and co-occurrence. Against a known language distribution that recovers terms at high rates.
Access patterns build a relationship graph — which messages match the same query, and which users search for the same things — even when no term is ever recovered.
Injection breaks it outright. An adversary who can add messages to the corpus can plant crafted ones and observe which queries return them, recovering query terms from very few injections. In this product the server can post, and so can every channel member. That is precisely the threat model in which searchable encryption degrades worst.

6.3 Shared, opaque index blobs¶

Instead the index is stored as an opaque encrypted blob per channel, which any member can extend and every member can use. The server stores bytes it cannot interpret: no tokens, no IDs, no result sets. Leakage collapses to blob size and update timing.

Sharing is safe because the index is a pure function of the channel's messages. Every member can already read every message it was built from, so an index encrypted under the channel key tells a member nothing they could not compute themselves. It only saves them the work.

Channel mode	Index blob encrypted under
`rolling_key` / `peer_provided`	The epoch key for the epoch it covers
`server_escrowed`	The channel KEK — server holds it, and still does not read it in the normal path

Chunks must align with epoch boundaries¶

For rolling_key channels the index is partitioned per epoch, encrypted under that epoch's key. This is not an optimisation — it is what stops the index becoming a history leak. A member who joined at epoch 5 cannot decrypt epochs 1–4, so they cannot read those index chunks either, and their search results match their message access exactly. A single index spanning all epochs would hand new members a searchable summary of history they are not allowed to read.

Escrowed channels have no rotation, so chunking is purely about update size: one chunk per channel per time bucket (a month works) means a new message rewrites only the current chunk.

Concurrent writers¶

Two members indexing at once must not clobber each other. Use the same optimistic-concurrency pattern as epoch rotation: a version counter on the chunk, 409 to the loser, who re-fetches and re-applies. Each chunk records the message range it covers so a client can see what is already indexed and do only the gap.

A shared index is untrusted input¶

Any member can write it, so a malicious member can poison it — omitting messages to hide them from search, or adding entries pointing at messages that do not contain the term.

Treat search hits as hints and verify them against the actual message before displaying. That reduces false entries to wasted work rather than incorrect results. Omission cannot be prevented, only detected: a client that searches its own local copy can notice hits the shared index missed. Rely on the shared index for speed, never for completeness guarantees.

Verifying hits the client never loaded¶

Verification needs the message, and the whole value of a shared index is finding messages this client never loaded. So the client resolves the misses: GET /api/channels/{id}/messages?ids=a,b,c returns exactly that set — sealed, channel-scoped, behind the same permission check as a normal page, capped at 100 ids. The client decrypts them locally and verifies. The server learns which message ids were fetched, which it already knows how to correlate; it does not learn the query or the plaintext.

A hit that cannot be fetched or decrypted is dropped, not shown on the index's word alone.

6.4 Boundary: moderation does not use this index¶

Nothing above is a moderation capability, and the shared index must not become one. Two separate reasons:

It cannot work. The blob is opaque to the server, and its per-epoch chunks are readable only by members holding that epoch key. A moderator who was not in the channel at the time cannot decrypt it.
It should not work. Repurposing a convenience cache into a surveillance index is exactly the drift this document exists to prevent.

Server-wide moderation search stays what §3.1 describes: a deliberate decrypt with an audit_log entry naming the actor and reason. That is the right shape anyway — it is the operation that most deserves to be visible.

7. CSAM: local scan, consented report, verified provenance¶

7.1 Scanning — unchanged, client-side only¶

The server never scans proactively. It distributes an encrypted hash DB (GET /media/csam/hashdb); the client hashes locally with no network call per image. Opt-out, on by default. On match: blur, warn, offer Report.

7.2 The verification problem¶

A report must prove the submitted plaintext is the file the accused actually sent. Otherwise anyone can submit arbitrary content plus a victim's user ID and have them suspended.

This is not hypothetical: the implemented handler takes reported_user_id, attachment_id and decrypted_blob entirely from the client and verifies no relationship between them.

Two approaches that do not work:

Re-encrypt and compare ciphertext. crypto.Seal uses a fresh random DEK and AES-GCM uses a random nonce, so encrypting the same plaintext twice never reproduces the stored bytes. Forcing a match requires reusing both DEK and nonce, and AES-GCM nonce reuse is catastrophic — it leaks the XOR of plaintexts and permits tag forgery. Disqualified outright.
A plaintext hash stored at upload. Works when the server can decrypt (it can check the hash is honest). Useless when it cannot: a malicious uploader submits a bogus hash and becomes permanently unmatchable.

7.3 The solution: the reporter discloses one file key¶

The reporter already holds the file's DEK — they decrypted the file to view it, which is how the local scan ran at all.

1. Client detects a hash match, prompts, user consents to report.
2. Client sends:
     reported_user_id, channel_id, message_id, attachment_id
     decrypted_blob   — the plaintext
     file_dek         — that one file's DEK, unwrapped from the reporter's grant
     content_hash     — perceptual hash
     ncmec_hash_match — which DB entry matched
3. Server verifies, in order:
     a. attachment exists and attachment.uploader_id == reported_user_id
     b. AES-GCM-Open(stored_ciphertext, file_dek) succeeds
     c. constant-time compare of the result against decrypted_blob
   Any failure → reject the report. Do not store evidence.
4. On success: seal decrypted_blob under the isolated evidence KEK, store in
   object storage, insert csam_reports row, publish to NATS.

Why (b) is sufficient proof. The binding comes free from the AEAD. A malicious reporter cannot construct a DEK that makes someone else's stored ciphertext authenticate to plaintext of their choosing — that is tag forgery, and it is infeasible. Successful decryption under a supplied key is proof the ciphertext really is that plaintext.

Why this satisfies §1. The server holds no standing decryption ability for rolling_key media. It gains the ability to read exactly one file, only when a recipient of that file consents to disclose it. Consent-scoped, per-file, verifiable.

For server_escrowed channels the server can skip the supplied DEK and decrypt the stored file itself — but it should still run checks (a) and (c), because the framing vector is identical in both modes.

7.4 Report API¶

client/moderation.md documents {attachment_id, evidence_note} — which is not what the server implements and is missing everything above. It must be replaced with the §7.3 body. Flagged in §10 as a doc that will contradict this one until updated.

8. What each mode gives up¶

State this in the UI, not just here. "Honest disclosure" is a stated project principle and it only holds if the tradeoff is visible at the moment of choice.

Feature	`server_escrowed`	`rolling_key`
Message search	✅ Client-side, shared index (§6)	✅ Client-side, shared index (§6)
Moderation search across a server	⚠️ On demand, decrypts + logs	❌
History when you join	✅ Full	❌ None
Thumbnails	❌ client-side (both modes)	❌ client-side
Push notification previews	✅	❌ metadata only
Trusted blocklist at upload	✅	❌ advisory
Readable under subpoena	⚠️ Yes	✅ No
Recoverable if you lose all devices	✅	⚠️ PIN backup only

That last row matters more than it looks. In rolling_key, losing every device without a PIN backup means the history is gone — there is no server-side copy of the key. The PIN backup is not optional UX polish; it is the only recovery path.

9. Current implementation status¶

Piece	Status
`channels.encryption_mode`, `ck_version`	Built (migration 000013)
`channel_keys` table + `GET`/`PUT /api/channels/{id}/keys`	Superseded by epoch keys (000015). Unused; still to be dropped.
Envelope encryption, Vault KEK provider	Built and in use
Keyserver: bundle, OTPKs, PIN backup, lockout	Built, client publishes and restores
Client X3DH + Double Ratchet + AES-GCM	Built, self-test passes, and in use — DMs run on it end to end.
DM E2EE UI	Built — conversation list, chat view, X3DH on first message, ratchet sessions and decrypted plaintext persisted device-locally
Safety-number / identity-change warning	Built — first identity key per peer is pinned, changes block sending until reviewed, 60-digit number to compare out-of-band (`e2ee.md` § Safety Numbers)
Key transparency	Not built, and not planned here — safety numbers only help users who actually compare them. A server that swaps a key for a user who never checks is still undetected; real protection needs a verifiable append-only log of identity keys
Per-device identities	Not built — every JWT carries `did: "default"`, so multi-device DM fan-out is implemented but untestable
Epoch tables / grants / `member_hash` validation	Built (migration 000015)
Client key-bundle upload, PIN backup UI	Built — Settings › Security
Channel key delivery to clients	Built — escrow key for `server_escrowed`, per-device sealed grants for `rolling_key`
Client message encryption + key publishing	Built — steps 1–5
Client-side media encryption	Built — `POST /api/media/sealed`, DEK wrapped by the channel key (migration 000014). `media_file_grants` / `public_dek` not needed with this design.
Client-side thumbnailing	Not built — images render full-size for now
Client-side search + shared index blobs	Built (§6) — migration 000016, `search-index` endpoints, `?ids=` hit resolution, live UI
`services/media/internal/processor/`	Retired — will not be written (see §5)
Argon2id + AES-GCM + PIN backup on the client	Built — KAT-verified against the Go implementation
CSAM report verification	Built — reporter supplies the DEK, server re-seals and compares
Server-side rotation on join/leave	Not built — clients own it, driven by gateway `MEMBER_JOIN`/`MEMBER_LEAVE`
`DecryptForModeration` `audit_log` row + HTTP route	Not built — logs via zap only
Enum rename to `server_escrowed` / `rolling_key`	Not done — decided in §2, still `persistent` / `secure` on the wire

10. Open decisions¶

Keep peer_provided? It has no delivery mechanism and rolling_key satisfies the requirement. Dropping it removes a mode from every doc and UI.
~~Rename the enum now or later?~~ Decided: rename, and use server_escrowed. See §2 for the surfaces that change. Still to schedule, but the target names are settled.
Per-device or per-user grants? The specs say per device; the code is per user. Recommendation: keep per device.

Per device contains a compromise — a stolen laptop exposes only that device's grants, and revoking it just means future epochs are not sealed to it. Per user means one compromised device exposes the user's key and revocation forces a re-key of everything they can see. Per device also means a device never holds the user's long-term private key, only its own.

The objection is history: add a phone, see nothing. But that cost comes from rotation policy, not grant granularity — you cannot read epochs issued before your grant either way. A new device is simply the same case as a new member. Per device makes it a routine, visible event rather than a rare one.

The lever is key transfer between your own devices:

Option	Gives history	Cost
PIN backup only	❌ restores identity keys, not epoch keys	Already specced and built server-side
Device-to-device epoch handoff	✅	An existing device seals its cached epoch keys to the new one — reuses the async grant issuance machinery in `e2ee.md`
Accept the gap	❌ new devices start from "now"	Zero work; what Signal does; easy to explain

Treat "new device gets history" as a separate feature built on grant handoff, not a reason to collapse to per-user keys. Whichever is chosen changes what "secure channel" promises, so it belongs in the UI copy. server_escrowed channels are unaffected — the server holds the key, so any device gets full history. 4. Message cipher. Specs say XChaCha20-Poly1305 for rolling_key; the code uses AES-256-GCM. Both are fine — pick one and make pkg/crypto and the Dart engine agree, since they must be byte-compatible.

Note the constraint this runs into: the vendored libsodium.js sumo build exports crypto_aead_aes256gcm_is_available but not the AES-GCM functions themselves, and that check returns false in browsers regardless. A web client therefore cannot do AES-256-GCM through libsodium — it must use the Web Crypto API (crypto.subtle), which is what the PIN backup client does. Argon2id is available (crypto_pwhash_alg_argon2id13), so key derivation has no such problem. 5. ~~Does server_escrowed stay the default?~~ Reversed on 2026-09-05: no. New channels default to secure. (Migration 000021 plus handler.CreateChannel.)

The original decision rested on two premises, and one of them stopped being true. Escrowed was called "the only mode that supports history-on-join and search" — but the encrypted shared index now covers rolling-key channels, keyed per epoch, so search works there too. What escrowed uniquely still buys is history for members who join later, and server-side moderation decrypt, which secure refuses by design.

The second premise was implicit and wrong: it reasoned about "the server" as a self-hosted box. ZippChat runs the main production server, so escrowed means the operator of a hosted service holds the key and can be compelled to produce plaintext — a materially different disclosure from the one §3.1 was written to cover. A product claiming Signal security cannot have that as what a user gets by not reading a dialog.

persistent remains fully supported as an explicit opt-out, and its disclosure obligation in §3.1 is unchanged and still binding. Existing channels keep their mode; nothing is silently re-keyed. See hosting-model.md.

11. Implementation order¶

Each step is blocked by the one before it. Steps 1–9 are shipped; the remainder of this list is kept because it records why the order was what it was. See §9 for what is actually built.

Client key-bundle upload + PIN backup. Nothing else is reachable without published public keys. Crypto core already exists and is proven.
Deliver channel keys to members. Extend /api/channels/{id}/keys to server_escrowed, wrapped per device.
Move message crypto to the client. Send sealed envelopes; List returns ciphertext; client decrypts. Breaking API change.
Stop leaking plaintext through the gateway. events.MessagePayload.Content currently carries plaintext to every subscriber.
Explicit server-side decrypt path for search and moderation only: decryptForModeration(ctx, actor, reason), an audit_log entry per use, and ideally no channel KEK in the API process at all (§3.1).
Client-side media encryption + media_file_grants, and client-side thumbnailing for all modes. Delete the server imaging.Resize path; do not write processor/.
CSAM report verification per §7.3, plus the uploader_id ownership check — which is worth doing immediately and independently, since the framing vector is live today.
Epoch keys + server-validated rotation for rolling_key.
Client-side search: local index, then shared encrypted index chunks aligned to epoch boundaries (§6). Depends on step 3 — there is nothing to index client-side until the client holds plaintext.

Step 7's ownership check is the only item here that is both small and urgent; it does not depend on steps 1–6.

What remains after step 9¶

Client-side thumbnailing (step 6's unfinished half).
audit_log row + an HTTP route for DecryptForModeration (step 5's).
Server-side rotation driven by MEMBER_JOIN / MEMBER_LEAVE; clients own it today.
The enum rename decided in §2.
Drop the superseded channel_keys table and its endpoints.