Backups and restores

backups backs up everything the instance needs to come back: every service's database and data folder, /etc/xvii, the release store, and every database apps manages.

Destinations

Destination	Notes
S3-compatible	any provider; a bucket and a key with access to it
SFTP	another server; backups makes an Ed25519 key pair, you add the public half there
Local folder	a second disk; not a protection against losing the server
Telegram channel	AES-256-GCM encrypted 19.9 MB parts and a manifest; full snapshots, no deduplication

S3, SFTP and local folders are restic repositories: encrypted, deduplicated, incremental. Use at least one destination outside the control plane's provider.

Plans

A plan has a schedule, its sources, its destinations and a retention per source. A plan with no sources takes everything, including services installed later. Each run makes one snapshot per source.

Source	Is
`/etc/xvii`	domain, desired services and versions, settings and generated secrets
releases	the release store
`db:xvii_<service>`	a platform service's database
data folders	each platform service's folder under `/var/lib/xvii` (keys, files)
`app:<id>`	a database apps manages, streamed from its node

The recovery kit

When a destination is created, backups shows a recovery kit: the repository passwords and storage keys needed to restore without a working instance. Download it and keep it somewhere that isn't the instance. Later it can be shown again only in a logged panel session.

Verification and drills

Weekly, each repository is checked with restic check --read-data-subset 5%.
Monthly, a drill restores every platform database into a scratch database and checks it.
Failures send backup.failed, backup.verify_failed to notify. Subscribe to them.

A backup you haven't restored is a hope, not a backup: restore one into a test server once.

Restore part of it

From the backups panel:

a database, into a new name or in place;
a folder;
one file, as a download;
an app's database, in place.

A restore replaces data. Stop writers, confirm the target, keep a fresh backup of what you are about to replace, and check the application afterwards with real reads and writes.

Restore the whole instance

On a fresh server, with the recovery kit:

xvii server restore root@203.0.113.7 --kit xvii-recovery.txt

Over SSH it installs restic, PostgreSQL, Caddy and xvii-node, then restores from the newest snapshots: /etc/xvii (the domain, services, settings and secrets), the releases, and every service's database and data folder. The services start with their data. Point the instance's DNS at the new server for certificates and sign-in to work.

Then check: sign in, open every panel, check that nodes reconnect (xvii apps nodes) and name servers pull. Nodes keep running their apps while the control plane is away and reconnect on their own, but only edge nodes keep serving them: a relay node's visitors come through the control plane's Caddy, so its apps are unreachable until the control plane is back.

What losing the control plane costs

Downtime: the panels, sign-in, the APIs and every app on a relay node, until the control plane runs again. The alerts come from it too (notify and status run there), so watch it from outside: an uptime check that runs somewhere else, on https://dvxch.team and https://id.dvxch.team.
Data: a daily plan can lose up to a day of platform data (sessions, env versions, docs edits, AI chats). If that is too much, add a plan with only the db: sources that runs every hour: restic deduplicates, so each run stores only what changed.
Time to come back: restore the instance onto a scratch server and time it. Write down how much you can lose (RPO) and how long you can be down (RTO), and check both after every such drill.

Keys

Several services seal secrets with a key in their data folder (env's master key, apps' seal key, domains' DNSSEC keys). Those folders are part of the backup. A database restored without its folder has secrets nobody can open, and a lost DNSSEC key with a DS still published is a DNS outage.