Back up and restore

Backup inventory

Track control-plane PostgreSQL, Forgejo metadata and repositories, signed artifacts, app databases, routing/policy metadata, and recoverable secrets/KMS material. Document the weaker KV recovery contract separately. Store recovery material outside the primary runtime volume with controlled access and encryption.

App backup proposals are frequent online recovery points targeting a 15-minute RPO, daily copies retained 30 days, monthly copies retained 12 months, and pre-migration snapshots retained at least 30 days. These are pilot defaults pending contract and drill validation.

Routine backup verification

1. Inventory every production app/environment and confirm a scheduled backup policy exists.
2. Check last successful recovery point, copy destination, encryption/key version, integrity result, and retention state.
3. Alert on failed backups, stale recovery points, missing keys, or retention violations. Assign an operator and affected app owner.
4. Restore samples into isolated validation environments; a successful upload alone is not evidence of recoverability.
5. Record backup and restore metrics without logging application rows, credentials, or sensitive export URLs.

App database restore

1. Identify the incident, app/environment, selected recovery point, current schema, and compatible artifact. Describe potential lost writes since that point.
2. Obtain the authorized human recovery decision; record it separately from artifact rollback approval.
3. Restore into a new DB instance. Verify tenant scope, decryption, integrity, migration checksums, and schema compatibility.
4. Attach the new instance to an isolated validation environment. Restrict access and integrations; validate approved workflow checks.
5. Review results and authorize cutover. Quiesce writes where required and record the final recovery boundary.
6. Atomically switch the DB binding to the validated instance and run production health checks.
7. Retain the prior DB under the approved recovery window. Record both instance IDs, approver, timestamps, and validation evidence.
8. If validation fails before cutover, leave the live binding intact. If writes occurred after cutover, do not switch back blindly; reconcile the competing write histories under incident control.

Platform reconstruction

Restore authoritative PostgreSQL and Forgejo from a compatible recovery set. Reestablish KMS/secrets access and cluster identity. Reconcile a replacement runtime from desired state, signed artifacts, route/policy metadata, and app DB recovery points. Check cluster assignment and tenant boundaries before opening traffic. KV caches may need regeneration under their documented policy.

Export and drills

Verify full Git-history export plus SQLite/SQL/CSV exports, migrations, and schema manifests can be restored independently. Scope and audit each export; record expiry and access control without storing secret-bearing URLs in evidence.

Measure achieved data-loss interval and recovery duration against the pilot targets of RPO ≤15 minutes and RTO ≤4 hours. Record failures and rerun after fixes. These targets are not contractual SLAs until validated and agreed.