Hosted Backups Design
Status: design. Target: ADR-0030.
Goal
Two independent backup paths so a single operator failure cannot lose founder data.
- Founder-owned GitHub mirror. Every commit on the tenant volume is pushed to a founder-owned GitHub repo within 60 seconds. This is the canonical recovery path.
- Encrypted daily snapshot to founder-owned S3. A nightly tarball encrypted with the tenant DEK, uploaded to a bucket the founder controls (BYO bucket + IAM role).
Why founder-owned
If we hold the only copy, we are the single point of failure. Both paths land in storage the founder controls. We hold a write role, not a read role.
Path 1: GitHub mirror
Setup
- Founder connects GitHub via OAuth (Settings → Backups).
- We create (or accept) a private repo
lycato-<slug>under the founder's account or org. - We store the install token in the tenant secret store (encrypted with the tenant DEK).
Per-commit push
- A
post-commithook in the tenant volume's git config pushes to the founder GitHub remote. - On push failure, we retry with exponential backoff (max 6 attempts over ~10 minutes), then alert (push notification + email).
- The push includes the full commit history, not just deltas — the GitHub repo is always a complete mirror.
Recovery
If the tenant container is gone, the founder clones the GitHub repo locally and runs:
git clone [email protected]:<founder>/lycato-<slug>.git company
cd company
lycato doctor # confirms repo health
lycato migrate # if schema version is behind
lycato cockpit # back to work
The founder is now running self-hosted. They can stay there or provision a fresh hosted tenant and push the repo in.
Path 2: Encrypted S3 snapshot
Why also have S3
The GitHub mirror covers the markdown + git history. It does NOT cover:
.founder/keys.env— gitignored on purpose..founder/runs/*.jsonl— gitignored telemetry.Lycato/cache/index.json— gitignored derived data.inbox/attachments/audio files — gitignored (large blobs).
A full S3 snapshot covers everything, encrypted, so a true "clone the box" recovery is possible.
Format
- Daily at 03:30 UTC.
tar czf - <tenant-volume>/piped throughage -r <tenant-recipient> -o snapshot-<date>.tar.gz.age.- Uploaded to
s3://<founder-bucket>/lycato/<tenant-slug>/snapshot-<date>.tar.gz.age.
Encryption
- We use
age(not GPG) for the tarball encryption. - The recipient key is the tenant's public key; the private key is derived from the tenant DEK at restore time.
- We do not hold the founder's S3 bucket key. We have a write role via IAM trust — the founder can rotate or revoke at any time.
Retention
- Founder plan: 7 daily snapshots, 4 weekly snapshots.
- Team plan: 30 daily snapshots, 12 weekly snapshots, 12 monthly snapshots.
- Lifecycle policy enforced via S3 lifecycle rules (we set them on the bucket on first backup).
Restore
Founder-initiated only:
- Cockpit → Settings → Restore from snapshot → pick date.
- We provision a temporary container.
- Container fetches the encrypted tarball, decrypts with the tenant DEK, mounts a new volume.
- Founder reviews; on confirm, the new volume replaces the live one.
No operator-initiated restore. The founder owns the path back.
Disaster scenarios
| Scenario | Recovery path |
|---|---|
| Container crash | Auto-restart; data preserved on volume |
| Volume corruption | Latest S3 snapshot → restore |
| Hosted operator data loss | Founder's GitHub repo → self-host |
| Founder loses GitHub access | S3 snapshot (founder still owns bucket) |
| Founder loses both | We retain a 90-day operator backup as last-resort, encrypted with our HSM key, restored only on signed founder request via privacy@Lycato. Not a replacement; insurance. |
Audit
Every backup attempt + every restore writes to:
- Tenant audit log (
.founder/audit/YYYY-MM.jsonl). - Control-plane
audit_opstable (operator-visible). - Push notification to founder for restores.
What's intentionally absent
- Point-in-time restore (PITR). Daily granularity is sufficient.
- Cross-region replication of our operator-side last-resort backup. Out of scope at launch.
- Customer-managed encryption keys for the S3 path. Founder owns the bucket; that's already a strong control.