discord_bot/specs/SPEC-012-ops.md

SPEC-012 — Operations hardening

Runtime + host operability (FDB-012). Backups and host wiring are manual coverage; the in-process alerting is test.

OPS-13 — Consistent DB backups (coverage: test)

deploy/backup_db.py writes a gzipped snapshot of bot.db using the sqlite3 online-backup API — consistent even while the bot writes (WAL-safe) — with 0600 permissions. Restoring a snapshot yields a readable database with the same rows.

OPS-14 — Backups are rotated (coverage: test)

The newest backup-keep (default 14) snapshots are kept; older ones are deleted. Timestamped names sort chronologically so rotation is a pure list operation.

OPS-15 — Backup cron on each host (coverage: manual)

Each host runs backup_db.py daily via cron, writing to ~/backups/<bot>/ (outside ~/fjerkroa_bot, so deploys and service restarts never touch it). Verified by presence of the cron line and a fresh snapshot.

OPS-16 — Repeated API errors alert staff (coverage: test)

The responder counts consecutive OpenAI request failures; at api-error-alert-threshold (default 5) in a row it fires one staff alert (rate-limited like all staff alerts) so a silently-broken bot (cf. the gpt-5.6 tools/reasoning incident) surfaces within minutes instead of hours. A success resets the counter.