ops: consistent rotated db backups + cron, consecutive-api-error staff alert

specs/SPEC-012-ops.md

@@ -0,0 +1,32 @@
+# SPEC-012 — Operations hardening
+
+Runtime + host operability (FDB-012). Backups and host wiring are
+`manual` coverage; the in-process alerting is `test`.
+
+### OPS-13 — Consistent DB backups (coverage: test)
+
+`deploy/backup_db.py` writes a gzipped snapshot of `bot.db` using the
+sqlite3 online-backup API — consistent even while the bot writes
+(WAL-safe) — with 0600 permissions. Restoring a snapshot yields a
+readable database with the same rows.
+
+### OPS-14 — Backups are rotated (coverage: test)
+
+The newest `backup-keep` (default 14) snapshots are kept; older ones
+are deleted. Timestamped names sort chronologically so rotation is a
+pure list operation.
+
+### OPS-15 — Backup cron on each host (coverage: manual)
+
+Each host runs `backup_db.py` daily via cron, writing to
+`~/backups/<bot>/` (outside `~/fjerkroa_bot`, so deploys and service
+restarts never touch it). Verified by presence of the cron line and a
+fresh snapshot.
+
+### OPS-16 — Repeated API errors alert staff (coverage: test)
+
+The responder counts consecutive OpenAI request failures; at
+`api-error-alert-threshold` (default 5) in a row it fires one staff
+alert (rate-limited like all staff alerts) so a silently-broken bot
+(cf. the gpt-5.6 tools/reasoning incident) surfaces within minutes
+instead of hours. A success resets the counter.