structured memory: facts/pinned/episodes, batched consolidation, participant-scoped recall

specs/SPEC-002-memory.md

@@ -0,0 +1,79 @@
+# SPEC-002 — Structured memory
+
+Replaces the single-string per-answer LLM rewrite (lossy, O(convo)
+cost — the old `memoize` path). Layers, simplified per plan v4:
+**user facts** (durable, provenance-tracked), **pinned facts**
+(operator-set, global or per channel), **episodes** (rolling channel
+summaries with decay). Channel-facts deferred until recall proves
+insufficient. Raw feed = **observations** (messages, reactions,
+edits, deletes); an async consolidation pass turns observations into
+facts + episodes. The memory system is active only when both a store
+(`history-directory`) and a `memory-model` are configured — otherwise
+the legacy memory string is used read-only (MEM-10).
+
+### MEM-01 — Events become observation rows (coverage: test)
+
+User messages, bot answers, reactions (add/remove/clear), edits and
+deletes are recorded as observation rows (channel, user, kind,
+content excerpt) — cheap writes, no LLM call per event.
+
+### MEM-02 — Consolidation is batched, never per message (coverage: test)
+
+Consolidation triggers when `memory-consolidate-every` (default 20)
+unconsumed observations have accumulated for a channel; it runs as a
+background task guarded by a lock (no overlapping runs), consumes the
+observations it processed, and leaves them in place when the model
+call fails (retry next trigger).
+
+### MEM-03 — Only self-authored facts persist (coverage: test)
+
+The consolidator stores a fact only when its subject user is among
+the authors of the consumed observations, with provenance
+`source='self'`; facts the model attributes to absent third parties
+are discarded and logged. Operator pins carry `source='operator'`.
+"Bob says Alice likes X" must never become Alice's profile
+(review consensus C2).
+
+### MEM-04 — Recall is participant-scoped (coverage: test)
+
+The memory block assembled into the system prompt contains: pinned
+facts (global + this channel), user facts of **conversation
+participants only** (current author + authors in the recent history
+tail), and the channel's recent episodes. Facts of non-participants
+never enter the prompt — the model cannot leak what it cannot see.
+
+### MEM-05 — Episodes decay (coverage: test)
+
+At most `memory-episodes-per-channel` (default 10) episodes are kept
+per channel; consolidation drops the oldest beyond the cap.
+
+### MEM-06 — User facts have a retention limit (coverage: test)
+
+Facts not updated within `memory-fact-retention-days` (default 180)
+are purged during consolidation. Durable is not indefinite (GDPR
+storage limitation).
+
+### MEM-07 — Staff review and edit memory (coverage: test)
+
+Staff commands: `!bot memory <user>` lists the user's facts with ids;
+`!bot forget-fact <id>` deletes one; `!bot pin <channel|global>
+<fact>` adds an operator pin; `!bot unpin <id>` removes one.
+
+### MEM-08 — Legacy memory strings migrate to episodes (coverage: test)
+
+Schema v3 migration copies existing per-channel memory strings into
+an episode row each; pickle migration does the same. Deployments keep
+their accumulated context through the upgrade.
+
+### MEM-09 — !forgetme erases facts, observations and episode traces (coverage: test)
+
+`!forgetme` now deletes the user's facts, their observation rows, and
+episodes mentioning the user's name — in addition to the SAF-08
+history purge. This completes the erasure that SAF-08 v1 could not.
+
+### MEM-10 — Memory system off degrades gracefully (coverage: test)
+
+Without `memory-model` (or without a store) no observations are
+written, no consolidation runs, and `{memory}` falls back to the
+legacy memory string — no crash, no behavior change for
+unconfigured deployments.