structured memory: facts/pinned/episodes, batched consolidation, participant-scoped recall

This commit is contained in:
Oleksandr Kozachuk
2026-07-13 14:12:21 +02:00
parent 02c989946b
commit 3d2289496b
14 changed files with 608 additions and 90 deletions
+8 -6
View File
@@ -69,9 +69,10 @@ is the channel the message came from.
### ENV-10 — System prompt template substitution (coverage: test)
`message()` substitutes `{date}` (YYYY-MM-DD), `{time}`, `{memory}`
(current memory string) in the system prompt; `{news}` is replaced
with the news file content when the configured file exists and stays
literal when it does not.
(the assembled memory block — legacy memory string while the
structured memory is inactive, see MEM-10) in the system prompt;
`{news}` is replaced with the news file content when the configured
file exists and stays literal when it does not.
### ENV-11 — Per-channel history shrink prefers busy channels (coverage: test)
@@ -92,10 +93,11 @@ back-to-back (D1).
records the clearing in the channel's memory. (D7: the previous
handler declared `(reaction, user)` and crashed on dispatch.)
### ENV-14 — update_memory persists its argument (coverage: test)
### ENV-14 — update_memory persists its argument (coverage: withdrawn — successor SPEC-002)
`update_memory(memory)` sets the responder's memory to the passed
value and persists that value. (D12: the argument was ignored.)
Withdrawn 2026-07-13 with FDB-007: the per-answer memory rewrite
(`update_memory`/`memoize`) is deleted; structured memory (MEM-01+)
replaces it. The D12 defect died with the code.
### ENV-15 — retry-model is used after a rate limit (coverage: test)
+79
View File
@@ -0,0 +1,79 @@
# SPEC-002 — Structured memory
Replaces the single-string per-answer LLM rewrite (lossy, O(convo)
cost — the old `memoize` path). Layers, simplified per plan v4:
**user facts** (durable, provenance-tracked), **pinned facts**
(operator-set, global or per channel), **episodes** (rolling channel
summaries with decay). Channel-facts deferred until recall proves
insufficient. Raw feed = **observations** (messages, reactions,
edits, deletes); an async consolidation pass turns observations into
facts + episodes. The memory system is active only when both a store
(`history-directory`) and a `memory-model` are configured — otherwise
the legacy memory string is used read-only (MEM-10).
### MEM-01 — Events become observation rows (coverage: test)
User messages, bot answers, reactions (add/remove/clear), edits and
deletes are recorded as observation rows (channel, user, kind,
content excerpt) — cheap writes, no LLM call per event.
### MEM-02 — Consolidation is batched, never per message (coverage: test)
Consolidation triggers when `memory-consolidate-every` (default 20)
unconsumed observations have accumulated for a channel; it runs as a
background task guarded by a lock (no overlapping runs), consumes the
observations it processed, and leaves them in place when the model
call fails (retry next trigger).
### MEM-03 — Only self-authored facts persist (coverage: test)
The consolidator stores a fact only when its subject user is among
the authors of the consumed observations, with provenance
`source='self'`; facts the model attributes to absent third parties
are discarded and logged. Operator pins carry `source='operator'`.
"Bob says Alice likes X" must never become Alice's profile
(review consensus C2).
### MEM-04 — Recall is participant-scoped (coverage: test)
The memory block assembled into the system prompt contains: pinned
facts (global + this channel), user facts of **conversation
participants only** (current author + authors in the recent history
tail), and the channel's recent episodes. Facts of non-participants
never enter the prompt — the model cannot leak what it cannot see.
### MEM-05 — Episodes decay (coverage: test)
At most `memory-episodes-per-channel` (default 10) episodes are kept
per channel; consolidation drops the oldest beyond the cap.
### MEM-06 — User facts have a retention limit (coverage: test)
Facts not updated within `memory-fact-retention-days` (default 180)
are purged during consolidation. Durable is not indefinite (GDPR
storage limitation).
### MEM-07 — Staff review and edit memory (coverage: test)
Staff commands: `!bot memory <user>` lists the user's facts with ids;
`!bot forget-fact <id>` deletes one; `!bot pin <channel|global>
<fact>` adds an operator pin; `!bot unpin <id>` removes one.
### MEM-08 — Legacy memory strings migrate to episodes (coverage: test)
Schema v3 migration copies existing per-channel memory strings into
an episode row each; pickle migration does the same. Deployments keep
their accumulated context through the upgrade.
### MEM-09 — !forgetme erases facts, observations and episode traces (coverage: test)
`!forgetme` now deletes the user's facts, their observation rows, and
episodes mentioning the user's name — in addition to the SAF-08
history purge. This completes the erasure that SAF-08 v1 could not.
### MEM-10 — Memory system off degrades gracefully (coverage: test)
Without `memory-model` (or without a store) no observations are
written, no consolidation runs, and `{memory}` falls back to the
legacy memory string — no crash, no behavior change for
unconfigured deployments.
+2 -3
View File
@@ -66,9 +66,8 @@ still goes out).
`!forgetme` removes the requesting user's messages from all live
responder histories and from the store, then confirms in-channel.
v1 limitation, stated in the confirmation: the single-string channel
memory may still contain summarized traces — full fact-level erasure
arrives with the structured memory (FDB-007).
Since FDB-007 the purge extends to memory itself — facts,
observations and episode traces (MEM-09).
### SAF-09 — !privacy states the data practice (coverage: test)