add spec system: sdd/bdd/tdd loops, trace enforcement, envelope+config specs
This commit is contained in:
Oleksandr Kozachuk
@@ -0,0 +1,69 @@
|
||||
# SPEC-000 — Development process (binding)
|
||||
|
||||
This repo is developed spec-driven + behaviour-driven + test-driven.
|
||||
|
||||
## The three loops
|
||||
|
||||
1. **Spec-driven**: every behavior exists first as a numbered
|
||||
requirement (`<AREA>-NN`) in `specs/SPEC-NNN-*.md`. No code without
|
||||
a requirement. Requirements are never deleted — a dead requirement
|
||||
is marked `(withdrawn: <successor or reason>)` in its title line
|
||||
and keeps its ID forever.
|
||||
2. **Behaviour-driven**: every *user-visible* requirement gets at
|
||||
least one Gherkin scenario tagged `@<ID>` in `features/*.feature`,
|
||||
executed by pytest-bdd.
|
||||
3. **Test-driven**: implementation starts at a red test. Unit tests
|
||||
cover the non-visible requirements (arithmetic, invariants,
|
||||
parsing).
|
||||
|
||||
Change flow: spec change → feature/test red → implementation green →
|
||||
refactor.
|
||||
|
||||
## Requirement format
|
||||
|
||||
```
|
||||
### ENV-01 — Model answer reaches the user (coverage: feature)
|
||||
|
||||
Normative statement first. Rationale after.
|
||||
```
|
||||
|
||||
Coverage classes:
|
||||
|
||||
- `feature` — needs an `@<ID>` tag in some `.feature` file.
|
||||
- `test` — the ID must appear in a test file (docstring or comment
|
||||
of the covering test).
|
||||
- `manual` — needs a row in `manual-verification.md` with date and
|
||||
result before the increment demo.
|
||||
|
||||
## Enforcement
|
||||
|
||||
`tools/trace.py` (stdlib-only) runs in `make check` (and `make
|
||||
trace`). It fails the build when a declared requirement lacks its
|
||||
coverage class artifact, when a feature tag references an undeclared
|
||||
ID, or when an ID is declared twice. Code fences in spec files are
|
||||
ignored by trace (the example above does not declare ENV-01).
|
||||
|
||||
## Test substrate
|
||||
|
||||
BDD scenarios run against the **responder seam**: a
|
||||
`FakeModelResponder` subclass with scripted model output — no live
|
||||
Discord, no live OpenAI (see DECISIONS.md D-002). Discord-event
|
||||
behavior is covered by unit tests with mocked discord.py objects.
|
||||
|
||||
## ADRs
|
||||
|
||||
Decisions inside the set architecture go to `DECISIONS.md` as
|
||||
`D-NNN — title — one-paragraph rationale`. The stack itself is not
|
||||
re-litigated there.
|
||||
|
||||
## Spec map
|
||||
|
||||
- SPEC-000 process (this file, no IDs)
|
||||
- SPEC-001 responder + envelope — ENV-NN
|
||||
- SPEC-002 memory — MEM-NN (lands with FDB-007)
|
||||
- SPEC-003 safety/privacy/abuse — SAF-NN (lands with FDB-014)
|
||||
- SPEC-004 images — IMG-NN (lands with FDB-009/010)
|
||||
- SPEC-005 self-tasking — TSK-NN (lands with FDB-011)
|
||||
- SPEC-006 staff/operator controls — OPS-NN (lands with FDB-005)
|
||||
- SPEC-007 deploy/hosts — DEP-NN (lands with FDB-017, mostly manual)
|
||||
- SPEC-008 config — CFG-NN
|
||||
@@ -0,0 +1,133 @@
|
||||
# SPEC-001 — Responder + response envelope
|
||||
|
||||
Characterization of the current (v2) responder contract in
|
||||
`fjerkroa_bot/ai_responder.py`. The model answers with a JSON
|
||||
envelope: `answer`, `answer_needed`, `channel`, `staff`, `picture`,
|
||||
`picture_edit`, `hack`. FDB-005 will replace the transport of this
|
||||
contract (structured outputs); the *behavioral* requirements below
|
||||
survive that change unless marked otherwise.
|
||||
|
||||
### ENV-01 — Model answer reaches the user (coverage: feature)
|
||||
|
||||
When the model envelope carries a non-empty `answer` and
|
||||
`answer_needed` is true, `AIResponder.send()` returns an `AIResponse`
|
||||
with that answer text and `answer_needed == True`. This is the core
|
||||
loop: user talks, bot answers.
|
||||
|
||||
### ENV-02 — Suppressed answer stays silent (coverage: feature)
|
||||
|
||||
When the model envelope sets `answer_needed` to false (and the
|
||||
message is not direct, mentions nobody, and no staff note is set),
|
||||
the returned `AIResponse` has `answer_needed == False`. The bot may
|
||||
observe without butting in.
|
||||
|
||||
### ENV-03 — Staff note forces delivery (coverage: feature)
|
||||
|
||||
When the envelope carries a non-null `staff` text and a non-null
|
||||
`answer`, the returned response preserves the staff text and has
|
||||
`answer_needed == True`. Staff alerts must never be silently
|
||||
dropped.
|
||||
|
||||
### ENV-04 — Direct messages are always answered (coverage: feature)
|
||||
|
||||
When the incoming `AIMessage` is marked `direct` and the model
|
||||
returns a non-null answer, `answer_needed` is forced to `True`
|
||||
regardless of the model's own `answer_needed`. A user addressing the
|
||||
bot directly gets a reply.
|
||||
|
||||
### ENV-05 — Short-path rules skip the model (coverage: feature)
|
||||
|
||||
When a configured `short-path` `[channel-regex, user-regex]` pair
|
||||
matches the message, `send()` appends the message to history, trims
|
||||
history to the limit, persists it, and returns an empty response
|
||||
(answer `None`, `answer_needed False`) **without calling the model**.
|
||||
Cheap archival of noisy channels.
|
||||
|
||||
### ENV-06 — Malformed model output is repaired (coverage: withdrawn — successor ENV-18)
|
||||
|
||||
Withdrawn 2026-07-13 with FDB-005: strict structured outputs make the
|
||||
repair model obsolete. Malformed output now counts as a failed
|
||||
attempt — see ENV-18.
|
||||
|
||||
### ENV-07 — History is trimmed to the limit (coverage: feature)
|
||||
|
||||
After a completed exchange, `len(history) <= history-limit` holds.
|
||||
Trimming happens both before the model call and after appending the
|
||||
new question/answer pair.
|
||||
|
||||
### ENV-08 — Markdown links are unwrapped (coverage: feature)
|
||||
|
||||
In the final answer text, `[label](url)` becomes `url` and
|
||||
`@[label](url)` becomes `label`. Discord renders raw URLs; markdown
|
||||
link syntax from the model reads as noise.
|
||||
|
||||
### ENV-09 — Missing channel falls back to the message channel (coverage: feature)
|
||||
|
||||
When the envelope `channel` is null/none/empty, the response channel
|
||||
is the channel the message came from.
|
||||
|
||||
### ENV-10 — System prompt template substitution (coverage: test)
|
||||
|
||||
`message()` substitutes `{date}` (YYYY-MM-DD), `{time}`, `{memory}`
|
||||
(current memory string) in the system prompt; `{news}` is replaced
|
||||
with the news file content when the configured file exists and stays
|
||||
literal when it does not.
|
||||
|
||||
### ENV-11 — Per-channel history shrink prefers busy channels (coverage: test)
|
||||
|
||||
`shrink_history_by_one()` removes the oldest entry whose channel has
|
||||
more than `history-per-channel` (default 3) entries; when no channel
|
||||
exceeds the cap, the oldest entry overall is removed.
|
||||
|
||||
### ENV-12 — Exhausted retries raise, attempts are spaced (coverage: test)
|
||||
|
||||
When the model returns no usable answer three times in a row,
|
||||
`send()` raises `RuntimeError`. Consecutive attempts are separated by
|
||||
an exponential-backoff sleep — failures never hammer the API
|
||||
back-to-back (D1).
|
||||
|
||||
### ENV-13 — Reaction-clear events are recorded (coverage: test)
|
||||
|
||||
`on_reaction_clear(message, reactions)` — the discord.py signature —
|
||||
records the clearing in the channel's memory. (D7: the previous
|
||||
handler declared `(reaction, user)` and crashed on dispatch.)
|
||||
|
||||
### ENV-14 — update_memory persists its argument (coverage: test)
|
||||
|
||||
`update_memory(memory)` sets the responder's memory to the passed
|
||||
value and persists that value. (D12: the argument was ignored.)
|
||||
|
||||
### ENV-15 — retry-model is used after a rate limit (coverage: test)
|
||||
|
||||
After a rate-limited attempt, the next `chat()` attempt uses the
|
||||
configured `retry-model` instead of `model`; a successful attempt
|
||||
switches back. (D2: the fallback was assigned to a local and never
|
||||
took effect.)
|
||||
|
||||
### ENV-16 — Model calls are never served from a disk cache (coverage: test)
|
||||
|
||||
`openai_chat`/`openai_image` call the client every time. The pickle
|
||||
response cache (`openai_chat.dat`) is a test-era artifact and must
|
||||
not exist in the production path (D3/D4).
|
||||
|
||||
### ENV-17 — IGDB tool execution runs in a worker thread (coverage: test)
|
||||
|
||||
`_execute_igdb_function` executes the synchronous IGDB library off
|
||||
the event loop (worker thread), returning identical results. The
|
||||
event loop keeps serving Discord events during lookups (D5).
|
||||
|
||||
### ENV-18 — Malformed or refused output is a failed attempt (coverage: test)
|
||||
|
||||
Model output is requested as schema-validated JSON (strict structured
|
||||
outputs). Output that still fails to parse, or a model refusal,
|
||||
counts as a failed attempt (backoff + retry per ENV-12) — there is no
|
||||
repair model, no `fix()` path, no relaxed-JSON fallback. Replaces
|
||||
ENV-06.
|
||||
|
||||
### ENV-19 — The envelope schema is pinned (coverage: test)
|
||||
|
||||
Every chat call carries `response_format` = strict JSON schema named
|
||||
`envelope` with exactly the fields `answer`, `answer_needed`,
|
||||
`channel`, `staff`, `picture`, `picture_edit`, `hack` — all required,
|
||||
`additionalProperties: false`, nullable where the protocol allows
|
||||
null. Tool-followup calls carry the same format.
|
||||
@@ -0,0 +1,23 @@
|
||||
# SPEC-008 — Configuration
|
||||
|
||||
TOML config per deployment (`kroa.toml`, `ggg.toml` — both untracked;
|
||||
`config.toml` in the repo is the placeholder sample). Loaded by
|
||||
`FjerkroaBot.load_config`, hot-reloaded by a watchdog observer
|
||||
(defect D9 — reload race — is tracked in FDB-004 and will refine
|
||||
these requirements).
|
||||
|
||||
### CFG-01 — TOML config loads into a plain dict (coverage: test)
|
||||
|
||||
`FjerkroaBot.load_config(path)` parses the TOML file and returns its
|
||||
top-level table as a dict; responders read raw keys from it.
|
||||
|
||||
### CFG-02 — Per-channel system prompt override (coverage: test)
|
||||
|
||||
A responder bound to channel `X` uses `config["X"]` as its system
|
||||
prompt when that key exists, else `config["system"]`. One deployment
|
||||
can speak differently per channel.
|
||||
|
||||
### CFG-03 — Missing news file leaves the placeholder untouched (coverage: test)
|
||||
|
||||
When `news` points to a non-existent file, the `{news}` placeholder
|
||||
stays literal in the system prompt (no crash, no empty substitution).
|
||||
Reference in New Issue
Block a user