Oleksandr Kozachuk
98 lines
4.1 KiB
Markdown
98 lines
4.1 KiB
Markdown
# SPEC-004 — Image generation
|
|
|
|
Generation on `image-model` (default `gpt-image-2`), base64 end to
|
|
end — no URL downloads, no expiring CDN links in the generation path.
|
|
Optional knobs: `image-size` (default 1024x1024), `image-quality`
|
|
(passed through only when set). The Leonardo path stays behind
|
|
`leonardo-token` until parity is confirmed, then dies. The input
|
|
pipeline (attachment cache, vision, edit/remix) is FDB-010 / IMG-10+.
|
|
|
|
### IMG-01 — Images arrive as base64 buffers (coverage: test)
|
|
|
|
`draw_openai(description, count)` requests `count` images and returns
|
|
a list of decoded image buffers straight from the API response; every
|
|
generated image is metered in the ledger (SAF-05).
|
|
|
|
### IMG-02 — The envelope carries picture_count (coverage: test)
|
|
|
|
The envelope gains `picture_count` (integer). `post_process` clamps
|
|
it to 1..4 and defaults to 1 when absent (legacy history entries,
|
|
old-model output). ENV-19's field list is revised accordingly.
|
|
|
|
### IMG-03 — Multiple images, one message (coverage: test)
|
|
|
|
`picture_count` images are attached as multiple files to a single
|
|
Discord send (the last part when the answer is split, per BEH-06).
|
|
|
|
### IMG-04 — Legacy image models degrade safely (coverage: test)
|
|
|
|
When `image-model` is not a `gpt-image-*` model (e.g. `dall-e-3`),
|
|
the count is clamped to 1 and `response_format="b64_json"` is
|
|
requested explicitly (gpt-image models return base64 natively and
|
|
reject the parameter).
|
|
|
|
### IMG-05 — Picture prompts go to the API untouched (coverage: test)
|
|
|
|
The translate-before-draw step is deleted: the model's picture prompt
|
|
reaches the image API verbatim (current image models handle
|
|
Norwegian/German natively). The `translate()` method and its
|
|
`fix-model` dependency are gone (closes D-009).
|
|
|
|
## Input pipeline (FDB-010)
|
|
|
|
Attachments live in a content-hash cache
|
|
(`<history-directory>/images/<sha256>.<ext>`, index in the store,
|
|
schema v4). Active only with a store; without one the legacy CDN-URL
|
|
path remains.
|
|
|
|
### IMG-10 — Attachments are ingested at message time (coverage: test)
|
|
|
|
Every image attachment is downloaded immediately (timeout, size cap
|
|
`image-max-bytes` default 8 MB) and stored under its content hash.
|
|
Only sniffed png/jpeg/gif/webp bytes are accepted — extension and
|
|
declared MIME are ignored (attacker-controlled). Rejected content is
|
|
dropped and logged (D11 root fix + cache-abuse hardening).
|
|
|
|
### IMG-11 — Vision reads from the cache, never CDN URLs (coverage: test)
|
|
|
|
Vision parts are `data:` URLs built from cached bytes. Discord's
|
|
signed, expiring CDN URLs never reach the model or the history.
|
|
|
|
### IMG-12 — The cache is capped and aged (coverage: test)
|
|
|
|
`image-cache-mb` (default 500) LRU-evicts oldest-first;
|
|
`image-cache-ttl-days` (default 90) ages entries out. Eviction always
|
|
removes file and index row together.
|
|
|
|
### IMG-13 — picture_edit edits the newest channel images (coverage: test)
|
|
|
|
`picture_edit=true` calls `images.edit` with up to the 4 newest
|
|
cached images of the answer channel as inputs (API max is 16; 4 keeps
|
|
prompts sane). An empty cache falls back to plain generation — the
|
|
flag alone must never fail a reply.
|
|
|
|
### IMG-14 — Deletion propagates to the cache (coverage: test)
|
|
|
|
Deleting a Discord message purges its cached images; `!forgetme`
|
|
purges all of the user's images — files and rows (extends
|
|
SAF-08/MEM-09).
|
|
|
|
### IMG-15 — Generated images join the cache (coverage: test)
|
|
|
|
Bot-generated images are ingested like uploads (user `assistant`), so
|
|
"make a variant of that" remix chains work on the bot's own output.
|
|
|
|
### IMG-17 — Image-only messages are cached (coverage: test)
|
|
|
|
A message consisting only of attachments (no text) is ingested into
|
|
the cache and recorded as an observation, even though no reply is
|
|
produced — the image must be available for later `picture_edit` and
|
|
vision follow-ups. (Previously the empty-text early-return dropped
|
|
such posts entirely.)
|
|
|
|
### IMG-16 — The prompt announces editable images (coverage: test)
|
|
|
|
When the answer channel has cached images, the context suffix states
|
|
how many and that `picture_edit=true` edits the newest — the model
|
|
cannot use a capability it does not know about.
|