discord_bot/specs/SPEC-013-news.md

SPEC-013 — News digest

Replaces the broken pre-1.0-openai news_feed.py. A CLI (python -m fjerkroa_bot.news --config <cfg>) fetches the news-feeds and writes a compact digest to the news file that AIResponder.message injects into the {news} slot. Feeds are external input and operator-configured.

NEWS-01 — RSS (2.0 and 1.0/RDF) and Atom parse to items (coverage: test)

parse_feed(bytes, label) extracts {title, link, source} from both RSS (<item>) and Atom (<entry>) documents, tolerates malformed XML (returns an empty list, logs), and never raises.

NEWS-02 — Digest is sanitized and bounded (coverage: test)

render_digest caps at news-max-items, and every headline passes sanitize_external_text (SAF-03) — a feed cannot inject @everyone or control characters into the prompt via a headline.

NEWS-03 — Feeds are SSRF-guarded and deduped (coverage: test)

NewsFetcher.collect skips any feed URL the SSRF guard rejects, skips feeds that fail to fetch (one bad feed never sinks the run), and drops duplicate headlines across feeds.

Webhook posting (ggg model)

--post mode fetches feeds mapped to channels and posts NEW items to the channel's Discord webhook — replacing the py3.8 getnews.py (dead play3 feed, 35 MB substring-scan state file, HTML-redirect cruft). Config: news-post-feeds = [[url, label, channel], …], news-post-webhooks = {channel = url}, news-post-state.

NEWS-04 — Only unseen items post, then are marked seen (coverage: test)

NewsPoster.run_post posts each item whose key (link, else title) is not in the seen-set, adds it to the set, and posts to the mapped channel's webhook. Re-runs over the same feed post nothing new.

NEWS-05 — First run seeds without flooding (coverage: test)

With no prior state file (seed_only), every current item is marked seen but nothing is posted — migrating off getnews.py never dumps a backlog into the channels. news-post-max-per-run caps steady-state posts per run.

NEWS-06 — Post failures and bad channels are survived (coverage: test)

A feed the SSRF guard rejects, a feed that fails to fetch, an item whose channel has no configured webhook, and a webhook POST that raises are each logged and skipped — one failure never sinks the run, and the seen-set still advances for successfully-processed items.

NEWS-07 — Item summaries are extracted (coverage: test)

parse_feed also captures each item's short description — RSS <description>, Atom <summary> or <content> — with HTML stripped, entities unescaped, and whitespace collapsed, so an item carries what it is about, not only a headline. Missing descriptions yield an empty summary, never an error.

NEWS-08 — The digest carries summaries (coverage: test)

render_digest appends the sanitized, length-capped (news-summary-chars, default 200) summary after each headline, so the bot's ambient {news} context knows the gist of each story, not just its title. A zero cap restores the title-only digest.

NEWS-09 — Fetched news is stored, deduped, and rolled over (coverage: test)

Both the digest run (kroa) and the posting run (ggg) upsert every fetched item into a news table keyed by link (or title), so the same story is stored once. After each run the store is pruned to the newest news-keep rows (default 400), a rolling window that bounds growth while keeping recent history searchable.

NEWS-10 — get_news is offered as a tool (coverage: test)

When enable-news-tool is true and a store is configured, the chat call's tools list includes a get_news function (optional topic, source, limit) next to the other tools. Without a store or the flag it is absent.

NEWS-11 — get_news retrieves filtered, sanitized items (coverage: test)

get_news returns recent stored items, newest first, optionally narrowed by topic (every keyword must appear in the title, summary, or source label — so topic: "Nordland" finds items from that source) and/or an exact source; limit is clamped to 1..30. Each result's title and summary are passed through sanitize_external_text. The bot can then fetch_url a returned link for the full article.

NEWS-12 — get_news is metered per user (coverage: test)

Each get_news call increments a per-user daily counter; over news-daily-per-user (default 30) the tool refuses with an error result without touching the store. The budget gate (SAF-04) still applies to the surrounding model calls.