igdb search + capped http reads: native fulltext, full-page fetch

search_games used `where name ~ "<q>"*`: prefix-only, diacritic- and
word-order-sensitive -- "MARVEL Tōkon" found nothing though IGDB has
it. Switch to IGDB's native `search` clause (relevance-ranked,
diacritic-insensitive).

fetch_url and image downloads read bodies with content.read(n), which
returns only the first buffered chunk (~7 KB): pages collapsed to
their <title>. httpread.read_capped collects chunks up to the byte
cap; image downloads read limit+1 so over-limit files are still
rejected instead of cached truncated (IMG-10).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Oleksandr Kozachuk
2026-07-13 19:29:16 +02:00
parent d4eec4088d
commit 2caa18a17f
7 changed files with 134 additions and 8 deletions
+2 -1
View File
@@ -18,6 +18,7 @@ from urllib.parse import urljoin, urlparse
import aiohttp
from .ai_responder import sanitize_external_text
from .httpread import read_capped
DEFAULT_MAX_BYTES = 2 * 1024 * 1024
DEFAULT_MAX_CHARS = 6000
@@ -115,7 +116,7 @@ class URLReader:
current = urljoin(current, response.headers["Location"])
continue
response.raise_for_status()
return str(response.url), await response.content.read(max_bytes + 1)
return str(response.url), await read_capped(response, max_bytes)
raise ValueError("too many redirects")
async def fetch(self, url: str, channel: str, user: str) -> Dict[str, Any]: