igdb search + capped http reads: native fulltext, full-page fetch
search_games used `where name ~ "<q>"*`: prefix-only, diacritic- and word-order-sensitive -- "MARVEL Tōkon" found nothing though IGDB has it. Switch to IGDB's native `search` clause (relevance-ranked, diacritic-insensitive). fetch_url and image downloads read bodies with content.read(n), which returns only the first buffered chunk (~7 KB): pages collapsed to their <title>. httpread.read_capped collects chunks up to the byte cap; image downloads read limit+1 so over-limit files are still rejected instead of cached truncated (IMG-10). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -18,6 +18,7 @@ from urllib.parse import urljoin, urlparse
|
||||
import aiohttp
|
||||
|
||||
from .ai_responder import sanitize_external_text
|
||||
from .httpread import read_capped
|
||||
|
||||
DEFAULT_MAX_BYTES = 2 * 1024 * 1024
|
||||
DEFAULT_MAX_CHARS = 6000
|
||||
@@ -115,7 +116,7 @@ class URLReader:
|
||||
current = urljoin(current, response.headers["Location"])
|
||||
continue
|
||||
response.raise_for_status()
|
||||
return str(response.url), await response.content.read(max_bytes + 1)
|
||||
return str(response.url), await read_capped(response, max_bytes)
|
||||
raise ValueError("too many redirects")
|
||||
|
||||
async def fetch(self, url: str, channel: str, user: str) -> Dict[str, Any]:
|
||||
|
||||
Reference in New Issue
Block a user