igdb search + capped http reads: native fulltext, full-page fetch
search_games used `where name ~ "<q>"*`: prefix-only, diacritic- and word-order-sensitive -- "MARVEL Tōkon" found nothing though IGDB has it. Switch to IGDB's native `search` clause (relevance-ranked, diacritic-insensitive). fetch_url and image downloads read bodies with content.read(n), which returns only the first buffered chunk (~7 KB): pages collapsed to their <title>. httpread.read_capped collects chunks up to the byte cap; image downloads read limit+1 so over-limit files are still rejected instead of cached truncated (IMG-10). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -14,6 +14,7 @@ from typing import Any, Callable, Dict, List, Optional
|
||||
|
||||
import aiohttp
|
||||
|
||||
from .httpread import read_capped
|
||||
from .persistence import PersistentStore
|
||||
|
||||
DEFAULT_CACHE_MB = 500
|
||||
@@ -79,7 +80,9 @@ class ImageCache:
|
||||
async with aiohttp.ClientSession(timeout=timeout) as session:
|
||||
async with session.get(url) as response:
|
||||
response.raise_for_status()
|
||||
return await response.content.read(limit + 1)
|
||||
# limit + 1: an over-limit body must stay over-limit so
|
||||
# ingest_bytes rejects it instead of caching it truncated
|
||||
return await read_capped(response, limit + 1)
|
||||
|
||||
def data_url(self, sha256: str, ext: str) -> Optional[str]:
|
||||
path = self._path(sha256, ext)
|
||||
|
||||
Reference in New Issue
Block a user