Re:API Improvement Proposals
by sillysaurusx on Saturday July 03, 2021 @04:49PM
(#27724799)
Nothing wrong with that. That's how I've built scraper systems in the past. It's nice for getting historical snapshots of live data.
The trouble comes in when you want to do something that you know will modify the result. (The "request things that need updating" part was surprisingly hard for me; if an item is modified but the old cached copy is still being served, it became problematic for me to rely on it.)
Instead I revolve around getIdAndFreshnessList(filter) and getItems(ids).
I've built tools to store items locally, and check if the id is cached and the date is fresh, and only request things that need updating.
I'm digging myself into a hole of being "too different" again. This is just another convention I'll be judged negatively against.