News for hackers. Stuff that matters.
Re:Websites and web developers mostly don't care about clien... by snowwrestler on Saturday August 23, 2025 @08:08AM (#44995384)
Yes, it’s true. Most sites don’t have a forever cache TTL so a crawler that hits every page on a database-backed site is going to hit mostly uncached pages (and therefore the DB).

I also have a faceted search that some stupid crawler has spent the last month iterating through. Also mostly uncached URLs.

Re:Websites and web developers mostly don't care about clien... by n3storm on Saturday August 23, 2025 @08:38AM (#44995562)
Yeah, or an event plugin where spiders walks every day of several years...