News for hackers. Stuff that matters.
Re:Measuring LLM citations with server logs might not work a... by boatbuilder on Thursday August 28, 2025 @01:08PM (#45054543)
I assumed you could measure LLM citations by watching server logs. Our experiment showed otherwise: ChatGPT and Perplexity often cite pages without fetching them, instead serving from a global cache.

We ran this on Vercel/Next.js middleware (before the edge cache) to rule out local caching—browser requests log normally, but many citations never appear.

This doesn’t mean measurement is impossible, but the “citation = visit” assumption doesn’t always hold. Curious if others have observed similar patterns.

There are no comments on this comment yet.