News for hackers. Stuff that matters.
Re:SSE sucks for transporting LLM tokens by bjt on Saturday December 13, 2025 @02:23PM (#46257133)
The user's last prompt can be sent with an idempotency key that changes each time the user initiates a new request. If that's the same, use the cache. If it's new, hit the LLM again.

There are no comments on this comment yet.