Re:SSE sucks for transporting LLM tokens
by verdverm on Saturday December 13, 2025 @03:19PM
(#46257601)
It's likely better to just store them in the database, it could be hours or days later that I want to look at the session again. If you're going to want a database for conversation anyway, then it doesn't make sense to cache and query individual messages
[ Parent
| on HN ]
Re:SSE sucks for transporting LLM tokens
by ivan_gammel on Saturday December 13, 2025 @03:57PM
(#46257976)
A database can be the implementation of choice for the cache. But not all use cases do require long-term storage like that.
Re:SSE sucks for transporting LLM tokens
by verdverm on Saturday December 13, 2025 @05:28PM
(#46258804)
yup, the ADK framework I use handles all that bookkeeping for me, has a few options built in, and pretty easy to add new implementations. I'm extending it to attach filesys+exe changes during a session, persisted as container layers via Dagger