News for hackers. Stuff that matters.
Re:Observability for LLM apps with structlog and DuckDB by OutOfHere on Thursday June 06, 2024 @02:20PM (#40600657)
The stronger need for LLM apps is for persistent response caching and reuse. Once this is available at scale with well-defined cache expiration policies, printing logs selectively is easy. The solution ought to also support individual key invalidation as needed.
Re:Observability for LLM apps with structlog and DuckDB by idomi on Thursday June 06, 2024 @02:32PM (#40600784)
I think it really depends on your need, if you're working on your local and trying to get something to work, caching, scale etc, might be an overkill.