Re:Observability for LLM apps with structlog and DuckDB
by OutOfHere on Thursday June 06, 2024 @02:20PM
(#40600657)
The stronger need for LLM apps is for persistent response caching and reuse. Once this is available at scale with well-defined cache expiration policies, printing logs selectively is easy. The solution ought to also support individual key invalidation as needed.
[ Parent
| on HN ]