Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Show HN: An open-source AI Gateway with integrated guardrails. Parent comment
So there's 2 levels of cache - the LLM request itself might be cached (simple and semantic) and the guardrail response can be cached as well.
We use a mix of a distributed kv store and a vector DB to actually store the data
There are no comments on this comment yet.
So there's 2 levels of cache - the LLM request itself might be cached (simple and semantic) and the guardrail response can be cached as well.
We use a mix of a distributed kv store and a vector DB to actually store the data