News for hackers. Stuff that matters.
Re:Has LLM killed traditional NLP? by minimaxir on Saturday January 18, 2025 @03:02PM (#42750989)
So there's a bit of an issue with prompt caching implementations: for both OpenAI API and Claude's API, you need a minimum of 1024 tokens to build the cache for whatever reason. For simple problems, that can be hard to hit and may require padding the system prompt a bit.

There are no comments on this comment yet.