Re:Better prompt caching for GPT‑6
by OutOfHere on Tuesday September 22, 2026 @06:14PM
(#49808969)
The biggest continuing limitation I see is that the cached input has to be at least 1024 tokens. This is terrible. It means a lot of good prefixes that are smaller will go uncached for no good reason. The threshold should have been 128.
[ Parent
| on HN ]
There are no comments on this comment yet.