Re:Ask HN: What is the best way to provide continuous contex...
by bluegatty on Friday January 16, 2026 @05:31PM
(#46653097)
Prefill is 10x faster than generation without caching, and 100x faster with caching - as a very crude measure. So it's not a matter of 'only the case'. Those are different scenarios. Some hosts are better than others with respect to managing caching, but the better one's provide decent SLA on that.
[ Parent
| on HN ]
There are no comments on this comment yet.