News for hackers. Stuff that matters.
Re:It's OpenAI's world, we're just living in it by dabockster on Friday October 10, 2025 @07:54PM (#45545120)
> although time to first token is... not great

The token cache works on CUDA too. So yeah, the initial loadout sucks, but almost everything from then on is solid.

Re:It's OpenAI's world, we're just living in it by int_19h on Saturday October 11, 2025 @10:23PM (#45554601)
This works for scenarios like a simple chat. But the moment you start throwing large documents on it, or lots of code, it's back to waiting.