Re:It's OpenAI's world, we're just living in it
by int_19h on Friday October 10, 2025 @07:35PM
(#45545015)
Don't forget about Macs. Top models can run even fairly large LLMs (although time to first token is... not great). But even midline can certainly run LLMs in the ballpark of ~30B params very well, which is where things start to get interesting IMO.
[ Parent
| on HN ]
Re:It's OpenAI's world, we're just living in it
by dabockster on Friday October 10, 2025 @07:54PM
(#45545120)
> although time to first token is... not great
The token cache works on CUDA too. So yeah, the initial loadout sucks, but almost everything from then on is solid.