Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Ask HN: Affordable hardware for running local large language models?.
For larger models, or GPU accelerated inference, there is no “cheap” solution.
Why do you think everyone is so in love with the 7B models?
It’s not because they’re good. They’re just ok, and it’s expensive to run larger models.
There are no comments on this comment yet.
For larger models, or GPU accelerated inference, there is no “cheap” solution.
Why do you think everyone is so in love with the 7B models?
It’s not because they’re good. They’re just ok, and it’s expensive to run larger models.