News for hackers. Stuff that matters.

A comment on Ask HN: Recommendations for Local LLMs in 2024: Private and Offline?. Parent comment

Re:Ask HN: Recommendations for Local LLMs in 2024: Private a... by daghamm on Saturday April 06, 2024 @04:13PM (#39955098)
With ROCm 6 I think new-ish AMD GPUs give you much better bang for the buck.

https://ollama.com/blog/amd-preview

[ Parent | on HN ]
Re:Ask HN: Recommendations for Local LLMs in 2024: Private a... by Havoc on Sunday April 07, 2024 @06:48AM (#39959918)
Indeed...bought AMD shares on the bet that they'll catch up.

However this is still true only in a fairly narrow space - inference - once you stray off that very narrow path they're still a fair bit behind

[ Parent | on HN ]