News for hackers. Stuff that matters.

A comment on Google “We Have No Moat, and Neither Does OpenAI”. Parent comment

Re:Google “We Have No Moat, and Neither Does OpenAI” by lhl on Thursday May 04, 2023 @08:34PM (#35823958)
I've seen reports of people being able to run LLMs at decent speeds on old Nvidia P40s. These are 24GB Pascal GPUs and can be bought for as low as $100 (although more commonly $200) on eBay.
[ Parent | on HN ]
Re:Google “We Have No Moat, and Neither Does OpenAI” by outofpaper on Thursday May 04, 2023 @09:05PM (#35824147)
Link to report please
[ Parent | on HN ]
Re:Google “We Have No Moat, and Neither Does OpenAI” by lhl on Thursday May 04, 2023 @09:18PM (#35824217)
https://www.reddit.com/r/Oobabooga/comments/126dejd/comment/...
[ Parent | on HN ]
Re:Google “We Have No Moat, and Neither Does OpenAI” by kiratp on Thursday May 04, 2023 @10:15PM (#35824547)
You can do it on CPU now.

Benchmarks: https://github.com/ggerganov/llama.cpp/issues/34

[ Parent | on HN ]