News for hackers. Stuff that matters.

A comment on Llamafile lets you distribute and run LLMs with a single file. Parent comment

Re:Llamafile lets you distribute and run LLMs with a single ... by rgbrgb on Wednesday November 29, 2023 @04:53PM (#38465845)
In my experience, if you're on a mac it's about the file size * 150% of RAM to get it working well. I had a user report running my llama.cpp app on a 2017 iMac with 8GB at ~5 tokens/second. Not sure about other platforms.
[ Parent | on HN ]

There are no comments on this comment yet.