News for hackers. Stuff that matters.

A comment on M2 Ultra can run 128 streams of Llama 2 7B in parallel. Parent comment

Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by nathancahill on Wednesday October 11, 2023 @05:59PM (#37850808)
I'm so happy MacMiniColo is still around. I had projects hosted there when they were just getting started. They've stayed true to their mission. I love that their website still feels like Wordpress 3.0.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by hamandcheese on Wednesday October 11, 2023 @06:12PM (#37850974)
They aren't accepting new Mac minis for colocation. Which means if you want to go from, say, 8gb to 16gb of ram then that's an extra $90 a month for the privilege.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by jzombie on Thursday October 12, 2023 @05:58AM (#37855226)
I am using https://www.macminivault.com for colocation.
[ Parent | on HN ]