News for hackers. Stuff that matters.

A comment on M2 Ultra can run 128 streams of Llama 2 7B in parallel. Parent comment

Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by whalesalad on Wednesday October 11, 2023 @05:06PM (#37850267)
Amazon has Apple servers for rent. There is also https://macminicolo.net and https://www.macminivault.com
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by nathancahill on Wednesday October 11, 2023 @05:59PM (#37850808)
I'm so happy MacMiniColo is still around. I had projects hosted there when they were just getting started. They've stayed true to their mission. I love that their website still feels like Wordpress 3.0.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by hamandcheese on Wednesday October 11, 2023 @06:12PM (#37850974)
They aren't accepting new Mac minis for colocation. Which means if you want to go from, say, 8gb to 16gb of ram then that's an extra $90 a month for the privilege.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by jzombie on Thursday October 12, 2023 @05:58AM (#37855226)
I am using https://www.macminivault.com for colocation.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by cyberge99 on Wednesday October 11, 2023 @06:06PM (#37850903)
There’s also MacStadium.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by detourdog on Wednesday October 11, 2023 @06:19PM (#37851052)
I have a fiber connection, the space and the experience to run a Mac mini colo. If people are really interested. I would want to do it as a co-operative that helps with the expense of the physical location.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by ruph123 on Wednesday October 11, 2023 @07:56PM (#37851911)
MacMiniColo and MacStadium merged afaik.
[ Parent | on HN ]
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel by JCharante on Thursday October 12, 2023 @06:18AM (#37855350)
Doesn’t macos require you to rent it out with a one month minimum? At that point just buy the hardware
[ Parent | on HN ]