Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel
by hamandcheese on Wednesday October 11, 2023 @09:01PM
(#37852302)
Yeah, to be clear I wouldn't advocate this route for your core product. But running CI workers? Sure. Especially macs, which have onerous usage restrictions in the cloud that negate most of the elasticity benefits you might otherwise see.
[ Parent
| on HN ]
There are no comments on this comment yet.