News for hackers. Stuff that matters.

A comment on Ask HN: Resources for General Purpose GPU development on Apple's M* chips?. Parent comment

Re:Ask HN: Resources for General Purpose GPU development on ... by morphle on Wednesday December 25, 2024 @04:33PM (#42511307)
Here a video of testing Exo to run huge LLMs on a cluster of M4 Macs[1] more cheaply than with a cluster of NVDIA RTX 4090s.

[1] https://www.youtube.com/watch?v=GBR6pHZ68Ho

[ Parent | on HN ]
Re:Ask HN: Resources for General Purpose GPU development on ... by menaerus on Friday December 27, 2024 @06:09AM (#42521350)
They show a test-run of a 1B llama-3.2 model. Doesn't that fit in a single mac? Distributing the workload in this case must be slower than running it on a single machine.

However, this is interesting and I'm confused why aren't they showcasing the test-run of a larger model that actually necessitates distributing the workload across the cluster.

[ Parent | on HN ]