Re:Llama3 implemented from scratch
by miki123211 on Monday May 20, 2024 @09:50AM
(#40415466)
This is hard because you need high bandwidth between the GPUs in your cluster, bandwidth far higher than broadband could provide. I'm not even sure whether the time spend synchronizing between far-away machines would offset the increase in computational power.
[ Parent
| on HN ]
There are no comments on this comment yet.