News for hackers. Stuff that matters.

A comment on Apple releases CoreNet, a library for training deep neural networks. Parent comment

Re:Apple releases CoreNet, a library for training deep neura... by blackeyeblitzar on Tuesday April 23, 2024 @11:01PM (#40140006)
> It looks like a mid-level implementations of training and inference

I’m not familiar with how any of this works but what does state of the art training look like? Almost no models release their training source code or data sets or pre processing or evaluation code. So is it known what the high level implementation even is?

[ Parent | on HN ]
Re:Apple releases CoreNet, a library for training deep neura... by spott on Tuesday April 23, 2024 @11:06PM (#40140037)
https://github.com/NVIDIA/Megatron-LM

This is probably a good baseline to start thinking about LLM training at scale.

[ Parent | on HN ]