News for hackers. Stuff that matters.
Re:Open source solution replicates ChatGPT training process by sdenton4 on Sunday February 19, 2023 @05:38PM (#34862605)
Yeah.... Having spent a lot of cycles replicating ML work, it's much more difficult than taking a stab at replicating a paper. It's typically doable (results really do replicate) but it can take a few good brains a year to pull it off. There's typically a lot of small decisions that add up, and a lot of hyperparameter sweeps to land in a good region of the optimization space.

There are no comments on this comment yet.