News for hackers. Stuff that matters.
Re:Megatron-Turing NLG 530B, the World’s Largest Generative ... by sanxiyn on Tuesday October 12, 2021 @08:12AM (#28838745)
> What's worse, the trend to supersize language models is never justified, either theoretically (ha ha) or empirically in the relevant literature

What? That's absurd. Large language models are motivated by empirical scaling law. It is actually better justified than other ML research.

Scaling Laws for Neural Language Models: https://arxiv.org/abs/2001.08361

There are no comments on this comment yet.