News for hackers. Stuff that matters.

A comment on YaLM-100B: Pretrained language model with 100B parameters. Parent comment

Re:YaLM-100B: Pretrained language model with 100B parameters by wjnc on Thursday June 23, 2022 @06:50AM (#31847310)
log(300) / log(2) = only 8.2 doublings away. That's near future material.
[ Parent | on HN ]
Re:YaLM-100B: Pretrained language model with 100B parameters by thfuran on Thursday June 23, 2022 @07:12AM (#31847460)
Maybe at 90s hardware growth rates, but not now.
[ Parent | on HN ]
Re:YaLM-100B: Pretrained language model with 100B parameters by rictic on Thursday June 23, 2022 @07:30AM (#31847604)
The dream of the 90s is alive in the GPU market: https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-fl...

Moore's law didn't stop, just Dennard scaling. Expect graphics and AI to continue to improve radically in performance/price, while more ordinary workloads see only modest improvements.

[ Parent | on HN ]
Re:YaLM-100B: Pretrained language model with 100B parameters by thfuran on Thursday June 23, 2022 @08:13AM (#31847939)
GPU TDP seems on the verge of going exponential, cost per transistor isn't really decreasing so much at the very latest nodes, and even that article seems to suggest it'd likely be decades before 300x flops/$
[ Parent | on HN ]