Re:YaLM-100B: Pretrained language model with 100B parameters
by H8crilA on Thursday June 23, 2022 @06:30AM
(#31847177)
Indeed. What "everyone" can use is a ~$200 smartphone, so there's a ~300x gap to be bridged.
[ Parent
| on HN ]
Re:YaLM-100B: Pretrained language model with 100B parameters
by ben_w on Thursday June 23, 2022 @08:29AM
(#31848089)
Most of the cost of a phone isn’t the processor, so probably closer to x1000. Hardware may get that much cheaper, but it was never guaranteed, and we’re not making progress as fast as we used to.
Re:YaLM-100B: Pretrained language model with 100B parameters
by wjnc on Thursday June 23, 2022 @06:50AM
(#31847310)
log(300) / log(2) = only 8.2 doublings away. That's near future material.
Re:YaLM-100B: Pretrained language model with 100B parameters
by thfuran on Thursday June 23, 2022 @07:12AM
(#31847460)
Maybe at 90s hardware growth rates, but not now.
Re:YaLM-100B: Pretrained language model with 100B parameters
by rictic on Thursday June 23, 2022 @07:30AM
(#31847604)
The dream of the 90s is alive in the GPU market: https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-fl...
Moore's law didn't stop, just Dennard scaling. Expect graphics and AI to continue to improve radically in performance/price, while more ordinary workloads see only modest improvements.
Re:YaLM-100B: Pretrained language model with 100B parameters
by thfuran on Thursday June 23, 2022 @08:13AM
(#31847939)
GPU TDP seems on the verge of going exponential, cost per transistor isn't really decreasing so much at the very latest nodes, and even that article seems to suggest it'd likely be decades before 300x flops/$