Re:AI training is outpacing Moore’s Law
by hyperpallium2 on Wednesday December 08, 2021 @05:10PM
(#29491209)
You're right about literal Moores Law, but its benefits are much diminished, of performance, power usage and cost. My understanding is the main problem is heat dissipation.
I've long predicted macrochips to solve this, brute force of huge quantities of silicon, but it hasn't really happened.
Re:AI training is outpacing Moore’s Law
by rictic on Wednesday December 08, 2021 @06:15PM
(#29491837)
Graphics cards and neural accelerators have continued an exponential improvement in performance per watt dollar, which is the intuitive thing that people often mean when they say Moore's law.
Re:AI training is outpacing Moore’s Law
by adverbly on Wednesday December 08, 2021 @05:25PM
(#29491362)
If we hit a trillion transistors by 2030 without increasing chip size, I'll be shocked.
I guess it's inevitable that we'll reach a "hard" limit at some point, with exponential increase in density (electrons and photons have a "size"). But I don't know how that translates to current densities (how many atoms in a gate on the Apple m1 chips, and what is the theoretical minimum?).
Re:AI training is outpacing Moore’s Law
by bitshiftfaced on Thursday December 09, 2021 @10:33AM
(#29498478)
Moore forecasted a doubling of transistor points but a halving of costs. The costs haven't been keeping on pace. https://qph.fs.quoracdn.net/main-qimg-81fb704374c332ad5c5e27...
https://www.quora.com/If-the-Moores-law-is-slowing-down-can-...
Did he make a clear statement on cost? On a skim of: https://newsroom.intel.com/wp-content/uploads/sites/11/2018/... I see costs mentioned - but I can't immediately see a clear statement - to tie it in with transistor increase - but I might be overlooking something?
Re:AI training is outpacing Moore’s Law
by bitshiftfaced on Friday December 10, 2021 @10:36AM
(#29511085)
On page two he charts cost per transistor dropping roughly an order of magnitude every 5 years. Since the the "law" of doubling transistors is on a two year time span, this roughly matches.
http://www.transistorcount.com/
Slope seem a bit more gentle - but certainly not gone?