News for hackers. Stuff that matters.
Re:Why Computers Won’t Make Themselves Smarter by gwern on Tuesday March 30, 2021 @10:18PM (#26642842)
And there is no particular reason to believe that capabilities are merely the sum of a bunch of random iid variables! If you look at how DL scales with model size or data or FLOPS, it's a powerlaw (or lognormal), and in countless places in the natural world, things are lognormal rather than normal. Similarly, wealth, scientific productivity, lots of things are lognormal rather than normal. Even for human intelligence, the normality is due more to the fact that we all come off the assembly line as Mark 1 human units, practically identical, with the differences coming from environmental randomness & randomness in genetic mutation assortment.

There are no comments on this comment yet.