Re:Have We Reached Peak AI?
by benlivengood on Monday March 18, 2024 @08:03PM
(#39751537)
The scaling laws don't show an inflection point yet [0], DeepMind and OpenAI are producing rudimentary agents, and inference keeps getting cheaper.
It's certainly possible that we passed the capability inflection point months ago and we'll only see a slowing of progress in the future, but even if so there's no evidence that AGI will be significantly less capable than humans when we approach the asymptote of the S-curve we're on. Models that can achieve median human performance will be far cheaper to operate than the median human salary. All the SOTA benchmarks are against experts, not median humans (as they should be). Currently, benchmarks that may be platouing[0] are above human performance.
I've always been less interested by the individual SOTA results themselves than by the curves they're following.
It's certainly possible that we passed the capability inflection point months ago and we'll only see a slowing of progress in the future, but even if so there's no evidence that AGI will be significantly less capable than humans when we approach the asymptote of the S-curve we're on. Models that can achieve median human performance will be far cheaper to operate than the median human salary. All the SOTA benchmarks are against experts, not median humans (as they should be). Currently, benchmarks that may be platouing[0] are above human performance.
I've always been less interested by the individual SOTA results themselves than by the curves they're following.
[0] https://contextual.ai/plotting-progress-in-ai/