Even if they reach above average our whole society is structured around managing billions of above average humans. What's novel?
There's a lot of bragging, sure. But what I've seen from the OpenAI models is more like a university level intern or a fresh graduate.
Not that it matters, as the point is to aim for where the ball is going rather than where it is now, but I'd say that this makes them above the average human performance (at least, within its domain).
> Even if they reach above average our whole society is structured around managing billions of above average humans.
That's putting a binary cutoff in an arbitrary place; half the population is always above average on any measure (or indeed topic), what I wrote was "significantly above average".
To crudely approximate with IQ, which is a bad measure for humans and much worse measure for AI, there's just over a billion humans of IQ > 115, around 182 million of IQ > 130, and around 8 million of IQ > 145.
I'm not sure how much to trust estimates of politician's IQs given the tribalism involved, but surveys of other groups tend to show that around half of the highest performing leaders (CEOs etc) with the most power, are in the top 1% by IQ.
(Who knows when, or if, someone will make an AI that's good at the important tasks within of each of those roles).
> What's novel?
Two things:
1) Breadth: the current vogue is very general models, so they're interns sure (even though I rank them as a higher level of intern than you), but they are interns at everything — there is no human you can hire with a even a mediocre grasp of Mandarin and Arabic and Welsh and python and CSS and calculus and magnetohydrodynamics and psychology and …
2) Price: there is no human who can read a million tokens for a price equal to two days of the UN abject poverty threshold, or write a million tokens for six days of the UN abject poverty threshold.
You physically can't work that fast, even if you were typing [a] then [space] ten times a second for 18 hours a day; and you basically can't feed yourself if you earn less than that per day.
Right now, the best LLMs are interns not experts at each of these things; I don't expect the transformer model to be good enough to reach the top level in each of these things, but then again I didn't expect anything like Stable Diffusion to happen before self driving cars.
Some random member of the, say, Australian parliament, someone I've never heard of, is (relatively speaking) much more powerful than 99% of the planet.
A journalist who has an audience of millions, likewise.
A software developer whose work is economically valuable either through a broad audience or through being used to control something of great value, also powerful.
Not sure I agree with the 99% of planet. Even if a single member of the Australian parliament could command all of Australia, it would still only amount to commanding 0.3% the global population (or maybe 2% of NATO plus Japan). Maybe within Australia the individual member's power is more than 99% of population?
I don't know much about Australian politics, but in some places vanilla members of parliament or not very powerful within the parliament (compared to party leadership, whips, etc.).
Software developers need people that sell stuff, admin for larger organizations etc. They also very rarely have unchecked ability to change things when those things are high risk etc. Zuckerberg, for example, is a huge exception, not the norm.
Journalists, even with a large readership, typically would not have much ability to get their readership to just act a certain way. There is not a great command relationship.
Most individuals just don't hold much power.
I think that's the wrong way to calculate the value?
OK, so, how to phrase this…
Imagine we were talking about money rather than power (because money can be quantified more easily, and is an imperfect token of power).
I believe the base salary for Australian MPs is AU$211,250/year, which is ~ USD 140,700/year. This means they earn more than 99% of the world precisely because:
> Most individuals just don't hold much power.
Still works when substituting in "money" for "power".
Saying that each random MP has more power than 99% of the world population, is very different than saying that each of them individually controls 99% of the power, which would be tautologically impossible.
It's an analogy to explain the point.
I'm saying the ranked list of people by power, random* MPs are in the 99th percentile.
I mean, is there even a single politician in the world who represents less than 99 people? (There used to be**, but any current examples?)
* OK, so that doesn't work over literally all polities — a Tuvalu MP might not have as much power as the parking attendant nearest to the Australian parliament building — but hopefully this at least helps clarify? Perhaps?
If they were really super effective broadly, then Trump would have won any popular vote easily.
Trump is up against all the other equally effective communicators who are both allowed to and motivated to hold that specific office.
Do you think I was claiming someone might be able to demonstrate absolute power, rather than mere relative power? Even Stalin didn't have that, but he would've been one of the closest.
You do have to be significantly above average to get there. Current generation LLMs are a bit above the human average, enough to be interesting, but probably not enough to be a personal Goebels for every would-be dictator.