But we have:
- LLMs pass the Turing test more convincingly than anything before
- LLMs are vastly more popular than any AI/ML methods before it
- "But they don't reason!" -- we're advancing chain of thought
LLMs are also ML (using statistical methods)
I know this sounds like moving the goalposts, but maybe all this does is show that the Turing Test was not sufficient?
> LLMs are vastly more popular than any AI/ML methods before it
Are we back in high school? Popularity contests?
The implications of passing the Turing Test are profound. Tricking a user with a 5 minute conversation isn't the same. The examiner is allowed to be an expert questionner.
For one thing, software engineering would quickly be taken over as a client can just chat with an AI to get the code. That is far from happening, even with all the recent LLM advances.
Similarly for literature, science etc. Currently, you can detect a difference between a competent human & machine in all these fields just by text chat. Another way of saying this is that the Turing Test is AI-complete and is a test for AGI.
https://courses.cs.umbc.edu/471/papers/turing.pdf
> "The Imitation Game"
> "Can machines think?"
We may need to redefine what these words mean.
Whether machines can think, and whether we believe so, are distinct.
If we're just a kind of machine, we can just say that LLMs have not surpassed us.
While LLMs still give away obvious signatures (their flaws are LLM-y and not humanlike), they play the imitation game better than any other machines before them. I lived long enough and played enough with simpler chatbots to believe that the simple text-based imitation game that Turing suggested will be won by computers before machines achieve reasoning ability that surpass the smartest humans.
> [...] the Turing Test is AI-complete and is a test for AGI.
Since Turing never used the term AGI, this is an interpretation.
Since we cannot determine whether a machine can think by comparing their anatomy to ours and drawing a conclusion that whatever we do, they must do the same, we are bound to compare their ability to imitate us. Which means we can't ultimately know, we can just feel ourselves convinced.
So there's no objective point at which to place the goalpost.
So constantly moving it is an expression of acclimatisation.
I'm impressed by LLMs, in spite of the hype and the homework cheating.
If anything, LLMs ability to imitate human work how far from thinking a lot of human behavior is.
That is not true. Even our cells, with features that look a lot like machinery such as a proton pump, is on the whole several orders of magnitude more complex than any machine. Even a single human cell is more like an ecosystem than a machine. Let alone entire humans or even just the human brain. Consider that both cells and humans are capable of reproduction.
> If anything, the ability of LLMs to imitate human labor
Within extremely narrow confines and quite often going over into just conjuring up nonsense.
Good point. Not to mention the massive reliance on organisms like mitochondria and bacteria that don’t even share the host’s DNA.
> [imitate human labor] within extremely narrow confines and quite often going over into just conjuring up nonsense
Much like human labor.
So the intelligent, reflective and thoroughly iterated work is hardly replicated at all, and the poorly imitated, easily repeatable coursework and boring paper sludgework excellently so.
So we don’t just get to criticise LLMs for not actually being intelligent. We similarly get to criticise humans for not being so when we might think we are, either.
So? Does your car stop being a machine just because it's a complex systems of moving parts, many of which are dynamically balanced through feedback loops, which involve components sourced from different vendors, and substances that are not part of the original manufacturing data sheet?
Exactly what insight does this give us? I feel this is trying to contrast a single machine "unit" against a complex system, while also sneakily committing a naturalistic fallacy by using "machine" vs. "ecosystem" to imply "machine" vs. "life" in the magical sense (i.e. as if life was something beyond a physical process).
Why? I'd say it's been self-evidently true for at least three decades now.
> Even our cells, with features that look a lot like machinery such as a proton pump, is on the whole several orders of magnitude more complex than any machine.
How many is several? I think that, at the level of a proton pump, individual components comes close to the order of complexity humanity deals with in man-made machines.
Also, at this level, things really look like machines, act like machines, quack like machines - there's no reason to not call them machines, given they obviously are. It's naturally originating molecular nanotech.
> Even a single human cell is more like an ecosystem than a machine.
Certainly. But then, an ecosystem is defined as a system made of bunch of varied stuff interacting with each other, finding balance through a set of feedback loops. An ecosystem of machines is still an ecosystem, and is arguably a machine in itself, too - after all, the term "machine" also applies to self-balancing / feedback driven systems, ever since we invented control theory and formalism to describe feedback loops.
> Let alone entire humans or even just the human brain.
Complex machines don't stop being machines when you keep adding moving parts to increase complexity. Or, at which point between a protein pump and a human being you believe the assemblage of molecular nanotech stops being a machine?
> Consider that both cells and humans are capable of reproduction.
What's that supposed to tell us? Human reproduction involves cell reproduction.
Further, we can even disallow decisions based on the human/machine signature. The computers don't need to imitate human frailty and examiners decide based on the quality of the output, not whether it feels human or machine. I think that this is also why one of the stipulations is that the human/machine is not physically visible.
But even then, this goal is far away. We are not asking just about simple short chat imitation games but whether a computer can operationally do everything humans are capable of in the textual medium. This often requires experts.
The chat game version can be solved. But, the foundational goal is far away.
So it's quite neat that we've managed to beat the test by brute-forcing the right NN architecture on a goal function that literally is just "give the same response as a thinking human", in full generality of that phrase.
> But even then, this goal is far away. We are not asking just about simple short chat imitation games but whether a computer can operationally do everything humans are capable of in the textual medium. This often requires experts.
That is moving the goalposts. Or just another game entirely - which is fine on its own, as a further milestone.
Just like the original test, there are 3 participants in a test: an examiner, a human, and an AI.
The examiner may ask any question or series of questions to the human &/or AI, and receive responses. This is done via text or other means which prevent the examiner from directly seeing the body of responder.
Both AI and human are to attempt to convince the examiner that they're the human.
Unlike the original test, it then needs to repeat with different examiners.
If there exists an examiner which can distinguish an AI from a human (for all humans) in a Turing test with probability non-negligibly different from 50%, then the AI fails this meta-Turing Test.
Just because an AI can fool some people doesn't mean it's a human-equivalent intelligence.
Existence is too strong a criterion. Even correcting for random chance, if there's 1, 10, 100, even 1 000 000 examiners who could distinguish an AI from a human with high probability, that steal means the AI passed the test in practice, because even a million examiners is still a rounding error compared to the human population.
Being able to converse (or convince someone the conversation is natural) is not the same as being able to perform a technical task well. The same applies to humans as well, of course, which perhaps brings us back to the outsourcing comparison — how often does that work out great or not?
> LLMs are vastly more popular than any AI/ML methods before it
Popular is not what I'm looking for. I've seen the decisions the general public make elsewhere, I'm not letting them chose how I do my job :)
> "But they don't reason!" -- we're advancing chain of thought
Call me back when we've advanced chain of thought much more than is currently apparent, and again: for being correct in technical matters, not conversation.
This has been going on for decades. It is called outsourcing development. We've just previously passed the work to people in countries with lower wages and conditions, now people are increasingly passing the work to a predictive text engine (either directly, or because that is what the people they are outsourcing to are doing).
Personally I'm avoiding the “AI” stuff¹, partly because I don't agree with how most of the models were trained, partly because I don't want to be beholden to an extra 3rd party commercial entity to be able to do my job, but mainly because I tinker with tech because I enjoy it. Same reason I've always avoided management: I want to do things, not tell others to do things. If it gets to the point that I can't work in the industry without getting someone/something else to do the job for me, I'll reconsider management or go stack shelves!
--------
[1] It is irritating that we call it AI now because the term ML became unfashionable. Everyone seems to think we've magically jumped from ML into _real_ AI which we are having to make sure we call something else (AGI usually) to differentiate reasoning from glorified predictive text.