Good voice transcription AI already do that too; that's why they work best if they know which language they're operating in, as that means they can use the language to create a model of the most likely words.
I think the most recent WWDC from Apple even has a video about adding custom vocabulary for their speech engine to pick up on that covered some details in this exact topic, though I can't search right now.
Absolutely, AI is very rarely human in its failure modes, and often has novel and exciting failure modes instead.
But, on average… or so the marketing claims… it makes fewer mistakes.
For a while, it was possible to improve upon super-human chess AI by pairing them with a human; the combination was called a centaur. Eventually the AI got too good even for that as they stopped making the sorts of mistakes humans could spot, but in the meantime, even though they were superhuman, they had failure modes that we could help out with.
If it is or not, depends on the specific use of the transcription.
Consider "I went to Lenny's" being transcribed wrong by a human as "I went to Denny's" or by an AI as "Ivan to Lenny's".
Both are wrong, but if you get a human to check, we can be oblivious to the human mistake for the same reason it was made in the first place plus the effect where seeing text alters our perception of what we hear; the AI error being inhuman means we can spot it when the human error is imperceptible.
The answer is that we have programmed these systems to do what we require. They cannot exceed but they fail becasue of errors that we have placed in these systems.
All of the tasks that you have mentioned have been programmed that way. It has taken human ingenuity to work out how to do this programming. The end result is a machine (non-sentient, non-intelligent) that is doing what we require.
If you look at game playing, a system was created to play Go and won and yet that same system fails to win against humans under many circumstances. The literature is there, yet not publicised for all the world to see. A result of keeping the hype in play.
If you look at speech recognition, these systems still fail when we humans work against them and yet, we humans still recognise what the machines fail at.
Just keep in mind that a tractor can move a greater amount of material than a human can, but it is still only a tool. A plane can travel faster and fly higher that a human can, but it is still only a tool.
We use these systems to augment our abilities and yet they are all limited in so many ways that we are not.
The upshot is that we can do amazing things with the things we create, but none of those things exist without us and all those things fail without us.
The successful Go AI were programmed to learn; we still can't program a decent Go AI with rules humans come up with.
> The literature is there
Do you have a link? Two Minute Papers just had a video about an AI systematic finding ways to confound other AI, but I thought we'd passed the point where the best Go AI could be so manipulated by humans…
I have spent forty years dealing with all sorts of computer systems - designing, building, maintaining, repairing, redesigning and rebuilding. One thing I have learnt over that time is that none of the systems ever built has been error free in terms of the logic entailed within them. All to often, I have seen systems that were used to make decisions with and those using them assuming that the outputs were correct or reasonable. Yet on investigation, the logic entailed in them was completely rubbish.
We make assumptions and often we do not carefully check that those assumptions are actually real. I don't trust anything I write until I have gone over it with a fine tooth comb and then I will try to document all my assumptions and this usually shows up various logic errors or conditions that I didn't think about. I don't see this happening much out in the real world.
What do you mean?
AlphaZero was trained entirely on self-play, and is a generic reinforcement learning algorithm. All it starts with are the rules (Chess, Go, Shogi) and a few million games later it beats — so far as I can see from a quick Google — all the humans, and most matches against AlphaGo Zero which learned the same way and which in turn beat AlphaGo Lee in every match, and that (unlike the aforementioned) was trained on examples of human matches in addition to self-play… but still learning from those examples as there's no known useful[0] set of rules that even says if a Go game is over let alone which moves are good.
There are AI which can find and exploit its weaknesses, but I've not seen anyone else suggest humans can defeat it.
> I didn't keep the link for the discussion on the confounding of the Go AI system. What the discussion covered though was that there were simple Go configurations that the GO AI failed abysmally on when playing a human - it didn't learn here.
Do you remember the name of the AI?
A bit of rummaging got me KataGo, but the humans had to use another AI to discover the weaknesses of KataGo rather than figuring it out for themselves.
And yes, KataGo absolutely does learn. The fact you can trivially stop the learning process is a feature not a bug for AI, precisely because it means any safety testing of the sort you're calling for is actually possible (albeit rather different than formal logic).
[0] pathological cases are easy — "board empty == not finished" — but not helpful.
Whose intelligence programmed this system?
> Do you remember the name of the AI?
If I recall correctly - Go AI.
They used a simple regular pattern and the system failed to beat the human. It didn't [learn] from this.
All such systems use a set of rules (whether specific or pattern based or mathematically based - there is some form of logic involved, even when using probabilistic functions), you and I can make choices based on illogical decisions - irrational decisions if you like. No computational system is capable of irrational decisions, the decisions may be surprising but of you look at the code then that option was always there somewhere, It cannot take a path that does not exist.
We can create a completely new path not previously available.
I see.
Well, that's too generic to even be searchable.
> They used a simple regular pattern and the system failed to beat the human. It didn't [learn] from this.
Anything written like that would struggle against an amateur.
The machine learning based Go AI don't do that, and do beat humans.
> All such systems use a set of rules (whether specific or pattern based or mathematically based - there is some form of logic involved, even when using probabilistic functions), you and I can make choices based on illogical decisions - irrational decisions if you like. No computational system is capable of irrational decisions, the decisions may be surprising but of you look at the code then that option was always there somewhere, It cannot take a path that does not exist. We can create a completely new path not previously available.
Whatever standard I use for logical or illogical decisions, wherever I put that line, humans and AI seem to be on the same side.
We have electrical impulses flowing though messy networks, crossing tiny chemical barriers where they can be influenced by neurotransmitters; to me, that's not different enough from information flowing through an artificial neural network with weights and biases that have been automatically modified through feedback after winning and losing millions of games to say that the machine "isn't learning" — or that humans and machines aren't on the same side of "logical", at the fundamental lowest level we can't violate chemistry any more than transistors can violate physics, at the highest level the real logic of each can be random.
AI are inhuman, certainly, but still learning.
…
Just to check, you are aware that the weights and biases of an artificial neural network are basically never set by humans? That this process has to be automated?
You might want to search what articles were discussed here on the subject.
> Whatever standard I use for logical or illogical decisions, wherever I put that line, humans and AI seem to be on the same side
You can make an illogical decision. No computer program can make an illogical decision, though the pathways followed through the code may give rise to something unusual, it still follows the logic written in the program, it cannot do otherwise. Even if you have random values occurring on inputs, the code itself will follow predictable paths. If you write self-modifying code, this too is done in accordance with specific rules.
Your example of neurotransmitters and comparing to neural networks is unfortunately flawed. We developed the idea of neural networks based on a very simplified understanding of what is happening in living neural systems. Our neural network are (whatever you might think of them) extremely simple and follow very specific logical paths. We may not be able to identify what those paths will be a priori but we can analyse what has happened after the event and it will still be based on the underlying logic we put into these systems.
> Just to check, you are aware that the weights and biases of an artificial neural network are basically never set by humans? That this process has to be automated?
Yes. The automatic processes still relies of specific logical rules.
If we use only goto's, we can write all sorts of "unpredictable" programs based on random inputs. Does this generate any intelligence? If we know the code and the inputs, we do have the ability to determine what has happened.
> AI are inhuman, certainly, but still learning
I would say it this way AS systems are simply artificial constructs and the data generated is simply generated and not learnt.
There's a lot of stuff we consider to be "common sense", sometimes those things are used to criticise AI and sometimes they're used to criticise other humans for not knowing them, but that is a category that we don't even think about until we notice the absence.
For the things not considered common sense, like playing chess (beats all humans) or speaking/reading foreign languages (more than I can name to a higher standard than my second language), to creating art (even if it regularly makes the common sense mistake of getting the number of fingers and limbs wrong it's still better and not just faster than most humans), to arithmetic (a Raspberry Pi Zero can do it faster than all humans combined), to symbolic maths, to flying planes…
A dev conference I was at recently had someone demonstrate how they hooked up their whatsapp voice calls to speech recognition, speech synthesis trained on their own voice, and an LLM, and the criticism of the people who got the AI replies was not "you're using an AI" (he had to actively demonstrate his use of AI to conversation partners who didn't believe him) but "you can't have listed to my message, you replied too quickly to have even played it all back."
We make machines that are stronger, faster, and can have much finer motor control than we have as individual abilities. No machine we have created has the dexterity that we have.
Every computational system can be analysed in fine detail to determine the limits that we have built into them. It may take an enormous amount of time and effort to do so, but we can do it. No computational system that we have built is able to exceed the limited programming we place in it.
There is an enormous amount of hype that goes on about the current generation (and future generations) of these systems, but all of them are in the abilities that we have programmed into them. They are in all essentials completely stupid (in the worst possible way - non-sentient, non-intelligent).
Every logic error that we have made in building these systems is hidden in that code. One day, those errors will come back and bite us, but there is nothing intelligent or sentient in these systems. It is our errors, for which we are responsible, that will cause those problems.
We can use them as adjuncts to our sentience and intelligence - but all they are are tools, never anything more.
However, if we cede control to these systems, we are ceding control to something that is no better than fire (a good servant - a horrendous master). After forty years, I have seen far too often, hype by humans convince other humans to cede control to the systems that humans have made and the result has been various levels of chaos.
If anything, what we need to be careful of is how humans use these systems against other humans. This is the perennial problem that we face as we build new technology.
> However, we can enumerate all the things that we create can do.
Not really, no. Even before AI, "Turing Complete" makes things extremely hard to enumerate; see Busy Beaver numbers for how small a system can be and still outside our ability to fully comprehend — needing to use up-arrow notation because exponentials aren't big enough is always good for a laugh.
You example of the Busy Beaver numbers, which was a recent interesting read, is a good example of what I was trying to point out. We have a definition and even if we cannot enumerate each number, we discuss and think about these in a rational way. At the moment, I am quite interested in Computer Algebra Systems (of which there are a variety) and I find it interesting just how limited these systems are and just how difficult it is to program into them the capabilities that humans use to solves various problems. The various discussions have been quite enlightening.
Mathematics is an interesting subject and I think shows up the intractability of ever getting that highly feared singularity.
All artificial computing systems are limited in ways we are not. Your "Turing Machine" example is one such case. The Halting Problem being a class example.
I think that far too often, we fail to recognise that what we create is not that great. We often stand in awe of the things we make without comprehending that these things are a very poor reflection of what is around us and what we ourselves are.
Every time some hype comes about these artificial stupidity systems, I look at my youngest granddaughter and see in her, capabilities that far exceed anything that we have created. Even my old buck of a goat demonstrates capabilities far, far in excess of anything we have created in all of our computational systems.
As I have said elsewhere here, we have to be careful that we do not cede control of our lives to systems that we think are more than they really are - systems that are limited, fragile and prone to failure.
You appear to be asserting that humans can tell if a loop will end, when that loop is defined so that if it does it doesn't and if it doesn't it does.
> Even my old buck of a goat demonstrates capabilities far, far in excess of anything we have created in all of our computational systems.
How so?
Not saying this is necessarily false — GPT-3 is about as complex as the brain of a rodent, so it wouldn't exactly be surprising even though the LLM only does text and completely different AI do other things — but still, what exactly do goats do that's "far in excess"?
We can determine by looking at certain problems (The Halting Problem is one such example) what the outcome will be without actually having to execute that code. The Halting Problem is one of the simpler problems that cannot be solved by computational means, which includes all artificial computational system.
You ask [How so?] to my comment about my goat. I would suggest that to understand this you need to go and observe what happens in the environment with such beasts, whether it be a cow in a local paddock or a pet dog or cat. Take time to observe the interactions that occur and think about how little [training] is involved here.
Watch children around you, take some serious time and observe them in their interactions and I think that when you think about how we program our various artificial stupidity systems that we are still at the caveman stage in our computational systems. We have barely discovered fire so to speak.
As for [GPT-3 is about as complex as the brain of a rodent], I don't think GPT-3 has even reached a single bacterium cell state of intelligence.
I would like you to try the following: Using your index finger on your left hand, touch the tip of your nose.
Now think about this: How did you do that very simple task? When did you learn and how did you learn to do that simple task?
If you think about it carefully, the task that I describes is incredibly complex.
Now what would be required to get an artificial stupidity system (AS) to do the same task? What programming do we need to do to achieve this task? What programming was done to you to achieve that same task?
When you start asking questions like this, it becomes very clear that all of our computational systems (including all of the AS systems) are incredibly simple and not at all comparable to what we find within ourselves.
We can build very useful tools that we can use to good purpose. But no tool is ever more than a tool for us.
I suppose what concerns me about our current state of affairs is that we are far too impressed by our caveman antics. There is not a single industrial system built by mankind that comes close to the integrated control and manufacturing systems found in a single living cell. None of our communication systems come close to what is found in the various control/communication systems found in even the simplest of chordate organisms.
It was very obvious, 40 odd years ago during my engineering undergraduate days, just how fragile much of our technological base was then. It is far more fragile today and yet we appear to be enamored by our [current technological prowess] which is actually far more fragile than it was 40 years ago.
No, we definitely can't do that in general, and that lack of generality is the halting problem.
> Take time to observe the interactions that occur and think about how little [training] is involved here.
Apart from "all of evolutionary history", (though everyone agrees AI are slow learners when counting how many examples they need if that's your point?), there's continuous feedback from pleasure and pain and probably a lot more emotions that don't necessarily map onto any human qualia.
> I think that when you think about how we program our various artificial stupidity systems that we are still at the caveman stage in our computational systems. We have barely discovered fire so to speak.
I'd use a different analogy; dinosaur perhaps.
> As for [GPT-3 is about as complex as the brain of a rodent], I don't think GPT-3 has even reached a single bacterium cell state of intelligence.
I think my Roomba-clone does that: touch an obstacle, back off, rotate a few degrees, go forward again.
Now that, I'm fairly sure was programmed rather than learned (in the robot; still learned in the bacteria via evolution).
> I would like you to try the following: Using your index finger on your left hand, touch the tip of your nose.
> Now think about this: How did you do that very simple task? When did you learn and how did you learn to do that simple task?
How: a network of neurons, if I remember right about 40 deep, integrating mostly proprioceptor input as it's continuously updated when my muscles move.
When: unclear, either as an infant before and autobiographical memories, or genetic (which is arguably "not me").
I'm not sure this matters, either way though, as we do have robots which are navigating entirely by proprioception, and which again learned by training rather than being programmed.
> We can build very useful tools that we can use to good purpose. But no tool is ever more than a tool for us.
When tools stops being mere tools — regards of this is mere perception, or when peasants and slaves revolt, or when (Australia) pest-control animals themselves become pests — we generally have big problems.
I agree the world is more fragile; I don't know if AI will help or not.
This seems like the logical fallacy of "begging the question" since it is far from apparent to me that they are "more capable than us at the majority of tasks."