In practice they are basically simplified programming languages. If this, then that. The application of rules can lead to surprising emergent behavior. You might be able to find a pattern in it but you’re more likely to get wrong results and need to update the rule set.
There is also the problem of inputs. You end up needing to be an “expert” on the system to get good results.
That’s what I was thinking about when I mentioned emergent behavior.
The “fuzziness” of expert systems fascinates me. :)
What do you mean by ... winning out? It seems to have made one novel diagnosis, which is great for the individual in question. But how many times has it diagnosed something completely absurd? What's the ratio of signal to noise here?
If it's anything like trying to use it to solve difficult engineering problems it must have hallucinated turbocancer hundreds of times over.
Symbolic AI has been less performant than NN AI for decades now and by a large margin. Language, speech, images etc.
>It seems to have made one novel diagnosis
Ok so let's get this out the way. GPT-4 is good at diagnosis
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10425828/
>Six patients 65 years or older (2 women and 4 men) were included in the analysis. The accuracy of the primary diagnoses made by GPT-4, clinicians, and Isabel DDx Companion was 4 of 6 patients (66.7%), 2 of 6 patients (33.3%), and 0 patients, respectively. If including differential diagnoses, the accuracy was 5 of 6 (83.3%) for GPT-4, 3 of 6 (50.0%) for clinicians, and 2 of 6 (33.3%) for Isabel DDx Companion
The point isn’t to replace doctors but to empower them. They still need to come up with tests to rule out conditions.
We also have an algorithm for the tests: information gain. If you think of diagnosis like a game of Guess Who, we need to come up with tests which can narrow down the space of possible conditions. (It’s a bit more complicated of course since tests have a margin of error too.)
The thing is, a model doesn't have to be accurate to be useful. Expert-derived systems tend to operate closer to how doctors understand the problem, making them more interpretable.
If all a doctor is doing is following a flow chart, you can be damn sure that a machine programmed with the same flow chart would do it more effectively.
But also the patient usually has plenty of time to answer additional questions. You can cover a ton of ground in less than an hour.
Patient doesn’t have to be sitting in the exam room to run out of time.
I can see a short term future where “doctors” are replaced with non-doctor medical personal who gather data. Blood draws, temperature checks, collecting stool samples, photographing lesions, these are all easier with two hands. Analyzing the data will be done for the masses by machine.
And of course because of how the USA healthcare system works, it will somehow manage to be evens more expensive.
Not the first in the tome, but pg 240[2] contains a great example of such a decision tree.
1. https://archive.org/details/harrisons-principles-of-internal...
2. https://archive.org/details/harrisons-principles-of-internal...
I don't think humans do that in the same way as code would. And why would they? That's exactly how they're more valuable!
We use them at work for the diagnosis of mechanical parts rather than human bodies. We don't call them expert systems but they are exactly that. There is a database of potential failures, repairs and verifications, and it will suggest a troubleshooting procedure taking cost into account. It means that it may try unlikely causes first if they are easy to check. It also has simple learning capabilities.
The problem with expert systems is that it needs a curated and complete database. It is possible to do with mechanical parts as they are well specified and failure modes are well documented. The top-down approach comes naturally because that's how components in a mechanical system are designed. By comparison, human bodies are a complex mess.
As for ChatGPT, it is a pure product of technical advances. Neural networks are an old idea, they knew about the theory in the 60s, and we had implementations in the 80s. The problem is that with little data and limited computing power, they were little more than toys. They have been made practical in the last few years only because of the "rich internet" on one side and lots of powerful GPUs on the other. And the way I understand it, recent advances in deep learning technology is not really fundamental research, more like trying a bunch of stuff and see what sticks, again made possible by lots of data and computing power.
Perhaps this was the camp of AI that thought you could just build a decision tree (top down) ... that appears to have failed?
Instead the bottom-up (neural networked) AI seems to be winning out.
I wonder though if a ChatGPT would have been possible in 1985 with no rich internet to feed it data. I wonder if it would have been possible with even 2000 internet?