Re:Where machine learning meets rule-based systems
by chongli on Friday July 07, 2017 @09:44PM
(#14723583)
Training it to mimic humans means it will always be worse than humans.
The success of AlphaGo says otherwise. The distinction is that you're not mimicking one human but some large set of humans and you're using your fitness function to guide you toward a maximum 'meta human'.
We don't need self driving cars to be perfect drivers. If they're better than any human driver then that's a good enough reason to replace human drivers.
Re:Where machine learning meets rule-based systems
by akvadrako on Saturday July 08, 2017 @10:04AM
(#14725437)
AlphaGo was not primarily trained to mimic humans, but to win games. This included playing many games against itself and semi-random tree searches for better strategies. If it was only mimicking humans it probably would have lost to the world's best.
Re:Where machine learning meets rule-based systems
by Houshalter on Saturday July 08, 2017 @03:57PM
(#14726931)
AlphaGo was only initialized by mimicking humans. It then played millions of games against itself. And learned what moves were more likely to lead to wins. And then it was combined with a tree search algorithm that let it explore many more moves into the future than humans could consider.
Training a bot just to predict what move a human would make, would always lose to that human. It will just absorb all the human's mistakes and weaknesses. And on top of that it will add it's own, since no AI algorithm is anywhere near 100% perfect.
Re:Where machine learning meets rule-based systems
by cbames89 on Friday July 07, 2017 @08:53PM
(#14723430)
"Training it to mimic humans means it will always be worse than humans." This is not necessarily true, there is work in learning from demonstration where robots have exceed the "experts" reward. I'll post some papers later if you're interested.
Re:Where machine learning meets rule-based systems
by YeGoblynQueenne on Saturday July 08, 2017 @12:01PM
(#14725847)
Please do. I'm not the OP but I'm interested.
Re:Where machine learning meets rule-based systems
by cbames89 on Wednesday July 19, 2017 @04:29PM
(#14807622)
Sorry for the delay. Here's some early work from Pieter Abbeel where a helicopter does aerobatic manuevers better than the demonstrations it received: http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=EAE...
And even making a fitness function is a pretty difficult task. You can't exactly let a self driving car get into thousands of accidents until it learns not to do that. Making a good simulation is extremely difficult and very expensive. Training it to mimic humans means it will always be worse than humans.
As AIs get smarter this will become more of a problem. They will become more clever at exploiting the reward functions and even fooling human supervisors. One could imagine a self driving car causing accidents indirectly in some way. But it doesn't care because the car itself didn't collide with anything.
In the extreme case, AIs much smarter than humans would be extremely difficult to trust. They could figure out how to manipulate humans to get them to press their reward buttons or whatever equivalent. The famous thought experiment is an AI in a paperclip factory that desires to maximize paperclip production. So it invents nanotech, tears apart the Earth, and eventually turns the entire solar system into a massive paperclip factory.
Perhaps that's a long way off. But it's not comforting to know that AI ability is increasing faster than our ability to control AI.