News for hackers. Stuff that matters.
Re:Using Keras and Deep Deterministic Policy Gradient to pla... by modeless on Tuesday October 11, 2016 @04:13PM (#12688113)
This is cool, but unlike some other recent neural game players [1] [2], it doesn't look at the screen at all. It also has an objective function which is tightly coupled to this game. Therefore this work is not easily generalizable to other video games, nor would it be useful for a real-world self-driving car.

[1] http://vizdoom.cs.put.edu.pl/ [2] https://deepmind.com/research/dqn/

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by fchollet on Tuesday October 11, 2016 @04:18PM (#12688160)
It would be easy to replace the input features with representations from a pixel-level convnet, to make this is a "real" self-driving car, going from pixels to commands.

Anyone interested in this type of research: consider cloning the repo and implementing this modification, it would make a great starter project.

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yanpanlau on Tuesday October 11, 2016 @05:58PM (#12688861)
It is quite easy to change the input features as pixels and fit into convnet under Keras (That's why I love Keras so much). However, gym_torcs only support 64x64 pixels and it is hard to see by human eyes, IMHO.

https://github.com/ugo-nama-kun/gym_torcs/issues/4

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by modeless on Tuesday October 11, 2016 @07:06PM (#12689245)
Interesting, yeah that seems a bit small. Would be a fun project to fix!
Re:Using Keras and Deep Deterministic Policy Gradient to pla... by hardmaru on Tuesday October 11, 2016 @02:03PM (#12686881)
Projects like these reminds me of why I got into AI research in the first place.
Re:Using Keras and Deep Deterministic Policy Gradient to pla... by bluetwo on Tuesday October 11, 2016 @02:30PM (#12687189)
Very interesting.

Given enough training, would the car learn to find the apexes of turns?

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yazr on Tuesday October 11, 2016 @03:09PM (#12687583)
IMHO yes.

It is probably an easy local maxima with relatively fast convergence

Far harder are

a. "planning" - finding the optimal path through sequential turns

b. generalizing the learned experience to a new, unseen situation

Would love the get feedback from the author on this

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by sp332 on Tuesday October 11, 2016 @03:30PM (#12687744)
No, the function rewards staying in the middle of the track.
Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yanpanlau on Tuesday October 11, 2016 @06:03PM (#12688887)
Staying in the middle of the track is not a necessary requirement in the reward function. The reason I include it is to speed up the learning time in the beginning. You can remove it once you learn a reasonable policy and see it the agent can find the optimal apex path. I will do a test tonight.

Just like in human world : You first learn how to drive before you learn how to drift the car.

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yazr on Thursday October 13, 2016 @11:14AM (#12701259)
Can you please explain you are learning a SPECIFIC track or a random one ?

How would the existing agent fare on a brand new track ?

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yanpanlau on Sunday October 16, 2016 @02:49PM (#12720245)
Hi~I used Aalborg track as my training dataset and I used Alpine1 track as my validation dataset. The Alpine1 track is 3 times longer than Aalborg. As you can see on the video, the agent can drive reasonably OK on the validation dataset.
Re:Using Keras and Deep Deterministic Policy Gradient to pla... by bluetwo on Friday October 14, 2016 @03:21PM (#12710440)
Any news on the test? I'm curious.
Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yanpanlau on Tuesday October 25, 2016 @11:30AM (#12788784)
Please find the result below. I modified the reward function such that staying in the middle of the track is no longer required.

https://youtu.be/Tb5gASEJIRM

Re:Using Keras and Deep Deterministic Policy Gradient to pla... by yanpanlau on Sunday October 16, 2016 @02:52PM (#12720262)
Hi Bluetwo, I am currently travelling to the San Francisco right now. Can you send me a e-mail [email protected] so I can contact you and e-mail you the result directly when I back to Hong Kong?