Re:Deep reinforcement learning is a waste of time (2019)
by lysozyme on Saturday July 10, 2021 @04:41PM
(#27796287)
That’s a really interesting way of looking at the difficulty of the problem that’s being solved. I’m curious, how do you arrive at the number 10^9?
[ Parent
| on HN ]
Re:Deep reinforcement learning is a waste of time (2019)
by throwawaysea on Saturday July 10, 2021 @07:29PM
(#27797205)
The GP specified a robot with three joints in his illustrative example. Each joint has three axes (physical dimensions) along which it can move. Each movement has an arbitrary scale of movement of 0-10, since the GP discretized continuous space into 10 buckets arbitrarily for sake of illustration. So 10 choices for each of 3 joints for each of 3 dimensions. That comes out to 10 choices for Joint 1 Dimension 1 x 10 choices for Joint 1 Dimension 2 x … 10 choices for Joint 3 Dimension 3 which comes out to 10^9.
Re:Deep reinforcement learning is a waste of time (2019)
by hervature on Saturday July 10, 2021 @06:02PM
(#27796780)
You have 3 joints with 3 dimensions. So 9 variables you can control. If each variable has 10 choices, and you need to make a choice for each, that is 10^9 possible combinations.
Re:Deep reinforcement learning is a waste of time (2019)
by whimsicalism on Saturday July 10, 2021 @06:09PM
(#27796819)
"3 joints" -> What are the 3 dimensions that joints move along? I naively would have thought one or two (ie. how much spin on one axis and how much spin on the other axis) just by naively looking at my elbow for a second.
For an arm of fixed length, polar coordinates have 2 dof.
Re:Deep reinforcement learning is a waste of time (2019)
by hervature on Saturday July 10, 2021 @08:34PM
(#27797524)
Yes, there are multiple types of joints. In this case, you could think of a ball joint like a shoulder or hip. They move "up-down" and "left-right" but also rotate.
Re:Deep reinforcement learning is a waste of time (2019)
by phunehehe0 on Saturday July 10, 2021 @06:03PM
(#27796786)
I presume it's 3 dimensions for each joint, so 10^3 * 10^3 * 10^3
Re:Deep reinforcement learning is a waste of time (2019)
by liuzhy71 on Thursday July 15, 2021 @03:43AM
(#27842322)
I think the problem is exaggerated. Even with three ball joints, the action space is not that large since there are constraints on the velocity of the joints. They have to move gradually. So the actual action space is a lot smaller. A lot RL problem has similar continuity constraints, cuz in real world we are dealing with time series signals. I am not a expert in the RL domain yet, so I open myself to any opinion.