News for hackers. Stuff that matters.
Re:Deep reinforcement learning is a waste of time (2019) by phonebucket on Saturday July 10, 2021 @04:53PM (#27796387)
I have to disagree with a good chunk of this article.

The article completely misses the main advantage (to me) of reinforcement learning:

Reinforcement learning allows you to optimise on non-differentiable outcomes.

I can't differentiate real life, but I want to optimise a process within real life. This feels tantalisingly close to AGI. If I can figure out a reward function, I can use reinforcement learning.

Yes, this requires a reward function to be defined. Yes, this is a challenge to AGI. But to say that the big labs are not aware that this is a challenge to AGI is unfair. DeepMind is actively investigating open ended learning: https://deepmind.com/research/publications/open-ended-learni....

Just because the labs haven't tackled all the questions doesn't mean that they're not busy tackling difficult questions.

Re:Deep reinforcement learning is a waste of time (2019) by toisanji on Sunday July 11, 2021 @09:42AM (#27800711)
DeepMind just published a paper "Reward is enough": Furthermore, we suggest that agents that learn through trial and error experience to maximise reward could learn behaviour that exhibits most if not all of these abilities, and therefore that powerful reinforcement learning agents could constitute a solution to artificial general intelligence.
Re:Deep reinforcement learning is a waste of time (2019) by phonebucket on Sunday July 11, 2021 @11:49AM (#27801787)
> DeepMind just published a paper "Reward is enough"

That paper does not mean that DeepMind believes that reward is enough. It is one of their many papers that is seeking to espouse a point a view. It does not mean it is their exclusive point of view.

For example, DeepMind are actively hiring for researchers in open-ended learning (check their website). This is not a new thing for them, e.g. they were quickly on the ball with apprenticeship learning too (https://arxiv.org/abs/1706.06617).

Re:Deep reinforcement learning is a waste of time (2019) by soVeryTired on Saturday July 10, 2021 @05:53PM (#27796720)
Can you clarify what you mean by 'non-differentiable outcomes'?
Re:Deep reinforcement learning is a waste of time (2019) by ArnoVW on Saturday July 10, 2021 @06:48PM (#27797016)
My guess: a function that you can evaluate but that you can not differentiate. Differentiate being "know the slope".

Differentiable functions are great because you can run gradient descent on them in a very optimized way. Example: if your objective is to have a very high value, search in the direction that has a mounting slope.

Though maybe I'm missing something 'cos it seems to me you can run gradient descend on non-differentiable functions. It just requires more evaluations.

Re:Deep reinforcement learning is a waste of time (2019) by jeeeb on Sunday July 11, 2021 @12:19AM (#27798443)
> Though maybe I'm missing something 'cos it seems to me you can run gradient descend on non-differentiable functions

Isn't that exactly the point though? If you don't have an analytical solution for the gradient of the loss (reward) wrt the parameters - yes - you could brute force a numerical solution but as the number of parameters grows that quickly becomes infeasible. Approaches such as RL and GA provide a more intelligent way to search the parameter space.

Re:Deep reinforcement learning is a waste of time (2019) by LeegleechN on Saturday July 10, 2021 @07:24PM (#27797183)
Neural network models have millions of parameters, which means you are trying to optimize in spaces with millions of dimensions. If you can't use differentiation you get hit really hard with the curse of dimensionality.