nrmn

Yes, it feels like we have squeezed most of the performance out of current algorithms and architectures. OpenAI and deepmind have thrown tremendous compute against the problem with little overall progress (overall, alpha go is special). There was a big improvement in performance by bringing in function approximators in the form of deep networks. Which as you said can scale upwards nicely with more data and compute. In my opinion as an academic in the deep RL, it feels like we are missing some fundamental pieces to get another leap forward. I am uncertain what exactly the solution is but any improvement in areas like sample efficiency, stability, or task transfer could be quite significant. Personally I’m quite excited about the vein of learning to learn.

Why do you believe this to be the case?

Mosh is great, but it does not support port forwarding. For some people, including myself, this is a deal breaker. It has been an open issue/feature request since 2012[1] and even has a ~600 USD bounty on it[2].

[1] - https://github.com/mobile-shell/mosh/issues/337

[2] - https://www.bountysource.com/issues/4471419-ssh-port-forward...

Can jackfruit be eaten raw too?

fyi to the downvotes: o7 is a farewell salute.

https://www.reddit.com/r/EliteDangerous/comments/3sdixv/what...

Working on research for my thesis, which will be wrapped up by years end. The current project focuses on improving sample efficiency in deep reinforcement learning. I am researching how best to merge the options framework with the adaptability of meta-reinforcement learning.

In my spare time I write and research algorithmic trading strategies. I’ve been sticking to the traditional techniques, with a small toe into statistics for modeling.

With whatever time is left, I’ve been learning rust and have enjoyed it quite a bit so far.

From my viewpoint, as both a researcher and someone who has built frameworks around environments/games:

- Each step within the game has to be extremely fast. I.e the game should be able to be run as fast as the machine allows while keeping physics etc. consistent.

- Runnable via library import such that there is no drawing to the screen.

- Should be easy to reset the environment to an initial state.

- RNG state should be seedable.

- I highly recommend supporting an identical interface found in OpenAI's gym. Check their docs out. Even better would be to have your game importable as an environment in gym.

- Configurable screen resolution would be great (eg. output 120x100)

- The environment is "hackable" eg. the maps or levels can be modified or loaded say via some ascii map.

- Should support multiple copies of the game running at once.

- A nice to have would be if the current environment state could be exported and loaded later.

- Expose some information/signals such that a reward signal can be created. Or better yet you define one as the game creator.

PLE as well! https://github.com/ntasfi/PyGame-Learning-Environment

(disclaimer I'm the author)

No they all use the same general principal of backpropagation to do the training. Different flavours of optimizers exist with different tweaks and additions to speed training up.

Relevant file in project: https://github.com/tflearn/tflearn/blob/0.1.0/tflearn/optimi...

Author here.

Thought I'd share a library I've been using with my personal work and projects. Its an interface around PyGame which makes it painless to start doing RL based work. Deciding to share it now as I'd like to get feedback from others on how to adjust the library and make it useful. Plus I was afraid of developing "just-one-more-thing" syndrome causing further delays.

If you want to start using it right away General Deep Q RL[1] currently supports PLE out of the box.

[1]https://github.com/VinF/General_Deep_Q_RL

Not OP but I believe the low score is due to not enough training time and incorrect parameters such as frameskip. Space Invaders was mentioned as one of the few games they needed to lower the frameskip (from 4 to 3 or 2?) on because of the flashing lasers. I'm assuming OP left the parameters as is from the implementation by Nathan Sprague, which has frameskip at 4, and trained for a few epochs.

I've been trying to read a paper a day since midsummer. These are a few of the interesting, for me personally, since then:

Generating Sequences With Recurrent Neural Networks - http://arxiv.org/abs/1308.0850 Older one, but important to understand deeply since other recent ideas have come from this!

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks - http://arxiv.org/abs/1511.06434

Unitary Evolution Recurrent Neural Networks - http://arxiv.org/abs/1511.06464

State of the Art Control of Atari Games Using Shallow Reinforcement Learning - http://arxiv.org/abs/1512.01563 Interesting discussion in section 6.1 on the shortcomings/issues of DQN done by Deepmind

Spectral Representations for Convolutional Neural Networks - http://arxiv.org/abs/1506.03767

Deep Residual Learning for Image Recognition - http://arxiv.org/abs/1512.03385

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) - http://arxiv.org/abs/1511.07289 I wish they did more comparisons between similar network architecture with only the units swapped out. Eg. Alexnet, Relu vs Alexnet, Elu.

On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models - http://arxiv.org/abs/1511.09249

Just a few from my list :)

from the flipboard article:

"The final use case that we thought of was saving bandwidth. A smaller image could be sent to the client which would run a client side version of this model to gain a larger image."

Can be applied to gifs and videos, but this really depends on the usage case and if the client would tolerate such a thing.

Torch (7 at least) uses Lua as the language you program in.

+1 on Localytics. Interviewed with them a while go. Super nice group of people and interesting product.

Heads up, I interviewed with these guys before and was shafted on reimbursement for travel when I told them I took another offer. It wasn't a ton of money but its annoying as a college student.

You can get them if you look through the network request the browser makes. Largest I saw was 4096x3096, which is just perfect for desktop backgrounds!

For anyone actually interested in implementing DNN's I wrote up a quick blog post (essentially a brain dump) of general guidelines to adhere to when training DNNs. The source for this information is primarily from videos given by Geoffrey Hinton as well as various papers.

http://343hz.com/general-guidelines-for-deep-neural-networks...

+1 on this. I've used it in a previous project of mine. I did experiment with the other Redis-go libraries but Gary's felt the most complete and solid.

You're missing Thalmic Labs in Waterloo! They are the makers of the Myo! ( https://www.thalmic.com/en/myo/ )

Any love for interns?

Would you have any good resources to conv nets? I've done the tutorial on theanos site and the related stuff on Andrew ng random wiki (ufdl?). I'm still confused on the filters part.

"Decaf (CPU) is 2 times faster than Theano with GPU"

Have you managed to reproduce this? Thats awesome if it is true! I thought Theano was already very fast.

Quick question about your code for the conv net, why do you resize the images down to 32x32? I thought one of the big features of conv nets was the fact that they input does not have to be the same, it just slides a window around the image. Am I complete wrong with this one?

Would you be willing to maybe print out the weights for each layer? I'd be interested to see what features your conv net is capturing.

Could you explain the filtering "their inputs through fixed or random nonlinearities"? I haven't heard of this before.

I feel a little apprehensive about sharing my code online, Im not sure if its an ego thing or from seeing some of the rude comments people have made over twitter about X pull request being made on github. Am I just being silly about this?

Some calculus and linear algebra. The majority is linear algebra.

Interesting article. I wonder what will be the final straw that ends up bursting the bubble, my assumption would be the students defaulting on their debts in large waves but I'm unsure when that wave will hit.