tejask

Yes this is very much inspired by Geoff's work.

This is an interesting question. Technically, we capture a probability distribution in the code layer (between encoder and decoder). So you can sample from it multiple times and assess uncertainty. However, we have not really studied this.

In order to approximately learn a "real" graphics engine with support for basic physics, just feed-forward computation might not be sufficient. A more natural way to learn graphics/physics might be to learn the temporal structure more explicitly. On the other hand, it might also be interesting to just add temporal convolution-deconvolution structure in the existing model. This is work in progress though.

thanks for the references! I like that many people are doing such things. After looking at the chairs paper, it seems like they render images given pose,shape,view etc (supervised setting). However, in our model, there is a twist as it is trained either completely unsupervised or biased to separate those variables (but it is never given the true values of those parameters ... just raw data).

there are many ways to parametrize the decoder. One of the ways is to constrain it to output an explicit mesh or volumetric representation and express the rendering pipeline so that it's differentiable. The encoder will then effectively learn an "inference algorithm" to get the best output. A feedforward neural network is not enough and recurrent computations will eventually be necessary.

To the best of my knowledge, not much at all. It is an open question. Besides, a feedforward net is not going to be enough.

One of the authors here. You are absolutely right! In fact, I am currently doing something similar but it is not working as well yet. As far as this work is concerned, we wanted to see how model-free can we go.

In summary, the most interesting part for the general audience might be the following question -- can we learn a 3D rendering engine just from images or videos without any hand-engineering?

Apart from the interesting applications for computer graphics (like rendering novel viewpoints of an object from various viewpoints), this can also be directly used for vision applications. This is because computer vision can be thought of as the inverse of computer graphics.

Goal of computer graphics: scene description -> images

and

Goal of vision: images -> scene description.

Therefore, training a neural network to behave like a graphics engine is interesting from both these perspectives. We are a LONG way from even scratching the surface.

The first advice you give is a good reminder for people like me who have tons of unfinished projects.

My programming experience was unexpressed before college as I was mostly interested in basic sciences - mainly physics. It was only after high school did I seriously get into CS type of stuff as I started developing an intesrest in AI/algorithms. So for me, it is not hard to believe that there could be a lot of people with a hidden aptitude for programming. After all, programming is a means to an end. After high school, understanding the brain and developing AI was my "end", which obviously requires strong programming skills and thus subsequently triggered a stronger interest for programming.

This is awesome!

For an average teenager who is about to go to college, online education is a less favorable choice as it would probably be deemed less prestigious due to issues mentioned in the article. In the near future, the education industry could primarily influence markets which deal with professional or people who would use it for the sake of learning (few and difficult to indulge them over long periods).

Good concept. I would have liked to see a demo page (or video) before subscribing.

The reason I said it's one is from the perspective that the software is a combination of spiking potentials and chemical reactions. If you haven't read "On Intelligence" by Jeff Hawkins, you should. Though his results/theory may not even be close to the real explanation behind our brain, it sets a good stage for the topic.

From what you say, you seem to hint that we will never be able to produce a human-perfect unsupervised learning rule that can handle all modalities we perceive and make inferences. As far as classical machine learning is concerned, many learning rules are domain specific and training/learning heuristics changes with modalities. However, we are seeing an emergence of neuroscience informed machine learning, which is slowly uncovering importance of vital structure within the brain such as hierarchies within the visual cortex and its actual impact on learning. As far as consciousness is concerned, it is again tied with the problem of solving learning.

Thanks for reminding again. I wanted to do this for a long time and I just did.

One obvious mistake is on Page 2 (pseudo-code).

@Result = @Result | @ClauseResult

But @Result is 0 by default.

If you look at it this way, then you would need to have a timeout element associated with each entry. In neuroscience, long term memories are formed by neural genesis in hippocampal region - meaning new synapses form upon reinforcement learning. These connections decay over time, which means you slowly start "forgetting" things. If the timout is a small value and there is no stimulus to increase it, we will quickly forget leading to short term memory.