Re:Neural scene representation and rendering
by formalsystem on Thursday June 14, 2018 @05:34PM
(#17315593)
If any of the authors are on this thread, am wondering if there are any to plans to release source code this? This can potentially make generating environments for games or VR trivial by just taking photos IRL and then importing them into some game engine to generate a scene
[ Parent
| on HN ]
Re:Neural scene representation and rendering
by modeless on Thursday June 14, 2018 @05:53PM
(#17315755)
The important part is the dataset, which in this case is generated by DeepMind Lab, which is already open source: https://github.com/deepmind/lab
Reimplementing the rest of the paper shouldn't be tremendously difficult. These techniques tend to be fairly simple at their core. But training it could be expensive. In any case, it is a long, long way from here to "make generating environments for games or VR trivial by just taking photos IRL and then importing them into some game engine". Many years of research remain.
Re:Neural scene representation and rendering
by purplebear on Friday June 15, 2018 @06:08AM
(#17318656)
Hi, you can find the actual datasets used for the paper here: https://github.com/deepmind/gqn-datasets
Re:Neural scene representation and rendering
by formalsystem on Thursday June 14, 2018 @06:07PM
(#17315866)
Hey James, since you seem to know a lot about graphics and ML. Am wondering which specific problems remain open before what I mentioned becomes more feasible product wise?
Re:Neural scene representation and rendering
by modeless on Thursday June 14, 2018 @07:11PM
(#17316180)
The domain of images used in this research is extremely limited. These are very low resolution artificially generated images of small scenes with simple lighting, simple textures, simple geometry, and very restricted camera positions and parameters that are known exactly (which is not the case for most natural photos). Each of those restrictions needs to be lifted before this will work on realistic natural scenes, and that will require many orders of magnitude more data. It's not clear that this approach will easily scale up to that amount of dataset variation. It's likely that a much fancier neural net architecture and training scheme will be required, and probably faster hardware too.
This is not intended as a criticism of this research, which I think is really great.