News for hackers. Stuff that matters.
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by pcmonk on Thursday May 21, 2015 @02:35PM (#9584556)
The code generator is awesome. There's hardly a syntax error. The file headers are the best.

Nitpick: although tty == tty is, as you say, vacuously true in this case, that's just because tty is a pointer. If tty were a float, this wouldn't be the case, since it could be NaN. I wouldn't be surprised if it learned to test a variable for equality against itself from some floating point code.

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by MrZongle2 on Thursday May 21, 2015 @06:48PM (#9585840)
If nothing else, the RNN would be great for generating bogus source code for use in television programs and movies.

It would drive those who attempt to understand & reference it absolutely crazy. :D

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by zxyzzxxx on Thursday May 21, 2015 @05:12PM (#9585408)
The code is nonsense. Their method is good for fuzzy logic like recognition, but this approach with code will never work for anything other than an art project.
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by teraflop on Thursday May 21, 2015 @05:41PM (#9585543)
Currently it doesn't work, but saying it'll never work is pretty strong.

This kind of demo shows that deep neural networks can capture the structure of language, if not the semantics, in a very general way. And we have separate evidence that they can (in principle) capture semantic meaning and algorithmic reasoning as well, for example: http://arxiv.org/pdf/1410.5401v2.pdf (the "neural Turing machines" paper from DeepMind)

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by kpil on Thursday May 21, 2015 @07:34PM (#9585964)
This is better, but you get pretty far with just markov chains with probabilities for letters actually.
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by mikkom on Friday May 22, 2015 @01:22AM (#9586654)
Show me markov chain implementation that can write code letter by letter and I'll give you a car.

(And I mean plain markov chain, not something with additional logic that understands code structure)

comment by samizdatum shows pretty well how well markov chains work without some tweaking.

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by nomel on Thursday May 21, 2015 @06:18PM (#9585707)
Feed it all of github, and I'm sure you could come up with some interesting auto complete code generation tools. Of course, coming from github , it'll be poorly documented and filled with buffer overflows :D
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by saidajigumi on Thursday May 21, 2015 @07:18PM (#9585923)
I'll agree that this is interesting, but it seems like a lot of people in this thread miss the point: we're working with multi-layer tools now. This enables modeling of multi-layer processes. The code generation as it stands is a obviously a toy, but what happens if we actually think about the real processing layers?

Take this example of code processing, and instead front it with a parser that generates an AST. For now, an actual parser for a single language. Maybe later, a network trained to be a parser. The AST is then fed to our network. What could we get out of the AST network? Could we get interesting static analysis out of it? Tell us the time and/or space complexity? Perhaps we discover that we need other layers to perform certain tasks.

This, of course, has parallels in language processing. Humans don't just go in a single (neural) step from excitation of inner ear cells ("sound") directly to "meaning". Cog sci and linguistics work has broken out a number of discrete functions of language processing. Some have been derived via experiment, some observed via individuals with brain lesions, others worked out by studies of children and adult language learners. These "layers" provide their own information and inspiration for building deep learning systems.

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by zxyzzxxx on Thursday May 21, 2015 @06:51PM (#9585852)
But how will you find that needle in the haystack that works. This is effectively producing random code samples that look syntactically correct.

There is no need for to produce readable code, it makes it easy for humans, but computers have no problems with generating and subsequently understanding unreadable assembly.

Re:The Unreasonable Effectiveness of Recurrent Neural Networks by joelthelion on Thursday May 21, 2015 @03:09PM (#9584780)
Could be interesting to plug this kind of generator into American Fuzzy Lop.
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by simias on Friday May 22, 2015 @05:36AM (#9587303)
There's not a lot of floating point in the kernel though.
Re:The Unreasonable Effectiveness of Recurrent Neural Networks by ChuckMcM on Thursday May 21, 2015 @06:47PM (#9585831)
Yes, and feed it into hackertyper.net and you can entertain an 8 year old for hours :-)