News for hackers. Stuff that matters.
Re:MuseNet by yeldarb on Thursday April 25, 2019 @03:59PM (#19751941)
I'm working on this.

I've been training on Stack Overflow and the model has already learned the syntaxes and common coding conventions of a bunch of different languages all on its own. Excited to see what else it's able to do as I keep experimenting.

Some sample outputs (you'll probably want to browse to some of the "Random" questions because by default it's showing "answers" right now and I haven't trained that model as long as some of the older question-generation ones): https://stackroboflow.com

Re:MuseNet by lucidrains on Thursday April 25, 2019 @04:05PM (#19751995)
I've tried it as well and got good syntactic results. For more sensical programs, I think we will need more layers & attn heads. Perhaps someone will fork gpt-2 and add the sparse transformer to it.
Re:MuseNet by mgalgs on Thursday April 25, 2019 @10:19PM (#19754313)
These are actually a lot of fun to read. Kudos!