News for hackers. Stuff that matters.
Re:Geocities Forever – neural network generated geocities pages by versteegen on Friday May 06, 2016 @06:02PM (#11647009)
The generated English (and a number of other languages) doesn't make a lick of sense (certainly not comparable to decent NN language models), and there's HTML and JS visible all over the place. But they certainly look a lot like typical geocities pages. How do you generate HTML and JS with NNs? I wish there were an explanation somewhere. Maybe that's the reason that there's so much invalid HTML.
Re:Geocities Forever – neural network generated geocities pages by gwern on Friday May 06, 2016 @06:29PM (#11647173)
It could just be char-rnn output inside a very simple HTML wrapper. I found that a char-rnn can produce a lot of syntactically valid CSS with lots of URLs, so CSS+some HTML+image URLs is entirely possible: http://www.gwern.net/AB%20testing#training-a-neural-net-to-g...

The template here looks like it would go

"<script src="jquery.js"></script> <script src="sha1.js"></script> <script src="links.js"></script>

<script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-4209567-5', 'auto');
  ga('send', 'pageview');
</script> <html>

...[until generation limit, ~1600 characters; note the absence of any valid closing tags]

The low quality of the text might reflect the demands of simultaneously modeling text+HTML+CSS in a too-small RNN or insufficient training (training a high quality char-rnn/torch-rnn to convergence on the entire Geocities corpus, or even just a few hundred megabytes, could take weeks).

Re:Geocities Forever – neural network generated geocities pages by gwern on Friday May 06, 2016 @09:35PM (#11647906)
Yes, it turns out to be char-rnn/torch-rnn: 'RNN was 512-node, 3-layer' https://twitter.com/aanand/status/728738421758414849

He wasn't training on his own computer so aside from being much too small (for modeling at least 3 languages, text+HTML+CSS, I would start with 1k neurons and go up from there), it probably wasn't trained to convergence either. The 50MB size is maybe a bit on the small size but probably not the root cause of the low text quality.

Re:Geocities Forever – neural network generated geocities pages by jnpatel on Friday May 06, 2016 @06:24PM (#11647149)
Agreed about wishing there were an explanation.

It's not clear to me that NN are the best way to get interesting Franken-pages. Maybe a Markov chain instead?

Re:Geocities Forever – neural network generated geocities pages by yolesaber on Friday May 06, 2016 @08:03PM (#11647625)
The best way is to have the NN generate the HTML / CSS / JS code and then use a markov chain to create content to populate the page with.

It also has to do with quality of input. The texts on geocities pages aren't exactly pinnacles of clear and legible language. Same with the HTML.

Re:Geocities Forever – neural network generated geocities pages by drdeca on Friday May 06, 2016 @07:07PM (#11647352)
Markov chains can't keep track of nested stuff though I thought?

I suppose that NNs have some sort of limit to how many layers they handle, but I thought they tended to learn nesting fairly well, unlike markov chains?

I could be wrong about this.

Re:Geocities Forever – neural network generated geocities pages by jxdjsks on Friday May 06, 2016 @10:55PM (#11648174)
It looks a lot like the stuff I produced a while back with an rnn. I just fed the rnn raw web pages for a giggle. It does surprisingly well, often managing to close braces correctly. It really goes to show how good the browsers are at dealing with junk html. I was thinking about trying to jointly train the html and an image of the webpage, in the vague hope that you might be able to go from image to webpage. One day. Probably quite soon...