News for hackers. Stuff that matters.
Re:Geocities Forever – neural network generated geocities pages by gwern on Friday May 06, 2016 @06:29PM (#11647173)
It could just be char-rnn output inside a very simple HTML wrapper. I found that a char-rnn can produce a lot of syntactically valid CSS with lots of URLs, so CSS+some HTML+image URLs is entirely possible: http://www.gwern.net/AB%20testing#training-a-neural-net-to-g...

The template here looks like it would go

"<script src="jquery.js"></script> <script src="sha1.js"></script> <script src="links.js"></script>

<script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-4209567-5', 'auto');
  ga('send', 'pageview');
</script> <html>

...[until generation limit, ~1600 characters; note the absence of any valid closing tags]

The low quality of the text might reflect the demands of simultaneously modeling text+HTML+CSS in a too-small RNN or insufficient training (training a high quality char-rnn/torch-rnn to convergence on the entire Geocities corpus, or even just a few hundred megabytes, could take weeks).

Re:Geocities Forever – neural network generated geocities pages by gwern on Friday May 06, 2016 @09:35PM (#11647906)
Yes, it turns out to be char-rnn/torch-rnn: 'RNN was 512-node, 3-layer' https://twitter.com/aanand/status/728738421758414849

He wasn't training on his own computer so aside from being much too small (for modeling at least 3 languages, text+HTML+CSS, I would start with 1k neurons and go up from there), it probably wasn't trained to convergence either. The 50MB size is maybe a bit on the small size but probably not the root cause of the low text quality.