News for hackers. Stuff that matters.
Re:Generate Hacker News Comments from Titles by antpls on Sunday April 21, 2019 @03:45AM (#19711025)
Good work, the github repository is also clean and easy to follow.

May I ask : What is the size of the live model of the website, in MByte ? (or maybe it would be nice to add that info in the README, just to have an idea)

Also, would be interesting to see how it compares to the recent GPT-2 (https://github.com/openai/gpt-2)

Re:Generate Hacker News Comments from Titles by leod on Sunday April 21, 2019 @05:18PM (#19714644)
Thank you!

The model weighs in at 1.2GB with 100M parameters, which is similar to the smallest GPT-2 model.

I wouldn't be suprised if GPT-2 small (+ finetuning on HN data) performed better than what I have trained. Other than hyperparameters, I think there are two main differences: First, I pretrained the model solely on Wikipedia data, while GPT-2 used more general web data. Second, I used an encoder-decoder model, while GPT-2 is a language model. I'm suspecting that the encoder is not very useful for this task.