This was built in '17: https://intoli.com/blog/hacker-news-title-tool/

HN discussion: https://news.ycombinator.com/item?id=14400603

Interesting and good to see that as a reference. However, I suspect that approach overfits, and the author does not mention using a validation set.

Yes, apparently "Rust, Rust, Rust, Rust!" almost guarantees 1st page according to the model.

I did one some years ago (basic attempt using XGBoost) and AUC was high 0.6s or low 0.7s.

I think I treated the title as a string rather than words and each column/feature was one letter. That is, if there were 256 possible characters and each title could only be 80 characters long, there'd be 256 * 80 * 2 columns (title + title reversed).

If you are doing a classification model, what’s the score threshold?

The trick with that approach is if you set the threshold high, it’s very easy to get a model with a high accuracy since classes will be imbalanced. (The vast, vast majority of HN submissions do not do well)

If I remember correctly, HN's data dump included a made_front_page flag (not exact name). Also, XGBoost includes a feature that attempts to normalize against imbalanced classes. As for thresholds, there's only including submissions with at least average +/- standard_deviation votes.