I fed 24 years of my blog posts to a Markov model

Re:I fed 24 years of my blog posts to a Markov model by Legend2440 on Sunday December 14, 2025 @02:13PM (#46265857)

Google made some very large ngram models around twenty years ago. This being before the era of ultra-high-speed internet, it was distributed as a set of 6 DVDs.

It achieved state-of-the-art performance at tasks like spelling correction at the time. However, unlike an LLM, it can't generalize at all; if an n-gram isn't in the training corpus it has no idea how to handle it.

https://research.google/blog/all-our-n-gram-are-belong-to-yo...

[ Parent | on HN ]

Re:I fed 24 years of my blog posts to a Markov model by tqian on Sunday December 14, 2025 @11:10PM (#46270391)

I have this DVD set in my basement. Technically, there are still methods for estimating the probability of unseen ngrams. Backoff (interpolating with lower grams) is an option. You can also impose prior distributions like a Bayesian so that you can make "rational" guesses.

Ngrams are surprisingly powerful for how little computation they require. They can be trained in seconds even with tons of data.

[ Parent | on HN ]