News for hackers. Stuff that matters.
Re:I fed 24 years of my blog posts to a Markov model by tqian on Sunday December 14, 2025 @11:10PM (#46270391)
I have this DVD set in my basement. Technically, there are still methods for estimating the probability of unseen ngrams. Backoff (interpolating with lower grams) is an option. You can also impose prior distributions like a Bayesian so that you can make "rational" guesses.

Ngrams are surprisingly powerful for how little computation they require. They can be trained in seconds even with tons of data.

There are no comments on this comment yet.