This concept of making the best use of any available data to infer useful probabilities seems like a crude form of some ideas behind current LLMs, BUT they (Peters, Graham, Robinson and Woodhead) were doing this work 20 years earlier.

Bayes theorem dates back 250 years.

This is what engineering is all about. Applying what it long known to solve new problems, or old problems better.

LLMs are a triumph today because someone made a keystone observation

Hi I'm the aforementioned Gary Robinson. Very sad to hear of Tim's passing. A really nice guy. And he wrote the following code for the SpamBayes project: https://www.linuxjournal.com/files/linuxjournal.com/linuxjou...

The reason that's especially interesting is that I was using an AI to write something that needed that functionality, and it spewed out that same Python code, slightly modified with respect to naming and comments because, I assume, it was stored slightly imperfectly in the model's weights.

I loved SpamBayes! It’s amazing how effective that was, all those years ago. I never learned Python, but I never had trouble using it to build small useful scripts. I never knew I had Tim to thank for both.