A diff makes these kind of errors much easier to catch.

Or maybe someone from XEROX has a better idea how to catch subtly altered numbers?

I verify all dates manually by memorizing their offset from the date of the signing of the Magna Carta

HN is no place for chicanery.

>Not code, but I once pasted an event announcement and asked for just spelling and grammar check. LLM suggested a new version with minor tweak which I copy pasted back. Just before sending I noticed that it had moved the event date by one day.

This is the kind of thing I immediately noticed about LLMs when I used them for the first time. Just anecdotally, I'd say it had this problem 30-40% of the time. As time has gone on, it has gotten so much better. But it still makes this kind of problem -- lets just say -- 5% of the time.

The thing is, it's almost more dangerous to rarely make the problem. Because now people aren't constantly looking for it.

You have no idea if it's not just randomly flipping terms or injecting garbage unless you actually validate it. The ideal of giving it an email to improve and then just scanning the result before firing it off is terrifying to me.