Yes, because no politician wants to be the one who says "I propose a bill to make adultery legal. It's not an issue for the courts."

There have been less that a dozen misdemeanor prosecutions in the last half century in those 21 states combined.

So yes, in those states, it's illegal. Just as in many cases, so was homosexuality, and anal sex.