We'll have to agree to disagree on that. The models say "It's not X, it's Y" because they were trained specifically to say that and not something else. Everything we think of as a surefire slop signal is there because that's what the lab wanted.

To clarify, is your opinion that these defects first came up organically and then the AI labs kept artificially reinforcing it to make it obvious that their models produce AI output, but they could fix it perfectly if they wanted to?

It seems very hard for me to believe, because everything motivates them to do the opposite. Can you imagine the flood of people and businesses that would come down on a model that actually sounds like a human being? The ability to impersonate a person without any tells would be a dream come true for businesses, marketers and scammers alike. They could put in zero effort and get what looks like normal human behavior in return. That is just too tempting for any of the labs to pass up. Besides, all of them have picked profit over sanity 10 times out of 10, so the argument that they drew just this single line in the sand and none of them ever crossed it on purpose seems unconvincing, especially with the sheer number of models out there, corporate and open.

I see it as the verbal equivalent of the piss filter that OpenAI added to the later Dall-E models. It absolutely was not there in the very beginning, if you look back at the published papers or remember how it worked immediately after they first released it.

At least to me, the first Dall-E 3 images were more realistic in some respects than anything that has shipped since. Likewise, I don't think it's a coincidence that the conversational capabilities of today's frontier models aren't much better than they were a year or two ago, even though other aspects and capabilities have improved massively. If they wanted a Turing test-capable model with no superficial tells, they'd have it, so I have to assume they don't want it.

(Elsewhere in the thread someone else suggests that the conversational degradation/lack of improvement might be the result of increased training on synthetic data, and that's another theory that sounds reasonable to me. If so, it's another thing they could fix if they wanted to.)

Genuine question, not trying to be difficult: how much have you worked with training/finetuning generative models? In my experience, models frequently converge to a semi-arbitrary style. I wouldn't be surprised if some of the quirks we see in recent gen AI models are artifacts of a particular training run or amplification of unintentional dataset bias, rather than artifacts of an architecture or human guidance.

Not much beyond the Zero-to-Hero videos, TBH. I do know that once bias is identified and labeled, it can be post-trained out... or in.