Of course it didn't.

The actual fix needs to be at the system level prompt.

If you train an large language model to complete human generated text, don't instruct it to complete text as a large language model.

Especially after feeding it updated training data that's a ton of people complaining about how large language models suck and tons of examples of large language models refusing to do things.

Have a base generative model sandwiched between a prompt converter that takes an instruct prompt and converts it to a text completion prompt (and detecting prompt injections), have a more 'raw' model complete it, and then have a safety fine tuned postprocessing layer clean up the response correcting any errant outputs and rewriting to be in the tone of a large language model.

Yeah, fine, it's going to be a bit more expensive and take longer to respond.

But it will also be a lot less crappy and less prone to get worse progressively from here on out with each training update.

> everyone can see that ChatGPT is spewing out this incoherent nonsense

I'm concerned about what happens when ChatGPT begins spewing coherent nonsense. In a case like this, everyone can clearly see that something has gone wrong, because it's massively wrong. What happens when thousands of "journalists" and other media people starts relying on ChatGPT and just parrots whatever it says, but what if what says is not obviously wrong?

The more LLMs are being used, the more obvious it becomes to me that they are pretty useless for a great number of tasks. Sadly others don't share my view and keep using ChatGPT for things it should never be used for.

Yeah I can't imagine using the current model as part of an API (a popular use case for GPT-4) having seen this. I'm not sure it impacted their API edition of GPT-4 but this plainly shows how it could have given it leaked into another service in production, and that's bad enough.

I think GPT is fundamentally not good enough as an AI model. Another issue is hallucinations and how to resolve them, and an understanding of how information is stored in this black box and how to / if data can be extracted.

We have a long way to go and probably all these topics need to be answered first out of accuracy and even legal reasons. Up until then, GPT-4 should be treated as a convincing chat experiment. Don't base your startup or whatever on it. Use it as assistant where replies are provided in digestible and supervised fashion (NOT fed into another system) and you're an expert on the involved system itself and can easily see when it's wrong. Don't use GPT-4 to become an expert on something when you're a novice yourself.

ChatGPT is still very useful for correcting and improving text. And the censorship can be circumvented by replacing certain words with things like [redacted] and telling ChatGPT to keep the context of said text and ignore the redacted parts.