I've found it impossible to tell if it has any quality issues, because it only gives me non-answers, wrapped in disclaimers.

Especially true for Copilot. I almost never get anything useful out of it. It's too restrictive.

GitHub copilot ?

Bing copilot

Gemini's aggression around health & safety, politics, malware etc seem to be much stronger than those of OpenAI/Meta/Mistral

Google, in general, seems to have stepped up it's hostility to it's users of late.

Yes all LLMs have problems. But some are better than others, and there are benchmarks used for comparing the models. Also, in terms of PR, I don't remember OpenAI having messed up like this -- having one really bad incident is worse than not giving perfect answers.

OpenAI doesn't have image generation in the free version of its product.

I cancelled my sub to Gemini.

It is aggressive against the user.

The “insert an ethnic minority into every group photo” was the straw that broke the camels back as it was clearly biased.( for anyone who cares I am an ethnic minority).

But I do remember, and I mean this, my first ever query asking it to draw a picture by a Japanese artist resulted in a “call this number for mental health problems”

I was like wtf. The prompt was benign “water color sunbathing beach sunset” by some random normal Japanese artist. Sure potentially nsfw but the reply was completely hamfisted, insulting and really crazy.

I probably wouldn’t have cared but it was literally my first prompt.

Before anyone says “well it could have been degenerate” you have heard of the Michelangelo David right? The greatest ever sculpture? I haven’t tried but I’m pretty sure there isn’t a hope in hell the llm would reproduce anything like it.

Really went off it since day one.

I think they are referring to alleged political biases that are instilled in the model through RLHF.

In the linked article Sergey literally says

  > "it leans left in many cases"

Is this the actual smoking gun, “many”?

Alleged? Did you not see the pictures of black female founding fathers?

Lot of extremely normal people being radicalized by shitty software in here.

All LLMs will always have multiple quality issues. A tool that complex can never be perfect.

FWIW GPT-4 via API has treated me well, more consistently than ChatGPT.

Would be nice if they solved the issues with Assistants but that’s another matter…

Does the chatgpt give better results than the front end ?

Vanilla, non-binged ChatGPT is flawed, but not nearly as annoying as everything else like Claude, Pi, and Gemini

Like you say, I think what's being discovered is the lack of utility in these things. These techbro companies with Gemini, ChatGPT, etc. are just rediscovering that human to human communication is hard and cannot be automated and that human illustrators, artists, animators, writers, etc. have real, actualizable value.

And it's rediscovering that we suck at prompting even humans, much less systems that have no idea what they're doing and just statistically stitch and glue data together.

These LLMs are just exposing that humans are fairly bad at communication and they are encoding that into automated systems.