Humans hallucinate as well. Benchmarks count.

But with less power consumption.

I'm willing to bet $100 that a human consumes at least 10x times energy than a latest LLama (picking the open source model so that it can be easily verified) to produce two pages of text. All of this "AI is destroying the environment and consuming too much power" is about total consumption, both training and inference. Inference itself is cheap and green.

Yeah, too bad it’s about quality not quantity.

Those goalposts just keep sliding

What percentage of humans produce high-quality text?

Less than half, and that's being generous.

We can control for quality too, of you want. A lot of real life uses for Chatgpt are really trivial. I regularly ask it for some basic recipes based on my groceries and likes, quality is basically 100% hits so far.

It doesn't matter if it costs time.

It matters if it is better than what you have.

If it breaks a cup but is 10x cheaper than a human, go figer

You’re a very poor user of LLMs if they’re not a net time saver for you.

So the No True Scotsman fallacy of LLM productivity?

Most people do see productivity gains from using LLMs correctly. Myself included. Just because some people don’t learn how to use them correctly doesn’t mean LLMs aren’t helpful. It’s like when internet search came out and a handful of laggards tried it once, failed to get the exact perfect result, and declared “internet search is useless”. Using a tool wrong is not evidence of the tool being useless, it’s evidence that you need to learn how to use the tool.

hallucinations are literally the finger in the dam. if these models could sense when an output is well-founded and simply say “i dont know” otherwise… say goodbye to your job

Googling a question and finding an incorrect answer every now and then doesn’t mean that googling is useless. It means that you need to learn how to use google. Trust but verify. Use it for scenarios where you aren’t looking for it to be the trusted fact checker. It excels at brainstorming, not at fact giving.

I agree with your last sentence strongly, but a lot of the benchmarks are based on factual accuracy

> say goodbye to your job

How many times do you think I've heard that over the past three decades? And you know what? They've been right every time, except for this one little fact:

The machine cannot make you give a shit about the problem space.

It's a real issue! But only for people who built the habit of typing in address bar, clicking the first stack overflow link and copy paste the first answer. Maybe break that habit first?

It depends on what you're using it for, but you may well be holding it wrong.

Probably closer to "You're Holding It Wrong"