Intelligence isn't the same as "can exactly replicate text". I'm hopefully smarter than a calculator but it's more reliable at maths than me.

Also there's a huge gulf between "some people claim it can do X" and "it's useful". Altman promising something new doesn't decrease the usefulness of a model.

What you are describing is "dead reasoning zones".[0]

    "This isn't how humans work. Einstein never saw ARC grids, but he'd solve them instantly. Not because of prior knowledge, but because humans have consistent reasoning that transfers across domains. A logical economist becomes a logical programmer when they learn to code. They don't suddenly forget how to be consistent or deduce.

    But LLMs have "dead reasoning zones" — areas in their weights where logic doesn't work. Humans have dead knowledge zones (things we don't know), but not dead reasoning zones. Asking questions outside the training distribution is almost like an adversarial attack on the model."

https://jeremyberman.substack.com/p/how-i-got-the-highest-sc...

saddest goalpost ever

Would you hire a PhD to copy URLs by hand? Would them having PhD make it less likely they’d make a mistake than an high school student doing the same?

I would not hire anyone for a role that requires computer use who does not know how to use copy/paste

Grad students and even post docs often do a lot of this manual labour for data entry and formatting. Been there, done that.

Manual data entry has lots of errors. All good workflows around this base themselves on this fact.

A high school student would use copy/paste and the urls would be perfect duplicates..

> A high school student would use copy/paste and the urls would be perfect duplicates..

Did the LLM have this?

LLMs aren’t high school students, they’re blobs of numbers which happen to speak English if you poke them right. Use the tool when it’s good at what it does.

And the people who are causing this confusion are the CEOs of the companies saying that the newest model is a PHD in your pocket.

They are lying, because their salary depends on them lying about it. Why does it even matter what they're saying? Why don't we listen to scientists, researchers, practicioners and the real users of the technology and stop repeating what the CEOs are saying?

The things they're saying are technically correct, the best kind of correct. The models beat human PhDs on certain benchmarks of knowledge and reasoning. They may write 70% of the easiest code in some specific scenario. It doesn't matter. They're useful tools that can make you slightly more productive. That's it.

When you see on tv that 9 out of 10 dentists recommend a toothpaste what do you do? Do you claim that brushing your teeth is a useless hype that's being pushed by big-tooth because they're exaggerating or misrepresenting what that means?

> When you see on tv that 9 out of 10 dentists recommend a toothpaste what do you do? Do you claim that brushing your teeth is a useless hype that's being pushed by big-tooth because they're exaggerating or misrepresenting what that means?

Only after schizophrenic dentists go around telling people that brushing their teeth is going to lead to a post-scarcity Star Trek world.

It's a new technology which lends itself well to outrageous claims and marketing, but the analogy stands. The CEOs don't get to define the narrative or stand as strawman targets for anti-AI folks to dunk on, sorry. Elon has been repeating "self driving next year" for a decade+ at this point, that doesn't make what Waymo did unimpressive. This level of cynicism is unwarranted is what I'm saying.

You shouldn’t - that’s the point of the comparison. If some insane dentists started saying this you should not stop brushing your teeth!

So questioning the utility of LLMs for knowledge work is now akin to a conspiracy theory?

Not what I said at all. Question it all what you want. But disproving outrageous CEO claims doesn't get you there. Whether LLMs are AGI/ASI that will replace everyone is seperate from whether they are useful today as tools. Attacking the first claim doesn't mean much for the second claim, which is the more interesting one.

I'm questioning the basic utility. They are text generation machines. This makes them unsuitable for any work that requires accuracy or understanding, which is the vast majority of knowledge work.