I don’t think there’s a person alive who wouldn’t carefully and accurately count the number of legs on a dog if you ask them how many legs this dog has.

The context is that you wouldn’t ask a person that unless there was a chance the answer is not 4.

You deeply overestimate people.

The models are like a kindergartner. No, worse than that, a whole classroom of kindergartners.

The teacher holds up a picture and says, "and how many legs does the dog have?" and they all shout "FOUR!!" because they are so excited they know the answer. Not a single one will think to look carefully at the picture.

It's hilarious how off you are.

You have never seen the video of the gorilla in the background?

That's a specific example that when you draw a human's attention to something (eg: count the number of ball passes in this video), they hyper-fixate on that, to the exclusion of other things, so it seems like it makes the opposite point that I think you're trying to?

Exactly this. Humans are primed for novelty and being quizzed about things.

Ok? But we invented computers to be correct. It’s suddenly ok if they can look at an image and be wrong about it just because humans are too?

My point is that these llms are doing something that our brain also is doing. If you don't find that interesting, I can't help you.

Well, they’re getting the same result. I don’t particularly see why that’s useful.

All automation has ever been is an object doing something that a human can do, without needing the human.

The result is still wrong, though! It needs to be right to be useful!

The analogy should be of an artist that can draw dogs but when you ask them to draw a dog with three legs they completely fail and have no idea how to do it. That likelihood is really low. A trained artist will give you exactly what you ask for, meanwhile GenAI models can produce beautiful renders but fail miserably when asked for certain specific but simple details.

No, the example in the link is asking to count the number of legs in the pic.

Ok, sure, but I'm trying to point out the gap in expectation, i.e. it's an expert artist but it cannot fulfill certain specific but simple requests.