Even when using the web search tool GPT-4 becomes stupider.

Do they use a dumber model for tool/vision?

I’m guessing that it’s just a much harder problem. Images often contain more information but it is far less structured and refined than language.

The transformation process that occurs when people speak and write is incredibly rich and complex. Compared to images which are essentially just the outputs of cameras or screen captures — there isn’t an “intelligent” transformation process occurring.

I think images also have much higher information density than words, or at least they can. There is a reason a picture is worth 1000 words.

This is news to me. Any good examples of this outside of the above?

Vision language models are blind (192 comments) https://news.ycombinator.com/item?id=40926734

On the other hand if I take pictures of circuits, boards, electronic components, etc GPT4o is pretty reliably able to explain the me the pinouts, the board layouts, reference material in the data sheets, and provide pretty reasonable advice about how to use it (I.e., where to put resistors and why, what pins to use for the component on an esp32, etc). As a beginner in electronics this is fabulously helpful. Its ability to pass vision tests seems like a pretty dumb utility metric when most people judge utility by how useful things are.