anything not on your list
It's the thing most people even in this thread don't seem to realize has emerged in research in the past year.
Give a Markov chain a lot of text about fishing and it will tell you about fish. Give GPT a lot of text about fishing and it turns out that it will probably learn how to fish.
World model representations are occuring in GPT. And people really need to start realizing there's already published research demonstrating that, as it goes a long way to explaining why the multimodal parts work.
>we are very, very far and this depresses me. What is the way forward? :( Maybe I should just do a startup
and was a founding member of OpenAI just a few years later in 2015
I would bet good money that when we can test prompting with our own unique images, GPT4 will not give similar quality answers.
I do wonder how misleading their paper is.
What is funny is neither GPT-4 nor the host noticed that (or maybe the host noticed it but didn't want to bring it up due to it being "inappropriate" humor).
They literally sent it 1) an a screenshot of the Discord session they were in and 2) an audience submitted image
It described the Discord image in incredible detail, including what was in that, what channels they subscribed to, how many users were there. And for the audience image, it correctly described it as an astronaut on an alien planet, with a spaceship on a distant hill.
And that image looked like it was AI created!
These aren't images it's been "trained on".
There's easily a 10:1 ratio of "it doesn't understand it's just fancy autocomplete" to the alternative, in spite of published peer reviewed research from Harvard and MIT researchers months ago demonstrating even a simplistic GPT model builds world representations from which it draws its responses and not simply frequency guessing.
Watch the livestream!?! But why would they do that because they already know it's not very impressive and not worth their time outside commenting on it online.
I imagine this is coming from some sort of monkey brain existential threat rationalization ("I'm a smart monkey and no non-monkey can do what I do"). Or possibly just an overreaction to very early claims of "it's alive!!!" in an age when it was still just a glorified Markov chain. But whatever the reason, it's getting old very fast.
Curious, source?
EDIT: Oh, the Othello paper. Be careful extrapolating that too far. Notice they didn't ask it to play the same game on a board of arbitrary size (something easy for a model with world understanding to do).
I wasn't expecting GPT-4 to be able to correctly answer "What is funny about this image?" for an image of a mobile phone charger designed to resemble a VGA cable - but it can.
(Note that they have a disclaimer: "Image inputs are still a research preview and not publicly available.")