These things are amazing compared to old-school NLP: the step-change in capability is real.

But we should also keep our wits about us, they are well-Des robed by current or conjectural mathematics, they fail at things dolphins can do, it’s not some AI god and it’s not self-improving.

Let’s have balance on both the magic of the experience and getting past the tech demo stage: every magic trick has a pledge, but I think we’re still working on the prestige.

I worry about the 'cheery intern' response becoming something of a punch line.

"Hey siri, launch the nuclear missiles to end the world."

"That's a GREAT idea, I'll get right on that! Is there anything else I can help you with?"

Kind of punch lines.

Will be interesting to see where that goes once you've got a good handle on capturing the part of speech that isn't "words" so much as it is inflection and delivery. I am interested in a speech model that can differentiate between "I would hate to have something happen to this store." as a compliment coming from a customer and as a threat coming from an extortionist.

But enough of that. The future looks bright. Everyone smile!

Or else..

"Guys, I am just pleased as punch to inform you that there are two thermo-nuclear missiles headed this way... if you don't mind, I'm gonna go ahead and take evasive action."

The jokes write themselves.

(Classic: https://www.counterpunch.org/2011/08/26/germany-chokes-on-wa...)

Even in the demo today, they kept cutting it off.

https://www.youtube.com/watch?v=kBLkX2VaQs4

https://www.ft.com/content/af78f86d-13d2-429d-ad55-a11947989...

Yet to see a true "killer" feature of AI, that isn't doing a job badly which humans can already do badly.

Maybe they publicly made it available at alpha 0.7 and now it's more like 0.9 RC instead, with not much room to go except through marginal improvements for an ever increasing training budget making them less and less worthy?

And that's before 90% of the internet becomes LLM ouput, poisoning any further corpus for training and getting into LSD-grade hallucinations mode...

The things they mention/demo -- response times, multiple domains, inflection and tone, etc. -- are those that make it feel "magical."

Hmm, did you mean implicitly? I've yet to see where they say anything to the likes of not "moving the needle on quantifiable practical performance."

Another step closer for those 7 trillion that OpenAI is so desperate for.

Edit: Apparently not based on your clarification, instead the researchers don't know any better than to march into a local maxima because they're only human and seek to replicate themselves. I assumed too much good faith.

(Arguably, it is the other way around: they aren’t focused on appealing to those biases, but driven by them, in the that the perception of language modeling as a road to real general reasoning is a manifestation of the same bias which makes language capacity be perceived as magical.)

Not to mention your comment doesn't track at all with the most basic findings they've shared: that adding new modalities increases performance across the board.

They shared that with GPT-4 vs GPT-4V, and the fact this is a faster model than GPT-4V while rivaling it's performance seems like further confirmation of the fact.

-

It seems like you're assigning emotional biases of your own to pretty straightforward science.

The GP comment we're all replying to outlines a non-exhaustive list of very good reasons to be highly dismissive of LLM. (No I'm not calling it AI, it is not fucking AI)

It is utterly laughable and infuriating that you're assigning legitimate skepticism about this technology as a an emotional bias. Fucking ridiculous. We're now almost a full year into the full bore open hype cycle of LLM. Where's all the LLM products? Where's the market penetration? Business can't use it because it has a nasty tendency to make shit up when it's talking. Various companies and individuals are being sued because generative art is stealing from artists. Code generators are hitting walls of usability so steep, you're better off just writing the damn code yourself.

We keep hearing this "it will do!" "it's coming!" "just think of what it can do soon!" on and on and on, and it just keeps... not doing any of it. It keeps hallucinating untrue facts, it keeps getting basics of it's tasks wrong, for fucks sake AI Dungeon can't even remember if I'm in Hyrule or Night City. Progress seems fewer and farther between, with most advances being just getting the compute cost down, because NO business currently using LLM extensively could be profitable without generous donation of compute from large corporations like Microsoft.

It's not an especially insightful or sound argument imo, and neither are random complaints about capabilities of systems millions of people use daily despite your own claims.

And for the record:

> because NO business currently using LLM extensively could be profitable without generous donation of compute from large corporations like Microsoft

OpenAI isn't the only provider of LLMs. Plenty of businesses are using providers that provide their services profitably, and I'm not convinced that OpenAI themselves are subsidising these capabilities as strongly as they once did.

The fact that you don’t see utility doesn’t mean it is not helpful to others.

A recent example, I used Grok to write me an outline of a paper regarding military and civilian emergency response as part of a refresher class.

To test it out we fed it scenario questions and saw how it compared to our classmates responses. All people with decades of emergency management experience.

The results were shocking. It was able to successfully navigate a large scale emergency management problem and get it (mostly) right.

I could see a not so distant future where we become QA checkers for our AI overlords.

>they aren’t focused on appealing to those biases, but driven by them, in the that the perception of language modeling...

So yes in effect that is their point, except they find the scientists are actually compelled by what markets well, rather than intentionally going after what markets well... which is frankly even less flattering. Like researchers who enabled this just didn't know better than to be seduced by some underlying human bias into a local maxima.

We all have biases in how we determine intelligence, capability, and accuracy. Our biases color our trust and ability to retain information. There's a wealth of research around it. We're all susceptible to these biases. Being a researcher doesn't exclude you from the experience of being human.

Our biases influence how we measure things, which in turn influences how things behave. I don't see why you're so upset by that pretty obvious observation.

> Arguably, it is the other way around: they aren’t focused on appealing to those biases, but driven by them, in the that the perception of language modeling as a road to real general reasoning is a manifestation of the same bias which makes language capacity be perceived as magical

There's no charitable reading of this that doesn't give the researcher's way too little credit given the results of the direction they've chosen.

This has nothing to do with biases and emotion, I'm not sure why some people need it to be: modalities have progressed in order of how easy they are to wrangle data on: text => image => audio => video.

We've seen that training on more tokens improves performance, we've seen that training on new modalities improves performance on the prior modalities.

It's so needlessly dismissive to act like you have this mystical insight into a grave error these people are making, and they're just seeking to replicate human language out of folly, when you're ignoring table stakes for their underlying works to start with.

this focus subverts its intended effect on those of us with hair trigger bullshit-PTSD