> whatever will come after transformers

I feel like this is a point that is not being talked about enough. Yes, OpenAI gave us GPT and DALL-E. But had sama and gdb remained there, would we have gotten anything new that is as groundbreaking as the original GPT and DALL-E, or would we have continued getting GPT-12 and DALL-E-19? Sure, iPhone 15 sells, but some may say Apple has stagnated since iPhone was released.

From the various sources it appears that they’re being fired because they were trying to push the envelope TOO HARD, not the other way around.

And, outside of the Cynicism-Is-Intelligence hackernews crowd, basically everyone has been fawning over the breakneck speed of progress coming out of OpenAI, even at the recent OpenAI devdays.

Sorry to nitpick, but --

OpenAI was releasing innovations in the GenAI space at a breakneck pace. Remember, GPT-1 didn't change the world, it was GPT-3.5/4 from earlier this year. OpenAI was at peak innovation when sama and gdb left.

And folks used to say Apple was stagnant, but after Apple Silicon completely upended the personal computer world (along with some other things) the dissidents have been mostly silent.

Apple Silicon made x86 silicon look bad, but what has it really upended? Macs are taking over more of the personal computer market, but hard to say what the factor is there. I think it's mostly network effects, partially due to their shameless proprietary approach. PCs, Apple or other, are kinda generally good, no matter what the price or combo, and disappearing at the same time, a lot can be done with just a browser on any foundation. Apple seems to be years behind or nonexistent where things are really changing, AI and cloud.

> Apple Silicon completely upended the personal computer world

How did it upend the personal computer world? Apple's chip developments are an amazing technological achievement, but they don't have anything innovative to put them in. Apple slaps them in grossly thermally limited form factors, where the chips can't operate anywhere close to their capability. It's kind of a silly exercise, in my opinion. At the end of the day, Apple has made the same computers, phones, and tablets for the past 10 years. I'm not sure where the innovation is.

> Apple slaps them in grossly thermally limited form factors, where the chips can't operate anywhere close to their capability.

Checks temp of iMac, while encoding video and maxing all cores.... nope.

Saying the chips haven't enabled any sort of innovative products it's like driving down the interstate wearing goggles, err, blinders. Let's check back in four months.

What are they then? Have I missed something?

You have! The Apple Vision Pro is slated to be released by the March/April 2024. The device has only been made possible by the steady march of Apple Silicon.

That and a big battery pack.

Oh, right. VR, that everlasting elixir of promise of hype and transformation. And I guess we're going to ignore that several VR sets already exist.

But now, are we even going to get GPT-4 or GPT-5 with the same level of polish that sama would have put into it?

I'd argue right now that we're at the "iPhone 3G" point on the technology curve, with significant improvements to come over the next few years as the tech gets polished.

I don't know gdb very well, but I did get to chat with him a bit about what he was working on around the time of this article, which was mostly infrastructure grunt work, removing obstacles and procedural rough edges—basically anything to make the researchers and engineers as happy and productive as possible. It is so, so easy to undervalue that kind of work done by a totally brilliant and capable technologist. For an early stage startup it's gold. For a later stage startup it's gold.

Why do you think people undervalue it? Very curious.

It's the impression I got from "who have not done any transformer research", as well as the fact that sama wrote this article.

You can’t demo it and sell it.

It's gold, but it's not singular. There are many people who have been doing that work for decades and are able to step into the role. The same can't be said for the R&D work he's supporting, as comparatively few have deep insight or experience for working with the innovative tech yet.

So while Greg's work would have been extremely valuable, it's value is on a lesser order of magnitude than many of the other researchers and engineers who OpenAI had collected into its ranks. More essential innovative value will be lost to the bleed of loyalists and startup bettors who will peel off from those ranks.

I'm suggesting that there are not many people who have been doing that work, at least not at the same level or to the same effect. He did it with Stripe and OpenAI, back to back.

I think that there is some kind of elitism around AI researchers. Yes they are very valuable, but someone helping everyone else be more productive is absolutely critical.

Having a car might be critical and acquiring a car might be expensive, but there are a lot of them and they are ultimately replaceable. If your is lost and you still have cash, you can generally go find a new one the same day and get borrow a ride from someone if you really need to.

That's not necessarily true for (say) the rare high-end graphics board you use for running local inferences. It's also expensive -- even less expensive than the car -- but replacing it can be a bigger deal and cause a complete interruption.

There are countless experienced late-career generalists who can keep projects moving by contributing to critical, smart support. I'm one of them. We're extremely valuable indeed.

But there really are far fewer people who were ahead of the curve and years-deep into the AI research central to OpenAI's entire existence. Those people are beyond critical, they're essential.

That doesn't make them better people, or smarter people, or in any other way elite. It just means that in the context of OpenAI those people are much harder to come by and can be much more disruptive when lost.

Apologies, I'm afraid I wasn't very awake reading your comment. You're right, my issue is more about the dismissive attitude that I often see: for some reason I mistook you as someone having that attitude as well.

That's the job of any good professor for their PhDs and postdocs.

I don't even know how people like this get valued so much. Why do people treat Silicon Valley "entrepreneurs" and investors as if they're made out of some sort of intellectual adamantium? Aren't they, generally speaking, just people looking to make a name and buck for themselves, primarily driven by ego rather than intellectual or philosophical pursuits?

I think it's fair to say that OpenAI retains its AI talent so the loss of the founding team may not result in lost research progress.

The founding team does however bring tactical experience that is (maybe) unmatched. They also have experience solving problems that turn cutting edge AI models into usable products. It's easy to devalue non research contributions, but they have legitimate value and are instrumental in OpenAIs success.