They are losing tons of money on it. Simple as.

But is it a race to zero? Maybe AI will make very expensive things (eg software engineering) very cheap, but there will be enough competition that the margins are tiny and don't actually justify putting crazy dollar amounts into research

That's exactly what's happening. Software -> 0 marginal cost (obviously still need engineers to actually make the architecture). LLMs -> Price of Energy / Amortized Cost of GPUs. "AI Labs" -> Owned by Creditors

From an engineer, who’s devastated by this, i have to admit/raise my concern: why do we still need engineers to make architecture? Why is everyone saying that’s a must? For me the toughest problem was coding a huge repo/pkgs, architecture should be much of an easier task to do. I’ve seen it do sys design, scaled architecture better than 10X engineers. With good planning, context and prospective info -> it can produce extremely good architecture even for complex systems.

Im not sure I would say architecture specifically. Just anything on the input and output.

It cant lmao (skill issue)

The problem is the "if". There is no guarantee that there's a "better, more efficient approach" to be found.

So far, the attempts to replace the transformer LLM with something else have yielded either "the same transformer LLM but with this one tweak that makes it 10% better" or "a radically different architecture that's a bitch to work with, and performs, best case, about the same as the transformer LLM".

By now, I'm firmly in the camp of "the architecture is good enough". Rifle is fine. Transformer is fine. Get better at the transformer before you look at anything else.

And getting the transformer better may make finding the next thing easier

I think it’s more likely that some future LLM model finds a more efficient architecture than a human does, at this point. So yeah I’d go all in on LLMs too.

Big corporations, by nature, are not rational in the decisions they make.

That's why they'll always be room for startups.

Me neither. When revenues are growing every year, a fixed chunk of researchers' salaries (maybe growing a few % each year) doesn't really hurt, but R&D offers a nice upside.