It's not that Intel doesn't have people that recognize the issues, but rather the people who do have that foresight are drowned out by people who don't realize the game has changed. Intel, to be fair, does have the best autovectorizer--but designing vector code from scratch in a purpose-built vector language is still going to produce better results, as shown when ispc beat the vectorizer.
But Nvidia can also get drunk on its kool-aid, just as Intel has been. Nvidia's marketing would have you believe that switching to GPUs magically makes you gain performance, but if your code isn't really amenable to a vector programming style, then GPUs aren't going to speed your code up, and the shift from CPU-based supercomputers to GPU-based supercomputers are going to leave you happy. There's still room for third-way architectures that is anyone's game.
Bad management is when your company evaporates because you make one bad call.
Management gets points for surviving and then fighting back despite being wrong. And when you look at Intel's history, there are few companies on the planet who have managed to do that multiple times. They have a good mix of people who know what they are doing technically AND people who do whatever it takes to keep the company from sinking when those bad technical calls happen.
If Nvidia survives whatever their next bad call maybe expect them to start looking more and more like Intel.
That's a pretty low bar!
AMD could never made the same gamble as Intel did for Itanium. There's a long technical argument as to whether the world is better off in a technical CPU design sense because of that, but I disagree that it's necessarily good management of Intel that's allowed it to recover from disaster.
The best management can play the hand they're dealt perfectly and still lose. However bad management can play the best hand poorly and still win.
CUDA's sweet spot lies between embarrassingly parallel (for which ASICs and FPGAs rule the world because these are generally pure compute with low memory bandwidth overhead) and serial (for which CPUs are still best), a place I call "annoyingly parallel." There are a lot of workloads in this space in my experience.
But if you don't satisfy both of the aforementioned requirements and/or you insist on doing this all from someone else's code interfaced through a weak-typed garbage collected global interpreter locked language, your mileage will vary greatly cough deep learning frameworks cough.
Finally, it doesn't matter who's doing it, marchitecturing(tm) drives me nuts too.
Controlling the language certainly helps Nvidia's economic moat.
Arguably one of the things that Nvidia really got right was learning from those past failures at other companies and making it easier for developers to utilize the platform starting from a standpoint that they were familiar with and helpfully nudging them towards what would run fast in parallel.
I think the big disconnect is thinking they had to make people rewrite code. CUDA often targets entirely new codebases, and in some cases new types applications.
The "rewriting of code" is mostly for things like AV processing and codecs where there was such a sellable benefit in performance it would have been insane for them not to invest the effort.
Intel was doubly hindered here, because they wanted everything to use x86. Intel had trade secret and patent protections from competitors, and a critical mass of marketshare.
Parallel programming was something that had to fit into that "x86 for everything" mindset rather than being a separate/competing technology to x86. The company that pushed winmodems and software sound cards wasn't going to be able to lead the disruption there.
Only now, more than a decade later they realize their mistake and try to correct the juggernaut's course. Such glacial mistakes in this industry can cast death blows to even the largest entities.
They drowned out Pat Gelsinger late 00s, then Justin Rattner and many others Retired in early 10s, the rest is history.
Reminds me of "First they ignore you, then they laugh at you, then they fight you, then you win".
So many companies have this reaction (e.g. RIM with BlackBerry). Wondering if this is some kind of "corporate instinct".
There is a sort of survivors bias in focusing in the 2% and assuming that's the norm. Corporations act this way because it generally works.
Like the OP said though, it does lead to arrogance over time, and that's when a fall happens.
Corporations can't really be as successful at innovating as a startup. A startup is free to build or reshape itself into anything, focus on a single thing, and pivot on a dime. In a corporation the same structures that hold it up and moving are the ones that resist it changing direction or promoting something new. Easy to lose focus and get lost in the red tape.
And that's before you consider the risk a CEO sees in potentially cannibalizing their own (currently successful) business or just throwing money down the drain at 98 losing ideas. Like you said, 2% of ideas may be successful so a corporation would rather let the startup play it out and then buy it if it has potential. Easier to justify to investors.
So corporations innovate when they have nothing to lose and any risk is worth taking. See MS trying to somewhat successfully reinventing themselves after seeing mobile and FOSS booming. Private companies also have an easier time innovating because they have no investor pressure. They may be behemoths but at least they can avoid suffering from the "too many cooks in the kitchen" syndrome.
I know this to be true, however I cannot understand for the life of me why this is the case.
If I was the CEO or CTO and had, say, 5k people under me, you had better believe there would be dozens of little 3-4 person teams doing hard research on threats and coming up with recommendations to get in front of them.
I mean this is basic 1st year MBA SWOT Analysis stuff.
And having lots of teams "innovating" is also not that great. You'll just end up with a stack of 100 great ideas on your desk but only 2 that might make money. Your job is to guess which 2. Any decision you take will be heavily scrutinized by everyone in the company and shareholders. You may just go the safe way, that worked over the past few years and put a bonus on the table.
A 10-20-100 person startup with everybody in the same office and a very flat structure will be a lot more agile. The people are all there for that one single purpose, and the dynamic is quite different. Once the goal is reached many just move on. This provides a very different motivation vs. the typical corporate employee.
[0] https://qz.com/846530/something-weird-happens-to-companies-w...
holding the place of an incumbent has advantages and disadvantages. Sometimes you can't leverage the advantages, and that's when a company can get buried by the upstart the worst.
That somebody might be Nvidia. I believe that Nvidia is still battling and has not yet paid the 1.06 billion Euro fine to AMD imposed on it by the UK courts in 2009. Hearsay claims that it was the similar US fine that basically paid for Zen R&D...
It seems gratuitously confusing for readers, and doesn’t seem to have any benefit I can see.
Stock symbols
But instead of stepping up to the plate and igniting a Red Queen's Race that would have benefited everyone, INTC first tried to discredit the technology repeatedly, then they built an absolutely dreadful series of decelerators that demonstrated how badly they didn't understand manycore. Eventually, they gave up, and now they're playing catch-up by buying companies that get within striking distance of NVDA rather than building really cool technology from within.
Now if someone threw a large pile of money at AMD again, things could get really interesting IMO. But the piles of stupid money seem biased towards throwing ~$5M per layer at the pets.com of AI companies these days.