From the report 80% is "twice the rate of failure for information technology projects that do not involve AI" (https://www.rand.org/pubs/research_reports/RRA2680-1.html) so seems that a 20% hit-rate isn't actually that great. Possibly it's a quirk of how they're normalizing the success/failure count?

Most of those non-AI projects are likely using well-established practices and technologies. From that perspective a 20% success rate seems pretty good to me.

Sure that makes sense, overall "success of a technology product" seems like a very fuzzy thing to try and measure so I imagine one could spin the numbers basically however you want

> research and development.

I think that is the question, how much legitimate R&D is really going on here vs trying to shove an LLM into some random hardware and ship it?

Or shove an LLM in some app trying to solve a problem that it isn't capable of.

No doubt that creating these models are hard, having the data is hard. But how much of the AI startups is actually that vs just shoving the OpenAI API in something.

I think we’d need to dig into the 80% that failed and what kind of “AI project” they were. Is this really R&D? Or were you trying to insert AI into something that didn’t need it, and failed because no one is using your expensive and annoying “chat with us” popup that your VP insisted would keep your company competitive?

It's not a hit rate I think, it's a not yet failed rate. A lot of these were started just recently. And a lot is deep in red but is supported by the VC IV line. And by supported I mean totally. If a Word competitor needs to cut costs it theoretically can scale down and survive. If a neural network startup needs to cut costs it has to shut down operations, due to incredible price to train models and run inference.