You are probably right that in the six months since the release of ChatGPT, we do not have academic surveys of the productivity gains of LLMs in real-world scenarios.

But that is hardly the measure of a new technology.

That’s how folks initially thought of cars too - the accident rates, lack of speed etc. The point is, as a technology, this is going to change the future. It’s a valid point that the current state might be less than what you expect. The only way to improve it is to double down on it.

Comparisons to crypto are a bit shallow - except that they both are ‘new’ technologies and need GPUs, there’s not much.

One was just posted: https://news.ycombinator.com/item?id=36152510

I already have a massive improvement in productivity from GitHub Copilot that I don’t find it hard to imagine that fine tuned LLMs will be able to automate the bulk of mundane work in software (maybe even all of it). Programming, and other computer based activities that involve writing are the exact types of things that language models are good at.

Luckily, I was just reading such a study a few days ago: https://www.itnews.com.au/news/westpac-sees-46-percent-produ...