So it depends on how you measure progress? Scientifically and advancement of knowledge, or success in applying such knowledge to business problems and use cases. That the deep learning community insists on publishing most of these breakthroughs as research is the basis of the critique being made in the OP.

There's a tremendous amount of know-how required to successfully build an airplane, a jet-engine, a processor, or to successfully train a large neural network. And while theoretical physics no doubt can explain it all, it's not sufficient to build something that works. I think the fact that we're seeing so much progress in deep learning without any theoretical basis is another example of this.

[1] Antifragile, Chapter: History Written By the Losers

No amount of compute power will let a RNN/LSTM outperform a Transformer.

No amount of compute power will let an AlexNet or Inception style network train as well as a ResNet.

I don't understand what you mean by this? Any neural network is first build in theory and then tried out practically by coding and testing. I think practitioner and scientist play equal role.

Did it? Deep Learning provided us with some mindblowing performance art (FaceApp, DeepFake, etc.), but as far as I know any actual business value that claims to be derived from 'AI' is fake news.

If https://en.m.wikipedia.org/wiki/Google_Translate does not seem like having business value to you, then I don't understand what you mean by "business value".

There's no way Google Translate will ever be monetized. So yes, it's effectively an expensive art project for Google, not something that brings real business value.

Hotword detection ('ok google') is a practical application of AI, but that's really slim pickings considering how hyped 'AI' was. (Also, the costs of data collection for this feature are still way too high.)

Let me close my argument: neural network techniques developed in the last ten years are super useful here and now, they are used by billions of people every day and they do make their lives easier.

Not really. My point is simple: as of 2020, any proposition for a business to 'invest in AI' is a money-losing proposition. (Just like 'blockchain'.)

"We collect information to provide better services to all our users — from figuring out basic stuff like which language you speak, to more complex things like which ads you’ll find most useful, the people who matter most to you online, or which YouTube videos you might like..."

Obviously I don't have access to Google's accounting sheets, but there's no way in hell Google Translate pays for itself if you take all the costs of R&D into account.

Reason being that professionals who might pay big money for it want stuff other than just shiny 'AI' - they might want stuff like professional dictionaries and cited sources, etc.

And obviously people who translate random webpages for the lulz won't be paying much for this service.

If you all of a sudden start using Google Translate for Spanish and you think the phrases you are translating won't get used to serve you more targeted adverts for your upcoming vacation, you're being a bit naive.

Google Translate is being monetized. It's as easy a guess as yours to say that there's no way in hell Google Translate stays around if it didn't already pay for itself a long time ago. But let's stop guessing please.

I personally know companies spending tens of thousands per month on Google Translate.

2 have been around text and one on video.

In my little world attitudes have soured on 'AI'; 'AI' is posed to become the next 'blockchain'.

A few orders of magnitude is potentially very significant.

It's worth remembering that despite the hype and fake-clever systems playing chess and constructing plausible sentences, we still have nothing remotely approaching AGI at the moment, and no reason to think it is on the horizon.

But I agree that the AI tools we have have made significant progress in the line of powerful un-intelligence, and will have significant impact in automation.

That's an interesting POV, "if it looks like a duck... it doesn't matter that we don't understand why".

I agree with your point that (as per evolution) there is no reason why we can't just set the initial conditions and set it running and with luck/wise choices (tho the latter implies some understanding, which we don't have) an AGI will emerge.

However, I don't see any evidence of that despite the impressive power of GPT etc.

One reason why is that while I agree that "meaning is use", the use in the human case is in our interaction with the world and our inner thoughts.

The use in the GPT case is more of a Chinese Room style affair, isolated, unthinking and without motivation or interest.

During the typing of this comment I've considered a myriad of things, some related, some not - for instance, going to get a coffee in a minute, turning away to respond to a Whatsapp message. This motivated, conscious and subconscious context switching is just one example - let alone the myriad emotions, memories, feelings, inner life and experience that drive and inform it.

Intelligence as surface is only plausible in the philosophic zombie sense.

And we have nothing yet that comes near to mimicking even that general intelligence at surface level, just (massively impressive) powerful niche-explorers and domain entity generators like GPT.