He calls out ads/marketing/copywriting as one area where ChatGPT is effective. His point is that the scope of things beyond that use case is probably a lot more limited than people are assuming.
Web3 was a circle jerk within the crypto community. It only convinced fools.
Recent GPT/stable diffusion products universally blow people away with their output.
Engineers and everyone else regularly dismiss new tech for its shortcomings and completely miss the forest for the trees.
It's fine to disagree with him, but most people doing so will be from a much less informed position.
https://scholar.google.com/citations?hl=en&user=VfYhf2wAAAAJ...
His work on Keras is admirable, but that is about it.
Then why did Chollet dismiss it?
"If this had been a blog post and not a random spur-of-the-moment train of thoughts, I wouldn't have made the AI/web3 comparison. It was counterproductive, as it is what most folks are now focusing on."
"The two are, in fact, very much not the same."
I see a lot more naysayers than you. While I am impressed, plenty are unconvinced.
(I really need to make time to test it against random standard written qualifications; if anyone has some example exams and marking schemes, in any language, please let me know. I have Polish maths, plus English and Irish school leaving certificates).
I've yet to be blown away by either system. I guess I am just incredibly cynical but the output of ChatGPT and Stable Diffusion aren't all that impressive to me. A lot of Stable Diffusion art just looks like browsing random DeviantArt or ArtStation pages. ChatGPT reminds me of IRC Markov bots, it produces the same non-sensical crap that they would. Only when a prompt hews incredibly close to some training content does it seem to product cogent output.
Both will get better and might become more impressive to me but right now that's not the case. My main issue is neither system actually understands what it is being asked. They're relying on the viewer's tendency for apophenia and pareidolia do make them think the system understood what it was asked.
I think there's justification for heavy skepticism for the future of these systems because we're approaching the point where their incorrect/inaccurate output is going to end up feeding the next generation systems' training data.
He also said pytorch was just a hype. It looks like he just doesn't like things he didn't do himself.
"I wouldn't have made the AI/web3 comparison. ... The two are, in fact, very much not the same."
So if you learned it blindly you'd be wrong too.
I actually started trying to learn rust using chat gpt. I can just ask it to explain bits I don't understand ... and it does.
There's a lot of content generation in the legal and medical sphere that is probably also pretty much something where chat gpt could be helpful. Dangerous of course when it gets it wrong. But still, I could see this being a useful tool for researchers in all sorts of fields to quickly dig through a lot of information. Basically, chat gpt is trained non more stuff than a single human will be able to read.
I think this whole space is maybe bottlenecked on imagination. We have a lot of AI experts with not a whole lot of other expertise not seeing the forest for the trees. OpenAI employs a few geniuses but their product is basically a chat box and an API.
As I was remarking to a friend the other day: so they have chat gpt, pretty decent speech to text, and amazing text to speech .... so why can't I talk to chat gpt and listen to the answer? Such an obvious thing to do. Surely somebody thought of that. I mean inside OpenAI. I know there's a multitude of openai powered proofs of concept by third parties. But they don't seem to have the ambition to support a finished product so far.
I read the thread, it actually starts with him saying:
"If this had been a blog post and not a random spur-of-the-moment train of thoughts, I wouldn't have made the AI/web3 comparison. It was counterproductive, as it is what most folks are now focusing on. The two are, in fact, very much not the same."
So you can have the "appeal to authority" but the authority now says that was bad and should not have done it. :-)
He continues:
"I meant to compare the surrounding hype generation dynamics (as discussed towards the end of the thread). Expectations unmoored from reality becoming a universally accepted, self-evident canon once the same narratives have been repeated enough times in the echo chamber."
To me, it just sounds like more dystopian attention spam. If absolutely all marketing copy vanished overnight never to return, in my opinion the world would be a better place.
It would be the only possible defense against this dystopian crap.
Spam?
Like crypto, there are other potential use cases on the horizon for LLMs. The problem is the tech is still far too clunky, unreliable and impractical for the foreseeable future. LLMs are subject to a devastating hallucination problem, where they confidently report clearly erroneous data. And there is no solution in sight. Marketing copy is one of the sole major use cases, where random inaccuracies is mostly not catastrophic. But you're never going to trust LLMs to write code or legal contracts or technical documentation or even your own emails until the hallucination issue is drastically improved.
That leaves us with a few other niches that are, frankly not very monetizable. ChatGPT is very good at writing student essays, but students don't have very much money to spend. Makes a fun chatbot, but people are unlikely to pay much of anything to have a chat companion.
Like crypto, AI is a case of really intellectually interesting technology that nerdsniped a lot of smart people. Like crypto, AI has a few small (in terms of market size) use cases where there's clear PMF. Like crypto, there are potentially a lot of other much larger use cases that the nerdsniped smart people are imagining. Like crypto, these large use cases are going to require a lot of fundamental improvements to the tech itself before they become feasible. Like crypto in 2021, AI valuations in 2023 are treating those large scale use cases as imminent rather than long-term speculative.
(Note this isn't a statement about which tech has a brighter long-term future. It's still more than possible to believe (as many on HN do) that AI will be transformative over the next couple decades and crypto is vapor. But the point is AI valuations are implying that it's going to be transformative over years not decades, and it's pretty clear that's not the case. There are parallels with crypto in 2021, where regardless of the long-term promise, the short-term returns simply cannot support the valuations.)
> This is consistent with the primary learning from the 2020-2021 class of GPT-3 startups (a category of startups willed into existence by VCs and powered by hype), which is that commercial use cases have been falling almost entirely into the marketing and copywriting niches.
ChatGPT is trivially useful in a lot of cases...The other day I asked it to write marketing copy for a project and it wrote _better_ marketing copy than I could have written in an hour. On another project, I spent 10 minutes integrating the OpenAI library and was programmatically receiving incredible results with almost no effort.
The nature of predicting the future means that there will be periods of overconfidence. In the case of Web3 (taken to mean blockchain/crypto/digital coins), the overconfidence was fueled by a core ponzi scheme combined with truly extraordinary returns for early speculators. But AI has no ponzi scheme attached to it so the comparison breaks down. My uncle cannot gamble his retirement on a mysterious promise of überwealth from a man with wild hair and no financial experience.
AI companies will have lots of false starts but 2022 was a transformational year and we are only getting started.