Two issues here:
1) we are only about ~10 years into the deep learning boom
2) we've seen deep learning scale with compute over this 10 years, not only over the last 2-3 years.
It could be we've reached the end of the road for NLP, no one really knows. But generally we see breakthroughs in lockstep with big jumps in compute capability (typically, GPU releases, occasionally with architecture changes).
I was listening to the recent interviews with Sam Altman and the Anthropic guy who are familiar with current research and they are very not like that. It's more wow we've got so much to build, AGI in a couple of years. (For it seems to me a rather limited version of AGI - more can code well rather than can fix your plumbing.)
They are CEOs, half the job is public cheerleading.
this will be the new "fusion in 10 years", but with the added downside of expending a small-country's worth of carbon per day while not actually getting us there.
like, of course sam altman is going to talk about how close they are and how they need more money.
Yeah, we don’t know for sure. But it seems like there are signs the easy gains may be over.
On the other hand I think arts and entertainment is where AI will likely still survive scrutiny, as imprecision is tolerated reasonably.
I don’t understand, this seems like the opposite of an example of “hype” causing damage. These customers found a better/cheaper alternative.
There are tons of business that probably went through this.
This is going to happen to so many businesses and industries that a universal basic income will become a necessity if we don't want revolts.
There was value in the market. VCs destroyed it through dumping. And for all we know they’ll give up in a year and we’ll have nothing. No jobs and no existing solution.
That's a very charitable take on that site. I'd call it a site where people pay money for solutions to assignments they can't be assed to complete themselves.
Because you can no longer be a cheap artist, because you can no longer help students on easy problems en masse, because family businesses no longer need a webmaster.
That's a step in the right direction, maybe even towards UBI.
On growth, I disagree that we reached the plateau already. We won't fundamentally change things but larger context windows, speed, compute and cost? Obviously.
That in itself is a major evolution.
It looks like it is fading out of hype maybe, but that's just like all things. LLMs aren't going anywhere, just like Rails got version 8 out and it's better than ever.
They had no real original content of their own, just worked solutions to homework problems they pulled from textbooks. They were good at SEO and would appear at the top. You clicked on it because it lied to you: showing you part of the content you wanted. Just enough for the search engine preview. That probably boosted them further, wasting more time by others tricked by the same fake results.
To see the rest of the answer, they wanted you to pay money and hope it was what you wanted. Who would subscribe to that other than students desperate for homework answers?
Then ChatGPT comes in without any of the scammy tactics. Sure, it's often wrong, but so are Chegg and Quora.
I’m sure there are plenty of good places that were destroyed. I just don’t have verifiable anecdotes for any others at hand.
One product worked and made money.
The other is (effectively) being dumped on the market by VCs in hopes of being a valuable unicorn.
It may never turn a profit.
My argument is that that VC bet on something that may not work out may have destroyed lots of real jobs. It wasn’t harmless, even if you ignore all the other externalities about resource usage.
It’s not OK in my mind for VC to just stand-up companies that destroy parts of the economy and then go out of business because it turns out they were never profitable in the first place.
It would be insane for that to be considered good economic policy.
Edit: Also, the VCs that are just GPT wrappers deserve to die. The only people it hurts are the investors with more money than sense.
It’s their massive money injections that allows these distortions to happen.
If people had been having to pay for the resources ChatGPT (etc) uses to use it then they wouldn’t have had the deleterious effects they have.
One definition of dumping: “Selling goods at less than their normal price, especially in the export market as a means of securing a monopoly.”
Isn’t that exactly what’s going on? Grow too big to fail fast so you’re the winner and people have to use you?
Is this a real thing that happens? Can you name some historical examples? What is stopping from destroyed businesses from being rebuilt pretty quickly? Uber subsidized taxi rides for millions of people for years, but if Uber went out of business, legacy taxis would come right back. And even if they didn't for X amount of time, X would have to be > the amount of subsidy that was received in the meantime. And if the legacy business was so vulnerable that people were willing to try and replace it with some big matrixes, the replacement company might still escape the local maximum that the previous company was trapped in.
You express a lot of concern that these are just "VC pumped up companies" or something, as if that negates the technology. But it doesn't! The technology has already been developed because of these VC investments, and much of it is public.
Moreover, even if these companies go out of business tomorrow and aren't replaced, having years of consumers paying less money and getting a better product is a good unto itself. Yes, a company that made something inferior went out of business in the process - but if, after the big companies are shut down, there really is somehow no alternative - a new company based on the old method can always start again.
I just don't understand how you can spin as bad the idea that VCs are spending billions on unprofitable companies, meaning that they are spending billions that go straight into either innovation, or consumer's pockets. Who loses out here except for the VCs?
And while I have empathy and respect for people who lose their jobs, companies going out of business is an everyday occurrence. We should wish it only happened because better-for-consumers solutions came along.
> Just tons and tons of waste everywhere.
I worry about this not just for AI, but in general. That's capitalism right there, profit now - who cares later. And I am becoming radicalized against it.
I just saw a mention about how a homework help company called Chegg has had their stock drop 99% because everyone is just using ChatGPT.
They were a real functioning company, with hundreds or thousands of employees and contractors. All of whom are basically going to be laid off because some company burned through a bunch of VC money to lure everyone away with “the new thing“.
All the artists who lost jobs or commissions. All the companies who ended up wasting a ton of time trying to build AI or integrate AI features that aren’t actually useful. And maybe they’ll end up in a product in two years and by then no one will care or want them.
All the electricity, the silicon, the water for cooling, the new data center is being built that won’t be needed.
Just tons and tons of waste everywhere.
ChatGPT is neat. For all we know we’re near a local maxima of what we’re capable of achieving without another completely new approach that will take 10 or 15 years to figure out. There’s no proof that the acceleration and capabilities we’ve seen over the last 2 to 3 years will continue like that.
I know my company has been asked about adding AI into The main product I work on. I don’t see any benefit. I’ve been told when they ask the customers what it would do for them, they can’t say either. But they seem to have been trained to ask for it by the hype.
Remind me of all the nonsense about chat bots being integrated into every company’s webpage five or six years ago. They’re not helpful. But they were the thing.
ChatGPT has some uses, but is also way more expensive/wasteful.
I hope the hype moves on fast. I’d like this stuff that shakes out to stick around but what’s going on right now is just way too wasteful for my taste.
Feels like almost everyone is trying to build the biggest Z-Ray they can because they’ve been told it’s an amazing discover. No one actually knows what it is, or how to build one, but that hasn’t stopped trillions of dollars from being poured into it. And if we get there, it may not be worth anywhere near what was paid.