Can’t wait for AI 2.0 and ads :(
I see it as a chance for the capital class to sell everyone shovels and build railroads that will further cement their power and influence, all the while insisting software and art are more democratic than ever. All the while using the same tools to build surveillance infrastructure that will make any dissent impossible.
So yeah, exciting is one word you could choose.
Everybody is now excited about the top end: more parameters, larger context window. Justly so, but I think that it's more important in the long term what will happen at the low end, both in the software and hardware direction. Software obviously has started (c.f. DeepSeek). As for hardware, Google TPU ASIC is six years old, and has presumably 4096 ALUs and uses few watts of power to perform 4 TOPs. Several years later, I would hope that it's reachable goal for those 4 watts to do as many TOPs as say H20 (400 or so).
Once the models stabilize, I could see a low power inference peripheral design with huge TPU matrix with an embedded model parameter Flash EPROM feeding it. This could go into anything from white goods to computers. Imagine a thermometer that you speak symptoms to and ask for diagnosis; is it a good idea? I don't know... but is it possible? I could see that it is.
You don't think that Carnegies, Vanderbilts, Goulds, etc don't have outsized influence and wealth?
But like the OP said, we can’t even predict what’s going to happen even three years out so I’ve just resigned myself to “going with the flow” and enjoying the ride as much as I can. If the negative consequences are coming for us, might as well get as much benefit as we can now while we’re all still wide eyed and bushy tailed.
Ha this is so true. All the major LLMs have a surprisingly ad free experience right now. Likely because they’re all aggressively competing for customers. But as soon as one of them realises the money that can be made from ads and how that will pay for daily operation costs after the investors give up investing, then the game is up
Google is an advertising company.
Microsoft, in fits of jealousy and veiled rage, wishes it were an advertising company.
OpenAI is not profitable yet despite having high charge rates.
If they aren't capitalizing on ad infusion yet, their shareholders should rightfully be getting angry, right? Money left on the table.
Simply because you and I may not be able to buy ads does not mean they arent already there.
Dont forget the pg classic on the topic: https://paulgraham.com/submarine.html
Is there a source for such claims. I am also interested to know.
While I despise ads, it’s an interesting problem. Even on my personal site, I can’t really control how people learn about me.
After another MCP discussion today got into "Neo4j is the best database for almost every MCP agent, because…" I realized I may have "Semantic Web PTSD" and finally have a name for some of the less than excited gut reactions to "Agentic" I've been having for a while now. (Beyond also the terrible gut reaction to both the "word" "agentic" itself and also naming a key protocol after the villain from Tron.)
The trick I've always used in these circumstances is the cynical approach. That is assume nothing changes. If it does change, adopt late, rather than burn all your time, money and energy on churn and experimentation.
In the last 35 years of doing this, I've seen perhaps 10% of technology actually stick around for more than a few years. I'll adopt at maturity and discard when it's thoroughly obsolete.
Being fintech, no technology so far has fundamentally changed what business we do or even how it's done for a long time even if we pretend it does. A lot of the changes have just been a cost naively written off through arbitrary justification or keeping up with trends. 99% of what we do is CRUD, shit reports and batch processing, just like it was when it was S/390.
Even fewer things have had an ROI or a real customer benefit. Then again we have actual customers not investors.
Don't kid yourself. Skepticism is not neutrality. These days throwing shade is a growth industry. There's money to be made shorting just like there is going long. Neither is the objective, disinterested position, although skepticism always enjoys the appearance of prudence, at least to the ignorant.
Anyone who says they're not trying to sell you something lacks self-awareness about what they themselves have been sold.
It was a step forward, but it was incremental. Internet was also incremental. In every sense. Just because the general populace didn’t hear about it until then, didn’t mean that it was that “revolutionary”. Yes, they crossed a line which made them useful, something what people want. Sometimes mainly because of marketing. But still incremental.
This whole modern neural network saga started around 2011. Every step was incremental since then. Just because most people didn’t hear about these just in 2022, doesn’t mean that large LLMs were suddenly here from nothing. They still need to improve for example to not make the code quality plummets immediately when programmers start to use it. It was, and will be an incremental process.
No one, except Steve Ballmer, would describe it as a potential fad or question how good it can actually get before Apple goes bankrupt from all the investment into this new tech.
I like this new stuff we get now, but the iPhone felt like a clear win with no downsides of a potential societal collapse.
Pretty sure it's known. But, just like smoking, it's tolerated. Can't make the line go down.
By then, no one was saying that anymore.
The AI scepticism is still going strong.
Sent from my iMPC
Countless pundits and many heads of companies like Motorolla and Nokia, said exactly what you say "nobody except Ballmer" would say.
Mostly for the worse. Mental health crisis, depression, loneliness epidemic, antisocial tendencies, attentions destroyed, total 24/7 surveillance, hard dependency on a couple of mobile OS vendors...
People were lifting their ChatGPT prompts on their devices during graduation.
At this point we have half the industry being overly pessimistic and the other half being unreasonably optimistic. The median truth would be in the middle. But I don't think that's a very sound position to take either.
The reason is that I think we're actually dealing with a severe imagination deficit in society. That always happens around big technological changes. And this definitely looks and feels like such a thing. Ten years from now it might all seem obvious in retrospect. But right now we have the optimist camp predicting what boils down to the automotive equivalent of "faster horses" (AGI, I robot, self driving cars, and all the rest). It's going to be this wonderful utopia where no-one works and everything runs by itself. I'm not a big believer in that and I don't think that's how economies work.
And we have a bunch of pessimists predicting that it's all going to end in tears. Dystopia, everybody is going to be unemployed, and a lot of other Luddite nonsense.
The optimists basically lack imagination so they just reach for what science fiction told them is going to happen (i.e. rely on other people's science fiction). And then the pessimists basically are stuck imagining the worst always happens and failing to imagine that there might be things that actually do work.
It's fairly easy to predict/bet that both sides are probably imagining things wrong. Just like people did three years ago. Including myself here. So, not making a prediction here. But, kind of curious to see how the next few years will unfold. Lots of amazing stuff in the past three. I'll have some more of that please.
The researchers responsible for LLMs didn't see them coming either. The architecture is quite basic, if they knew it would have this success they'd do it way sooner as a PoC, even when hardware wasn't quite up to it.
The most important thing to do is play and build small things with all of them as they take turns being in the lead or running the fastest sideways in this horse race.
What's new to someone might be old to most etc. What's critical to the individual is knowing what's new to them vs what they know is out there.
I am not sure after 2 years of this if there will be a single clearcut winner, as much as they will all contribute best practices to something more emergent.
Webapps when they first began also didn't have much structure, frameworks, etc, and it evolved from just building a lot, and trying to build better.
You saw this thing and knew the future had changed.
It's very easy to forget Android just shamelessly ripped it off wholesale. The early version of Android looked nothing like the complete iPhone clone they ended up with.
And I have an android phone right now. But it only exists because of the revolutionary iPhone.
Credit where credit's due.
And right now AI feels like just before the iPhone. Like then we had touch screens, and we had new battery tech and we had mini-apps on the internet. And we had iPods showing small form factor MP3 players. But they didn't work together coherently.
And then the iPhone came along.
Like we're waiting for the iPhone of AI, to bring the tech together to an actually usable state.
I predict a 1999-dotcom-boom style crash of the AI hype within 3 years, slower incremental improvements, and increased rent seeking (ads, sponsored responses, etc.) and price jacking.
I suspect the bubble is self-sustaining, because everyone making investment decisions is confident that they won't be blamed if (IMHO, when) the bubble collapses, because "everyone was investing in AI" while simultaneously, even those who might be skeptical are terrified of missing the boat if LLMs really do live up to their hype, or at least convinced they can ride the bubble and find a greater fool before it pops.
Just use the tools we have at our disposal and create cool, profitable, or meaningful stuff.
It was never easier than today.
Build regular programs that are useful and will not get replaced by AI.
I don't think there's a human on this planet who can even predict the state of the industry in 3 years. In my entire time in the industry, I have always felt like I had a good line of sight three years away. Even when the iphone came on the scene, it felt like a generational increase rather than a revolution.
We just have no idea. We don't know the extent of how it can improve. We don't know if we are still on exponential improvement or the end of the S curve. We don't know what investment is going to be like. We don't know if there is autonomy in the future. We don't know if it's going to look more like the advancement of autonomous vehicles where everyone thought we were just a year or two away from full autonomy - or at least people bought the hype cycle.
Any anyone who says they know has something to sell you.