And even if they are first, why assume the first is going to be the best? Maybe 6 months later a different AI model will come out that stomps all over the first one. Maybe a competitors isn't any smarter but they can do it with half the power and can scale their intelligence twice as fast, so even being later they still end up in the lead.
For your second point, consider the definition of superintelligence. Nobody else would find such an advantage faster by definition, except through sheer luck. If they did, as soon as they offered any hint whatsoever about it, the existing superintelligence would instantly reverse engineer the strategy and execute it faster.
> But I do not see a way to rule it out as a possibility.
Your argument is unfalsifiable. Anything can happen by this logic and they could be valued whatever trillion $, like, maybe they’ll invent a way to travel through the multiverse. Maybe they’ll create the star trek replicator thing.
It is certainly not unfalsifiable. There are other scenarios in which this does not happen. For instance, the most obvious is that other organizations develop a similar capability at roughly the same point in time. This results in a "multipolar" economy that is mostly controlled by a small set of distinct owners of this kind of capability. The multipolarity prevents total capture and thereby prevents the rest of the scenario. Another obvious scenario is one in which the government acts soon enough to short-circuit this process before one organization gains a sufficiently decisive advantage that government intervention is no longer efficient enough to be effective. Yet another obvious scenario is one where a short-term financial crash deprives Anthropic of enough capital that this strategy is paused or slowed down enough to allow it to be countered. And yet another obvoius scenario is one where the scaling laws simply stop scaling or the amount of compute needed to scale them to this point cannot be physically obtained, thus allowing time for the strategy to be countered.
No, what that AI will do is that it will create an artificial reality that caters to the highest bidder.
Expect this to happen in the next 20 years.
A lot of manufacturing is already there - that's a part of why China undercuts Western manufacturers, a lot of cheap land and subsidies means you can be far cheaper than Western companies already, even more if you automate what you can. Farming, at least anything with growing plants, is also barely requiring humans any more - precision farming with RTK is absolutely insane. And what remains is trades (where, again, automation like prefabrication is a threat), parts of healthcare and education.
We will own nothing, rent everything and fight for the scraps Sam Altman hands to us.
Same as Goedel thinking that a logical loophole in the constitution is something that could turn America into a dictatorship legally. Guess what? In practice, somebody wanting to instate a dictatorship probably won't give a crap about some subtle logical fallacy. Same with the Keynesian economics that Americans perceive as law.
even so, what about network effects? how do they lure users away from TikTok and Instagram? AI-generated clips? or say Anthropic recreates Amazon. that's a software company, right? so how do they get all the warehouses, cloud infrastructure (they will need entirely different kinds of datacenters and hardware), relationships with third-party merchants, etc.? or say SpaceX - is your superintelligence going to be setting up massive autonomous factories and buying thousands of tons of aerospace materials and billions of dollars of machinery to make that happen? where's it get the money?
and even if this somehow does manage to accomplish all that, how politically popular will it be? what prevents the US government, lobbied by.. the majority of corporations, and hounded by constituents, from invoking the Sherman Act and busting this trust like AT&T or Carnegie Steel?
They already have solid control over logistics and inventory management.
It’s not a stretch that they then capture the means to control primary inputs and manufacturing.
Throw in an overly enthusiastic AI to continually optimise the supply chain, continually optimising a country’s supply of food and commodities, and we have all the ingredients for the classic sci-fi dystopia of a computer controlled society.
None of this is guaranteed to happen. But I do not see a way to rule it out as a possibility.