If they can crack that latter review/spec-check/assurance step, checking that what was built was what was demanded of the problem such that we don't have humans in the loop at that step either, then the bottleneck moves again. Then I think it moves to requirements capture and to product development, but that might depend on the industry.
Yeah, if this stuff actually worked that well already, OpenAI et al. would just run AI CEOs and engineers. Why get some other company to pay you at all when you can automate every other company out of existence and take all the money they make?
The fact of the matter is that while the tech has some uses, it sure as hell isn't a full scale replacement and you almost always actually have to massage the input into LLMs to get anything decent back out in practice. Some CEOs and managers can learn to do this, of course, and some already are... but that quickly turns into a second full time job. A "programmer" is still needed. The job might change from mostly hand-writing C++/JS/Python to prompt engineering + some manual coding to fix all the stupid fuck-ups that the bots can't solve themselves, but you still need someone to actually prompt the bot.
When that changes, it won't just be engineers losing work; there will be no reason to even have a human CEO any more.
The human race isn’t ready for that world IMHO. The only reason there is a middle class is because people have leverage in the form of their labor. When that becomes worthless … the people who own stuff and make their living from doing so won’t hesitate to get rid of everyone else - whom are now worthless to them.
I don't think there is any shortage of great ideas at these companies, they are just extremely bloated. And I don't think its something like indecision or bad PMs, it's "we have a finite amount of time and resources so we need to be conservative but also not too conservative"
If you have AI systems that can simply build out POCs in days, backtest on real data, show reliable results and numbers, you get a suite of product options you were never able to get before. If you have coding agents that can speed up implementation, you can build more stuff and choose the things that stick.
It changes the cost/benefit calculus of the entire business. I think you are exactly right in that: PMs/leadership are by their nature orchestration machines. Other roles are as well, but I think PM's are at a particular advantage here in that it will be quite awhile I would expect before core product decisions and creativity can be delegated to an AI, but not quite awhile until virtually everything that they're blocked on (legal approvals, POCs, wire frames, etc etc etc) will become less and less of a blocker
I'll also add this: within a large organization, you often need to interact with many different codebases owned by many different teams. Agents have made it much easier to wrangle by having the ability to deploy one to scope out your web of dependencies to learn about what would be needed for feature X, and how that integration can happen.
We've been doing far more away team work simply because it makes things move faster. It's easier to convince a team to sign off/review something than it is to get them to commit to the planning and eventual work.
It genuinely is helping things move faster inside large organizations. Or at least, it is for us, particularly since we're getting organizational prioritization to actually build the scaffolding to make those agents more effective at search.
1000x yes: you have touched on what I think is the single biggest factor here, that is the humongous value of POCs. they are gnarly to build without agents, and so we used to have to get everyone on board so we didn't get screwed in performance reviews, which was monumental task because that means convincing very busy PMs who have a lot on their plate and dont want to take risks on things they don't understand, and now it's like "can we scale this out" and you have a very nicely formatted proposal and POC. It de-risks things very quickly
Say I want to build a feature in a product.
- DS has to do a deep dive (need buy in) to opportunity size and derisk with data. That DS has to work with other DS (people may have left or moved teams) to figure out how to get the right data and figure out what the difference is between 10 different tables that have overlapping but inconsistent data. - Eng has to build up an actual simple demo (need buy in) - Design has to make it not hideous (need buy in) - Legal has to review what you're doing; POCs should involve real data where possible because otherwise no one will trust it, even if its just for user analysis on existing products
This plus about 6 internal system bugs for custom tools that are flaky and who's team has long been re-orged or laid off, 8 people who won't answer you, 2 PTO's for the stakeholders, 6 weekly meetings
no one did POCs, they just had ideas and tried to get PM's to put it on the roadmap so if it fell through at least it was bought into
You still want someone whose ass is on the line if they get it wrong.
The problem is they get killed by some other executive who is afraid of their department looking bad by comparison.
I think this is fairly illustrative of the challenges in AI becoming as impactful as the Internet. The bottleneck is not making things. There are plenty of people who are really good at making things and can easily be 10x or 100x as productive as the average corporate worker. YCombinator was founded on that premise - small teams of founders and early employees could be orders of magnitudes more productive than the 1000s of corporate employees at their competitors.
The bottleneck is on bringing your product to market. If your innovative new product is built within a corporate environment, it'll get killed unless the executive you work under can get a promotion out of it, and you'll be denied all sorts of help with approvals, launch process, PR, marketing, branding, etc. If it's a startup, they'll try to shut you out with exclusive distribution deals, legal threats, lobbying efforts to change the legal environment, PR campaigns, FUD, etc.
The Internet was revolutionary because it let millions of people bring products to market without asking permission. Instead of having to bid for retail shelf space among dozens of entrenched competitors that all had sweetheart deals with the retailer, you could just put up a website and sell it to anyone across the globe. Instead of following hundreds of regulations that governed existing commerce, you could just launch something and sort it out later. AI doesn't really have that property - if anything, it makes things more centralized, with more gatekeepers, and so seems more likely to destroy economic value than add to it.
> YCombinator was founded on that premise - small teams of founders and early employees could be orders of magnitudes more productive than the 1000s of corporate employees at their competitors.
I think this is still true, but the theory is:
1. You don't need YC-type funding to do YC-type business any more; 2. You don't need to scale the business past those small teams any more, you just buy more tokens.
For clarity YC still obviously has a place as an incubator, mentoring, and networking function. I just think that what was previously the inevitable conclusion that you have to hire all the people the second you hit PMF to keep up with scaling the business as you scale sales is no longer inevitable. If you didn't want to go that way before AI, you were a "lifestyle business" and not worth investing in. As more and more knowledge functions get capably implemented by AI, it's the preferred position: humans are vastly more expensive than tokens, so you want them doing the stuff the AI still can't do.
I don't think this necessarily translates to mass unemployment. I think it translates to masses of smaller businesses that are radically more efficient because the handoffs between business functions are tool calls, not emails to someone who doesn't want to help.
> The Internet was revolutionary because it let millions of people bring products to market without asking permission.
Think about it this way: if I am a small business owner but I think it makes sense to do something that previously only a team in a corporate environment could do but is now within the reach of AI, not only can I do it now, but I also don't have to ask anyone for permission! Who wins between the corporation and the small business in that scenario?
> AI doesn't really have that property - if anything, it makes things more centralized, with more gatekeepers, and so seems more likely to destroy economic value than add to it.
I think this will turn out to be backwards. I can see a version of this where the number of things you can do without needing to turn to a gatekeeper for help increases to the extent that the balance completely inverts.
The vast majority of businesses are small, and AI can give them tools which previously required corporate scale to make sense, without the inefficient hand-offs between busy, political humans. Which is also something that the internet did! Getting an advert in front of a national market pre-internet was Hard but sometimes you had to do it because your target market was "all Canadians who buy toothpaste" or whatever and that meant saturation-bombing the physical environment with physical billboard ads, posters, flyers, and so on. So you only did it if you were P&G-scale. Now you, personally, can do it, trivially, for better or worse.
> Valve
Arguably a monopoly. They've got a product that sells itself with very low infra overheads for the income.
> Hedge funds
Very different model. I don't think the same intuitions apply.
Obviously that includes whatever needs to be done to hoover in data from their marks and Meta also does the same thing without fail and both are really good at it. But outside their remit not so much.
I would agree but it's really minimized the building. More and more time is being spent on pre-coding work.
You'll find that most internal "innovation" teams are just lip service. In most cases, the "mothership" will be incapable of reproducing true innovation -- from a statistical perspective, culture perspective (mega corps are anti-scrappy; internal politics), and motivation perspective (startups aren't 9-to-5). It's much easier to have big M&A budgets, a VC arm, and some handwavvy internal innovation group.
Every now and again, you'll get real innovations (Waymo, transistors, GUIs), but even those have a spotty track record of commercialization when created internally.
Kubernetes is at 11 years ago, and is huge enough to be included there. The Google Pixel was just under 10 years ago. So... not nothing haha
Second to this are countless other areas that have a major impact on the companies bottom line that are entirely engineering driven, especially at google given they are a cloud provider and have meaningfully grown the workspace business and launched waymo in this time.
I would argue that that's been the case for quite some time before AI. As an example, what innovative amazing world-changing products have Google or Meta launched in the past decade with their very high numbers of very talented and highly-compensated engineers? The issue with most big tech companies are leadership, strategy, and product direction. I'm not saying that they don't make any profits, just that they probably aren't "building [the right thing]".
AI for product development and management would be far more impactful than automating rote coding tasks / building React UIs that mirror API structures IMO.