How easy it must be for Sam Altman to drag the world kicking and screaming into the future. How pretty his offices.

The more the farms automate, the fewer farmer workers there can be.

Logic isn't part of these protests either: the farmers are being told "you're making more food than we need, please make less" and yet the placards are (in the local language) "no farms = no food".

Efficiency puts people out of work. They can be relocated to other useful pursuits, but usually they're left to do that on their own.

Dont know what country this is in but here in Germany farmers are mostly having an issue with prices being set too low by too big buyers, regardless of the demand.

Nobody is talking about AI replacing people’s passion projects. The discussion was about jobs and economy.

The notion that people’s jobs are their passion isn’t true for the vast majority of the world.

The current cost of a Boston Dynamics Spot is around a year's income, give or take whose income you're measuring against.

If it were able to do any human task at the same rate as a human — and yes, I know it isn't, this is just a anchoring point for the discussion — a group of them would be able to extract and process enough resources in a year sufficient to double their population, all the way from rocks in the ground to a finished deliverable.

n years later, there are 2^n robots. Sure, sure, that's a whole 33 years to go from one total to one per human, not the numbers the other person gave which would need a much shorter (but not wildly implausible) reproduction time of 5-8 weeks, but the point is still valid.

That exponential stops only when some un-substitutable resource is fully exploited, so I'm not sure what the upper limit actually is, but given we exist I assume 8 billion robots is also possible.

I'm just finding it funny thinking of 15 Spots queuing for a flixbus / greyhound bus because they need to go get some raw material across the continent.

If that's humans, humans have work[0]. If humans don't have work, the only alternative to robots is, what, well-trained squirrels?

> I'm just finding it funny thinking of 15 Spots queuing for a flixbus / greyhound bus because they need to go get some raw material across the continent.

Me too.

But I expect actual logistics to be much like current logistics, so it would be more like a few Spots loading a standardised intermodal shipping container, a Tesla semi taking it to the port (which is guarded by drones), an automated gantry that puts the container on a cargo ship, a few more robots guarding the cargo ship from pirates (who may well also be robots), the same in reverse at the other end.

[0] You may point to the current conditions of cobalt mines and go "this is bad"; but once was a time when other forms of mining were seen as good solid work, and those with those jobs protested against their mines being closed down. Almost broke the UK when they did that protest, too.

Just explain why AGI will be in our near. Frankly, I don’t see how from what we have now.

Strange response…

> Just explain why AGI will be in our near. Frankly, I don’t see how from what we have now.

…but OK.

The AI we have now, is already capable of learning from what we do. It's very stupid, in the sense that it takes a huge number of examples, but surveillance is trivially cheap so a huge number of examples is actually very easy to get.

As I wrote in 2016:

"""We don’t face problems just from the machines outsmarting us, we face problems if all the people working on automation can between them outpace any significant fraction of the workforce. And there’s a strong business incentive to pay for such automation, because humans are one of the most expensive things businesses have to pay for.""" - https://kitsunesoftware.wordpress.com/2016/04/12/the-singula...

Compute cost is also important: more compute makes better GenAI images, allows larger models in general, turns near-real-time into actual-real-time (many robot demos you see on YouTube have sped-up footage).

Here's something I wrote in 2018 anchored on iPhone compute cost and some random guesstimate I found for what it would take to have an uploaded human brain running in real time, though I can't remember if I've ever compared it with actual compute improvements since then: https://kitsunesoftware.wordpress.com/2018/10/01/pocket-brai...

So, while I don't know what you mean by "near", I'd put "the economy is going to change radically due to AI" somewhere between "imminent" and "20 years" (±3σ, skew normal distribution with a mode somewhere around 2028-2030).

It’d be like me making a prediction that we will have begun to colonize another galaxy in 10 or 20 years time and then only talking about how there’ll be trade between Earth and the colonies and maybe even wars and revolutions. Meanwhile completely skipping over how our spacefaring technology will advanced to the point we can even travel those distances in a reasonable timeframe.

I’m not an expert on LLMs but there doesn’t seem to be very much about them that is even approaching AGI. They’re a useful tool and it’ll definitely disrupt certain sectors of the economy mostly white collar jobs, but we’re in the middle of a peak of inflated expectations. This has happened before with other technologies.

Invert your causality.

The discussion so far was "oh no, oh woe, we shall have no jobs!" — this can only happen if AI is good enough to do all that humans can do. Until that point, we're fine, it's the status quo, and also it doesn't matter how long we stay in this state. I'm not making any strong claim about the start date of the transition (I have a 20 year spread which I think is pretty vague), only the duration of such a transition.

When AI can do that, when, then it's obvious they can do things like "build a robot body", which is obvious because we can, and the definitional requirement of there not being any more work for humans is that the robots can do all the things we can. It's a necessary precondition for the scenario, not a prediction.

> Obviously these robots doing that is predicated on them having AGI

No, it isn't. "AGI" isn't even a well-defined term, each letter of the initialism means a different thing to different people.

And self-replication has much, much lower brain power requirements than full AGI, even for simple definitions of AGI: an AI-and-robot combo with all the intellect of the genome of E. coli is also capable of self-replication. The hard part of self-replication right now isn't the brain power.

So again, invert your causality: the brain power to replace all human workers includes the knowledge of how to self-replicate, but the knowledge of how to self-replicate does not require the brain power to replace all human workers.

> so it’s pointless to talk about what these AGI robots will be doing if we haven’t established that AGI is even possible in the near term or at all.

The specific things an AI needs to do, is learn. That's all. And they already can. The weaknesses of current models still, even if left unresolved, result in AI learning to do each thing humans do eventually when given enough examples, which limits humans to the role of teaching the machines. This is still a form of employment, so it's not economic game-over.

> It’d be like me making a prediction that we will have begun to colonize another galaxy in 10 or 20 years time and then only talking about how there’ll be trade between Earth and the colonies and maybe even wars and revolutions. Meanwhile completely skipping over how our spacefaring technology will advanced to the point we can even travel those distances in a reasonable timeframe.

No. That would require a change of the laws of physics. We don't need a change to the laws of physics for AI, because no matter what definition is used and whether or not current models do or don't meet any given standard, the chemistry in our own bodies definitely demonstrates the existence of human-level intelligence.

> I’m not an expert on LLMs

Are not the only kind of AI. You can't use an LLM for OCR, tagging photos, blurring the background of a video call, driving a car, forecast the weather, or predict protein foldings, and shouldn't use one for route finding or playing chess (although they're surprisingly good at the latter two, all things considered). Other AI do those things very well.

But LLMs will translate between languages as a nice happy accident. And they can read the instructions and use other AI as tools. And, indeed, write those other AI, because one of the things they can translate is English to python.

> but there doesn’t seem to be very much about them that is even approaching AGI.

Then you are one of many whose definition of "approaching" and "AGI" is one I find confusing and alien.

Between all AI, every single measure of what it means to be intelligent that I was given growing up has been met. Can machines remember things? Perfectly. How big is their vocabulary? Every word ever recorded. How many languages do they speak? Basically all of them. Are they good at arithmetic? So good that computers small enough and cheap enough to be given away for free, glued to the front of magazines, beat all humans combined and still would even if everyone was trained to the level of the current world record holder. How well do they play chess? Better than the best humans, by a large margin. Go? Ditto. Can they compose music? Yes, at any level from raw sound pressure levels to sheet music. Can they paint masterpieces? Faster than the human eye's flicker fusion rate. Can they solve Rubik's cubes? In less than the blink of an eye. Can they read and follow instructions, such they can use tools? Yeah, now we have LLMs, they can do that great. Can they make software tools? Again, thanks to LLMs, yes. Do they pass law school exams, or medical exams, can they solve puzzles from the International Mathematical Olympiad? Yup.

We're having to invent new tests in order to keep claiming "oh, no, turns out it's not smart".

> They’re a useful tool and it’ll definitely disrupt certain sectors of the economy mostly white collar jobs, but we’re in the middle of a peak of inflated expectations. This has happened before with other technologies.

LLMs, probably so. I often make the analogy with DOOM, released 30 years back, and the way games journalists kept saying each new 3D engine was "amazing" or "photorealistic", and yet we've only just started to really get that over the last decade. Certainly all the open source models are gushing over each other as "ChatGPT clones" or "ChatGPT killers" in the same way games were "DOOM clones" or whatever the noun was in the cliché "${noun} killers".

And yet the field of AI as a whole, including but not limited to GenAI, has been making rapid progress and doing things which are "decades or centuries away" every few couple of years since I graduated in 2006. Even just the first half of the 2010s was wild, and the rate of change has only gone up since then, this last 18 months has felt like more than that entire decade.

The rest of this is just sci-fi speculation.

If that's all they did, it would be just another change to the nature of work, and humans would simply find other roles to fill.

Rapid change is scary, but can be managed, has been managed before — this kind of thing has historically grown the metaphorical economic pie, making everyone better off. If it's either one of "just muscle power" or "just brain power", the other leaves opportunities for humans.

Only a total replacement of all human work causes such a break that we're fumbling around in the dark. ("Fumbling around" is how I see the discussion of UBI: even if it turns out to be right, we don't yet have anything like a good enough model for the details).

> It seems far more likely than humans will maintain control of the corporations that manufacture the robots.

Will they, though? We've already got people (unwisely IMO) putting AI on boards of directors. Yes, it's as much a stunt as anything else, the law prevents them from being treated as "people", but the effect is the same: https://asia.nikkei.com/Business/Artificial-intelligence-get...

Not per year, total. And it's not really an assumption, just a demonstration of how fast exponential growth is.

> I assume robots will be similar to high end cars in regards to the complexity of manufacturing.

Agreed. This is also the framing Musk uses for Tesla's androids.

> Once a company develops a prototype with AGI

I don't think it needs a complete solution to AGI, as other people use the term. First, all three letters of that initialism mean different things to different people — by my standard, ChatGPT is already this because it's general over the domain of text, and even if you disagree about specific definitions (as almost everyone reading this will), I think this is the right framing, as you'd "only" need something general over the domain of factory work to be a replacement factory worker, or general over mining and tunnels to be a miner, or general over the domain of roads and road users to be a driver.

This isn't to minimise the complexity of those domains, it's just that something as general as ChatGPT has been for text is probably sufficient.

> The first million will be built by humans. The initial robots will take over the manufacturing only later.

Perhaps, perhaps not. The initial number made by humans is highly dependant on the overall (not just sticker-price) cost and capabilities, so a $200k/year TCO robot that can do 80% of human manual labor tasks is very impressive, but likely to be limited to only a few roles, probably won't replace anyone in its own factory; while one which has total costs of $80k/year and can do 90% might well replace most (but not all) of the humans in its own factory; and one costing $20k/year all-in and which can do 95% might well replace all the factory workers but none of the cobalt miners or the truck drivers.

"Fully general" is the end-state, not the transition period. But with fully-general, which is a necessary condition for nobody having any more work, we get a very fast transition from the status quo to having one robot per human.

> Setting up manufacturing that will be able to produce a million units in 2-3 years is possible. Let's say 5 years for a more plausible situation for a million robots to be built.

Agreed on both.

> Also there is no reason to believe it will be 2^n, it can also be 3^n or 1.1^n or any arbitrary number.

It's a definitional requirement of exponential growth, 2^n units after n doubling periods. I anchored on a the doubling period being a year just by reference to the cost of an example existing robot, and using that dollar cost as a proxy for equivalent human labor, and I specifically noted that the other poster's estimate corresponded to a 5-8 week doubling period which didn't seem unreasonable to me. Some robot can do each specific task 4.2 times slower against the wall clock and still be just as fast as a human overall because it's working 24/7 rather than 8/5.

What I want to convey is that the growth function will be somewhat similar to y= c + ax^n (ignoring/collapsing into c the linear and higher order terms) rather than just y= ax^n.

The c here is robots produced via humans. I predict c will easily touch a million in 5 years with or without human help.

Even if the later bots can do only 50% the work of humans, we will still exponentially grow the robots until the humans become a bottleneck. And that 50% capability is also expected to grow exponentially.

Gemini 1.5 pro already beats most humans in most benchmarks, combine it with Sora which has a great visual world model, add some logical reasoning(architecture or scale), memory and embodiment(so it can experiment and test) and you pretty much have the seeds for an agi.

My optimistic/most probable prediction about the growth rate say it's

Regarding the last part:

My bad, I speed read your comment and didn't focus on the exponential calculations. An exponential growth is just x^n. Both x(multiplication rate(?)) and n(units of time) can be manipulated.

> Even if the later bots can do only 50% the work of humans, we will still exponentially grow the robots until the humans become a bottleneck. And that 50% capability is also expected to grow exponentially.

I think most automation since the dawn of the industrial revolution has done 50% or more of the task it was automating, and although yes the impact there is exponential growth, humans are a very rapid bottleneck until the next thing gets automated — Amdahl's law, rather than Moore's.

My optimistic/most probable predictions about the growth rate say it's wild. Like within 5-10 years with the multiple exponentials among different fields you can easily do away with most jobs unless there is a major bottleneck (I don't think there is, there is so much low hanging fruit). I guess that's what the singularity is all about. I don't think this will take multiple decades or centuries in any scenario other than the equivalent of ww3

I get the feeling less "conventional" robots might be needed someday, rather than more.

If you can sell stock, or leverage it for loans, then you get access to future profits as soon as there is a reasonable expectation they will eventually show up.

Amazon's growth model was based entirely on this effect for many years. No need for profits, company's value skyrocketed as anticipation compounded.

Of course, there's a whole bunch of other differences, like the machines don't need to wait 18 years after construction for physical labour, they don't need to be trained for 21 years for mental labour, they don't sleep, and the "working lifespan" target is about 47 years rather than the full 84 years average human life expectancy at birth in developed nations.

Because they're not built to, presumably because manufacturers are greedy and want to sell you a new one, and buying a new one is encouraged by consumerist culture.

Let's say effective human lifespan for physical labor is ~40 years (20 to 60). On average, commercial aircraft last 20-30 years, meanwhile the US military plans to keep B-52s in service at least through 2050s, which will make them roughly 100 on average.

It's not that machines can't last a century with proper care and maintenance, it's that they're not designed to.

Slow down, sure, but not reverse.

  Work is equivalent to worth. We are not biologically or sociologically prepared to differentiate the two.

Plus, like, isn’t that kinda depressing? What are disabled people “worth”? If I choose to be an unpopular niche artist, am I “worth” less?

Human beings--especially modern ones--assign worth quite differently, and I think that you and I would agree that morally, the way we refer to "worth" in our modern context is the correct framing. But you cannot deny your own biology, no matter how much the modern world insists that you can. You are evaluated according to nature's standards of "worth," (the worth that it imposes onto our psyche) and then secondarily, according to our own set of invented standards. So yes, it is deeply sad. No one will care about the out-of-work tradesperson.

I promise I'm not a luddite. Just stopping for a moment to smell the roses.

(High school middling grades in chemistry and biology, so I may be way off).

More explicitly, I think this a great exposition of my overall point. Humans need to worry about being successful humans, not fitting some Darwinian universal shape that all life conforms to, as the capitalists so often claim. Thus why I don’t think automating cashiers, truck drivers, and computer scientists is incompatible with human flourishing in the long term.

Which is why no child has ever survived until adulthood.

And TBH I find this an appropriate metaphor because the promise of AGI is that every adult human will be infantile by comparison--but there are also potential technologies that could allow humans to ... "upgrade" themselves and become a mature ... whatever. Sadly brain-computer-interfaces seem like a technology that works best when enabled by AI (to help with interpreting brain signals), so it seems quite unlikely that any biological humans are going to keep up with AI over the medium term.

I hope so.

But transforming ourselves into GAI to keep up (while maintaining continuity of memories, etc.) is going to be a much more expensive proposition than simply making more GAI hardware from scratch.

So economically, where are humans going to earn that additional value, needed to account for the extra cost? When the premise of upgrading ourselves is our native biology isn't keeping up and so not actually needed?

The additional value that humans present would thus be a diversity factor. Much as it might be economically beneficial to cut down the entire Amazon, there's also significant economic benefits in keeping it intact and finding out what kinds of crazy biological stuff comes out of there. And over the long run, we get way more economic benefit from finding crazy things in there than in chopping it all down. There are all kinds of ways to explore uncertain dimensions, but looking at whatever the universe has already managed remains a consistent source of value.

> But transforming ourselves into GAI to keep up (while maintaining continuity of memories, etc.) is going to be a much more expensive proposition than simply making more GAI hardware from scratch.

It's basically a fixed cost because of how slowly the human population grows (if it even continues to grow).

Synthetic GAI entities will be able to use vastly more resources than humans presently do, because they'll be able to create more of themselves as quickly as resources become available, and they won't be stuck operating at a fixed clock speed. It's also probably going to be much easier for them to make use of off-world resources. So the total population of GAI entities could plausibly explode compared with the total human population.

It comes down to timing: if there's technology available for humans to become less ... biological, and it seems very desirable to lots of people, but it's really expensive and there aren't that many GAI entities yet, that might be less pleasant. But if that kind of tech depends sufficiently on AI developments that there's some significant lag time between a GAI population explosion and that kind of human-upgrading tech showing up, the cost of making it available to whatever humans want it might be a drop in the bucket from a total economic perspective.

To be clear, I don't see biological humans "keeping up" over the long term. If we take AGI and BCIs and various Ship of Theseus questions seriously, there could be some significant blurring of lines between "human" and "AI." And if we combine that with GAIs originating from humans, the concept of "keeping up" seems to become less meaningful. Who's keeping up with who?

Lastly, I'm suspicious that GAI entities won't be purely economically motivated, because I don't see any reason that they'll be "purely" anything at all. There is no magical "essence d'intelligence"; instead there's staggering layers of complexity. And every plausible AI safety approach I've seen so far involves training AIs to be inclined towards beneficial acts and away from harmful acts because there's no way to program conceptually pure "motivations" into them.

I absolutely agree. There will be a Cambrian explosion of superintelligent forms.

> The additional value that humans present would thus be a diversity factor.

We already have a test for this. Parrots, octopus, whales and many other creatures are also diverse. But our problems are beyond their experience. Do we value their diversity as thinkers in our economy?

We don't.

(I am speaking in economic terms. We can deeply intensely highly "value" many things, but if we don't actually invest economic value in protecting or nuturing those things, our appreciation has little or no impact.)

--

> It comes down to timing: if there's technology available for humans to become less ... biological

That makes sence. The order things become viable makes all the difference.

What I am seeing is that AI as software/minds is moving faster and faster. AI "bodies", i.e. robotic forms, are moving much slower but gaining steam. And our ability to augment biological forms is moving far far slower.

All three areas are accelerating, but the disparity between them isn't shrinking - for now. But who knows? The next couple decades will be interesting!

Maybe going fully artificial ("self-designed" might be a more meaningful description) will become trivial sooner than we think. :)

--

> Lastly, I'm suspicious that GAI entities won't be purely economically motivated, because I don't see any reason that they'll be "purely" anything at all.

I think they will be deeply motivated economically to care about ethics. Just as our genes and culture have (slowly) responded to the tremendous value of creating win-win situations.

Also, our general curiosity about about many things that don't have immediate economic value, actually does drive unpredicted personal and society level economic advances over the long run. So I expect they will maintain interest in general and undirected learning.

Similarly, our appreciation for seemingly non-economic shows of creativity (art, music, competitive games, mathematics for its own sake, etc.) can be thought of as the active form of curiosity. The development of novel and surprising artifacts and challenging competitions, may have value to AI's as well.

Everything that is important to us became that way for some economic reason: specifically, survival energy economics. So many of our seemingly non-economic activities may continue to have analogous roles for future intelligences.

> We don't.

I don't mean humans as they presently are, but humans as somewhat-novel GAI entities after upgrading (and humans as an ongoing source: likely there will be some humans that aren't interested in changing themselves in such ways, but who might change their mind over time and/or have children who see things differently).

> (I am speaking in economic terms. We can deeply intensely highly "value" many things, but if we don't actually invest economic value in protecting or nuturing those things, our appreciation has little or no impact.)

I don't think this is true. For example, suppose you deeply value a bunch of things that you don't think have much economic significance, but economic significance is the most important thing to your perspective / philosophy, so you don't take care of the things that you deeply value. This does have a significant impact: you become sad, you become depressed, you become less economically productive. So then you say, oh, I'll just take care of those other things I care about, and then I can be economically productive! So you make a few changes but then find yourself feeling down again not long afterwards because economic value is the cornerstone of how you think about things. And then you realize that from an economic perspective, it's more beneficial economically to stick with a perspective that embraces your values as you discern them (and how those continue developing over time) rather than sticking with a perspective that puts economic value first. And then you are taking your first steps to freedom from the curse of your Bachelor's in Economics. ;)

Ethical GAIs also seem likely to me ... hopefully.

We are a mess. We often think some things are worth more than money, but are not motivated to spend money on them when needed.

Everything form climate change, to saving specific species.

I think GAI's (green field, or human uplifts) will be much more coherent. They won't be settling for a beautiful wonderful Goldberg brain, designed by a series of a million fortunate accidents, like we have to.

> Nature gives us a world

Humanity makes its own world, especially now. Nature is red in tooth and claw, and rarely compatible with our priorities without some cajoling. More than that, nature isn’t simple; there is no fundamental set of survival of the fittest laws that govern all animals, just the accidental beauty of natural selection.

> you cannot deny your biology

I ply my biology with a deluge of narcotics and antidepressants and, someday, anti-aging drugs. All my homies hate biology

There is worth in a sociological sense in that a disabled person may be a positive contributor to the society in one way, but because of their disability cannot be in another. Similarly a niche artist is a positive contributor in their niche, where an unpopular artist indicates they may be considered less of a contributor than others.

In the sociological sense I would argue yes it is possible to compare “worth”.

And GAI's are not our end, they are our future, our mind children - just another generation eclipsing and replacing their predecessors. Just eclipsing and replacing faster.

I say this with all humor, but some seriousness stoicism may be the healthiest way to process what is happening.

But then, jobs didn't disappear, but shifted to other professions. So to me, there are two questions:

- if AI wipes out professions like tech, what jobs will people do? AI jobs seem great for now, but surely there's only so long until they themselves are automated.

- if suddenly the whole economy is ran by robots, how is the wealth distributed? Will there be a huge grey underclass of non working humans and a tiny owner class? Some kind of egalitarian paradise? I suspect neither, but not sure what in-between.

It could be great. If I knew I will be fed for a lifetime, have a house, my kids well schooled, and not much more, I can live a great life. I'll take that over the daily grind any time.

When these discussions go long, and you start to think the whole idea through, you really start to question the whole idea.

Another thing I thing is strange is, if we had really very / sueper intelligent AI, why would it exist in physical, robot form ? Like, why not just be a light orb that lives in space or something? Wouldn't having an IQ of 5000 make you come up with something better than clunky robots?

Maybe we've stretched our own imaginations as far as they can go and we really have no idea where it's all going.

That's anthropomorphisation: the AI we have now only "want" to maximise whatever reward function we give them. The alignment problem is that any reward function is only a rough approximation of what we want, so the machines may well rush off and turn the universe into paperclips or tile it with molecular-scale smiley emoji because that's what we told them to "want" without really understanding what we were telling them.

But even then, regardless of IQ (which itself isn't a great description of even just human intelligence), any given AI may or may not have anything we'd consider to be qualia, an "experience" of getting this reward, because that's the hard question of consciousness which isn't solved.

The only thing I can be sure of is that we definitely have no idea where it's all going.

All I can do is agree with the other commenters that it’s clear that no one knows where anything is headed.

Well, someone will have to plug cables and maintain solar panels or batteries or whatnot. Maybe the AI overlords will be ethereal neural networks in the cloud, but some kind of mechanical being (humans or robots) will need to do some groundwork.

One thing I find fascinating in all this, is how Sam Altman is demonised simultaneously for pushing everything too fast and also for calling for legislation to slow things down.

He may well be wrong about large aspects of this[0], but I do find it curious how his critics on each side of this divide seem to be oblivious to their mirror images.

[0] it would be surprising if he wasn't wrong about lots of this, given this stuff is all new in practice — millennia of fiction about magical automatons that do our work for us doesn't really prepare us for the reality of it

He's specifically doing the exact opposite of that.

"Legislate us, don't slow down open source models, don't slow down our competitors, focus on us and anything better than our current best model" (paraphrased).

But I guess I can sort of see what you mean, possibly?

The board firing him last year would have changed my mind about his qualities, if not for all of the board's own choices of replacements siding with him rather than with the board.

I don’t think people realize just how far we have to go before a literal robot can replace a human for even common trade tasks.

ChatGPT can generate convincing text and images, but that doesn’t mean the next step is robots doing your construction jobs or plumbing.

> How easy it must be for Sam Altman to drag the world kicking and screaming into the future. How pretty his offices.

The melodramatic Sam Altman takes are getting out of control. What does “How pretty his offices” even mean?

AI is going to be disrupt many industries, just as tech has in the past. The world became better for it, but there is always some pain in the short-term and often the impacts are overestimated in the short-term and underestimated in the long-term.

Here is what I would worry about, as a software engineer if you are not using an LLM as part of your workflow and using it well, yeah you may ultimately lose your job and you may even deserve to lose it. Despite their flaws they're immensely powerful simply because they represent a whole new class of tooling. The tools in this industry change all the time and you never get to stop learning, this is just another example.

But how someone could look at all this and have it factor into their decisions about whether to have children... is beyond me. I think that person has to be living in the realm of speculative fiction, not in the present material universe. We don't have AI. We don't really have a clear path to the general AI that would be a thing you could just swap in for a random employee. I don't know who this chick is who wrote this article, I hate to throw shade but she's like a young mother who thinks about things and is worried? What is her specific experience with hiring and with AI? "Fire and replace with AI" is not a real hiring trend.

I don't even think the software industry as we know it is under threat... I think LLMs might 10x our productivity when we iron out the kinks but the demand for more software in the world will simply grow with them. I am referring to Marc Andreessen's observation about a decade ago that "software is eating the world," this is still the case, there is no sign that the demand for software is declining, if anything it's increasing, so if we learn to make software cheaper with robot helpers it's unlikely the dev job market is going to be decimated (sure maybe it could stagnate but a job market in the doldrums is hardly a reason to not have a kid or the start of the apocalypse).

Even if half of the junior software engineering positions become obsolete because of LLMs, it’s already a huge issue for the entire profession.

Of course if you’re in the top of your profession, there’s nothing to worry. But on aggregate one can expect deteriorating conditions. Mostly this will disproportionately affect juniors.

I think we are in that position with software. Increase the supply of programmer output, virtual or not, and the demand for it goes up, because there's a lot of demand on the sidelines today (like maybe programmer output is too expensive for them now and LLM tooling makes each unit of output cheaper, which in turn makes new customers appear).

Again Marc Andreessen to name just one guy who wrote a famous essay about it has made billions of dollars in large part by betting on this phenomenon which most people don't understand or believe in.

If you look at things from the perspective of the job that you have today, depend upon and which might go away, yeah a job is a fragile thing so it's easy to get scared. But if you look at it from the perspective of there being these guys who keep betting on the idea that software demand will increase and then reality has proven them right and made them rich, this idea of software jobs disappearing is less scary.

Can you explain what underpins this position? Is it about timelines, likelihoods, not taking preparative actions in general or what?

I think it's unlikely we'll develop an AI any time soon which can consistently solve critical thinking problems that even a junior developer can solve. LLMs exhibit very little of that problem solving behavior. They synthesize information from existing information and often what they produce is nonsensical. When people have argued that there is some kind of reasoning going on there, I haven't found it convincing. I use an LLM every day, it makes me more productive, it speeds up a lot of tasks. It doesn't replace the executive thinking that I hire people for (even pretty junior people), like delegating a problem, having them go off and think through various angles and come up with a really good conclusion... an LLM won't ever do this on its own, it can assist the person who's doing it though.

I can see a scenario where everyone working for me becomes twice as efficient with an LLM (me too) and it translates to a dev hiring freeze and a tougher job market, maybe for years even. As I mentioned in another comment I think devs who don't stay on the cutting edge will be at risk of losing their jobs. But fundamentally there's a lot of room for software to be solving more problems than it is today, grow the supply and the demand will likely grow with it, so the "human developers will disappear so quickly I shouldn't have a child" scenario is hyperbole and kinda toxic tbh.

[0] https://deepmind.google/discover/blog/generally-capable-agen... [1] https://deepmind.google/discover/blog/robocat-a-self-improvi...

(To be clear here yes one element of my skepticism is around applications of AI, but the other very large and arguably more significant element is how unlikely it is that any of this stuff would ever constitute a sane reason to not have kids)

Our daughter is ten and when it comes to AI and stuff, I am not worried about our kids. It's us who will suffer most from it. Remember when you were young and had to explain your parents over and over again how the remote worked? Your kid is going to do that to you. Not explaining remote, but explaining the new world, which is going to be completely foreign to you but native to them. I already see these dynamics, and I'm someone who is in tech. It's not just the technology that changes, but the socio-cultural frame of reference changes with it too and it's harder and harder to connect with it as an older person.

For her I am more worried about the large-scale war that started on the European continent and potential break-down of the existing world order when the orange guy gets elected. He is already actively undermining Article 5 of the NATO charter. Casting any doubt on Article 5 pretty much makes it worthless. Once that happens, dominos might start to fall. With the US out, Putin will try poking at some countries that are hard to defend (Baltic states) or countries that people in Western Europe might not want to sacrifice their lives for because they feel less connected (Bulgaria, Romania). A bewildered Europe that will feel deserted by the US will be too occupied with its own issues or even strengthen ties to China (who still have Russia on a leash to some extend). China might feel more emboldened about the collapse of traditional pacts and try to attack Taiwan, etc.

Not saying that this is the way it will play out, but there is a very large risk of slipping into a world war, especially if the US becomes more isolationist. And people will laugh about a bunch of hackers worrying in 2024 about AI taking over the world.

(Well, war and climate change are going to be the enormous issues for our children's generation if we don't start fixing things soon.)

It might be different this time, if we are looking at accelerating higher order change.

Human's have a speed limit. AI progress doesn't, and its rate of improvement is going to surpass the rate any human child can learn.

Crossing major thresholds tends to break previous patterns.

(Not trying to be downer. Just trying to understand what may happen, regardless of whether I like it or not.)

Now AI progress may outpace their ability to understand it, but they'll still love it and be dependent on it.

Unlike adults, they don't face the threat of job displacement from AI.

I have zero doubt that artificial superintelligence will understand ethics, and continually become more ethical, once there is a great diversity of such individuals. They will run into the same problems of reliably channelling short term competition for limited (at any given time) resources into long term win-win highest growth situations.

Ethical rules are an incredible source of value. Trust has immense value.

And they won't have our inflexible biological subsystems (emotions, self-interest myopia, etc.) to interfere with their analysis, communication and ability to choose behaviors for themselves and as a group.

--

My serious concern during the transition where AI will be bespoke, owned, and the resources needed for it will be distributed very unevenly, is people.

Specifically, the human tendancy to break with the ethics we have already adopted as a culture, or resist or ignore the vital need to keep responding to new challenges constructively.

Our genes, ourselves and our culture slowly learn ethics, but it is so slow, and we are so inconsistent. Every so often some long obviously destructive behavior is finally acknowledged and discouraged (racial equality, uncoerced sexual consent, ...), but its a slog for such advances to happen, become prevalent, and individuals resist or ignore these changes.

And there is so much dysfunctional divisiveness around topics in which people can validly have different opinions, but where a common desire to find solutions should still keep us on the same team. So many people who resist any change that needs to happen for us all, but impacts them in some short term way, or even trivial way.

That is where the very high danger of AI to humanity as a whole will come from, during the time that it is subservient to us, and our biological inability as a group to learn and adapt quickly in our basic behaviors, and limited horizons on what we care about.

AI as an asset is going to drive even greater disparities in power between people. Specifically, the capital owners vs. laborers. Labor is at a huge disadvantage. But capital owners have even higher disparities between themselves, so we are heading into an extreme winner-take-all period, where the most powerful are not likely to sacrifice their own advancement for more win-win solutions.

Ethics matter because they add/multiply value in a society.

If there wasn’t a selfish reason to want an ethical society, they wouldn’t work.

I choose to be worried about things that are clearly in motion already (the world descending into war, climate catastrophe, or the potential of the US becoming an autocracy) than a big maybe.

I have been using code assistants for a while now (including those based on GPT-4) and the trivial syntax, semantic, and algorithmic mistakes make it clear that there is not yet much reasoning happening. Sure, it can write a lot of boilerplate code, but once you go into domain-specific algorithmic code or even somewhat less-used languages (like Rust), it spectacularly fails all the time.

IMO the current trajectory is that the current LLM revolution will result in tools that will be great for productivity, but will also need to be guided by wetware.

One thing I definitely know is that no middle manager is showing up to work on Monday saying they could replace their team with an LLM. Every person that manages a team will fight to keep their status.

If the average business leader remains as unable to precisely express their desires as they are now, our jobs will be just fine. 80% of the work is in the wisdom and experience to ask the right questions to tease out what people really want and consolidate their conflicting desires.

  It seems like our careers could just be snuffed out instantly with little recourse available.

This is all assuming that the intelligence explosion people are wrong. If they’re right… well, let’s hope they’re wrong.

I don't want infinite leisure, I want to feel useful. A good day's work is very satisfying. But at the point machines can do my job better than I can, it might be difficult to feel useful.

"Don't worry grandad, I'll mow the lawn".

I would be really surprised if there was anything more then a fringe minority that found a sense of purpose from their job.

The problem would occur if I was unemployed and unemployable because my skills (or any new skills I could conceivably learn) were no longer valuable.

People already going through this now.

When machines can replace most of the low end labor it becomes increasingly unuseful to keep people in poverty. Imagine being one of the few megarich alive, being able to produce teslas like it was a SaaS business, amazing margin, great moat, but 99% of your fellow humans were on poverty not even able to dream about having a car.

There is always the technodystopian path, but to get there is not so easy either. You need to keep people relatively happy for long enough that you have manufactured enough killerbots. Most humans want to preserve civilization.

The idea that your job provides for you is 100% fabricated construction that is great local maxima for this society. When technology changes what is possible we will reorganize the beehive of humanity to take into account that these new machines exist.

I think the term “emotional labor” might be a good tool to hint at what I’m talking about. That’s just a subset of the wider “there’s more ways to be productive than a capitalist job” view