AI is just software, nothing new to see here.

AI safety is primarily a sandboxing problem.

There is no collective action problem, and every company should just slow down if they think they need to slow down.

There is no need for regulation because the existing incentives in the market keep companies from acting badly, which is why no company has ever done anything bad.

If something bad does happen, then we can regulate after the fact.

We will end up creating more jobs than we destroy, so don't worry about it.

Our kids might forget a whole bunch of stuff or never learn it in the first place. But don't worry, they'll come up with new things to learn instead.

The only way to get safety is to move faster because then we will more quickly arrive at safety.

Recursive self-improvement is just what we've always done.

The real danger is alarmism that might scare the public and the young people.

Nothing bad can happen, it can only good happen.

I don't know. Astroturf campaigns to subvert data center construction seem to have had an impact, for example, but ultimately I can only fall back on vague vibes picked up from the news and social media. I don't know the specific success rate.

Some of those campaigns seem justified to me (why in the world do these things have to be built in populated areas?) while other arguments made by the anti-data center movement are certainly propagated in bad faith (Water usage? Really?)

Seems like we're barreling forward, even though some of the FUD spreaders appear to be in charge of the main companies.

That's just so much kayfabe. Don't listen to what they say, watch what they do.

Do you, like Huang, see absolutely no risk in our current approach? Is everyone who disagrees with your perspective on this subject just plain wrong? Misguided, or actively malicious/in bad faith?

I can't speak for Huang as I don't know him personally, but he has to walk a tightrope between an infantilized POTUS with essentially unlimited power to wreck his business, and his own stockholders who have elevated his company's market cap to a level that probably doesn't make sense even to him. As far as I can tell he's just going the obvious, predictable, and safe route of trying to sell as many shovels and tents as he can while the selling is good.

I see a bigger downside in the precautionary principle than in accelerationism. We are going to face immense societal disruption due to AI over the next couple of decades. That is as unavoidable as an earthquake. It is going to happen regardless of who prevails in the political and social arguments going on today. There is no possible way that every actor in every country on Earth is going to be convinced or compelled to "pace the frontier." Consequently, all that matters is how we react, adapt, and take advantage of the new ground rules of reality.

As usual, there will be those who thrive and those who get steamrolled. But I see no reason why we won't all end up better off in the end, as with the rest of the history of technology since the invention of fire.

sounds like, based on your next-to-last paragraph, he’ll be good on this for next couple of decades

GP might have been thinking of things like the inventions of the illustrious Thomas Midgley Jr. and his vigorous promotion of tetraethyl lead, etc.

Yep, that would be a much better example. Defending Midgley is a task for superintelligent AI if there ever was one.

Still, the underlying technology is not "leaded gasoline" but "internal combustion engines." The human condition would certainly be better if you could erase Midgley from history, along with the people who invented shoe-store fluoroscopes and radium paint, but it would be worse if you erased Roentgen, Otto and Diesel. Horses weren't the global-max of mobility and human brains aren't the apotheosis of reason.

Who gives a fuck if some techno barons can deploy novel software at the drop of a hat if it means I lose my career in an environment without social safety nets.

It’s not progress from my perspective when my life quality regresses.

AI is the ultimate social safety net, if you'll just get out of the way of the people weaving it.

I see company doing layoffs left and right because they can do the same or more with less employees. I’m not seeing any companies feverishly hiring engineers because the productivity per hour of labor is higher.

There is also no guarantee that AI will lead to any social safety net, they are orthogonal concepts.

The only thing that would lead to a social safety net is political will and currently all the political will behind the technology firms building AI appears to be firmly behind dismantling democracy and becoming oligarchical leaders of their little fiefdoms while the rest of us can get fucked.

Better if you don't need saving at all, because you've been empowered to help yourself at your job. Ultimately the goal is to free you entirely from dependence on a day-to-day occupation.

You may have noticed that no one else is even trying to help.

Don’t hint at it. Lay out the mechanics of how you think AI is going to become a safety net for the average person.

Mechanically how is AI going to lead to a social safety net? You can’t just handwave the productivity increases when they are all owned by a handful of concentrated wealth

After this incarnation of the Industrial Revolution, those white-collar jobs will be mostly gone, and nothing will replace them. I personally don't think CEOs and politicians will end up being any more immune than junior programmers. New forms of organization will have to emerge on both the labor and management sides. (Democracy at the nation-state level is pretty much toast anyway, so something else is going to have to replace it, AI or no AI.)

Blue-collar labor will eventually be subsumed by robotics, but that will take longer. Things like UBI will make more sense in this environment than they do now, and I think that'll be a big part of it. Not to say that all gains will be equally distributed, or that they should be, but I don't things can get much worse than where they are headed today with regard to the tilted economic playing field.

The worst case scenario would indeed be if all the gains get concentrated and centralized, as the major AI labs seem to want, but I don't think it'll work out that way.

Youve laid out an idealistic view of how the world will work. You have not laid out how the transition between the current world where wealth is being concentrated somehow turns into UBI.

Even this sentence

> I personally don't think CEOs and politicians will end up being any more immune than junior programmers.

Implies that you think the capitalists who own these systems are going to be in charge since CEOs work for their boards and politicians largely follow their donors.

Again, lay out exactly how you think we move from our current system to UBI via AI technology.

>but I don't think it'll work out that way.

Is not you laying out how you think it will work, but a dogmatic religious belief with no evidence or logic behind it.

edit: Im sorry, I glossed over this sentence.

> Democracy at the nation-state level is pretty much toast anyway...

Are you a rationalist, e/acc, or someone related to TESCREAL, or especially anything touching on Curtis Yarvin? If so no need to answer, I think you guys are in a cult with the modern equivalent of Dianetics, so we are never going to see eye to eye.

Nor have you laid out how traditional political and social structures are going to save your job (or anyone else's) in the long run. You say you're "Angry and opinionated." We've seen the anger part, so let's hear some opinions.

Are you a rationalist, e/acc, or someone related to TESCREAL, or especially anything touching on Curtis Yarvin? If so no need to answer, I think you guys are in a cult with the modern equivalent of Dianetics, so we are never going to see eye to eye.

I'm e/acc-curious, you could say. Lots of philosophical overlap but I'm not much of a joiner. Not sure who the rationalists or TESCREAL are.

I know who Yarvin is, though, and would generally take the opposite side of any argument with him. If there's a "cult" in play here, it's the voters who were duped into electing and then re-electing Trump. Democracy simply cannot survive if it continues to grant stupid people the same political power as smart people. The stupid people will be herded to the polls by unethical smart people like Yarvin, using the power of mass media and social media as we've seen.

Meanwhile, the remaining smart people get along together like cats in a sack, ensuring they have only sporadic political influence that is bound to diminish further over time.

At the same time, democracy cannot retain the support of the people if it disenfranchises voters on grounds of education or intelligence. That leaves one obvious conclusion: that it won't survive. How AI will get us past that, I have no idea. But if you want to predict the future you're going to have to account for both the rise of AI and the fall of democracy.

I never stated that they would. That’s an opinion you manifested from thin air because I stated that I don’t think AI is going to lead to a social safety net on its own.

I think things are generally going to get worse for the average person with AI as all the benefits accumulate at the top.

Assuming the rise of AI and the fall of democracy doesn’t have a default mode of social safety nets, and we’re seeing leadership increasingly divorced from the will of the people already.

For reference on TESCREAL

https://en.wikipedia.org/wiki/TESCREAL

The destroyed jobs and the persons doing them will be different from the new ones created..

>If something bad does happen, then we can regulate after the fact.

How does regulating after the fact undo the harms caused?

> they'll come up with new things to learn instead.

Like they forgot how to communicate face to face, but have learned to communicate via social media?

How many of them can read x64 or ARM assembly emitted by their compilers?

How many of them will ever need to?

There's your answer.

It’s how I deeply understand what a RAM lookup vs having it already in a register means for optimization.

These are things you need to know if you want to work on high performance applications or in limited embedded systems.

So, yes, many of us need to and it’s important we keep teaching it to future students.

Me, too, but we should both understand what we're talking about: a hobby.

It's like teaching cursive to schoolkids. All well and good, but don't you dare complain about limited classroom time for instruction in other, more important subjects.

I built and deployed embedded systems robotics that protect water and oil pipeline infrastructure that costs billions of dollars if there is a failure.

That is not a hobby, that’s a career.

The tools I built just helped fix the fresh water access for my entire city of 2 million people. Explain to me what’s more important than people having access to water.

Humans put the tool in the pipe and analyse the data. This isn't newfangled shiny tech, it's old stuff designed and maintained from a decade ago.

There's no fancy AI, no magical automated robots. Just humans using EM tools to scan pipes.

This is a category error. LLMs are probabilistic. The ones run by an AI company over API, even more so.

Compilers are not.

Nobody cares. Deal with it and get over it.

>Nobody cares.

That is a different matter. And you are probably wrong here as well. Plenty of people care about a program doing exactly what they want. Nothing more, nothing less. If not every one would have been writing programs in Ruby (not that Ruby is non-deterministic) .

Get over that camperbob2

When I say that nobody cares about determinism, that's what I mean. Determinism is indeed important, but only at the delivery level. There are many routes to achieving it, none of which require you to write low-level code yourself. If you insist on doing that, you have a hobby, not a profession.

What exactly do you mean by this? Can you give an example..

My workflow used to be:

    1. Draft rough spec
    2. Write a bunch of C/C++ code
    3. Test manually

    1. Put some actual care into a detailed spec
    2. Write equally detailed test harness spec
    3. Hand both specs to clanker.  Surf HN for a while
    4. Review target code casually
    5. Review test harness very carefully
    6. Run test(s) manually (or, lately, get the clanker to do that too)
    7. Iterate if necessary, going back to step 1, 2, or 3 as appropriate

The analogy I like to use is Harold Black's work in the 1930s, trying to convince the patent office and his peers that yes, negative feedback is a huge, huge F'ing deal, because it only takes a small amount to make a large improvement in linearity. Anytime you have something that is 95% as good as it needs to be, it will be good enough if you can wrap a loop around it.

Lol, you are in for a world of pain when you run out of prompts!

But you can always throw the entire program, and start from scratch...You have every behavior captured in tests, right?...right?

Not necessarily. I'm still learning how to do that, just like the rest of the industry will have to. TDD in the age of AI is a skill that will emerge from disciplined practice as well as a lot of trial-and-error.

As for digging in manually, right now I still have to keep a pretty close eye on the C code the clankers emit, but then I'm old enough to remember when we had to debug compiled C code by stepping through x86 opcodes in Turbo Debugger. That need went away, frankly sooner than I thought it would. I almost never need to look at x64 assembly anymore, and I can't sight-read ARM at all. The same thing will happen with what we call high-level languages today.

At some point last year I noticed that I was finding fewer bugs in AI-generated parts of the code than the AI was finding in my own, and that trend clearly isn't going to reverse itself. When the clanker screws something up, it's usually because I gave it flawed or incomplete instructions, not because it's incapable of doing the job right.

Not always, but usually.

Wait, how do you debug now? Do you mean because of LLM assistance?

2008 triggered a glut of cheap experienced workers that slowly reengaged juniors but this time that the entire ladder is going to be more valuable than any grad.

This is an introductory programming course, designed for non-CS students e.g. engineers and scientists.

> Learning outcomes

> 2. Explain key concepts in AI-assisted programming, including Large Language Models (LLMs), prompting, problem decomposition, and top-down design.

> 3. Apply the workflow of AI-assisted programming and prompt-engineering techniques to guide and improve code generated by AI assistants.

This course used to be non-AI (last year), and they rewrote recently to incorporate AI tools, as they realised the writing's on the wall for non-programmers.

It must be quite challenging to write curriculum when the underlying technology (AI) is changing so quickly.

It is so, so easy to pick up LLM aided development.

What is way harder is self-managed slop mitigation which is only achieved with employees who know the A,B,C's of software development.

Jensen gets it!

Related:

https://en.wikipedia.org/wiki/Coupling_(computer_programming...

https://en.wikipedia.org/wiki/Abstraction_layer

https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...

https://en.wikipedia.org/wiki/Tower_of_Babel

https://en.wikipedia.org/wiki/Prat%C4%ABtyasamutp%C4%81da

Jensen Huang has been CEO of NVIDIA for 33 years. That is a role with a very specific type of information environment. He is sort of this weird combination of specialist and - he necessarily has to operate within a certain level of abstraction.

I’m somewhere in the middle, I’m a high achiever but not the highest. I would say I’m above average in my usage of AI at my tech job. The way I’ve learned systems thinking is by being a bit non-specific in what I learn. British history, psychology, software engineering, queuing theory, cooking.

The idea of learning systems but not basic math - the idea of being too discerning in what I’m willing to learn. The entire idea of passing up the ability to learn something like basic math.

So many mental models of the world are developed by engaging with things like basic math. How do you learn systems without learning patterns behind numbers?

If the whole argument is something like it’s now about taste or creativity or being a builder? The way you learn those skills is engagement with all the things. It’s not abandoning all the things to read a book on systems thinking and product management.

It’s not never focus, but if your default position is “maybe I shouldn’t be curious about that”. You’re operating from a deficit.