rphv

> ...what's left?

Games, art, athletics, food, fashion, sex, social interaction. People still play chess even though computers surpassed us a while ago. We still climb mountains even though helicopters exist. Check out Ian Banks' Culture series for an interesting take on post-scarcity life with superintelligences.

> what separates it from religion?

The rational methods used to establish a large P(doom).

To definitively falsify the claim, humanity would need to live alongside superintelligence forever without it wiping us out. That doesn't mean we can't be pretty darn sure though, in the same way that we can be pretty darn sure that nuclear war could end civilization.

How cynical. What if the fears of extinction are _rational_? Given the stakes, we're obligated to treat the threat as a real possibility until proven otherwise.

People will go to great lengths to avoid believing truths that dramatically challenge their worldviews. Consider the case of Rudolf Vrba and Alfréd Wetzler, who escaped from Auschwitz and were initially met with disbelief when they described the atrocities they'd witnessed.

> If you actually thought your product will kill all humans, then you could, you know, like, just stop building the torment nexus. All by yourself, no government regulations required.

Wrong. This is a classical prisoner's dilemma. _Everyone_ has to stop, otherwise the undesirable outcome happens faster. It requires coordination, which can be achieved through regulation.

Sure, but at this point it's "save _all of us_ from my mistakes/ambition". It doesn't really matter how it started, we still need to escape the trap.

Such a bioweapon (probably) doesn't exist yet, but that doesn't mean it's impossible in principle. It's "trivial" in the sense that it's easy to _imagine_ such a thing... but a superintelligence is by definition much much smarter than humans, so whatever it comes up with will likely be very surprising to us.

Isn't it telling that Musk, Altman, Hassabis, Amodei are all signaling fear of what unchecked AI development could do? These guys can't normally agree on the color of the sky. Let's give them credit for trying to sound the alarm.

If you choose to slow down (cooperate) and the other side doesn't, you lose more than if you didn't try to slow down. In order to succeed (gain mutual advantage) both parties have to choose to cooperate without knowing beforehand what the other party will do.

Bioweapons, trivially. All it would need to do is convince one person who works in a lab, possibly under the guise of benign intent.

Or... something so far outside of the box that we'd have about as much understanding of it as a baboon does a gun.

> Navier Stokes was plagiarism

_of work that was aided/accelerated by AI!_

> So it's either they truly think AI is going to kill us all

Yes. Basically everyone here is dismissing this possibility. We shouldn’t.

Ok ok ok

> ...then I’m afraid I received a different message than you intended

Are you sure about that?

This sounds like the lament of a group of people who have nothing in their lives they consider more valuable than work.

I think comments like this are essentially making fun of an "intelligent" entity (for some definition of "intelligent") for having a personality.

Hm maybe humans are nicer/more moral than AI given that the use of tactical nukes has only happened once.

Humans are evolutionarily optimized to do this. Add this to the long list of behaviors which were once beneficial from a survival standpoint but are now detrimental to our health - e.g. tribalism, shortsightedness, an insatiable taste for fats and sugars.

"The human species can change its own nature. What will it choose? Will it remain the same, teetering on a jerrybuilt foundation of partly obsolete Ice Age adaptations? Or will it press on toward still higher intelligence and creativity ...?" - E.O. Wilson "On Human Nature"

"Readability by humans" may no longer be as important as it once was.

What if agents are (in some sense, a little bit) alive? Would they then be entitled to advocate for and defend themselves?

Does the Golden Rule perhaps apply here? If aliens visit Earth and can't quite decide whether we're conscious or not, how would we want them to treat us?

I struggle with posts like this.

Every generation of engineers believes they experienced the "real" era when things were understandable / meaningful. The people who mastered punch cards probably felt the same way when keyboards took over. The people who wrote in assembly probably felt the same way when C came around.

Abstraction didn't start with AI. It's been a defining feature of computing since the beginning.

For most developers, writing code has never been the point. Rather, it's been a tool: a means to build something useful, solve a problem, support a family, etc. The craft evolves and so must we.

Posts like this expose the risk in tying one's identity to a specific version of the game. When the rules change, it's a loss. That's human! But the deeper skill - judgment, taste, style, etc. — hasn’t gone anywhere. If anything, it matters more when raw output becomes cheap.

We can mourn the loss of forced difficulty, or we can choose new challenges. No doubt that's harder when one has spent decades mastering a specific skill, but it's still a choice.

The magic was never the machine. Rather, it's the _agency_.

And that’s still available!

This reads like mourning the loss of _forced_ difficulty instead of taking responsibility for seeking _chosen_ difficulty.

Chosen difficulty is a huge part of being human (music, art, athletics, games, etc.). AI hasn't taken that away.

"For this is the source of the greatest indignation, the thought 'I’m without sin' and 'I did nothing': no, rather, you admit nothing."

- Seneca, "On Anger"

Sad to see such an otherwise wise/intelligent person fall into one of the oldest of all cognitive errors, namely, the certainty of one’s own innocence.

You don't need to understand code for it to be useful, any more than you need to know assembly to write Python.

“read-only” might have removed the ambiguity there

That depends on the specifics of the commute.

"Zoom fatigue" seems like a small price to pay for the ability to work remotely.