News for hackers. Stuff that matters.
Re:Anthropic IPO by Loquebantur on Thursday November 12, 2026 @12:15PM (#50470702)
When you have AI models autonomously self-improve, you implicitly lose your understanding of how they work and accordingly your supposed control of them?

AI can be a "tool", but only so long as you understand how it works and what the implications of its use are. Being an enormously powerful tool implies, the consequences of not understanding and not knowing are proportionally enormous as well.

Currently, LLMs are used practically exclusively by "professionals" who, more or less, know what they're doing. The idea of applying them "everywhere" completely misses the gap in understanding and responsibility for the vast majority.

Re:Anthropic IPO by bakugo on Thursday November 12, 2026 @12:18PM (#50470746)
[delayed]
Re:Anthropic IPO by Loquebantur on Thursday November 12, 2026 @12:25PM (#50470861)
The domains where they apply it, privately for instance, aren't of immediate import. Their fuck-ups stay local.

The consequences of "AI use" are manifold and very poorly understood. That doesn't make them less impactful, on the contrary.

Whether you look at "humanoid robots" or incorporeal LLMs, when they are more capable than yourself, you become their dependent counterpart, not their "master".

Re:Anthropic IPO by davidsainez on Thursday November 12, 2026 @12:30PM (#50470937)
Source? AFAIU most of Anthropic's revenue comes from enterprise usage, who presumably have an incentive not to generate pure, unvalidated slop

https://www.tradingkey.com/analysis/stocks/us-stocks/2623001...

Re:Anthropic IPO by mcmcmc on Thursday November 12, 2026 @12:27PM (#50470887)
[delayed]
Re:Anthropic IPO by SoftTalker on Thursday November 12, 2026 @01:08PM (#50471525)
Nice analogy. We know what the end result of people worshiping false gods is.
Re:Anthropic IPO by Loquebantur on Thursday November 12, 2026 @01:08PM (#50471526)
Don't you have one already? What improvement would that artificial one be and to whom?

Anthropic desribes itself as a "public benefit corporation" in its IPO. The irony. Or is it sarcasm?

A god isn't your subordinate. You cannot "program" your wishes into it, like a genie/jinn. They (or their investors) want to make themselves into gods. Over you.

Re:Anthropic IPO by tim333 on Thursday November 12, 2026 @01:18PM (#50471682)
I've often heard skeptics say AI enthusiasts think they are building a god but I've never seen any evidence. Are there any quotes implying Anthropic think they are building a god?
Re:Anthropic IPO by 10111001 on Thursday November 12, 2026 @01:53PM (#50472274)
> I've often heard skeptics say AI enthusiasts think they are building a god but I've never seen any evidence. Are there any quotes implying Anthropic think they are building a god?

Not Anthropic specifically, but there's this thing that I recently read:

https://www.nytimes.com/2026/11/09/opinion/ai-anthropic-doom...:

> A.I. company leaders often talk about how their work might lead to human extinction soon. This is astonishing. Even if you wanted to destroy humanity, you wouldn’t say so. If, however, you wanted to save the human species, wouldn’t you stop developing this technology?

> There is an explanation for why some tech leaders, so seemingly worried about an extinction-level event, are plunging ahead with A.I. development: Mankind’s survival, while important to many of them, is less critical than building something they believe to be not only conscious but godlike. A substantial portion of the A.I. community, in celebrating this belief, is quick to support any action or statement that reinforces the idea that the technology is like a deity. They might call for regulation, but only in specific ways — perhaps by deploying more A.I. to control A.I. — that adore the technology’s powers. It is that adoration that is nonnegotiable.

It's by the guy who apparently invented VR and was apparently mentored by Marvin Minsky. So he likely has the kind of access that gives his opinions credibility.

Re:Anthropic IPO by tim333 on Thursday November 12, 2026 @03:08PM (#50473421)
Thanks, although that still sounds a bit like a skeptical viewpoint.
Re:Anthropic IPO by 10111001 on Thursday November 12, 2026 @03:50PM (#50474072)
> Thanks, although that still sounds a bit like a skeptical viewpoint.

I think it's unlikely that an AI enthusiasts would literally claim "they are building a god," but such an attitude (or something very similar) is implied by their statements and attitudes, e.g.

1. Talk/hope of building something that so exceeds human capability, it's beyond human comprehension and control.

2. Taking existential risks for humanity (by their own statements) to try to get there, against all reason.

3. etc.

Re:Anthropic IPO by tim333 on Thursday November 12, 2026 @04:03PM (#50474241)
I guess. Anthropic's about page is more "We aim to build frontier AI systems that are reliable, interpretable, and steerable".

I guess they think AI will become smarter than people but I'm not sure that's a god.

Re:Anthropic IPO by sillysaurusx on Thursday November 12, 2026 @03:14PM (#50473498)
(Archive link: https://archive.is/SSgkC)
Re:Anthropic IPO by ben_w on Thursday November 12, 2026 @01:28PM (#50471880)
> When you have AI models autonomously self-improve, you implicitly lose your understanding of how they work and accordingly your supposed control of them?

This was always the case with ML, even absent RSI. Big bag of weights is a pain to figure out exactly why it can do whatever it is that it does, leading to all kinds of blind spots about how they go wrong.

> Currently, LLMs are used practically exclusively by "professionals" who, more or less, know what they're doing.

Even the people who made them don't really know what they're doing, hence https://www.felonybench.com