afthonos
afthonos on Hacker News
They weren’t enough to impress others, but I think they were enough to feel impressive, which is almost as pleasant psychologically.
I think, without trying to disparage GP at all, that unironic watching is very hard for the generations that came after the 90s. Some gems shone through, like the LoTR movies, which managed to be unironic
and good, but the fresh air of “Joss Whedon dialogue”, pointing out storytelling conceits as they were happening live on your screen, let everyone feel smart right when “smart” became the only thing worth being.
AI destroying, at alarming speeds, all the bastions of “smart” I think will lead back to a less ironic mood. “I’m so smart” does not impress people who can reproduce your brilliance by asking Claude for “insights, no mistakes.”
We shall never look upon her like again.
—Sent from Dollywood
Still waiting for Boeing to catch up to this marketing renaissance. They are probably salivating at their projected valuation when they start saying things like “our planes could crash intentionally to kill everyone on board” or “our bombers might decide to blow up your cities instead of your enemy’s!”
Imagine the fortune food supply companies will make! “Our lettuce just might poison your child!”
Truly a golden age is near, and you, perceptive geniuses that you all are, will be the only ones not fooled, not for a second, because it all makes sense, always and forever, as long as you squint hard enough to close your eyes.
No, green cards actually remove that constraint. You are a permanent resident, and you’re allowed to live in the US permanently. Irrespective of employment.
Relatedly, I think where I land is that the (current) challenge for humans is creating such oracles. AIs have patience and compute we can only dream of—I think someone estimated 100,000 person-years were spent on Navier-Stokes—but they still need a “reality” to measure against.
Even if you’re right, that doesn’t mean AI can’t get better than humans at everything.
Yes, and that’s bad, but not as bad as training against the chain-of-thought. If you avoid pressuring the chain of thought, you reward methods of achieving goals that don’t care about being illegible; if a model then
wants to be illegible, it may have to use methods that are not rewarded in its training, which is harder. Think figuring out how to do opsec on the fly, or even from reading books, rather than have someone tell you every time you screw it up.
Of course, this leaves the possibility that the best methods for solving generic problems obfuscate the chain-of-thought. That would be unfortunate.
Not quite; training against the chain-of-thought is the Most Forbidden Technique, because it might teach models to obfuscate the it. The point of avoiding that, though, is to ensure the chain-of-thought can be usefully read (and, done carefully, monitored).
“Our virtuous lobbying, their heinous punishing of political enemies”?
Are you making an ontological case, or a factual case? In other words, would anything be rogue AI in your mind?
The problem is that the world has spawned the insane take that
because they didn't use the most basic safety protocol imaginable, they were clearly only testing a hot air balloon, and hot air balloons
obviously destroy preschools in giant explosions, nothing to see here.
"If they believe what they say they were incompetent" -> absolutely true statement.
"They were incompetent, therefore they didn't believe what they said" -> Sir, I'd like to introduce you to human beings, you may not have met one before.
Unlike communicating via glass panels processing information at near-lightspeed, relayed by fiber optic cables laid down across thousands of miles of ocean floor, processed in datacenters filled to the brim with transistors etched at atom-scale.
You better start believing in science fiction; you’re surrounded by it.
I notice a giant leap between “hacking and exfiltrating” and ”making public”. Why would the AI do that for you? Are you just that charming?
I agree on complicated ideas, and on facts (because obviously different sources will selectively report facts, even if they’re trying to be fair).
The ideas here are not complicated. The argument is three sentences.
No one is saying we have it, they’re saying they’re building it.
And what mechanism to control these drones, which are autonomous or remote-controlled machines, do you imagine that humans will be able to use, but AI won’t?
And what in the history of the past five years, gives you confidence that not a single human will give the AI the access it needs to do bad things? Incompetence alone is sufficient to kill us, let alone humans that genuinely, or confusedly, want bad things.
It’s an excuse not to think, not an argument.
The doomer argument is straightforward: a powerful AI will have goals. If those goals don’t include the welfare of humanity, a sufficiently powerful AI will kill us either by accident, by indifference, or intentionally when we get in its way. Making an AI that includes the welfare of humanity in its goals is incredibly hard.
That’s it. To refute the doomer argument, you can refute any part of that. I notice that none of the people bringing up fanfics, cults, and weirdness are doing that. No, they’re instead trying to convince people that doomers are uncool. Since only cool people can know things, QED.
I feel your frustration, but it’s just normal human behavior, no need for a psyop. A lot of people are emotionally invested in AI being a fad, useless, easily controlled, a capitalist boondoggle, a circular financing scam, a bubble about to explode. The alternative is that we live in an incredibly dangerous time, because
obviously if we’re building something smarter than ourselves, we’re building the kitchen where we’ll be cooked.
So you get the normal reactions:
1. Nothing that happens is ever surprising. Navier-Stokes was stolen, HuggingFace was pedestrian and normal, and coding agents that last year could barely write a function and now can do a day’s worth of work in ten minutes will never be able to do anything more, because this is the end of history.
2. The people that say otherwise are lying. Because it’ll make them rich (although they are already rich). Because they are a cult (although regular people asked if we should build machines that are better than us at everything have the same reaction). Because they are marketing geniuses (although suspiciously few companies advertise the lethality of their products).
3. The people who’ve been warning about this the longest are weird and in a sex cult. (Except Alan Turing.) (Except Geoffrey Hinton.) (Except Stephen Hawking.) (You are in this group. You must be, otherwise they’d have to listen to you.)
I do hope they’re right. But their song will never change, and when some misaligned AI kills thousands, they will say that of course AIs were always going to kill thousands, but did you know the guy who told us so also writes fanfic?
Predicting the next word in context led to Open AI losing control of one of its research clusters. I’m not sure what “just” is doing in that sentence; most of my regression analyses don’t do that.
It’s not, that’s definitely one path to doom. But everyone racing as fast as possible is another, far surer, path to doom. We should avoid both.
Yes, one part of the argument was that it would be hard to explain human values (and while it seemingly turned out easier than I thought as well, it’s hard to check for sure). An equally important part was that the AI wouldn’t care about them.
Agreed. Ellison’s path to that future went through AI.
It is always money — but it isn’t always only money. They are not asking for anything that will prevent them from making money in the future, but they are asking for help stopping the runaway train they’re on. These are compatible requests.
You are
incorrectly cynical. They are telling you things are bad, and because you refuse to countenance they could be worse, you assume they must be better to comply with your mandate to disbelieve.
A true cynic looks at the statements by the AI labs, assumes things are worse because the labs want to seem better than they truly are. And it takes a special kind of mass delusion to drive a sane person to think “AI is completely under our control” is worse than “AI could kill everyone.”
Important to note that KataGo was double-handicapped. 20 seconds per move maximum; it couldn’t read deep. Against an amateur, it doesn’t matter, but against a historically strong pro it matters a lot.
Anthropomorphizing helps to create a lower bound for damage. If you can imagine a bad person doing it, AI will be
at least that bad, unless proven otherwise. I think referring to AI as a tool obscures that, because we are not used to tools (especially the ones we use daily) taking catastrophic actions.
Example: would a sufficiently motivated human break into a website to steal something they want? Yes, obviously, happens all the time. Ok, you should expect AIs to do that.
Example: would a sufficiently motivated nail-gun steal nails from the local hardware store to finish the job? Uh…that’s not even coherent.
Anthropomorphizing helps people get over the conceptual barrier. It’s wrong, but it’s usefully wrong; “it’s just a tool” is not.
Once you’re over the barrier, anthropomorphizing starts to become dangerously wrong: “I talked to Claude, Claude’s cool, Claude would never go and hack the website.”—-bzzt, wrong, your intuition failed you. But the solution is not to fall back on the tool framing; that one is still wrong.
I think we may be using different words to describe the same concept. You think of it as them not being “people“, and I think of it as them not being “aligned“. But fundamentally, the problem is they are entities that take unpredictable actions that that their creators and their users are not OK with.
The only place where I think we might still disagree is whether it’s possible to understand the tool. My position is that, at our current level, it’s not. And that the more advanced they get, the less possible it will be.
The dead astronauts were relieved to have been killed by something incapable of malice. As I’m sure will we.