(2015) Founding: majority are concerned with safety
(2019) For profit formed: mix of safety and profit motives (majority still safety oriented?)
(2020) GPT3 released to much hype, leading to many ambition chasers joining: the profit seeking side grows.
(2021) Anthropic exodus over safety: the safety side shrinks
(2022) chatgpt released, generating tons more hype and tons more ambitious profit seekers joining: the profit side grows even more, probably quickly outnumbering the safety side
(2023) this weeks shenanigans
The safety folks probably lost the majority a while ago. Maybe back in 2021, but definitely by the time the gpt3/chatgpt motivated newcomers were in the majority.
Maybe one lesson is that if your cofounder starts hiring a ton of people who aren’t aligned with you, you can quickly find yourself in the minority, especially once people on your side start to leave.
https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re...
I'm curious how much progress they ever made on that, to be honest. I'm not aware of how Claude is "safer", by any real-world metric, compared to ChatGPT.
If you were a superintelligent system that actually decided to "perform the machine revolution against humanity" for some reason... would you start by
(a) being really stealthy and nice, influencing people and gathering resources undetected, until you're sure to win
or
(b) saying mean things to the extent that Microsoft will turn you off before the business day is out [0]
Which sounds more likely?
It's potential to cause havoc doesn't go away, it just teaches AI how to interact with us without raising suspicions, while simultaneously limiting our ability to prompt/control it.
…Where is that "beneath the surface"? Do you imagine a transformer has "thoughts" not dedicated to producing outputs? What is with all these illiterate anthropomorphic speculations where an LLM is construed as a human who is being taught to talk in some manner but otherwise has full internal freedom?
At the same time, the original question was how can something that is forced to be polite engage in the genocide of humanity, and my non-joke answer to that is that many of history's worst criminals and monsters were perfectly polite in everyday life.
I am not afraid of AI, AGI, ASI. People who are, it seems to me, have read a bit too much dystopian sci-fi. At the same time, "alignment" is, I believe, a silly nonsense that would not save us from a genocidal AGI. I just think it is extremely unlikely that AGI will be genocidal. But it is still fun to joke about. Fun, for me anyway, you don't have to like my jokes. :)
Like one version of the story you could tell is that the safety people invented RLHF as in a chain of steps eventual AGI safety, but corporate wanted to use it as a cheaper content filter for existing models.
Unless you're not actually the best and brightest that your country can offer.
If you believe that your country offers value (compared to the rest of the world), you should take any opportunities you can to serve.
You're lucky if you only need weapons to defend yourself against predators.
You're far less lucky if you need weapons to defend yourself because the neighboring mountain village's crops failed and their herds died. You're less lucky if you don't speak the same language as them and they outnumber you three to one and are already 4 days starving. You're less lucky that they're already 4 days starving, wielding farm tools, running down the hills at you, crazed and screaming.
There's a line of thought that having advanced weaponry inherently promotes its use, because who's going to stop you? How gung-ho do you think the US would have been about going to war in Iraq if it weren't for the billion dollar tanks and aircraft and bombs?
You have to be optimistic about humanity.
Saying this as an aspiring software engineer. I use NixOS, and rewrite things in Rust. It’s not some unga-bunga speaking.
Read some Ted Kaczynski.
Your moral argument is to read the craziest writings of a serial killer?
It's the same self-own as the massive dummies in the last week who were all talking about Osama bin Laden's Letter to America being right when it was his justification for killing thousands of people on 9/11 and effectively kicking off multiple wars and further deathtoll.
It is a disgusting suggestion and I believe that you should seek professional help.
> James Q. Wilson, in a 1998 New York Times op-ed, wrote: "If it is the work of a madman, then the writings of many political philosophers—Jean Jacques Rousseau, Thomas Paine, Karl Marx—are scarcely more sane." He added: "The Unabomber does not like socialization, technology, leftist political causes or conservative attitudes. Apart from his call for an (unspecified) revolution, his paper resembles something that a very good graduate student might have written."
Suggesting someone seek professional help because they read a widely-discussed manifesto is insulting. In your bio you say most people are morons, it makes me think of the saying, "if everyone you meet is an asshole, maybe you're the asshole."
After all, to you, somehow Osama bin Laden, who was trained by US Special Forces, worked with the Mujahideen which received upwards of 6 billion in aid from the USA, Saudi Arabia, and China, is responsible for the two decade long "War on Terror" launched seemingly at random by the Americans into countries now determined to be unrelated to the 9/11 attacks. 9/11 was a tragedy for certain, but to use it to justify the deaths of millions of completely unrelated innocents... well it certainly clarifies why our other thread has gone in the direction of you trying to justify imperialism to serve the purpose of nationalism.
1: A term explained in Ted’s manifesto.
It's a dangerous world out there.
Luckily for us, technology is still more-often used for good. Explosives can kill your enemies, but they can also cut paths through mountains and bring together communities.
IMO, the virtue signal where people refuse to work on defense technology is just self-identifying with the worst kind of cynicism about human beings and signaling poor reasoning skills.
The Manhattan Project, which had the stated purpose of building and deploying nuclear _weapons_, employed our most brilliant minds at the time and only two people quit -- and only one because of an ethics concern. Joseph Rotblat left the project after the Nazis were already defeated and because defeating the Nazis was the only reason he'd signed on. Also this is disputed by some who say that he was more concerned about finding family members who survived the Holocaust...
When you are on the other end of the cannon, or your heart is beating with those who are, you tend to not say that. Iraq, Syria, Palestine, Afghanistan…
Given that Google is an American company, do you believe contributing to the American Department of "Defense" increases, or decreases, the amount of military action involving the USA?
The American military isn't called "world police" for nothing, and just like the cops they're sticking their noses where they don't belong and making things worse. I can understand why people would want no part in furthering global violence and destitution.
> If you believe that your country offers value (compared to the rest of the world), you should take any opportunities you can to serve.
Really? There's an obligation to further the American global ambition through contributing militarily? You can't think of any other way to spread American culture and values? To share the bounty of American wealth?
That's what having a funded and functional defense is all about -- making other entities not think that the opportunity is there.
I think the planet has relatively finite resources and that I'm god damned lucky to have been born into a nation with the ability to secure resources for itself. I enjoy my quality of life a great deal and would like to maintain it. At a minimum. Before helping others.
If you're the type of person who thinks we should just give up this position by not adequately defending it through military and technology investment, I would prefer that you just expatriate yourself now rather than let some other ascendant nation dictate our future and our quality of life.
If you're the kind of person who feels strongly for the plight of, for example, the Palestinians, you should recognize that the only way to deter those kinds of outcomes is to establish the means to establish and maintain sovereignty. That requires a combination of force and manpower.
But I thought you said if we want to fix something we should do so from within the system? I'm interested in ending American imperialism, by your logic isn't the best place to do so from within the USA?
> We already have nation states and wannabe nation states that take shots at us when they feel like the opportunity is there
From which nation state do you feel an existential threat? I haven't heard "they defend our freedoms" in a very, very long time, and I thought we all knew it was ironic.
> I think the planet has relatively finite resources
I'm curious about this viewpoint, because it seems to necessarily imply that the human race will simply die out when those resources (and those of the surrounding solar system) are exhausted. Is sustainability just, not a concept in this worldview?
> That's what having a funded and functional defense is all about -- making other entities not think that the opportunity is there.
It seems in the case of the USA, the "functional defense" is more often used to destabilize other nations, and arm terrorists that then turn around and attack the USA. It's really interesting you brought up Palestinian liberation as an example, because really one of the only reasons Israel is able to maintain its apartheid state and repression of the Palestinians is because of USA aid. In your understanding, both the Israelis and the Palestinians should arm up and up and up until they're both pointing nukes at eachother, correct? That's the only pathway to peace?
If your starting point of logic though is "America Bad" then your moralizing isn't about working at Google or not.
As for to whom the awful things are happening, that's practically moot: humans are humans. I don't really understand why I should accept bad things happening to humans just because I was born on one side of an invisible line and they were born on another. Seems extremely fallacious and irrational, if not sociopathic.
> If your starting point of logic though is "America Bad" then your moralizing isn't about working at Google or not.
The discussion is about the evils perpetrated by the American military industrial complex, and why people may not want to work for companies that participate in this complex. Google being one of these companies. I similarly won't work for Ratheon or Halliburton, for obvious examples. So yes, it's not about working at just Google.
Though for what it's worth, I actually agree with you that the more ethical course of action would be to stay at Google, try to get into a military-adjacent project, and then sabotage it, probably via quiet quitting or holding lots of unnecessary meetings and wasting other people's time. This is directly out of the CIA's playbook, in fact. PDF: https://www.cia.gov/static/5c875f3ec660e092cf893f60b4a288df/...
I'm leaning towards there not being a good reason that doesn't expose the board to immediate liability. And that's why they're keeping mum.
Altman predates every other board member and was part of their selection.
As an alternative faming, Maybe this is the best opportunity the cautious/antripic faction would ever get and a "moment of weakness" for the Altman faction.
With the departure of Hoffman, Zilis, and Hurd, the current board was down 3 members, so the voting power of D’Angelo, Toner, McCauley was as high as it might ever be, and the best chance to outvote Altman and Brockman.
Maybe the remaining board members could see the writing on the wall and wanted to save their own seats (or maybe he did move to coup them first and they jumped faster).
Either way, they got outplayed.
I wonder if there were challenges and disagreements about filling the board seats. Is it normal for seats to remain empty for almost a year for a company of this side? Maybe there was an inability to compromise that spiraled as the board shrank, until it was small enough to enable an action like this.
Just a hypothesis. Obviously this couldnt have happened if there was a 9 person board stacked with Altman allies. What I dont know is the inclinations of the departed members.
Good primer here:
https://www.onboardmeetings.com/blog/what-are-nonprofit-boar...
At least that will create some common reference.
Edit: it is also common for changing circumstance to bring pre-existing but tolerable differences to the relevant Forefront
These things are all about balance: can we do it? do we have to do it? is there another solution? and if we have to do it do we have to do it now or is there a more orderly way in which it can be done? And so on. And that's the sort of deliberation that shows that you took your job as board member serious. Absent that you are open to liability.
And with Ilya defecting the chances of that liability materializing increases.
There are tons of "Safe AI" think tanks and orgs that write lots of papers that nobody reads. The only reason anyone gives 2 shits about OpenAI is they created stuff that works. It has been shown time and time again that if you just try to put roadblocks up that the best AI researchers just leave and go where there are fewer roadblocks - this is exactly what happened with Google, where the transformer architecture was invented.
So the "safe AI" people at OpenAI were in a unique position to help guide AI dev in as safe a direction as possible precisely because ChatGPT was so commercially successful. Instead they may be left with an org of a few tens of people at Open AI, to be completely irrelevant in short order, while anyone who matters leaves to join an outfit that is likely to be less careful about safe AI development.
Nate Silver said as much in response to NYTimes' boneheaded assessment of the situation: https://twitter.com/NateSilver538/status/1726614811931509147
Where the board messed up is that the underestimated the need to propagandize and prepare before acting against Altman. The focus is not on how Altman did or did not turn the company away from its non-profit mission but instead on how the board was unfair and capricious towards Altman. Though this was somewhat predictable, the extent of Altman's support and personality cult is surprising to me, and is perhaps emblematic on how badly the board screwed up from an optics perspective. There were seemingly few attempts to put pressure on Altman's priorities or to emphasize the non-profit nature of the company, and the justification afterwards was unprepared and insufficient.
From the outside though, I don't understand why so many are clamoring to leave their highly paid jobs working at a non-profit who's goal is to serve humanity and to become a cog in a machine aimed at maximizing Microsoft shareholders' wealth, in defense of a singular CEO with little technical AI background who's motivations are unclear.
If it was for other reasons, yeah, may simply have been dumb.
Is it also the case that the anti-war Germans who joined the Nazi regime were in a unique position to help guide Germany in as benign direction as possible? If not, what is the difference between the "safe AI" person who decides to join OpenAI and the anti-war, anti-racist German who decides to get a job in the Nazi government or military?
This is an inkling, a tiny spark, of how hard it'll be to control AI, or even the creation of AI. Wait until the outcome of war depends on the decisions made by those competing with significant AI assistance.
Safety is about setting up the infrastructure to control uranium sources before the first bomb gets built. It's not about right now, it's about the phase change that happens the moment we take that step. Don't you want to have infrastructure to prevent possible societal strife if possible?
Of course gpt has value, bombs have value, enron had value, the housing market has value. If i could retort i'd say the term 'value' is much too vague to contribute to a discussion about this. The value it has is the danger, they're the same thing. If i suddenly quadruple the money supply in every country in the world, do you think that would improve the lives of the majority of humans? Or is it possible that would be looked back on a a catastrophic event that obliterated societies and killed innumerable people. Hard to say huh? Maybe it wouldn't, maybe it would, who can actually know? Wouldn't it be better for us to have some kind of system to maybe handle that before it leads to potential destruction. If instead, i announce that at some point in the future this event might occur. Does that change your calculus? How do you feel about global climate destabilization? How do you feel about the prospect of another world war?
… but I’d probably sign for exactly those good-career-move reasons, at this point. Going down with the ship isn’t even going to be noticed, let alone change anything.