par1970
par1970 on Hacker News
How did they know that people wouldn't find a way to use "the dataset, training code, or GPT‑2 model weights" to "generate misleading news articles, impersonate others online, automate the production of abusive or faked content to post on social media, automate the production of spam/phishing content."?
If I remember correctly, it seemed like a plausible outcome to me (especially spam and junk social media content). Was there some conclusive evidence that I was missing?
So you aren't claiming that they said GPT-2 was dangerous in the sense that it could disempower humanity, kill all humans, etc.
You are just claiming that OpenAI execs said that GPT-2 might "generate misleading news articles, impersonate others online, automate the production of abusive or faked content to post on social media, automate the production of spam/phishing content." Then, what is unreasonable or bad about the OpenAI execs saying this in 2019?
> those very same people had touted GPT-2 as a dangerous model
Where did they say this at? AFAIK this is the original GPT-2 announcement: https://openai.com/index/better-language-models/. Here are some direct quotes:
“We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate):
* Generate misleading news articles
* Impersonate others online
* Automate the production of abusive or faked content to post on social media
* Automate the production of spam/phishing content”
“Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT‑2 along with sampling code (opens in a new window). ”
You said this: "Screams of propaganda. Sama saying GPT-2 is too dangerous to release…all over again."
So show me Sam's "too dangerous to release" propaganda for GPT-2.
AFAIK this is the document that talks about GPT-2 being dangerous:
https://openai.com/index/better-language-models/Here are some direct quotes:
“We can also imagine the application of these models for malicious purposes , including the following (or other applications we can’t yet anticipate):
* Generate misleading news articles
* Impersonate others online
* Automate the production of abusive or faked content to post on social media
* Automate the production of spam/phishing content”
“Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT‑2 along with sampling code (opens in a new window). ”
Where is the ridiculous part? The fear mongering part? The epistemically weak part?
Show me.
This is the way.
And, this is just for us girls, notice that Anthropic just believing that they are making a machine god is sufficient for their public announcements to not jUsT bE mArKeTiNg.
So what is your credence that they will build a machine god in the next twenty years?
If I present as cynical, will people just believe me when I don't provide any justification?
Are you claiming that the most likely way to please the user is to do something that will lead you to having to say "I cheated."?
LLMs are just sophisticated PR generating tools for chip manufacturers and tools just do what we tell them to do, so you're wrong. qed
Putting aside whether Andrew's shock is appropriate, it seems like we agree that current agents at least occasionally do things that are straightforwardly against the interests of the prompter when given mundane prompts like "Get me into this gym class as soon as possible."
How does this look once agents are superintelligent?
That doesn't make sense. LLMs just do what we tell them to do. It's similar to if I ask you for twenty bucks because I forgot my wallet and then you rob some guy to give me the twenty bucks, that's just what I asked you to do.
> Seems to me like it was doing what it was asked to do?
Maybe it's what he asked it to do, but it's not what he wanted it to do. Which we know because (a) normal people don't want to break the law to get into a gym class, and (b) "But Andrew was shocked by what happened next." and "Alarmed, Andrew asked the agent to undo this."
> AI (in this form) will never be able to solve things we truly cannot solve yet.
Argument?
> LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text.
Argument?
Are LLMs close to being able to significantly help AGI researchers?
Which doomer argument have you found what problem with?
Do you deny the reported bug finding capabilities, or do you deny that they are dangerous?
I think this is roughly solved. Tell the agent to do all of its calculations in python.
Do you have a defense of why human-hammer-nail is a good analogy for human-chatgpt5.4-pwndsamsung?
But the service also tells criminals and adversaries about the bomb locations.
Yeah, maybe you are right. But is doing math and reasoning about Turing machines a priori? If so, then it seems plausible to me that reasoning about a codebase (without running it) is also ‘a priori’.
> What do you mean "a priori understanding codebases"?
I took him to be distinguishing between (1) just reading the code/docs and reasoning about it, and (2) that + crafting and running tests.
Did you tell it that it should test, or did you have it generate actual tests that you could run if you wanted to?
How much domain experience do you have? Is it helping you solve problems for paying customers?
If your project requires the solution of a tricky algorithmic issue, then is the AI system able to solve that part, or do you have to give it the solution?