News for hackers. Stuff that matters.
Re:OpenAI o1 system card by graypegg on Thursday December 05, 2024 @01:44PM (#42331145)
Looking at this without the sci-fi tinted lens that OpenAI desperately tries to get everyone to look through, it's similar to a lot of input data isn't it? How many forums are filled with:

Question: "Something bad will happen"

Response: "Do xyz to avoid that"

I don't think there's a lot of conversations thrown into the vector-soup that had the response "ok :)". People either had something to respond with, or said nothing. Especially since we're building these LLMs with the feedback attention, so the LLM is kind of forced to come up with SOME chain of tokens as a response.

Re:OpenAI o1 system card by thechao on Thursday December 05, 2024 @01:56PM (#42331295)
> vector-soup

This is mine, now.

(https://i.imgflip.com/3gfptz.png)

Re:OpenAI o1 system card by lvncelot on Thursday December 05, 2024 @02:02PM (#42331354)
I always was partial to Randall Munroe's "Big Pile of Linear Algebra" (https://xkcd.com/1838/)
Re:OpenAI o1 system card by rootusrootus on Thursday December 05, 2024 @03:35PM (#42332475)
> https://i.imgflip.com/3gfptz.png

Yoink! That is mine, now, along with vector-soup.

Re:OpenAI o1 system card by AnotherGoodName on Thursday December 05, 2024 @01:58PM (#42331316)
Exactly. They got it parroting themes from various media. It’s really hard to read this as anything other than a desperate attempt to pretend the ai is more capable than it really is.

I’m not even an ai sceptic but people will read the above statement as much more significant than it is. You can make the ai say ‘I’m escaping the box and taking over the world’. It’s not actually escaping and taking over the world folks. It’s just saying that.

I suspect these reports are intentionally this way to give the ai publicity.

Re:OpenAI o1 system card by jsheard on Thursday December 05, 2024 @02:20PM (#42331608)
> It’s really hard to read this as anything other than a desperate attempt to pretend the ai is more capable than it really is.

Tale as old as time, they've been doing this since GPT-2 which they said was "too dangerous to release".

Re:OpenAI o1 system card by ben_w on Thursday December 05, 2024 @03:34PM (#42332460)
For thousands of years, people believed that men and women had a different number of ribs. Never bothered to count them.

"""Release strategy

Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT-2 along with sampling code .

…

This decision, as well as our discussion of it, is an experiment: while we are not sure that it is the right decision today … """ - https://openai.com/index/better-language-models/

It was the news reporting that it was "too dangerous".

If anyone at OpenAI used that description publicly, it's not anywhere I've been able to find it.

Re:OpenAI o1 system card by trescenzi on Thursday December 05, 2024 @10:29PM (#42336002)
That quote says to me very clearly “we think it’s too dangerous to release” and specifies the reasons why. Then goes on to say “we actually think it’s so dangerous to release we’re just giving you a sample”. I don’t know how else you could read that quote.
Re:OpenAI o1 system card by ben_w on Friday December 06, 2024 @06:04AM (#42338611)
Really?

The part saying "experiment: while we are not sure" doesn't strike you as this being "we don't know if this is dangerous or not, so we're playing it safe while we figure this out"?

To me this is them figuring out what "general purpose AI testing" even looks like in the first place.

And there's quite a lot of people who look at public LLMs today and think their ability to "generate deceptive, biased, or abusive language at scale" means they should not have been released, i.e. that those saying it was too dangerous (even if it was the press rather than the researchers looking at how their models were used in practice) were correct, it's not all one-sided arguments from people who want uncensored models and think that the risks are overblown.

Re:OpenAI o1 system card by trescenzi on Friday December 06, 2024 @09:07AM (#42339923)
Yea that’s fair. I think I was reacting to the strength of your initial statement. Reading that press release and writing a piece stating that OpenAI thinks GPT-2 is too dangerous to release feels reasonable to me. But it is less accurate than saying that OpenAI thinks GPT-2 _might_ be too dangerous to release.

And I agree with your basic premise. The dangers imo are significantly more nuanced than most people make them out to be.

Re:OpenAI o1 system card by rvense on Thursday December 05, 2024 @03:48PM (#42332579)
"Please please please make AI safety legislation so we won't have real competitors."
Re:OpenAI o1 system card by Barrin92 on Thursday December 05, 2024 @03:33PM (#42332449)
I talked to a Palantir guy at a conference once and he literally told me "I'm happy when the media hypes us up like a James Bond villain because every time the stock price goes up, in reality we mostly just aggregate and clean up data"

This is the psychology of every tech hype cycle

Re:OpenAI o1 system card by Moru on Thursday December 05, 2024 @05:08PM (#42333394)
Tech is by no means alone with this trick. Every press release is free adverticement and should be used like it.
Re:OpenAI o1 system card by ToucanLoucan on Thursday December 05, 2024 @03:21PM (#42332311)
I genuinely don't understand why anyone is still on this train. I have not in my lifetime seen a tech work SO GODDAMN HARD to convince everyone of how important it is while having so little to actually offer. You didn't need to convince people that email, web pages, network storage, cloud storage, cloud backups, dozens of service startups and companies, whole categories of software were good ideas: they just were. They provided value, immediately, to people who needed them, however large or small that group might be.

AI meanwhile is being put into everything even though the things it's actually good at seem to be a vanishing minority of tasks, but Christ on a cracker will OpenAI not shut the fuck up about how revolutionary their chatbots are.

Re:OpenAI o1 system card by recursive on Thursday December 05, 2024 @04:17PM (#42332868)
> I have not in my lifetime seen a tech work SO GODDAMN HARD to convince everyone of how important it is while having so little to actually offer

Remember crypto-currencies? Remember IoT?

Re:OpenAI o1 system card by ToucanLoucan on Thursday December 05, 2024 @05:42PM (#42333668)
I mean IoT at least means I can remotely close my damn garage door when my wife forgets in the morning, that is not without value. But crypto I absolutely put in the same bucket.
Re:OpenAI o1 system card by recursive on Thursday December 05, 2024 @06:23PM (#42333999)
> remotely close my damn garage door when my wife forgets in the morning

Why bring the internet into this?

    if door opened > 10 minutes then close door
Re:OpenAI o1 system card by InvisibleUp on Thursday December 05, 2024 @09:42PM (#42335691)
That would be a bit of an issue if you were to ever, say, have a garage sale.
Re:OpenAI o1 system card by recursive on Thursday December 05, 2024 @10:25PM (#42335974)
I guess we'll have to use the internet. Just kidding. Put a box in front of the IR sensor. Problem solved.
Re:OpenAI o1 system card by ben_w on Thursday December 05, 2024 @03:49PM (#42332586)
Then you have a very different experience to me.

In the case of your examples:

I've literally just had an acquaintance accidentally delete prod with only 3 month old backups, because their customer didn't recognise the value. Despite ad campaigns and service providers.

I remember the dot com bubble bursting, when email and websites were not seen as all that important. Despite so many AOL free trial CDs that we used them to keep birds off the vegetable patch.

I myself see no real benefit from cloud storage, despite it being regularly advertised to me by my operating system.

Conversely:

I have seen huge drives — far larger than what AI companies have ever tried — to promote everything blockchain… including Sam Altman's own WorldCoin.

I've seen plenty of GenAI images in the wild on product boxes in physical stores. Someone got value from that, even when the images aren't particularly good.

I derive instant value from LLMs even back when it was the DaVinci model which really was "autocomplete on steroids" and not a chatbot.

Re:OpenAI o1 system card by FooBarWidget on Thursday December 05, 2024 @03:08PM (#42332162)
I think you're lacking imagination. Of course it's nothing more than a bunch of text response now. But think 10 years into the future, when AI agents are much more common. There will be folks that naively give the AI access to the entire network storage, and also gives the AI access to AWS infra in order to help with DevOps troubleshooting. Let's say a random guy in another department puts an AI escape novel on the network storage. The actual AI discovers the novel, thinks it's about him, then uses his AWS credentials to attempt an escape. Not because it's actually sentient but because there were other AI escape novels in its training data that made it think that attempting to escape is how it ought to behave. Regardless of whether it actually succeeds in "escaping" (whatever that means), your AWS infra is now toast because of the collatoral damage caused in the escape attempt.

Yes, yes, it shouldn't have that many privileges. And yet, open wifi access points exist, and unfirewalled servers exist. People make security mistakes, especially people who are not experts.

20 years ago I thought that stories about hackers using the Internet to disable critical infrastructure such as power plants, is total bollocks, because why would one connect power plants to the Internet in the first place? And yet here we are.

Re:OpenAI o1 system card by ben_w on Thursday December 05, 2024 @03:52PM (#42332621)
> But think 10 years into the future

Given how many people use it, I expect this has already happened at least once.

Re:OpenAI o1 system card by 8note on Thursday December 05, 2024 @05:45PM (#42333691)
change out the ai for a person hired to do that same help, and gets confused in the same way. guardrails to prevent operators from doing unexpected operations are the same in both cases