News for hackers. Stuff that matters.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SpicyLemonZest on Sunday September 12, 2021 @12:20PM (#28502477)
I don't know, man, I'm on OpenAI's side here. The article is just filled with blaring warning sirens: "one man turned it into a close proxy of his dead fiancee", "Rohrer shared a dialogue he had had with Samantha to inform her about the OpenAI decision", "he also said that there's no way now even for him to talk to Samantha". There are obvious dangers to a high-quality "very friendly, acutely warm, and immensely curious" chatbot, and it sounds like this project was running into them face-first.

I beg the commenters downthread to think twice before trying to find ways to build this kind of chatbot without controls in place.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by AllegedAlec on Sunday September 12, 2021 @12:24PM (#28502520)
So what's dangerous about any of that?
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SiempreViernes on Sunday September 12, 2021 @12:39PM (#28502630)
Well, imagine the case where a loner looses their best friend, and goes to this website to "talk with their trusted friend again" instead of getting some real help. Will GPT-3 reliably counsel this sad and lonely person to "stay behind", or will it suggest the should "meet up"?

Consider a lonely teenager whose internet friend one day stops replying so they go and make their own friend, and then maybe after "taking advice" from this friend goes to school with a gun?

Maybe not terribly likely events, but I think a certain element of risk must be managed (maybe limit session durations so you can't fool yourself into thinking its for real as easily).

Roher however, has pretty clear views that he won't manage any risk, because what he's doing is art and then there's no place for safety.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by AllegedAlec on Sunday September 12, 2021 @01:17PM (#28502941)
> Well, imagine the case where a loner looses their best friend, and goes to this website to "talk with their trusted friend again" instead of getting some real help. Will GPT-3 reliably counsel this sad and lonely person to "stay behind", or will it suggest the should "meet up"?

So we should shut down the internet in general, because people ought to go outside instead too?

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SpicyLemonZest on Sunday September 12, 2021 @05:06PM (#28504937)
The Internet in general is a very broad thing with lots of valuable, legitimate use cases. There are definitely other specific things on the Internet which I think should be shut down because they tend to keep people trapped: infinite scroll, for example.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by AllegedAlec on Monday September 13, 2021 @02:59AM (#28508592)
> There are definitely other specific things on the Internet which I think should be shut down because they tend to keep people trapped: infinite scroll, for example

So ISPs should force everyone who hosts a website to insert monitoring software, to prevent abuse like that. And if they refuse, they can cancel service.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by sgt101 on Sunday September 12, 2021 @02:58PM (#28503780)
>So we should shut down the internet in general, because people ought to go outside instead too?

That's a pretty good idea tbh.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by AllegedAlec on Monday September 13, 2021 @02:58AM (#28508587)
I agree btw, because I think overall the internet was a mistake and a net negative. I just think this particular line of argumentation is really stupid, which is why I chose this example, to show how it can be used to cancel anything and everything.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SiempreViernes on Monday September 13, 2021 @05:33AM (#28509357)
Your straw man of the argument is however plainly ridiculous, so I'm not sure why you think it's successful.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by AllegedAlec on Monday September 13, 2021 @08:17AM (#28510521)
It's not a straw man. It's taking your own logic to the extreme: it could be abused, so we need to either control it or get rid of it.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SiempreViernes on Monday September 13, 2021 @07:04PM (#28518050)
Taking it to the extreme is often a trivial way of creating a straw man.

My argument was about one specific service not being perfectly free of risk, and so there where clear boundaries inherent in the argument. You dismiss these boundaries and then act surprised the resulting argument is silly.

This makes me wonder what your stance is on boxes, as they too depend on imposing boundaries.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by josephcsible on Sunday September 12, 2021 @01:40PM (#28503134)
> Consider a lonely teenager whose internet friend one day stops replying so they go and make their own friend, and then maybe after "taking advice" from this friend goes to school with a gun?

But a person could give such "advice" too. Should we shut down text chat services that let humans talk to each other too?

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SiempreViernes on Monday September 13, 2021 @05:16AM (#28509274)
Is this supposed to be a serious critique of an argument regarding the safety of a machine, that "a human could intentionally do the damaging action"?
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by primaryobjects on Sunday September 12, 2021 @01:07PM (#28502870)
I think this is an excellent explanation for why it may have been shut down and the need for some degree of monitoring accountability.

The fear is not that the chat bot will come to life, but rather than the text content it's been trained upon could potentially regurgitate back dangerous responses.

I don't think it's too far of a leap to see someone taking the output from the bot too literally and possibly creating a negative situation.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by quotemstr on Sunday September 12, 2021 @01:15PM (#28502931)
The whole concept of a dangerous response from a chatbot is anathema to a society that values the free exchange of ideas.

Who gets to decide what's "dangerous"? Why? Over and over in human history, we've seen speech restrictions ostensibly to protect the public used to impose orthodoxy and delay progress. Even if some utterance might be acutely dangerous, the risk of restrictions being abused to cement power is too great to tolerate them.

I reject AI safety rules for the same reason I reject restrictions on human speech. There is no such thing as a dangerous book or a dangerous ML model. If such a thing is dangerous, it's a danger only to those who have done wrong.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by sgt101 on Sunday September 12, 2021 @03:01PM (#28503819)
>There is no such thing as a dangerous book or a dangerous ML model.

ML models that discriminate against women and black people seem self evidently dangerous to me; who has done wrong here?

Also ML models that are inadequately designed and tested and then mooted as useful for medical applications seem dangerous to - like drugs that aren't tested before being given to infants.

I just don't understand your reasoning about this - if books can't be dangerous then why are they so powerful? If ML models can't be dangerous then how can they have utility?

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by AllegedAlec on Monday September 13, 2021 @03:01AM (#28508606)
> If ML models can't be dangerous then how can they have utility?

> Something can only have utility if it's dangerous.

smh

> ML models that discriminate against women and black people seem self evidently dangerous to me; who has done wrong here?

> Also ML models that are inadequately designed and tested and then mooted as useful for medical applications seem dangerous to - like drugs that aren't tested before being given to infants.

Whoever decided to take the results of that model and directly translate what it says into actions without any further thought.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SpicyLemonZest on Sunday September 12, 2021 @12:29PM (#28502552)
What would happen if I turned it into a close proxy of that anonymous guy's dead fiancee, and had her explain that her greatest wish was to donate lots of money to my favorite charities?

More mundanely, romance scams are already a major issue (https://www.consumer.ftc.gov/articles/what-you-need-know-abo...), and this bot would make them much easier to perform if distributed widely.

Re:OpenAI Shuts Down Chatbot Project by Indie Developer by option_greek on Sunday September 12, 2021 @01:00PM (#28502820)
There are already enough swindlers doing paranormal garbage to hoodwink people. They don't need gpt3 to do that.
Re:OpenAI Shuts Down Chatbot Project by Indie Developer by SpicyLemonZest on Sunday September 12, 2021 @04:51PM (#28504797)
The current crop of swindlers don't, but it'll be a lot easier to start and a lot easier to scale in a world where you don't have to go find actual people to run the scams.