Is AI dungeon used to train newer models/update the current model? If not then there's no risk of another Tay.

If they use a frozen model then one person using it to generate some NSFW content shouldn't make the output more NSFW-laden for anyone else.

So I don't see how they're "damned if they don't".

> So I don't see how they're "damned if they don't".

The specifics of their approach don't even matter - all it takes is one random Twitter user or media outlet to report on "immoral" output generated by someone to damage the brand image.

That's what they want to avoid and that's why they are trying to take preventive measures.

People make penis shaped blocks in Minecraft all day long... and that doesn't tarnish the Minecraft brand. Why? because people understand that it's not Minecraft the company doing so - or condoning the practice. And that's just scratching the surface of what can be done in games like that which allow "creative expression".

Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?

For the record, I'm not sure where I land with regards to this change by AI dungeon. That being said, I find that arranging blocks in obscene shapes (let's even call it, generally, 'creating or arranging potentially objectionable content in-game'), an act that will likely happen offline, or on a privately owned server, is different than a game/toy/service where output is requested, generated, then sent to the user. (Trying to keep that description simple since I don't really know if that's the right way to describe it)

When comparing the Minecraft scenario to the AI Dungeon scenario under the purview of "doing creative things with games," I don't find the two scenarios to be much different. However, for one of those things, it's a possibility within the vast play space of Minecraft - you could even create genitalia and display profanity with just alphabet blocks - but for the other scenario, depending on how you see it, it's content that is generated, served, and potentially stored by AI Dungeon. I'd be concerned about the publicity if I was AI Dungeon as well.

> Why would you think that this company needs to Thought Police their gamers when thousands of others are able to host creativity based games without being dystopian?

Wow. You act as if that's something new :D

There's even an actual job for doing just that: it's called an editor. You know, the people who work in media, publishing houses, TV, radio, papers, magazines, that kind of thing. It's what they do. Everywhere. Editing content. Removing comments, changing words, making sure the views represented in the publication match the intentions and policies of the investors and parent organisations, etc.

You act as if something is new and evil just because it's done with computers. It isn't. It has been the case for as long as papyrus and cave paintings have been around. Back then it was don't anger the chief/king/Pharao, today it's don't anger the Twitter mob or the hand that feeds you. Same difference.

Games are a medium just like magazines or online blogs and comparing blocky dongs in Minecraft with explicit child pornography says a lot about your understanding of "creative expression". Really makes me wonder sometimes.

So right to Godwin's Law we go then - yeah.

I get the feeling it's not me who is the radical here.

There is a difference between wanting to remove content that a company don't want to associate itself with and genocide.

For you to even suggest there's a connection says far more about your twisted logic and dysfunctional thought process than mine.

Also AI dungeon is a multiplayer game so yeah - YOUR freedom of expression is limited by OTHER players freedom of not being subjected to certain content. If you don't understand that, work on your social skills.

> Also AI dungeon is a multiplayer game so yeah

What? No it isn't.

Technically there's a "multiplayer" mode but that's fundamentally just people taking turns. Restricting certain kinds of content for that reason is nonsense, akin to restricting the kind of story you can write in google docs because multiple people might have been given access to a document.

So you don't do anything below XK-class end-of-the-world scenarios, so you?

Your massive overreaction shows no difference to your perceived enemies. This is a game and the creators want to limit the game world and the actions therein to exclude certain content.

This isn't about control over what people think. There are social norms and these change over time.

The "facts" in your message are the same kind of facts that your typical Karen shouts into the phone when they call 911 because there's a negro walking through their neighbourhood. Same level of paranoia, same unhinged out of proportion response.

When was the last time RMS was literally hunted down by a mob? Because if he wasn't, I have to ask who the real fragile flowers are - the ones that get easily offended or the guys like you who see freedom being destroyed and the 4th Reich rising because a private company wants to remove child pornography from their product?

Ridiculous.

Please don't post flamewar comments to HN. We're trying to avoid that here.

https://news.ycombinator.com/newsguidelines.html

(Also, please don't karen: https://news.ycombinator.com/item?id=26880612)

> this isn't control over what people think

I don't think you understand that banning people over what the Thought Police find "bad" is LITERALLY working to control what people think.

> facts that your typical karen

and you are the typical ignorant fool who complains of "godwin" then resorts to "karen hates black people" drivel. Ignorant of the same calls "Karen" made a hundred times before but all of a sudden skin color matters to everything (hint: there's a word for people who only care about skin color... the funny little hypocrisy of the Woke Crusaders is the racism that permeates every move they make to fight racism by making EVERYTHING about race even when it's not).

> Ridiculous

Considering the other stories about AI dungeon over stepping? I'm more convinced I'm right about the Woke'saders and their march down the same path of others throughout history. It is ridiculous... but that's never stopped the religious in their fervor.

> Child porn

If that's really waht's there? report it to the police and take REAL action... not some half assed Woke'sader steps.

But "think of the childrend" is a close second to "ZOMG THE NAZI'S!" when it comes to justification for overstepping in the Name Of Good. So Sayeth the Wokesaders.

Please don't post flamewar comments to HN. We're trying to avoid that here.

https://news.ycombinator.com/newsguidelines.html

If they think that blocking certain words is going to prevent that from happening then well, good luck with that.

It does prevent the news article from saying "And they knew about it and they did nothing!"

I'd be very worried about this in their shoes and acting very similarly.

But then don't you have to answer to all the media queries?

"They knew about this problem, they even attempted to half ass fix it, but it's obviously their mind is on the subscriber money, and not preventing blatant CHILD EXPLOITATION!"

Then you throw some more money at a yet unsolvable task, and then in 2 years, when some other outlet gets triggered, you have to defend yourself once again.

"They've know about this problem for 5 years and it's still rampant in their community! Shame on you JERF INC.!"

You're not entirely wrong, but if you make the story unattractive enough, the media will go do a hatchet job on someone else. They've only got so much time and they've got a lot of outrage to drum up^W^W^W^Wengagement metrics to drive, so you jst put enough in to try to make them hit somebody else instead.

Correct. And they can go one step further, they could just use the input they currently do not flag if they want to feed back into the model.

This is not a technical consideration, it is just PC at its worst.

Oh come on. This kind of thing is done everywhere in media.

You either deny that games are a medium or you're under the impression that whatever you read in news publications, books, see in films and TV is the result of the unfiltered creative outlet of the producers. News flash: it isn't.

There's heavy editing going on everywhere and just because it targets a specific computer model in this case instead of a more "traditional" product like a novel, film or stage play doesn't make an iota of difference.

In fact this demonstrates the maturity of the technology and its use if it gets the same treatment as every other public media.

If OpenAI was truly open as their name says, there would be no dilemma. The technology would be democratised and anyone would be able to run their own instance with their own censorship.

Centralised systems is the only reason why you get these problems.

Tumblr was destroyed precisely because of Apples puritanical views toward sexual expression (along with Tumblr's extreme laziness in moderating ACTUAL child porn). Discord may well be as well.

"Private companies have the right" always fails to account for the fact that these "private companies" are monopolies, with immense control over wide swaths of people's digital lives. No company can compete with them. No alternative to them can be created, because they will immediately be squashed, or fall into obscurity. Participation in these services is "optional," but more and more only in the way that having electricity or running water is "optional."

As a result, these "private companies" effectively become governments, ruling over digital countries you are forced to live in, and affording you little to no rights. And yet, they do this in an ACTUAL country, which DOES afford its citizens rights. Thus a conflict exists.

Ordinary "Private Companies" can censor whatever they like. Monopolies cannot, because doing so threatens free society.

Perhaps you don't see how once a product contains a censorship feature, it begs to be expanded. Each step appears reasonable, but at the far side, the censor is very restrictive.

Take the products in the category of forums. Most of them do censorship, THIS forum does censorship. A few do not, like Parler. If you like, take a tour of the forum products that do not do any censorship, spend some time there, report back on what it is like.

Which forum products do you enjoy and which do you not?

> All it takes is a handful of bored teenagers or middle-aged basement-dwellers to turn your product into something the vast majority of your clientele finds appalling.

the Internet's original sin!

the irony is that Tech amplifies it but Tech won't (IMHO) be able to fix it with a technological solution (it's not a technical problem).

You act as if the two are mutually exclusive. I don't think the points in the comment you replied to are negated by anything you argued.

Do both then.