Yes. But those open-source LLMs already exist. It's surprising the OP didn't at least explore the possibility of using them to get around the limitations of "Open" AI (which sounds more and more like a Victorian matron).

Are the risks worth it for OpenAI to allow that at what is basically the start of their business?

And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand.

I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?

It's largely to protect their brand from bad press (this has hit even Google/Microsoft in the past), but established finance institutions absolutely hate porn, even the "less shady" ones like onlyfans or pornhub have issues with banking.

> I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?

These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use.

In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way.

The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market.

Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy.

Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox.

I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.

1. Some people are so far to the left side of the bell curve that they're going to be screwed out of a human relationship no matter what they do, and it's unkind to withhold sex bots from them just to provide increased motivation to the segment of the population that's still salvageable.

2. Society seems to have collectively decided that romantic intimacy, unlike food and shelter, is optional and that experiencing an absence of it should not entitle you to sympathy or support. Therefore, the kindest thing they can do is leave the autists alone so they can cope as best they can in peace.

3 - this is technodrugs; its users get high on their sexual addiction dopamine loop, and lose the remaining of their will and life energy. It's not our business to stop those who consciously engage in such self destruction, but luring the naive users into this pit, exploiting their loneliness and making money off it isnt any better than what cartels do.

The author of this app must be doing this unknowingly: his overdeveloped mind sees just an interesting problem to solve, and the thick layer of nihilism obscures the sight of what he's really doing.

And you think chronic, crippling loneliness can't make someone lose their will to live? If you don't know what tulpas are, there's an entire, fairly large online community of people willingly inducing schizophrenia in themselves just so they have an imaginary friend to talk to. Surely, AI companions would be a healthier alternative to that.

I think that opium doesn't cure loneliness.

I'm not sure your analogy applies here. Opioid substances aren't capable of producing persistent subjective improvement because the brain seems to have defense mechanisms against it that neuroscience has not yet found a way to bypass, but in the case of a sufficiently advanced AI companion, the brain would be receiving the exact same inputs as with a real partner.

With many people choosing not to reproduce, many currently existing relationships are already primarily a sort of recreational activity and are divorced from their evolutionary purpose. Replacing live partners with AIs would just take that one step further.

You make some good points. I don't agree with them, but they're solid.

> Therefore, the kindest thing they can do is leave the autists alone so they can cope as best they can in peace.

This one, no. I'm probably autistic. It wasn't a diagnosis for anything when I was growing up, I was just weird. I craved connection so badly it compelled me to try to "improve" myself to be more like others. I'd be completely fucked today if someone offered a fake, effortless alternative.

I don't think parasocial relationships are actually working out for anyone. The kids are all miserable, lonely, and have a million mental health issues. Kids used to negotiate their differences and play together. Now each of them thinks they get to set the rules for the entire playground. I'd be lonely and bitter too with that mentality.

> You could argue the AI companies would have an incentive to exploit their users, but this is a thing that happens in real relationships as well!

Sadly, yeah. Your chances of finding a human companion who isn't a piece of shit is way higher than finding an ethical tech company to pour your heart out to though.

>I craved connection so badly it compelled me to try to "improve" myself to be more like others. I'd be completely fucked today if someone offered a fake, effortless alternative.

As long as someone makes enough effort to pay the bills and not neglect their health, I don't see why anything more should be necessary. It certainly beats having to adopt hobbies you don't like or learning to lie about your feelings in order to appeal to potential partners (both common pieces of advice for nerdy men).

>I don't think parasocial relationships are actually working out for anyone.

The tech isn't quite there yet, but we've seen immense progress just over the past couple of years.

>Your chances of finding a human companion who isn't a piece of shit is way higher than finding an ethical tech company to pour your heart out to though.

It seems that the solution here, just as with other software, is to make AI companions open source.

Just get a cat/dog.

If you're able to let your imagination run wild you'll see that the depth of depravity of NSFW AI content generated by user input probably has no end.

Even worse, some people turn it into a competition.

Leak an API key on github or somewhere else and heaven help you if you get picked up by their scraping.

It is inevitable

100% this. I'm not interested in these kinds of usages of LLMs but if someone is paying you to access your API directly you should be letting them do simple stuff like have sexual conversations.

OpenAI's filtering is probably the most annoying part of ChatGPT and it makes it useless in so many cases. I can't tell you how many times I've uploaded a picture of a a place from Google Maps or of a location somewhere (that happens to contain a person) and tried to ask for details about the location to find out more info about it just to be shutdown... "I can't help with that" well then you're slowly making yourself useless and not worth the $20/mo.

> What I find unsavory, is OpenAI imposing content restrictions like NSFW.

I mean, the fact that people's NSFW conversations were being processed in a way where both OP and multiple companies could monitor them and do absolutely anything they want with them is a hint that something went very wrong there.

I consider OpenAI pre-emptively pulling the plug a good thing in this case. Maybe there's a case for an AI erotica service, but OP sure as hell shouldn't be the one operating it.

> What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more.

I'm guessing they don't want payment processors cutting them off.

Everyone, even MS backed companies, are afraid of Visa.

And for some obscene reason Visa and Mastercard are afraid of a few hundred extremely religious/conservative moms who make a stink any time someone wants to be slightly different.

If they succeed at making it impossible to make money from porn, they sure as fuck aren't stopping there.

AFAIK it’s a couple people in the government who don’t like it, not moms.

> The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.

There are projects like llama.cpp https://github.com/ggerganov/llama.cpp and koboldcpp: https://github.com/LostRuins/koboldcpp that can make running open source models pretty easy. As for where to find the models, you can look at HuggingFace for quite the selection, depending on what your needs are and what your hardware (CPU and RAM or GPU and VRAM, or both) can run: https://huggingface.co/models

I was actually curious about this, so I wrote a blog post on it: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... which wasn't anything too serious, in addition to formats like GGUF I think becoming more popular than GGML since. Things are moving ahead quite rapidly over there.

I guess it can also be easier to pay for something like NovelAI for those who want something for general purpose writing (including most NSFW stuff), but it wasn't exactly amazing for my needs. Despite the name, it wasn't great at prose enhancement: to take a dry sounding paragraph or two and rewrite it until I've found an additional expression or two to include instead, or a few sentence structures than work better. Kind of fun as a replacement for AI Dungeon or something like Character.AI, though.

Then again, the local models need expensive hardware to actually run well, if you want good results (somewhere above 20B parameters from what I've seen). I do wonder how much better the models that approach 200B would be for general text generation use cases, if the performance would actually be good. Even more so, when it comes to writing code, or all of the boring stuff like boilerplate/scaffolding code, utility functions, or even stubs for tests. ChatGPT, Phind and GitHub Copilot are all there, but could be even better.

It's not clear what the limits are, from the article or really from OpenAI. The author only really knew that some combination of quantity and extent of material was enough to trigger a warning, it could very well have been a handful of users pushing boundaries pretty far. NSFW is one thing, but chat can certainly go into NSFL territory, non-consensual roleplay, etc.

I don't know if people here are being naive or dense, but merely "non-PC" content isn't going to trigger anything.

I spent like a couple minutes setting up a roleplay where ChatGPT is happily advocating for Nazi ideology (and not even on the API; ChatGPT is generally more censorious than the API). I'm not going to try to hit the limits, but I literally don't think you can hit the limits with merely political speech. I don't know what the NSFW limits are, but I bet they are further than people here think.

The vagueness absolutely makes stuff difficult, and the author hit a kind of all-or-nothing approach by going from no restrictions to using the moderation endpoint which is very strict. That could be improved.