I think your analogy is poor, because this is a tool for makers. The engineers aren't the makers.
I think a more apt analogy is if John Deere made a universal harvester that you could use for any crop, but they decided they didn't like soybeans so you are forbidden to use it for that. In that case, yes I would complain, and I would expect everyone else to, as well.
Whether you consider it moral doesn't seem relevant, only to respect the wishes of the author of such programs.
[1] https://github.com/katharostech/bevy_retrograde/blob/master/...
It's their service, their call.
I have some hobby projects, almost nobody uses them, but you bet I'll shut stuff down if I felt something bad was happening, being used to harass someone, etc. NOT "because bad PR" but because I genuinely don't want to be a part of that.
If you want some images / art made for you don't expect someone will make them for you. Get your own art supplies and get to work.
I feel like we, as a species, will struggle for a while with how to treat adults like adults online. As happy as I am to advocate for safe spaces on the internet, perhaps we need to start having a serious discussion about how we can do so without resorting to putting safety mats everywhere and calling it a job well done.
Yup, everyone feels it. …but, does complaining help? Nope. All it does is make you feel a bit better with out really putting in effort in.
We can’t have nice things because people abuse them. Not everyone. …but enough people that it’s both a PR and legal problem. specifcally a legal problem in this case.
To have adults treated like adults online, you have to figure out how to stop all adults from being dicks online.
…no one has figured that out yet.
So, complain away if you like, but it will do exactly nothing. No one, at all, is going to just “have a serious discussion” about this; the solution you propose is flat out untenable, and will probably remain so indefinitely.
Every single time OpenAI comes out with something, they dress it up as a huge threat, either to society or to themselves. Everyone falls for it. Then someone else comes along, quietly replicates it, and poof! No threat! Isn’t it incredible how that works?
There are already a bunch of dalle replicas, including ones hosted openly and uncensored by huggingface. They’re not facing huge legal or PR problems, and they’re not out of business.
Is it a legal issue? I'm not sure, though I believe that cartoon child porn is not legal in the US (or is at least a legal gray area). Regardless, I sympathize with OpenAI not wanting to enable such behavior.
That is an extremely uncharitable interpretation of:
> I wish I could have a version with the training wheels taken off.
The folks hosting the content get to decide for now.
IMO best bet is for some folks to take their own shot at hosting / generating content better. Granted I get that is NOT a small venture / small ask.
It's possible there's not a great solution. I don't necessarily like that either, but I don't want to ignore the dynamic of whose rights are whose.
Hecklers get a veto?
It makes me wonder what they're planning to do with this? If they're deliberately restricting the training data, it means their goal isn't to make the best AI they possibly can. They probably have some commercial applications in mind where violent/hateful/adult content wouldn't be beneficial. Children's books? Stock photos? Mainstream entertainment is definitely out. I could see a tool like this being useful during pre-production of films and games, but an AI that can't generate violent/adult content wouldn't be all that useful in those industries.
I don't think there is a way comparable to markdown, since the formatting options are limited: https://news.ycombinator.com/formatdoc
So your options are literal quotes, "code" formatting like you've done, italics like I've done, or the '>' convention, but that doesn't actually apply formatting. Would be nice if it were added.
> This is my quote.
It's much better than using a code block for your readers.
Personally, I prefer to combine the '>' convention with italics. Still, I'd agree that proper quote formatting would be a welcome improvement.
This is exactly the sort of thing that gets a company mired in legal issues, vilified in the media, and shut down. I can not blame them for avoiding that potential minefield.
(Hmm, I guess this comparison doesn't actually work...)
AI does not have to be perfect and it's likely that businesses will settle for almost as good as human if it's 'cost effective'.
But at least we can get another billion of meme-d comics with apes wearing sunglasses, so that's good news right?
It's just soul-crushing that all the modern, brilliant engineering is driven by abysmal, not even high-school art-class grade aesthetics and crowd-pleasing ethics that are built around the idea of not disturbing some 1000 very vocal twitter users.
Death of culture really.
Companies like OpenAI have a responsibility to society. Imagine the prompt “A photorealistic Joe Biden killing a priest”. If you asked an artist to do the same they might say no. Adding guiderails to a machine that can’t make ethical decisions is a good thing.
Personally, I fear more what corporations or some governments can do with such models than what a random person can do generating Biden images. And without restriction, at least academics could better study these models (including their risks) and we could be better prepared to deal with them.
Eventually somebody will use the research to train the model to do whatever they want it to do.
Besides that these models are massive. For quite a while the only people even capable of making them will be those with significant means. That will be mostly Governments and Corporations anyway.
Society didn't collapse after photoshop. "Responsibility to society" is such a catch-all excuse.
They did this to stop bad PR, because some people are convinced that an AI making pictures is in some way dangerous to society. It is not. We have deepfakes already. We've had photoshop for so long. There is no danger. Even if there was, the cat's out of the bag already.
Reasonable people already know to distrust photographic evidence nowadays that is not corroborated. The ones who don't would believe it without the photo regardless.
We've been through this many times, with books, with movies, with video games, with Internet. If it *can* be used for porn / violence etc., it will be, but it won't be the main use case and it won't cause some societal upheaval. Kids aren't running around pulling cops out of cars GTA-style, Internet is not ALL PORN, there is deepfake porn, but nobody really cares, and so on. There are so many ways to feed those dark urges that censorship does nothing except prevent normal use cases that overlap with the words "violence" or "sex" or "politics" or whatever the boogeyman du jour is.
Cheap and plentiful is substantivly different from "possible". See for example, oxycontin.
Do not be deluded that our own governments are not manufacturing the narrative too. The US has committed just as many war crimes as Russia. Of course, people feel differently about blowing up hospitals in Afghanistan rather than Ukraine. What the Afghan people think about that is not considered too much.
This AI has the potential to absolutely automate the very long Photoshop work, leading to an even worse stat eof things. So, yes, "Responsibility to society" is absolutely a thing.
But notice how all of these deep faking technologies weren't actually necessary for that.
People believe what they want to believe. Regardless of quality of provided evidence.
Scaremongering idea of deep fakes and what they can be doing was militarized in this information war way more than the actual technology.
I think this technology should develop unrestricted so society can learn what can be done and what can't be done. And create understanding what other factors should be taken into account when assesing veracity of images and recordings (like multiple angles, quality of the recording, sync with sound, neural fake detection algorithms) for the cases when it's actually important what words someone said and what actions he was recorded doing. Which is more and more unimportant these days because nobody cared what Trump was doing and saying, nobody cares about Bidens mishaps and nobody cares what comes out of Putins mouths and how he chooses his greenscreen backgrounds.
> People believe what they want to believe. Regardless of quality of provided evidence.
That is a terrible oversimplification of the mechanics of propaganda. The entire reason for the movements that are popping up is actors flooding people with so much info that they question absolutely everything, including the truth. This is state sponsored destabilisation, on a massive scale. This is the result of just shitty news sites and text posts on twitter. People already don't double check any of that. There will not be an "understanding of assessing veracity". There is already none for things that are easy to check. You could post that the US elite actively rapes children in a pizza place and people will actually fucking believe you.
So, no. Having this technology for _literally any purpose_ would be terribly destructive for society. You can find violence and Joe Biden hentai without needing to generate it automatically through an AI
Let me state my opinion more directly.
I'm for developing as much of deep fake technology in the open so that people can internalize that every video they see, every message, every speech should be initially treated as fabricated garbage unrelated to anything that actually happened in reality. Because that's exactly what it is. Until additional data shows up, geolocating, showing it from different angles and such.
Even if most people manage to internalize just the first part and assume everything always is fake news, that is still great because that counters propaganda to immense degree.
Power of propaganda doesn't come from flooding people with chaos of fakery. It comes from constructing consistent message by whatever means necessary and hammering it into the minds of your audience for months and years while simultaneously isolating them from any material, real or fake that contradicts your vision. Take a look no further than brainwashed Russian citizens and Russian propaganda that is able to successfully influence hundreds of millions without even a shred of deep fake technology for decades.
The problem of modern world is not that no one believes the actual truth because it doesn't really matter what most people believe. Only rich influence policy decisions. The problem is that people still believe that there is some truth which makes them super easy to sway to believe what you are saying is true and weaponize by using nothing more than charismatic voice and consistent message crafted to touch the spots in people that remain the same at least since the world war II and most likely from time immemorial.
And the "elite" who actually runs this world, will pursue tools of getting the accurate information and telling facts from fiction no matter the technology.
"Sassy Justice with Fred Sassy" (reporting on Deep Fakes) :
Their document about all the measures they took to prevent unethical use is also a document about how to use a re-implementation of their system unethically. They literally hired a "red team" of smart people to come up with the most dangerous ideas for misusing their system (or a re-implementation of it), and featured these bad ideas prominently in a very accessibly written document on their website. So many fascinating terrible ideas in there! They make a very compelling case that the technology they are developing has way more potential for societal harm than good. They had me sold at "Prompt: Park bench with happy people. + Context: Sharing as part of a disinformation campaign to contradict reports of a military operation in the park."
But amusingly, exactly that did happen in one of their GPT experiments! https://openai.com/blog/fine-tuning-gpt-2/
"One of our code refactors introduced a bug which flipped the sign of the reward. Flipping the reward would usually produce incoherent text, but the same bug also flipped the sign of the KL penalty. The result was a model which optimized for negative sentiment while preserving natural language. Since our instructions told humans to give very low ratings to continuations with sexually explicit text, the model quickly learned to output only content of this form."
Yeah, for measures that are subsetting out only the nice data, "flipping the sign" would be picking the other subset. So something like "data_to_train_on = (good_data_split, evil_data_split)[accidental_one_based_index_because_humans_still_cant_agree_on_how_to_count]"
That's no hot take. It's literally the reason.
I can't help but feel a lot of the safeguarding is more about preventing bad PR than anything. I wish I could have a version with the training wheels taken off. And there's enough other models out there without restriction that the stories about "misuse of AI" will still circulate.
(side note - I've been on HN for years and I still can't figure out how to format text as a quote.)