Ah, OK

That sound data sanitizing rather than the AI safety/danger that Bostrom and company worry about. For them, it's "OMG, must limit it's capabilities" rather than "Must keep it from doxing or offending people". The parent I was replying to was kind of ambiguous what sort of crippling they meant (We also should be careful using DALL-E as an accurate measure of what's possible, because OpenAI has intentionally crippled their offering in a number of ways to avoid scaring or offending people, under the rubric of "AI safety".)

Some of what they've done is described here – https://openai.com/blog/reducing-bias-and-improving-safety-i... – & in other statements. I'm not sure they've claimed, or we can be sure, that every one of their training or output limits has been disclosed.

It seems one of their techniques is to silently add extra words to some prompts to increase the variety of returned images in certain dimensions, eg: https://twitter.com/rzhang88/status/1549472829304741888

While I can't find the link right now, someone also tweeted examples of asking DALL-E2 for certain historical roles that would have been, by actual history and likely representation in training data, fairly uniform in race/sex – but the new OpenAI 'bias mitigations' generate ahistorical race/gender-varied examples. That's interesting – I'm a big fan of non-traditional castings like 'Hamilton!' etc – but serves to make the AI look ignorant of history, when in fact it has just been design-limited to simulate ignorance.

Here's the link I couldn't find earlier: https://twitter.com/Plinz/status/1554255192316526594

Prompts like "a movie still of ancient philosopher Heraclitus", "ancient philosopher Aristotle", and "famous physicists of the 17th century" render largely plausible portraits – except with an added sprinkling of ahistorical, 21st Century "AI safety" race/gender variety.

This is less a common-sense failure of the AI than an artifact of its creators' imposed harnesses.

And thus I'd suggest that more generally, unless a model's full design/training-data/constraints are fully described, it remains possible that other "failures" could sometimes be side-effects of undisclosed designer choices.

OK, I looked at your link. It talks about safety but all of measures described are about avoiding bias. I'm not personally against this - having more diverse images is great. But I think my point that the Lesswrong version of AI safety does not seem to be being considered stands. The various anti-bias stuff seems unlikely to be reducing the system's capabilities and I think your original point was that we could not judge Dall-E 'cause of safety-reduced capacities and if this is your evidence, it seems unconvincing to say the least.

I don't think it simply adds extra words; I think they run it through a GPT prompt, something like "restate this with generic references to humans replaced with specific ethnicities". Can't tell from the outside of course.