I believe the concept is to stop users from prompting the AI to generate offensive stuff specifically, and then publishing the so-generated stream of offensive stuff as negative PR for GitHub, in the same way the generated stream of offensive stuff coming from Microsoft’s AI was a big PR disaster.

I suppose you’re referring to the AI Twitter bot that initially was very lovely and within a day 4chan had turned into a nazi. That was both very naive and hilarious.

https://spectrum.ieee.org/tech-talk/artificial-intelligence/...

The big difference in this case, however, is that this AI was constantly learning based on user input, however, which I do not think is the case for Copilot.

Copilot is indeed constantly learning based on user input (as detailed here [1]) but it seems to be more high-level ("did the user accept or deny suggestion XYZ" and potentially what changes you make to suggestions after accepting) versus just dumping everything directly back into the model a la Tay.

[1] https://docs.github.com/en/github/copilot/about-github-copil...

Maybe, but even if so, filtering the output would also prevent this.

Easily offended AI is exactly what the world needs

We have too many easily offended NPC's as is.

Yeah. If only we could cure all those delusional righties fantisizing about being the protagonist in their own personal RPG.

eyeroll

They probably don't want to repeat Microsoft's incident with Tay, though they seem to have created their own incident which dooms the product if it wasn't already