Where does Google Vision Cloud sit in your categorization?

https://algorithmwatch.org/en/google-vision-racism/

First group.

You’re observing that they aren’t doing a perfect job, which is true, but my grouping isn’t related to perfection of results.

> The deepmind team etc type of group who actually know what they’re doing and the boundaries of what they are working with.

You claim that they "know what they are doing and the boundaries of what they are working with" -- and yet they recklessly make public a racist vision product?

I have a PhD in neural networks, haven't used it in many a year, but some of the knowledge is still there. Some of the memories of racking my brains to understand what the hell is going on are still there, too.

It is easy to have a theory of what is going on, to model the processes of how things are playing out inside the system, to make external predictions of the system, and to be utterly wrong.

Not because your model is wrong, but because either the boundary conditions were unexpected, or there was an anti-pattern in the data, or because the underlying assumptions of the model were violated by the data (in my case, this happened once when all the data was taken in the Southern Hemisphere...)

In all these cases, you can know what you're doing, you can know the boundaries of what what you're working with, and you can get results that surprise you. It's called "research" for a reason.

The model can also be ridiculously complex. Some of the equations I was dealing with took several lines to write down, and then only because I was substituting in other, complicated expressions to reduce the apparent complexity. It's easy to make mistakes - and so you can know what you're doing, and the boundaries that you're working with, and still have a mistake in the model that leads to a mistake in the data ... garbage in, garbage out.

In short, this shit is hard, yo!

Forgive me because I myself do not have a PhD in ML, but if this is hard (making a non-racist system) why are there not serious guardrails you prevent releasing racist systems to the public?

Is it your contention that Google intentionally devised a racist system and imposed it on the public ? That would be quite the claim.

If instead it was a fuckup, well that seems adequately covered by “this shit is hard”.

If you are instead complaining about a lack of oversight, I don’t have a horse in that race. Ask someone else, I don’t care about the politics, I’m here for the technology.

Your argument is that knowing what you are doing means error free output.

It's more like applying the technology with caution and accountability when you already know beforehand that the output is not error-free.

They never promised that the output would be error free, having output with errors is still useful for many applications. And the issues you are talking about got fixed as soon as it was discovered and since then Google has made sure to always diversify their datasets by race. Nowadays that is common knowledge that you need to do it, but back then it wasn't obvious that a model wouldn't generalize across human races and it is much thanks to that mistake that everyone now knows it is an issue.

It was discovered by others, not them; they fixed the issue only retroactively when it was called out in public. This lack of oversight is part of what I mean with applying things with caution.

And why would they have assumed in the first place that the model _would_ generalize across human races, or any other factor for that matter?

I feel like we're missing the point here. The dangerous groups are those execs you mention who will have the decision about whether to move something into production or not.

When this technology gets into their hands with a dev leash it will be recklessly implemented and people will die.

The same this is happening in the academics.