The algorithms reflect and amplify biases in the data. They also go into feedback loops once they affect the reality being represented in data.
Imagine that an algorithm selects individuals at an airport for search. Contraband smugglers are caught. The dataset now includes them, evolving the bias. We have already seen this at play on social media, recommendation engines, fraud detection, etc.
The future is worrying. There are ratchets on technologies such as these. Unless there are extraordinary counter-reactions, they will be "affecting their own datasets" en masse very soon.
The difference is with technology these biases can at least try to be addressed directly and create an objective more fair system.
Technology can address biases but we need to make sure that they are addressed sufficiently, so that we do not end up causing more problems than we solve.
This is a really interesting point that I'll keep in the back of my mind, thanks for that! I hadn't even thought it through in such a concrete manner but I think this really hits the nail on the head.
> However, just from a cursory review of the mugshots, people of color were disproportionately represented in the mugshot database. So it’s not entirely fair to criticize the facial recognition technology for matching more people of color.
No, the point is the disproportionate representation, and it is a fair criticism, because it is a fundamental limiting factor in the use of the technology. You seem to be distinguishing the "technology" from the data source, but that is not possible. The quality of the technology depends on the quality of the data provided to it.
> I believe a curated list of mugshots with certain characteristics would result in a similar representation of mismatches. Nothing I have read about the technology suggests that there are inherit [sic, is inherent] bias.
This bias is in the data, not the algorithm per se. This is quite accepted in many, even most, criticisms of machine learning applications in the social sphere. It comes up a lot for example in NLP models, for example with assumptions on gender for professions.
It is a technology problem because it demands a technological solution, because removing any and all bias by hand-curating datasets is just not a scalable approach.