Re:Algorithm Helps New York Decide Who Goes Free Before Trial
by mindslight on Sunday September 27, 2020 @11:23AM
(#24607313)
There is no inherent problem with using an algorithm for these decisions. The growing problem seems to be the lack of algorithms!
If the specific rules for a decision are not understandable by humans, then we're not talking about an "algorithm" but rather an opaque model. An algorithm would be something like: Defendant gave address and phone number (1 point), Defendant has no prior convictions within past 3 years (3 points), and so on. With rules ultimately understandable by humans, who can debate and modify specific rules such as ones based on race, or ones that would have disparate impact like zip code. This is 1970's tech, not 2020's.
I don't know about NYC's system, and it's hard to tell specifics from the article which only makes references to a few input datums but never says how the inputs effect the output. If there are specific scoring rules that can be individually debated, then the article is trying to drum up worry by leaving them out. If NYC's system is based on inputting data to a proprietary tool that only outputs a score and citizens are unable to know how it works, then the article needs to drive home the black box unaccountability aspect of this (and stop using the word "algorithm"), rather than focusing on the imposition of requiring a phone number.
If the specific rules for a decision are not understandable by humans, then we're not talking about an "algorithm" but rather an opaque model. An algorithm would be something like: Defendant gave address and phone number (1 point), Defendant has no prior convictions within past 3 years (3 points), and so on. With rules ultimately understandable by humans, who can debate and modify specific rules such as ones based on race, or ones that would have disparate impact like zip code. This is 1970's tech, not 2020's.
I don't know about NYC's system, and it's hard to tell specifics from the article which only makes references to a few input datums but never says how the inputs effect the output. If there are specific scoring rules that can be individually debated, then the article is trying to drum up worry by leaving them out. If NYC's system is based on inputting data to a proprietary tool that only outputs a score and citizens are unable to know how it works, then the article needs to drive home the black box unaccountability aspect of this (and stop using the word "algorithm"), rather than focusing on the imposition of requiring a phone number.