If the author is on board with height, I can take him down all the way to a reductio where you give everyone the exact same credit terms.
I can very much appreciate your reductio argument, but if i had to wager, that's the way we're headed. Seems like aspiring lawyers should start cracking open the ML textbooks, because that case is a pandora's box.
"defensible" is a weasel word.
It goes back to the old "correlation vs causation"
Machine learning choose the "shape" of a decision surface. Optimizing parameters of that surface may be defensible, or may nor be. Your algorithm may learn that people with my skin color, or who live in my neighborhood, on have a certain average behavior on your objective function.
I would word your #1 and #2 as (1) baseless prejudice that midjudges people in general, (2) statistical prejudice that misjudges many individuals.
But it's still a leap to treat me as the "average" in that group, instead of judging me on the content of my character. What guided your choice of parameters? Even if you did the best within practical limits, it still may be unfair.
This second issue seems harder to resolve, because it will require lawmakers to clearly articulate a "bright line" law in the language of the algorithm itself. As the article acknowledges, algorithms can "learn" race, even if race is not an explicitly defined explanatory variable.
Looking forward to part two.
Edit: Why race? Because it's an obvious and easy example. I believe the author would be on board with your height example.