News for hackers. Stuff that matters.
Re:Software Needs Philosophers (2006) by alisonkisk on Monday December 14, 2020 @10:17AM (#25418473)
Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity. Software, being mostly based on math is a prime example of "map that is not the territory".
Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:24AM (#25418556)
> Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity.

(E: throwaway894345's comment is of much higher quality than mine.)

Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And, I know a few philosophers who also repeat this nonsense.

Those that do hold this view IME are almost always folks who are invested in a personal politics that is congruent with dismissing concerns about bias. Without exception, IME, they're also knee-jerk antagonistic toward other ideas about bias that don't involve computers (eg they're the first ones to complain about implicit bias training). These people also tend to be the annoyingly pedantic type ("bias has a technical meaning in statistics!" sort of thing).

In other words, they're politically invested, and no amount of input from philosophers -- or technicians/scientists -- is going to change their strongly held personal politic beliefs.

Again, anyone who has spent a moment around philosophers knows that they are just humans who are not immune from the exact same problem.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @10:40AM (#25418750)
To add more context to your point, originally the "algorithms are biased" people were arguing (or strongly implying) that all algorithms were biased and that they would always be biased so we should never use them (or something to that end, anyway). I don't know that their opponents were arguing that algorithms "aren't biased" as much as for a clearer understanding of what is meant by the claim "algorithms are biased" because there's no obvious definition for "bias" that lends itself to the conclusion that algorithms will always be more bias (or "worsely biased" in case that's not the same as "more bias") than people. The latter camp specifically argued that algorithms aren't inherently biased, but rather they "inherit" any biases encoded in the training data, and thus the "algorithms are biased" group would do well to frame their concerns in terms of concerns about training data sets and focus on how we can mitigate those concerns.

Over time, as is often the case with healthy debate, the "algorithms are biased" people did start to frame their concerns this way and now we have robust and productive conversations about ethically training algorithms (instead of a strict prohibition on algorithms). This is strictly a good outcome, except that some people are misremembering the argument as one group arguing that "algorithms can't be biased" in any sense of the phrase (although I'm sure there are some fringe few people for whom that's true as well, but as you point out, that was never a prominent position) and we forget that the "algorithms are biased" people weren't always making the nuanced arguments that they've been pushed to make today.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:44AM (#25418803)
Excellent clarification; thanks for taking the time to write this up!
Re:Software Needs Philosophers (2006) by jimktrains2 on Monday December 14, 2020 @10:26AM (#25418593)
> Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view.

And yet here we are selling software, like facial recognition, to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:28AM (#25418613)
> And yet here we are selling software, like facial recognition

Speak for yourself. (Also, last I checked, most CEOs, boards, and sales forces don't have technical backgrounds.)

> to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?

And how do you get oversight?

That's a political problem, not a philosophical problem. If you want to sway the opinions of regulators and legislators, please do not reach for the philosophy dept. Convincing regulators and legislators to do stuff is not the job of a philosopher. So what is the job of a philosopher?

This kind of brings us back to my original point from which we've now strayed. Philosophy is a field of study. The job of a philosopher is to do philosophy. In order to do that job well, philosophers require more input from technical experts than they currently receive.

This matters precisely because philosophy is important; good philosophy is the lifeblood of a healthy society, but it's increasingly difficult to do good philosophy without some scientific training.

Re:Software Needs Philosophers (2006) by jimktrains2 on Monday December 14, 2020 @10:52AM (#25418919)
I do not sell such software.

You blame companies, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree.

Regulation is besides the point. The comment I replied to said that software engineers have never held the belief that their algorithms aren't biased and I was simply pointing out that that is either false on a large scale or those engineers are complicit with the use of their software in unethical ways because of those bias issues in the software.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @11:00AM (#25419023)
> but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell?

Those engineers understand the obvious problems with facial recognition software. Everyone does.

The engineers building this software don't need philosophical training. Instead, they need a personal/professional ethics (or, in lieu of such self-regulatory instincts, government regulation). Again, neither of those presents a philosophical problem.

> Other engineering disciplins whole-heartedly disagree. Regulation is besides the point.

I disagree. Other engineers disagree precisely because they operate in highly regulated disciplines. PE codes of ethics are not (just) personal ethical commitments; rather, they are (also) backed by the force of law. BTW, lots of engineers build WMDs.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @11:45AM (#25419637)
I think it's helpful to talk about "bias issues" specifically. When we talk about them in the abstract, they always seem to carry this boogeyman-ish quality, but when we talk about them as concrete instances, we can almost always (and rather easily) come up with concrete mitigations. For example, some people have rightly expressed concern about statistical models of criminality derived from law enforcement data, so we augment and weigh that data with other sources, such as victim surveys, hospital records, and so forth. Of course, that doesn't eliminate all bias (we can never completely eliminate bias) but it's a significant improvement, especially when the alternative to "using a formal statistical model" is "using law enforcement officers' personal judgment". We can also make available certain information about the training data set and the methodology to provide an audit trail. Very rarely are these ethical issues really as insurmountable as critics seem to imply when they speak about "biased algorithms" in the abstract.
Re:Software Needs Philosophers (2006) by YawningAngel on Monday December 14, 2020 @10:37AM (#25418723)
We don't sell that software because it is the consensus of those qualified to judge that it's a good idea, we sell it because we live in a market economy, it is legal, and institutions want to buy it. The problem here is with the law and the institutions.
Re:Software Needs Philosophers (2006) by jimktrains2 on Monday December 14, 2020 @10:55AM (#25418957)
You blame companies and the market economy, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree.

My point is that either the statement that it's widely held that software can be biased is wrong (or more generously that the concern is not considered), or software engineers are complicit in unethical uses of their software because of known biases.

Re:Software Needs Philosophers (2006) by YawningAngel on Monday December 14, 2020 @11:04AM (#25419060)
I think individual engineers absolutely have a duty to act ethically, and that that duty includes not working on unethical things. However, I don't think that stating that the behaviour is unethical and that people shouldn't do it helps solve the problem, whereas I do think that we could ban bullshit ML woo or reform our institutions so that they don't use it to spy on people
Re:Software Needs Philosophers (2006) by true_religion on Monday December 14, 2020 @10:54AM (#25418944)
Algorithms are just encodings of their methodology of their creators. Here's an algorithm to detect foreigners in my country:

Step 1: Calculate skin tone ratio of person

Step 2: If light skinned then assign person to bin "foreigner".

This will work in 99% of cases, so by one standard its a good algorithm that requires little effort to give valid results.

However, it's just encoded the bias that light skinned people can't be citizens, and thus fails for mixed children, refugees and the thousands multigenerational dependents of expats.

No one can really say computers are unbiased. They simply do what you tell them to do. In my opinion, we computer scientists and practicing engineers understand that, but the general population has the mystique that the machine is somehow more right than a person, simply because you can't see the person(s) who programmed the machine.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @12:17PM (#25420112)
That's not how any real-world algorithms work (and maybe this is exactly your point); rather, provided they learned on a representative sample, they'd learn that there's a 99% chance that a given light-skinned person is a non-citizen (or more likely, that the lighter the skin, the less likely they are to be a citizen--i.e., continuous and not discrete). For a given light-skinned person, they'd return a confidence of 99% that the person in question is a non-citizen. "Bias" would be a person looking at that confidence interval and misinterpreting it as 100%.

Additionally, it's problematic because it's easily conned--for example, if it's used to determine who is eligible to vote (even if we take care not to round the confidence interval up to 100% by classifying 1% of light-skinned people as eligible and 1% of dark-skinned people as ineligible), someone could haul in a bunch of foreigners with dark skin to commit voter fraud. This also isn't "algorithmic bias", but rather foolishness in the application of the algorithm (of course, it's impossible to conceive of a legitimate purpose for an algorithm that maps statistically between skin color and citizenship).

Re:Software Needs Philosophers (2006) by true_religion on Monday December 14, 2020 @02:46PM (#25422246)
> provided they learned on a representative sample

From this, I think you're talking about deep learning algorithms.

However, the term algorithm predates practical AI, deep learning, neural nets, etc.

I would know because I sat for an algorithms exam when the fastest consumer computers had speeds measured in single-digit MHz.

An algorithm is essentially just a series of steps that a person (or more commonly now adays, a computer) can follow along with to achieve a specific goal.

Talking about stochastic systems, back propagation, or even simple probability and confidence intervals just muddies the issue. At its core, algorithms can merely be something defined deterministically, and programmed by hand by a human being.

By the way, the system I described isn't something in actual use by machines (I hope). It's just the thought pattern of biased people in my country, a sort of mental short cut used to determine what level of service you get.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @06:08PM (#25424641)
I thought we were talking about machine algorithms, but it doesn't really matter. We're talking about a simple statistical mapping between two boolean properties: skin tone and citizenship. You stipulated already that, for a given light-skinned person, the algorithm (machine or human) will return "non-citizen" with 99% confidence, and that that confidence interval is correct.

As far as I can tell, we can conclude from those facts that there is no bias (maybe I'm wrong, but I consider bias to be an errant belief); however, there are still potential issues:

* For what applications is 99% confidence acceptable? Can we do better by avoiding a statistical analysis altogether, e.g., by using citizenship records?

* Do we really believe/trust that humans are capable of 99% confidence (calling into question the fact that you established in your original post)?

* Are users of this algorithm rounding 99% up to 100%, thus disenfranchising one group 1% of the time (as opposed to distributing the error proportionately between the two groups)?

Re:Software Needs Philosophers (2006) by adamsea on Monday December 14, 2020 @11:10AM (#25419156)
Very good point. And, this algorithm would totally fail in my country! As the U.S. has a wide variety of citizens of various skin tones. Which I'm very happy about : ).

Which I think supports what you're saying; not only do algorithms have biases, but we also can't assume they are universally applicable across a domain (in this case that a single facial recognition algorithm will have the same degree of accuracy in all places across the globe).

Re:Software Needs Philosophers (2006) by true_religion on Monday December 14, 2020 @12:05PM (#25419915)
Yep, I agree. Facial recognition is a field where you commonly see overfitting to culture where the tech package was created in.

One dead stupid example, I can think of is biometric cameras being unable to differentiate between faces accurately on the default settings. The cameras are sourced from Europe, and universally use an exposure setting that's too short to do anything more than bring out the whites of someones eyes if you have dark skin tones and are in a normally lit room. Just setting the explore to be a little longer, results in decent results for dark skin tones (and of course, worse results for light skin tones).

Consumer technologies like iPhone camera's on the other hand get it right, because they were forced to start testing on different culture groups due to bad press, but their ubiquity leads to people forgetting that every single setting in a camera can lead to better or worse outcomes, and you can't always rely on having a fast processor like an iPhone to autocorrect for you.