News for hackers. Stuff that matters.
Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:13AM (#25418429)
I am neither a software developer nor a philosopher, but I am a Computer Science researcher who spends a fair amount of time talking with colleagues in the Philosophy department.

My view is mostly the opposite of the headline -- philosophers need scientists/engineers/practitioners more than the other way around. Asking and attempting to answer big questions about the world requires, first, understanding the world. And today, understanding the world requires immense technical expertise.

One example: philosophers are quick to opine on the problem of bias in machine learning, but almost all lack even a CS 1 level understanding of answers to basic questions like "what is an algorithm?", and therefore mistake these fairly straightforward questions as somehow profound [1]. And all but a precious few philosophers [2] are frankly technically unqualified for discussions about how to actually address bias in ML.

Things only get worse when we enter questions about AI & Theory of Mind, computer ethics, etc.

--

[1] Of course, "what is an algorithm?" can result in conversations that lead to deep questions; my point is that there is a standard and more-or-less fit for purpose definition that's given in every CS 1 class, and understanding that definition is a prerequisite for deeper interpretations of the question. Diving off the deep end without having e.g. written and analyzed a simple sorting algorithm will always be a waste of time.

[2] Often those with either excellent training in the analytic tradition or, more commonly, some serious background in a quantitative field such as Mathematics or Physics.

Re:Software Needs Philosophers (2006) by ntsplnkv2 on Monday December 14, 2020 @11:09AM (#25419139)
I disagree. You don't need to understand exactly how a nuclear bomb works to understand its impact.

We've let scientists/engineers/practitioners control the steering wheel for decades, and what do we have? Insidious adtech, lies spreading uncontrollably, populism at extreme highs across the world, etc. All to make more money. And you think we need more of that in this debate? We need philosophy first - not second.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @11:18AM (#25419260)
> I disagree. You don't need to understand exactly how a nuclear bomb works to understand its impact.

Sure, but critique is the easy part of anything (including philosophy). You do need to understand how a nuclear bomb works in order to eg design an effective monitoring program. And not just how it works, but how it is built.

BTW, you also don't exactly need a professional philosopher to explain the dangers of a WMD or the pitfalls of facial recognition. Broad critique is the easy part. Specific critique is what leads to solutions, and specificity requires expertise.

> We've let scientists/engineers/practitioners control the steering wheel for decades

But we haven't!!!

How many POTUSes have been STEM people?

How many representatives?

How many senators?

How many governors?

How many federal judges?

How many CEOs?

Very, very precious few people who control the actual machines of power in our society have any amount of STEM training.

So, no, technologists have not been at the healm of the United States. On the contrary, non-technical people -- lawyers and business people, mostly -- have been managing a highly technical world for decades. That's why we had no economic policy to manage the impact of automation in the 1990s. It's why we still have no climate policy. And it's why we had a complete dearth of privacy regulation in the early 2000s.

Re:Software Needs Philosophers (2006) by ntsplnkv2 on Monday December 14, 2020 @11:37AM (#25419528)
>Sure, but critique is the easy part of anything (including philosophy). You do need to understand how a nuclear bomb works in order to eg design an effective monitoring program. And not just how it works, but how it is built.

That's just logistics - it comes after your philosophical understanding of the item. We don't have to monitor things that aren't bad. We monitor nuclear proliferation because we've already made a philosophical judgement on it. It came first.

> But we haven't!!!

The United States is an open market system. What drives most innovation is the free market, not the government. The government is almost always reactionary by nature. Software engineers have free choice to work on what they please. They choose to work for big money on insidious technology and they know it. You can't just blame someone else.

> So, no, technologists have not been at the healm of the United States. On the contrary, non-technical people -- lawyers and business people, mostly -- have been managing a highly technical world for decades. That's why we had no economic policy to manage the impact of automation in the 1990s. It's why we still have no climate policy. And it's why we had a complete dearth of privacy regulation in the early 2000s.

This logic is circular. We live in a highly technological world that can only be created by scientists/engineers, but they have no power. So how did we get to this highly technological world if they had no influence or power? Clearly they had power and clout - to suggest otherwise is just, well, wrong. No one is holding a gun to any software engineer's head in the US. None of them have to work on adtech. But they do, to make money, more than most people need. So spare me that we need more of these types of "philosophers".

We don't have climate policy because people stand to lose money, just like we have software engineers that refuse to take pay cuts to work on actual helpful technology instead of adtech.

Re:Software Needs Philosophers (2006) by AnimalMuppet on Monday December 14, 2020 @12:33PM (#25420372)
> That's just logistics - it comes after your philosophical understanding of the item. We don't have to monitor things that aren't bad. We monitor nuclear proliferation because we've already made a philosophical judgement on it. It came first.

That's true, I suppose, but unskilled laborers routinely achieve the level of philosophical sophistication required to make that judgment. It doesn't help your argument about the primacy of philosophy if it means the kind of "philosophy" achieved by the man in the street.

> We don't have climate policy because people stand to lose money, just like we have software engineers that refuse to take pay cuts to work on actual helpful technology instead of adtech.

That's true, but we also don't have a society that will pay more for actual helpful technology than it will for adtech. Why blame the software engineers for not making themselves martyrs? Why not blame the CEOs for not making themselves martyrs? Why not blame the people for wanting the wrong things?

Re:Software Needs Philosophers (2006) by ntsplnkv2 on Monday December 14, 2020 @01:32PM (#25421231)
I'm not sure what exclusivity has to do with primacy.

In any case that is an easy example. But what about the example of the use of the atomic bombs in Hiroshima? That is much more debatable and isn't as clear cut - so we need more advanced discussions to make a better decision in that case.

> That's true, but we also don't have a society that will pay more for actual helpful technology than it will for adtech. Why blame the software engineers for not making themselves martyrs? Why not blame the CEOs for not making themselves martyrs? Why not blame the people for wanting the wrong things?

I'm not arguing that software engineers should be martyrs. I'm arguing that software engineers bear culpability as much as anyone else, and to suggest that they are more necessary than philosophy as incorrect.

You touch an important problem. We could have a discussion about the ethics - who is responsible, who is not. What do you know - philosophy.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @11:43AM (#25419607)
> That's just logistics - it comes after your philosophical understanding of the item. We don't have to monitor things that aren't bad. We monitor nuclear proliferation because we've already made a philosophical judgement on it. It came first.

But professional philosophers do not all agree on the morality of developing nuclear weapons, nor do those who do agree have particularly convincing answers about how to actually denuclearize. So I'm not exactly sure how professional philosophy is supposed to solve this problem -- they don't agree, and even those that do offer no particularly useful solutions.

"Software Needs Philosophers" is NOT the same as "software engineers should abide by a strict code of ethics that aligns with my personal ethics". The latter is simply equivalent to the observation that "the world would be better if only I had perfect fiat".

Sans such fiat, you need politics. And, again, philosophers do philosophy. Not politics.

Regarding the rest of the post, the USA is run by business people and lawyers. To look at who occupies the major seats of power in government and business but then conclude otherwise requires a tremendous amount of mental gymnastics. Lots of relatively powerless people do things that are crucial to society -- factory workers, migrant farm workers, etc.

Do you really think better philosophy would prevent software engineers from working on ethically dubious projects? I seriously doubt it. I also doubt that putting technologists in charge of everyone would result in a wonderful utopia. But don't blame the state of the world on technologists when business people and lawyers are the ones setting the rules and leading the teams that play by those rules.

Anyways, this is all quite far afield from the original thesis of my top-level post, which was simply that professional philosophers need STEM professionals in order to do philosophy, perhaps more so than STEM professionals need philosophy to do good engineering/science.

Which itself is, admittedly, quite far afield from the original article :)

Re:Software Needs Philosophers (2006) by ntsplnkv2 on Monday December 14, 2020 @12:57PM (#25420746)
> this is all quite far afield from the original thesis of my top-level post,

This is accurate. I am not going to refute your points directly, I don't think they're necessarily relevant for the crux of your argument and do a fair bit of obfuscation. In your post it seems more that you are arguing against philosophy - not for "philosophers needing stem professionals."

> which was simply that professional philosophers need STEM professionals in order to do philosophy, perhaps more so than STEM professionals need philosophy to do good engineering/science.

"good" engineering/science - what is that? Define it.

I just don't buy the argument that philosophy needs stem to do its job. Not knowing how every algorithm works doesn't mean you can't discuss its impacts, effects, if it's ethical or not.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @01:57PM (#25421585)
> "good" engineering/science - what is that? Define it.

You don't need to know the definition of a "good" joint to make a strong table. You just need to know how to make one.

> Not knowing how every algorithm works doesn't mean you can't discuss its impacts, effects, if it's ethical or not.

Do you work with professional philosophers regularly?

I recently had a half-hour conversation with a respected tenured philosopher who had strong opinions about ML but literally did not understand what a computer is.

He consistently confused the words "logarithm" and "algorithm".

He regularly humanized machines as if they are sentient AIs. This had practical effects; he drew all sorts of wild conclusions about how you might "fix" a "biased AI" that would be impossible to implement.

He did not understand what the word "data" means. E.g., we got into a discussion about whether a picture of a human face is "data". And not in some profound way. In the way that made it clear he doesn't understand how modern facial recognition algorithms are built/designed. In a way that was simply not a useful point for conversation.

In short, he was doing bad philosophy because he didn't have enough technical background to understand which philosophical questions were amusingly nonsensical and which were actually interesting/difficult. Or even the basic vocabulary to not sound like a fool.

We eventually had a good conversation, because he was smart and willing to learn. But without a STEM expert in the loop, his opinions were worse than useless because they implied solutions that any would elicit bemusement from anyone trying to actually implement a better system -- or even anyone trying to decide what is/isn't ethical to build!

> In your post it seems more that you are arguing against philosophy - not for "philosophers needing stem professionals.

Hm. I stated: "philosophers need scientists/engineers/practitioners more than the other way around. Asking and attempting to answer big questions about the world requires, first, understanding the world. And today, understanding the world requires immense technical expertise"

If I were "arguing against philosophy" [1], then why would I invest time in working with philosophers?! Good philosophy is needed now more than ever.

What is useless -- and not just useless, but even harmful -- is the sort of terrible philosophy that's often done by people who don't take the time to understand the thing they are opining about.

[1] "Arguing against philosophy", I think, is a bit of an oxy moron anyways. Wouldn't any argument against philosophy be a work of philosophy?

Re:Software Needs Philosophers (2006) by ntsplnkv2 on Tuesday December 15, 2020 @08:24AM (#25430041)
> You don't need to know the definition of a "good" joint to make a strong table. You just need to know how to make one.

That's not what I asked. Please answer the question: "good" engineering/science - what is that? Define it.

> Do you work with professional philosophers regularly? I recently had a half-hour conversation...

Do you normally make sweeping generalizations based on one man's half hour conversation with you? And yes, I do.

> If I were "arguing against philosophy" [1], then why would I invest time in working with philosophers?! Good philosophy is needed now more than ever.

And now we're full circle. You've gone so circular that you've agreed with my point - we need philosophers more than we need STEM professionals.

Perhaps what you're really trying to say, is, "we need more modern philosophers." They will come. Nothing is stopping you from becoming one - but you can't throw out the philosophy side, even if technical solutions to problems require specific expertise.

Re:Software Needs Philosophers (2006) by throwawaygh on Tuesday December 15, 2020 @09:11PM (#25438178)
Have a great week.
Re:Software Needs Philosophers (2006) by adamsea on Monday December 14, 2020 @11:23AM (#25419312)
Good point. And I just want to throw on to that (not saying folks are disagreeing w/this) that IMHO some amount of technical expertise is necessary but not sufficient for the kind of socially responsible decision-making we are talking about.

JFK (or whomever) didn't need to be a physicist, but he did need to understand a little physics. And math, etc.

Re:Software Needs Philosophers (2006) by adamsea on Monday December 14, 2020 @11:21AM (#25419300)
Do you think there is room for various kinds philosophical thought within Computer Science itself? Honest question.

As an undergrad studying Comp Sci and also taking philosophy/literature classes it struck me just how rich the Computer Science metaphors are, and how much creativity seemed to go into coming up with a lot of the fundamental ideas of Computer Science.

But of course it's also possible that I was applying my own bias; I know most of the Comp Sci folks from the 50s/60s/70s were pretty heavy math folks.

But pursuing a graduate Comp Sci degree is something I've considered on and off. Alan Kay has always been especially inspiring.

TBH I've also felt like there's some unexplored areas where Computer Science and Poetics, or Literary Theory, could make a really interesting combination. Again it's a question of rich metaphors and investing meaning into use-of-language in a very comprehensive and granular way (grammar, one verb or adjective versus another, theories of narrative, etc.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @11:41AM (#25419585)
> Do you think there is room for various kinds philosophical thought within Computer Science itself? Honest question.

Yes, of course!

Many areas of computer science grew out of various areas of philosophy, and that's still true.

I guess my main thesis in this thread is that, today, philosophers need STEM in order to do good philosophy more than STEM folks need philosophy to do good engineering/science. "More than" is comparative, not absolute, and of course STEM folks can benefit tremendously by interacting with philosophers (see: the first line of my top-level post).

> But pursuing a graduate Comp Sci degree is something I've considered on and off.

There are some great CS departments that interact heavily with Philosophy departments. Carnegie Mellon even has a joint PhD program: http://logic.cmu.edu/

Re:Software Needs Philosophers (2006) by adamsea on Wednesday December 16, 2020 @10:07AM (#25443411)
Thank you for your response - much appreciated! And for pointing out the Carnegie Mellon program.

What is the academic job market like this days for CS Phd's, if I may ask?

Re:Software Needs Philosophers (2006) by btrask on Monday December 14, 2020 @03:40PM (#25422971)
This article is such a great opportunity for introspection ("what do we need in order to do better?"). It's too bad that the top comment has turned it into an opportunity for egotism ("they need us!").
Re:Software Needs Philosophers (2006) by throwawaygh on Tuesday December 15, 2020 @09:17PM (#25438221)
This isn’t idle speculation. I’m not a software engineer anymore. Also, I publish with philosophers.

We do not need input from the typical professional philosopher to do better.

Re:Software Needs Philosophers (2006) by alisonkisk on Monday December 14, 2020 @10:17AM (#25418473)
Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity. Software, being mostly based on math is a prime example of "map that is not the territory".
Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:24AM (#25418556)
> Software people not philosophers promulgated the nonsense idea that "algorithms can't be biased", so I think software people need plenty of help understanding what an algorithm is, with help from philosophy and humanity.

(E: throwaway894345's comment is of much higher quality than mine.)

Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view. And, I know a few philosophers who also repeat this nonsense.

Those that do hold this view IME are almost always folks who are invested in a personal politics that is congruent with dismissing concerns about bias. Without exception, IME, they're also knee-jerk antagonistic toward other ideas about bias that don't involve computers (eg they're the first ones to complain about implicit bias training). These people also tend to be the annoyingly pedantic type ("bias has a technical meaning in statistics!" sort of thing).

In other words, they're politically invested, and no amount of input from philosophers -- or technicians/scientists -- is going to change their strongly held personal politic beliefs.

Again, anyone who has spent a moment around philosophers knows that they are just humans who are not immune from the exact same problem.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @10:40AM (#25418750)
To add more context to your point, originally the "algorithms are biased" people were arguing (or strongly implying) that all algorithms were biased and that they would always be biased so we should never use them (or something to that end, anyway). I don't know that their opponents were arguing that algorithms "aren't biased" as much as for a clearer understanding of what is meant by the claim "algorithms are biased" because there's no obvious definition for "bias" that lends itself to the conclusion that algorithms will always be more bias (or "worsely biased" in case that's not the same as "more bias") than people. The latter camp specifically argued that algorithms aren't inherently biased, but rather they "inherit" any biases encoded in the training data, and thus the "algorithms are biased" group would do well to frame their concerns in terms of concerns about training data sets and focus on how we can mitigate those concerns.

Over time, as is often the case with healthy debate, the "algorithms are biased" people did start to frame their concerns this way and now we have robust and productive conversations about ethically training algorithms (instead of a strict prohibition on algorithms). This is strictly a good outcome, except that some people are misremembering the argument as one group arguing that "algorithms can't be biased" in any sense of the phrase (although I'm sure there are some fringe few people for whom that's true as well, but as you point out, that was never a prominent position) and we forget that the "algorithms are biased" people weren't always making the nuanced arguments that they've been pushed to make today.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:44AM (#25418803)
Excellent clarification; thanks for taking the time to write this up!
Re:Software Needs Philosophers (2006) by jimktrains2 on Monday December 14, 2020 @10:26AM (#25418593)
> Very few serious software engineers -- and virtually zero computer scientists -- have ever held this opinion. It's an extreme minority view.

And yet here we are selling software, like facial recognition, to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @10:28AM (#25418613)
> And yet here we are selling software, like facial recognition

Speak for yourself. (Also, last I checked, most CEOs, boards, and sales forces don't have technical backgrounds.)

> to organizations that can ruin people and have little oversight based on how well those same algorithms are supposed to work?

And how do you get oversight?

That's a political problem, not a philosophical problem. If you want to sway the opinions of regulators and legislators, please do not reach for the philosophy dept. Convincing regulators and legislators to do stuff is not the job of a philosopher. So what is the job of a philosopher?

This kind of brings us back to my original point from which we've now strayed. Philosophy is a field of study. The job of a philosopher is to do philosophy. In order to do that job well, philosophers require more input from technical experts than they currently receive.

This matters precisely because philosophy is important; good philosophy is the lifeblood of a healthy society, but it's increasingly difficult to do good philosophy without some scientific training.

Re:Software Needs Philosophers (2006) by jimktrains2 on Monday December 14, 2020 @10:52AM (#25418919)
I do not sell such software.

You blame companies, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree.

Regulation is besides the point. The comment I replied to said that software engineers have never held the belief that their algorithms aren't biased and I was simply pointing out that that is either false on a large scale or those engineers are complicit with the use of their software in unethical ways because of those bias issues in the software.

Re:Software Needs Philosophers (2006) by throwawaygh on Monday December 14, 2020 @11:00AM (#25419023)
> but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell?

Those engineers understand the obvious problems with facial recognition software. Everyone does.

The engineers building this software don't need philosophical training. Instead, they need a personal/professional ethics (or, in lieu of such self-regulatory instincts, government regulation). Again, neither of those presents a philosophical problem.

> Other engineering disciplins whole-heartedly disagree. Regulation is besides the point.

I disagree. Other engineers disagree precisely because they operate in highly regulated disciplines. PE codes of ethics are not (just) personal ethical commitments; rather, they are (also) backed by the force of law. BTW, lots of engineers build WMDs.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @11:45AM (#25419637)
I think it's helpful to talk about "bias issues" specifically. When we talk about them in the abstract, they always seem to carry this boogeyman-ish quality, but when we talk about them as concrete instances, we can almost always (and rather easily) come up with concrete mitigations. For example, some people have rightly expressed concern about statistical models of criminality derived from law enforcement data, so we augment and weigh that data with other sources, such as victim surveys, hospital records, and so forth. Of course, that doesn't eliminate all bias (we can never completely eliminate bias) but it's a significant improvement, especially when the alternative to "using a formal statistical model" is "using law enforcement officers' personal judgment". We can also make available certain information about the training data set and the methodology to provide an audit trail. Very rarely are these ethical issues really as insurmountable as critics seem to imply when they speak about "biased algorithms" in the abstract.
Re:Software Needs Philosophers (2006) by YawningAngel on Monday December 14, 2020 @10:37AM (#25418723)
We don't sell that software because it is the consensus of those qualified to judge that it's a good idea, we sell it because we live in a market economy, it is legal, and institutions want to buy it. The problem here is with the law and the institutions.
Re:Software Needs Philosophers (2006) by jimktrains2 on Monday December 14, 2020 @10:55AM (#25418957)
You blame companies and the market economy, but is that to mean that engineers are supposed to turn a blind eye to ethical issues in the products their employers sell? Other engineering disciplins whole-heartedly disagree.

My point is that either the statement that it's widely held that software can be biased is wrong (or more generously that the concern is not considered), or software engineers are complicit in unethical uses of their software because of known biases.

Re:Software Needs Philosophers (2006) by YawningAngel on Monday December 14, 2020 @11:04AM (#25419060)
I think individual engineers absolutely have a duty to act ethically, and that that duty includes not working on unethical things. However, I don't think that stating that the behaviour is unethical and that people shouldn't do it helps solve the problem, whereas I do think that we could ban bullshit ML woo or reform our institutions so that they don't use it to spy on people
Re:Software Needs Philosophers (2006) by true_religion on Monday December 14, 2020 @10:54AM (#25418944)
Algorithms are just encodings of their methodology of their creators. Here's an algorithm to detect foreigners in my country:

Step 1: Calculate skin tone ratio of person

Step 2: If light skinned then assign person to bin "foreigner".

This will work in 99% of cases, so by one standard its a good algorithm that requires little effort to give valid results.

However, it's just encoded the bias that light skinned people can't be citizens, and thus fails for mixed children, refugees and the thousands multigenerational dependents of expats.

No one can really say computers are unbiased. They simply do what you tell them to do. In my opinion, we computer scientists and practicing engineers understand that, but the general population has the mystique that the machine is somehow more right than a person, simply because you can't see the person(s) who programmed the machine.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @12:17PM (#25420112)
That's not how any real-world algorithms work (and maybe this is exactly your point); rather, provided they learned on a representative sample, they'd learn that there's a 99% chance that a given light-skinned person is a non-citizen (or more likely, that the lighter the skin, the less likely they are to be a citizen--i.e., continuous and not discrete). For a given light-skinned person, they'd return a confidence of 99% that the person in question is a non-citizen. "Bias" would be a person looking at that confidence interval and misinterpreting it as 100%.

Additionally, it's problematic because it's easily conned--for example, if it's used to determine who is eligible to vote (even if we take care not to round the confidence interval up to 100% by classifying 1% of light-skinned people as eligible and 1% of dark-skinned people as ineligible), someone could haul in a bunch of foreigners with dark skin to commit voter fraud. This also isn't "algorithmic bias", but rather foolishness in the application of the algorithm (of course, it's impossible to conceive of a legitimate purpose for an algorithm that maps statistically between skin color and citizenship).

Re:Software Needs Philosophers (2006) by true_religion on Monday December 14, 2020 @02:46PM (#25422246)
> provided they learned on a representative sample

From this, I think you're talking about deep learning algorithms.

However, the term algorithm predates practical AI, deep learning, neural nets, etc.

I would know because I sat for an algorithms exam when the fastest consumer computers had speeds measured in single-digit MHz.

An algorithm is essentially just a series of steps that a person (or more commonly now adays, a computer) can follow along with to achieve a specific goal.

Talking about stochastic systems, back propagation, or even simple probability and confidence intervals just muddies the issue. At its core, algorithms can merely be something defined deterministically, and programmed by hand by a human being.

By the way, the system I described isn't something in actual use by machines (I hope). It's just the thought pattern of biased people in my country, a sort of mental short cut used to determine what level of service you get.

Re:Software Needs Philosophers (2006) by throwaway894345 on Monday December 14, 2020 @06:08PM (#25424641)
I thought we were talking about machine algorithms, but it doesn't really matter. We're talking about a simple statistical mapping between two boolean properties: skin tone and citizenship. You stipulated already that, for a given light-skinned person, the algorithm (machine or human) will return "non-citizen" with 99% confidence, and that that confidence interval is correct.

As far as I can tell, we can conclude from those facts that there is no bias (maybe I'm wrong, but I consider bias to be an errant belief); however, there are still potential issues:

* For what applications is 99% confidence acceptable? Can we do better by avoiding a statistical analysis altogether, e.g., by using citizenship records?

* Do we really believe/trust that humans are capable of 99% confidence (calling into question the fact that you established in your original post)?

* Are users of this algorithm rounding 99% up to 100%, thus disenfranchising one group 1% of the time (as opposed to distributing the error proportionately between the two groups)?

Re:Software Needs Philosophers (2006) by adamsea on Monday December 14, 2020 @11:10AM (#25419156)
Very good point. And, this algorithm would totally fail in my country! As the U.S. has a wide variety of citizens of various skin tones. Which I'm very happy about : ).

Which I think supports what you're saying; not only do algorithms have biases, but we also can't assume they are universally applicable across a domain (in this case that a single facial recognition algorithm will have the same degree of accuracy in all places across the globe).

Re:Software Needs Philosophers (2006) by true_religion on Monday December 14, 2020 @12:05PM (#25419915)
Yep, I agree. Facial recognition is a field where you commonly see overfitting to culture where the tech package was created in.

One dead stupid example, I can think of is biometric cameras being unable to differentiate between faces accurately on the default settings. The cameras are sourced from Europe, and universally use an exposure setting that's too short to do anything more than bring out the whites of someones eyes if you have dark skin tones and are in a normally lit room. Just setting the explore to be a little longer, results in decent results for dark skin tones (and of course, worse results for light skin tones).

Consumer technologies like iPhone camera's on the other hand get it right, because they were forced to start testing on different culture groups due to bad press, but their ubiquity leads to people forgetting that every single setting in a camera can lead to better or worse outcomes, and you can't always rely on having a fast processor like an iPhone to autocorrect for you.