News for hackers. Stuff that matters.
Re:AI ethics research conference suspends Google sponsorship by JumpCrisscross on Wednesday March 03, 2021 @10:30AM (#26329605)
> AI ethics researchers are like hiring hackers working in security

One of these can be objectively evaluated for skill. There may be unwanted side effects. But the skill is there or it isn’t. That does not apply to the other, which makes their hiring and firing fundamentally different.

Re:AI ethics research conference suspends Google sponsorship by currymj on Wednesday March 03, 2021 @01:22PM (#26332397)
AI Ethics isn't a non-technical field. See the accepted papers at the conference in question:

https://facctconference.org/2021/acceptedpapers.html

it's a mix of "soft" humanities, applied ML, methodology, algorithms/theory, and HCI.

it's not any harder to evaluate an AI ethics researcher than it is any other kind of researcher.

Re:AI ethics research conference suspends Google sponsorship by jjaammee on Wednesday March 03, 2021 @11:16AM (#26330402)
Hacked or not hacked is easier to say than ethical or not. Your ethical standards maybe racism to others, everyone have their own interpretation of ethics.
Re:AI ethics research conference suspends Google sponsorship by KMag on Wednesday March 03, 2021 @01:30PM (#26332476)
Expanding on this, if you had a machine that let you see 100 years in the future, you could nearly perfectly and objectively evaluate your security team's performance by looking at the number/severity/value lost through data breaches over that period.

With AI ethicists... if you could look 100 years in the future.. there's not an objective measure of ethical outcomes you could use to evaluate the performance of your AI Ethics team. I suppose a cynic would look at the total cost spent defending and paying settlements / judgements in ethics-related lawsuits. As far as a non-cynical objective metric for AI Ethics performance, I can't think of any.

Re:AI ethics research conference suspends Google sponsorship by 8note on Wednesday March 03, 2021 @02:44PM (#26333357)
An "objective" metric would be customers and dollars lost due to losing trust.

You could also look at compliance costs?

Re:AI ethics research conference suspends Google sponsorship by KMag on Wednesday March 03, 2021 @10:59PM (#26338442)
Don't get me wrong. I think ethics work is hugely important, particularly in this new field. However, ethics are much more abstract than security, and therefore much harder to evaluate objectively.

> dollars lost due to losing trust.

Trust, ethics, and public moral outrage are hopefully related, but not the same thing. Also, measuring loss of trust and allocating changes in revenue to that change is very difficult.

> compliance costs?

I think many people would agree that there's often a big difference between legal requirements and ethics.

Re:AI ethics research conference suspends Google sponsorship by Ar-Curunir on Wednesday March 03, 2021 @10:56AM (#26330057)
So there is an objective measure for deciding how good a programmer is? Why don’t we just adopt that measure in interviews?
Re:AI ethics research conference suspends Google sponsorship by Guest42 on Wednesday March 03, 2021 @12:40PM (#26331797)
I’d say a better measure is whether the person can currently handle most of what will be asked and how quickly they can get up to speed on new things. It’s easier to figure out and more useful for the company. It also de-prioritizes those that create a buzzword salad (sometimes out of necessity) on their resume.
Re:AI ethics research conference suspends Google sponsorship by 8note on Wednesday March 03, 2021 @02:45PM (#26333372)
Ok, so what's the unit of this number, so I can rank all the members of my team?
Re:AI ethics research conference suspends Google sponsorship by 0-_-0 on Wednesday March 03, 2021 @11:14AM (#26330376)
Because an interview only lasts for a few hours.
Re:AI ethics research conference suspends Google sponsorship by ncann on Wednesday March 03, 2021 @10:40AM (#26329773)
> But the skill is there or it isn’t

If I understand what you saying correctly, you're saying AI ethics researchers have no skill?

Re:AI ethics research conference suspends Google sponsorship by bluefirebrand on Wednesday March 03, 2021 @10:49AM (#26329924)
Charitably, I think what they are trying to say is that the output of an AI ethics researcher is not directly related to the application of a measurably skill.

It is instead related much more closely to a person's sense of right and wrong.

Of course AI ethics researchers have skills. They have skill in researching, they have knowledge of AI, and presumably education in formal logic and ethics. They have skill in writing and communicating the results of their findings.

However the product they produce is not writing and it's not research. It's the commentary on the research, the recommendations based on the results they have found. It's very subjective.

It's very different than a skilled hacker applying their skill and knowledge to uncover a zero-day exploit in your software that needs to be fixed. That's a much more objective process.

Re:AI ethics research conference suspends Google sponsorship by 8note on Wednesday March 03, 2021 @02:47PM (#26333395)
So their product is peer review? That sounds like they're incredibly valuable
Re:AI ethics research conference suspends Google sponsorship by angrais on Wednesday March 03, 2021 @05:56PM (#26336016)
That depends ... Peer review is known to be a "lottery" because it is so subjective. Also known to be gained by writing papers about hot topics, etc.
Re:AI ethics research conference suspends Google sponsorship by nightowl_games on Wednesday March 03, 2021 @10:43AM (#26329826)
I agree that his statement is inflammatory, but we can steel man the statement into something truthful and palatable:

It is far easier to evaluate a security researcher's output than an ethics researcher.

I think many of us are familiar with this concept, as many of us likely preferred math over english in school as we preferred being deterministically evaluated.

Re:AI ethics research conference suspends Google sponsorship by mcguire on Wednesday March 03, 2021 @10:57AM (#26330064)
Is it? If the security researchers don't find vulnerabilities, does that mean they don't exist? If the AI ethics researchers embarrass you, aren't they doing their jobs?
Re:AI ethics research conference suspends Google sponsorship by keypusher on Wednesday March 03, 2021 @11:51AM (#26330990)
Just as a security researcher’s job is to find vulnerabilities and promote best practices, it’s not to hack and then publicly shame their employer. Ethics researchers should be the same, no? Find ethical issues and promote best practices, embarrassing your employer seems counterproductive except perhaps as a last resort.
Re:AI ethics research conference suspends Google sponsorship by joshuamorton on Wednesday March 03, 2021 @12:58PM (#26332064)
This depends. Google project zero routinely discovers vulns in Google products.

However they're empowered to fix them, essentially no questions asked. So you don't often get p0 posts about unfixed bugs in Google products.

Re:AI ethics research conference suspends Google sponsorship by 8note on Wednesday March 03, 2021 @02:48PM (#26333421)
I imagine that depends on your contract?
Re:AI ethics research conference suspends Google sponsorship by asdfasgasdgasdg on Wednesday March 03, 2021 @11:06AM (#26330238)
Since you, the employer, decide what the job is, maybe or maybe not. If you define the job as, "develop ideas and frameworks where we can accomplish our goals in the most ethical way possible," then publicly embarrassing the company might not be doing your job.
Re:AI ethics research conference suspends Google sponsorship by frenchy on Wednesday March 03, 2021 @11:12AM (#26330337)
A good ethicist & communicator will probably be able to embarrass you if you're doing something wrong, but so will a comedian.

I don't think OP is claiming that there ethicists have no skill, but rather that it's fundamentally a lot more difficult to evaluate.