News for hackers. Stuff that matters.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by version_five on Friday July 21, 2023 @10:18AM (#36814069)
There's a post by OpenAI that was posted earlier:

https://openai.com/blog/moving-ai-governance-forward

In it they mention:

  Bio, chemical, and radiological risks, such as the ways in which systems can lower barriers to entry for weapons development, design, acquisition, or use 
  Cyber capabilities, such as the ways in which systems can aid vulnerability discovery, exploitation, or operational use, bearing in mind that such capabilities could also have useful defensive applications and might be appropriate to include in a system
  The effects of system interaction and tool use, including the capacity to control physical systems
  The capacity for models to make copies of themselves or “self-replicate” 
  Societal risks, such as bias and discrimination 
I think the risks are mostly unfounded but dont really see a problem considering them. What I don't like is the "societal risks" - AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity". If we're going to make policy there needs to be crystal clarity about what this stuff means, it can't just be a proxy for stuff that does against a particular political view. Things like discrimination in employment are already illegal, I see this more as codifying a particular world view.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by phillipcarter on Friday July 21, 2023 @10:27AM (#36814189)
> AI ethics always end up just throwing out "bias" as an ill defined term that could mean whatever they want it to, the way people like to use "harm" and "toxicity"

There's a lot of specific examples of different kinds bias in models and things that can be done to improve them. A lot of that work was published in the past 10 years. One small example: https://arxiv.org/abs/1712.00193

I don't believe there's a lack of specificity going on here, at least not by those whose boots on the ground doing work on the topic.

[dead]
Re:A.I. Companies Agree to Safeguards After Pressure from th... by ke88y on Friday July 21, 2023 @10:52AM (#36814498)
Wait, what in the world are you talking about?!

The paper is about improving face attribute detection by using demographic information. They improve performance on some standard benchmarks by doing this.

The only way your characterization possibly makes any sense is if detecting facial characteristics is actually objectively more difficult for certain genders/races, which:

1. AFAICT isn't the case for most people (actually, I don't know anyone for whom this is the case, but I suppose it's possible such people exist).

2. Even if this were the case for humans, which it's not, why would we want algorithms to also be artificially handicapped at smile detection? That would be like building a calculator that messes up multiplication every once in a blue moon and takes a long time to do certain division operations and is worse at division than multiplication. Makes no god damn sense. Why the fuck would you want that? We know the right answer to "is this person smiling?". Why would we want a computer program that is bad at answering that question for particular subgroups?

> Oops the data must be wrong

The data isn't wrong. The baseline model's prediction is wrong (about simple shit like "is this person smiling?"). Using demographic representations while withholding demographic inference from the downstream face attribute detection improves the model's performance. At being correct. About simple shit like detecting smiles.

Seriously... what exactly are you claiming is wrong with the paper's methodology/setup/motivation? Did we read the same paper? Do you have trouble detecting smiles on women/men? And if so, do you think computers should have the same difficulty?

Re:A.I. Companies Agree to Safeguards After Pressure from th... by ml-anon on Friday July 21, 2023 @11:07AM (#36814704)
What exactly are “the facts evident in the data”?

For example if one were to train an LLM on HN comments aside from just learning the syntax of language, the model might be much more useful if there was a way to somehow weight utter dross like the contents of your comment vs comments from domain experts.

Re:A.I. Companies Agree to Safeguards After Pressure from th... by phillipcarter on Friday July 21, 2023 @10:49AM (#36814463)
Well, you're certainly entitled to your opinion. My point is that people doing work in this space are identifying specific problems and suggesting how to solve them, which is addressing the OP's concerns about a lack of specificity. If you don't like the methods, you can certainly engage with the authors of this work rather than write angrily about it on this forum.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by ke88y on Friday July 21, 2023 @11:23AM (#36814877)
"Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise. I mean, people will agree to this statement, but a lot of the evaluation and research methodology makes definite political commitments that practitioners won't admit are political.

The flip-side is also true: a lot of people on the "anti-ethics" side of this debate are too coy about the fact that there are definitely parts of the definition of toxicity that the vast majority of people will agree upon.

And even if we could define toxicity, what to do about it isn't obvious. E.g., let's take something uncontroversial: graphic depictions of violent rape of children. Is this something we should suppress? Not necessarily. It depends on the context. E.g., in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them. But those same descriptions shouldn't be co-opted into erotica, for example.

One way "out" is to say something like: "look, what we're interested in is providing tools for enforcement of community norms when communicating with a given audience; the community/culture gives us its definition of toxicity and we provide the tools to prevent toxic generation". But those tools aren't neutral: that could be a description of guardrails for children, and could also be a description of overtly political censorship.

It's a difficult and fraught area, and I think all sides of the debate could benefit from more empathy for the other sides of the debate. In particular, this includes presumption of honest intent. There is nothing wrong with filtering toxic content per se -- we can almost all agree that it's reasonable if a company doesn't want to buy a customer service AI that sometimes quotes Mein Kampf to customers with Jewish last names. But the techniques for doing so are not context-free goods, either.

Re:A.I. Companies Agree to Safeguards After Pressure from th... by klooney on Friday July 21, 2023 @11:46AM (#36815203)
> in descriptions of war crimes and genocides, we shouldn't censor victims who want the audience to know what happened to them

Even this is a political decision, and is generally made based on which side the decider is on.

Re:A.I. Companies Agree to Safeguards After Pressure from th... by ke88y on Friday July 21, 2023 @11:48AM (#36815252)
Agreed. It's an effective strong-man example in the context of western democracies, though.

At least if you include the word genocide. And are also careful to not explicitly point out that Native American relocation and Black Slavery were both genocides that included lots of rape [2]. And are not in a school board meeting [1]. Etc.

[1] https://www.foxnews.com/media/author-sex-slavery-book-graphi...

Re:A.I. Companies Agree to Safeguards After Pressure from th... by klooney on Friday July 21, 2023 @12:48PM (#36816177)
Honestly, those are talked about on a daily basis in all the mainstream news outlets not owned by Rupert Murdoch. Howard Zinn won a long time ago.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by ke88y on Friday July 21, 2023 @01:16PM (#36816577)
And a huge chunk of the electorate (in more than one country even!) is hugely influenced by or at least aligned with Rupert Murdoch.

These questions are political, even if you've made up your mind about them.

Re:A.I. Companies Agree to Safeguards After Pressure from th... by klooney on Saturday July 22, 2023 @09:43AM (#36826161)
I just don't agree that it's hidden by our politics. "How many Yemenis died in the war, and how" is actually an obscured and hidden topic.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by dragonwriter on Friday July 21, 2023 @11:55AM (#36815341)
> "Toxicity" in the LLM space is much less well-defined and there seems to be a reticence on the part of the research community to admit that defining "toxicity" is a necessarily subjective and often political exercise.

Is it the research community or the AI-selling industry? Because I see researchers and AI ethics advocates making that point (that toxicity is both subjective and more broadly contextual) all the time, and not just in a detached way but it is central to their arguments about openness and disclosure and public research on functionality; on the other side, I see censorship and adherence claims around particular political views of “safety” alongside opaqueness as the industry-leaders common response.

Re:A.I. Companies Agree to Safeguards After Pressure from th... by ke88y on Friday July 21, 2023 @11:56AM (#36815359)
Yeah, I should have been more specific. I think people agree to this when you say it out loud but the recognition is not actually reflected in favored evaluation and research methodology.
Re:A.I. Companies Agree to Safeguards After Pressure from th... by torginus on Friday July 21, 2023 @02:18PM (#36817434)
I feel like this is the modern equivalent of Colin Powell waving a vial of Anthrax in front of the UN assembly.