News for hackers. Stuff that matters.
Re:Keep your AI claims in check by IncRnd on Monday February 27, 2023 @06:03PM (#34963257)

  Are you aware of the risks? You need to know about the reasonably
  foreseeable risks and impact of your AI product before putting it
  on the market. If something goes wrong – maybe it fails or yields
  biased results – you can’t just blame a third-party developer of
  the technology. And you can’t say you’re not responsible because
  that technology is a “black box” you can’t understand or didn’t
  know how to test.
I'm reminded of every single AI product that has been released... Good job, FTC.
Re:Keep your AI claims in check by calr on Monday February 27, 2023 @06:10PM (#34963337)
Ignorance of your "black box" excuses not
Re:Keep your AI claims in check by mr_toad on Monday February 27, 2023 @06:23PM (#34963491)
> Ignorance of your "black box" excuses not

Hopefully the FTC understands linear separability. Because I often get the impression that people who want ML models to be explicable don’t, and are expecting the mathematically impossible.

Re:Keep your AI claims in check by pretendgeneer on Monday February 27, 2023 @06:36PM (#34963622)
If the system your building is "mathematically impossible" to explain in a way that would be expected of every other system. Maybe you shouldn't be building that system.
Re:Keep your AI claims in check by foxes on Monday February 27, 2023 @06:45PM (#34963711)
Mathematical explanation is not required in many areas.

Perhaps you can’t perfectly explain it but you can at least understand it statistically.

For example is not currently possible to mathematically explain people’s behaviour, but there is statistical evidence and also accountability on an individual. Eg a doctor making a decision about a scan result.

Re:Keep your AI claims in check by witnesser2 on Monday February 27, 2023 @07:01PM (#34963874)
I raise another example, to question, you can statistically solve the famous diffusion problem.
Re:Keep your AI claims in check by jrumbut on Monday February 27, 2023 @08:17PM (#34964475)
It's even simpler than that. You don't need any particular level of understanding, you just can't lie about what your level of understanding is.

The following would seem to be OK: "This model performs well in a set of tests we devised, your mileage may vary."

Re:Keep your AI claims in check by mr_toad on Monday February 27, 2023 @06:46PM (#34963726)
Non-linear models often perform better. You’re never going to approach anything like GPT with a linear model.
Re:Keep your AI claims in check by rcme on Monday February 27, 2023 @07:27PM (#34964105)
Theoretically you could use a polynomial model which, while not linear, is pretty high on the explainability scale.
Re:Keep your AI claims in check by witnesser2 on Monday February 27, 2023 @06:56PM (#34963833)
I don't think mentioning non-linearality means you can bypass the singularities you may cause typically CUSP, buckling etc and omit the analysis intentionally or negeliccantly.
Re:Keep your AI claims in check by edoloughlin on Monday February 27, 2023 @06:51PM (#34963782)
I rather think that they (the regulators) are saying that you can’t expect to use mathematical impossibility as a legal argument to avoid liability if your AI does material harm.
Re:Keep your AI claims in check by concordDance on Monday February 27, 2023 @07:31PM (#34964141)
Doesn't this kill AI as a product?

Or at least mean that if you're too poor for a doctor or lawyer you can't fall back to the unreliable AI?

Re:Keep your AI claims in check by zelphirkalt on Monday February 27, 2023 @07:39PM (#34964213)
Not if you "keep your claims in check" and have proper oversight of the output your AI generates. Then you will be ready at all times to step in, should something go south. Yes, maybe it does not scale up unlimited with proper oversight in place, but this is exactly what Google and co are missing these days and the reason they extract cash to no end. Proper oversight is completely missing.
Re:Keep your AI claims in check by cma on Monday February 27, 2023 @07:38PM (#34964205)
Aren't they really only saying you have to disclose it in your claims about the product?
Re:Keep your AI claims in check by codemac on Monday February 27, 2023 @07:59PM (#34964351)
Exactly my read - they're just saying that if you want to pretend you are selling magic that's not allowed.

Quantifying your risk does not mean you eliminated all your risk.

We allow the sale of alcohol, tobacco, and firearms too.

Re:Keep your AI claims in check by nasseri on Monday February 27, 2023 @07:22PM (#34964068)
that is correct.
Re:Keep your AI claims in check by visarga on Monday February 27, 2023 @07:15PM (#34964010)
Actually, if you ask GPT3 to explain step by step how it arrived to the conclusion, it works pretty well and even has higher accuracy.
Re:Keep your AI claims in check by mr_toad on Monday February 27, 2023 @07:22PM (#34964071)
GPT is making it up.

It’s cobbling together an explanation from texts that explain how to solve a problem. It’s like a student who copies an answer, then copies the explanation as well.

Re:Keep your AI claims in check by visarga on Tuesday February 28, 2023 @12:22AM (#34965938)
Chain-of-thought is not the same with direct prompting. Step by step is better.

> Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. The empirical gains can be striking.

https://arxiv.org/abs/2201.11903

Re:Keep your AI claims in check by l33t233372 on Monday February 27, 2023 @07:25PM (#34964098)
But there’s no reason to believe that is actually how the result was generated.

The result was generated via a deep neural network, not by whatever explanation or reasoning it prints out when asked.

Re:Keep your AI claims in check by crote on Monday February 27, 2023 @07:22PM (#34964072)
From the FTC's perspective, stuff like that is just a technical detail. It is on the business to make sure that the final result can be explained.

If you can't explain the ML part due to whatever math detail, you better make sure you write a good wrapper around it to catch bad output which you can explain.

Re:Keep your AI claims in check by evrydayhustling on Monday February 27, 2023 @07:25PM (#34964097)
Lots of businesses deal with high complexity and open ended problems that lead to "inexplicable" risks. They address this by showing diligence in monitoring and mitigating risks, evolving their approach as new concerns are understood. AI companies should employ similar motions (and some do).
Re:Keep your AI claims in check by gfd on Monday February 27, 2023 @07:49PM (#34964265)
As someone who don't understand linear separability (at least in the way you're using it), can you explain?
Re:Keep your AI claims in check by hanniabu on Monday February 27, 2023 @08:13PM (#34964432)
Why bother quoting when it's the entire comment? Never understood that
Re:Keep your AI claims in check by reaperducer on Monday February 27, 2023 @06:20PM (#34963435)
Good job, FTC

Hopefully it goes after some people.

I once worked for a founder who was running afoul off some FTC regulations. When the team brought it to his attention, he claimed to have never heard of the FTC. When we explained what the FTC is, he started arguing with us saying it had no authority over him, and he'll make the decisions for his business.

Money really does make some people crazy.

Re:Keep your AI claims in check by cyanydeez on Monday February 27, 2023 @06:38PM (#34963659)
The FTC requires political headwinds measured in decades to do appreciably good work
Re:Keep your AI claims in check by shredprez on Monday February 27, 2023 @06:38PM (#34963651)
Does the money make them crazy or does the system reward the crazy?

Markets can produce some unexpected winners...

Re:Keep your AI claims in check by Oxidation on Monday February 27, 2023 @06:27PM (#34963535)
Are you sure that wasn't Grimes' ex and the SEC?
Re:Keep your AI claims in check by foobarian on Monday February 27, 2023 @07:14PM (#34964003)
And then the FTC came along and fined them... <pinky to mouth> one... thousand dollars!
Re:Keep your AI claims in check by mc32 on Monday February 27, 2023 @07:02PM (#34963884)
I'll be elated if the FTC would go after advertisers and ad industry in general. There are almost no ads without exaggerated claims or ad with innuendo and insinuation of function but of course would not deliver on such.
Re:Keep your AI claims in check by gwright on Monday February 27, 2023 @07:04PM (#34963900)
Re:Keep your AI claims in check by SomaticPirate on Monday February 27, 2023 @07:10PM (#34963965)
Just knowing that they have this stance makes them seem even more toothless than before… Atleast before this I assumed AI claims were outside of their regulatory domain.