The Dutch benefits scandal: a cautionary tale for algorithmic enforcement

Re:The Dutch benefits scandal: a cautionary tale for algorit... by defrost on Monday February 06, 2023 @07:06PM (#34686489)

    Systems that were meant to detect misuse of the benefits scheme, mistakenly labelled over 20,000 parents as fraudsters. 

    Amongst the upheaval, little attention was brought to the fact that the tax authority was making use of algorithms to guide its decision-making.

    In a report by the Dutch Data Protection Authority, it became clear that a ‘self-learning’ algorithm was used to classify the benefit claims.

Whether it's pure AI or simple big spreadsheet chopping off tens of thousands that "match some filter" fraud detection is rarely that simple.

When services impact real human beings and false positives have consequences that fall on the least resourced it's important to follow through with human eyeballs and good channels of appeal.

In Australia the parallel mistake was the Robodebt Scheme:

    On 29 May 2020, the Morrison Government announced that it would scrap the debt recovery scheme, with 470,000 wrongly-issued debts to be repaid in full.

    Initially, the total sum of the repayments was estimated to be A$721 million, however in November 2020 this figure expanded to A$1.2 billion after the Australian government settled a class-action lawsuit before it could go to trial.

https://en.wikipedia.org/wiki/Robodebt_scheme

https://robodebt.royalcommission.gov.au/

[ Parent | on HN ]

Re:The Dutch benefits scandal: a cautionary tale for algorit... by bxjx on Monday February 06, 2023 @07:36PM (#34686796)

I too was going to mention Robodebt.

In addition to wasting money, it caused all sorts of trauma to already vulnerable people - sadly in some cases leading to suicide (https://en.wikipedia.org/wiki/Robodebt_scheme#Allegations_of...).

My team at GetUp built a tool to automate initiating the process of challenging these requests https://fraudstop.com.au/

[ Parent | on HN ]

Re:The Dutch benefits scandal: a cautionary tale for algorit... by RileyJames on Monday February 06, 2023 @08:18PM (#34687184)

I’d be very keen to know more about fraudstop. How it was built, the development process of the product and the green lighting of the project within getup. Was it internal only? Was anything open sourced?

I’m aware of GetUp, but this is the first I’ve heard of a product they’ve developed. And that they have a team that’s capable of such developments.

[ Parent | on HN ]

Re:The Dutch benefits scandal: a cautionary tale for algorit... by defrost on Monday February 06, 2023 @08:32PM (#34687323)

Good on you all for making tools to assist.

Robodebt was brutal in it's application and general air of intimidation, I wasn't personally affected but I fielded a few calls on behalf of several people I knew who felt crushed by what for them were significant debts (several times any savings they had in situations where they had zero income over enough to barely cover very basic cost of living).

[ Parent | on HN ]

Re:The Dutch benefits scandal: a cautionary tale for algorit... by EdwardDiego on Monday February 06, 2023 @10:12PM (#34688239)

Kudos to your team.

...and holy hell, they charged a 10% recovery fee?

At least in NZ, when our social welfare system recovers an overpayment, there's no interest or fees. The only punitive measures used are criminal prosecution if you overtly committed fraud to a significant extent.

[ Parent | on HN ]

Re:The Dutch benefits scandal: a cautionary tale for algorit... by thombat on Tuesday February 07, 2023 @03:43AM (#34690284)

The Deputy Minister of Information explains it well in Terry Guillam's "Brazil":

INTERVIEWER: And the cost of it all, Deputy Minister? Seven percent of the gross national produce ...

HELPMANN: I understand this concern on behalf of the tax-payers. People want value for money and a cost-effective service. That is why we always insist on the principle of Information Retrieval Charges. These terrorists are not pulling their weight, and it's absolutely right and fair that those found guilty should pay for their periods of detention and the Information Retrieval Procedures used in their interrogation.

And later on a well-meaning guard advises the protagonist:

               Don't fight it, son .... confess
               quickly ...  If you hold out
               too long you could jeopardise your
               credit rating.

[ Parent | on HN ]