Not sure if it made into the final text. But some drafts of the GDPR have provisions for rights for explaination

Say your business is obviously identified as violating ToS, e.g. turns out to be a prohibited category, they can tell you and we've clearly seen cases on HN where explanations had been provided on account closure or asking support.

There are cases like money laundering suspicions where they cannot though.

Because what would they say? The weight #9534 in neural cell #12956 in layer #34 exceeded value of 0.62f?

> Because what would they say? The weight #9534 in neural cell #12956 in layer #34 exceeded value of 0.62f?

No, obviously they would present the results of manual review conducted by humans after some ML algorithm flagged it. They do manual human review before randomly banning their customers, right? RIGHT?!

A company could come up with a more understandable (and more actionable) explanation by running some what-if scenarios. Make small tweaks to the input to the neural net until the net stops rejecting the account, then tell the customer how those inputs differ from their actual data.

For example, if tweaking the input to remove 5% of the chargebacks and the account passes you tell the customer their account was suspended for having too high a chargeback rate.