DARPA open sources resources to aid evaluation of adversarial AI defenses

Re:DARPA open sources resources to aid evaluation of adversa... by Nbox9 on Tuesday December 21, 2021 @06:35PM (#29643461)

“ART provides tools that enable developers and researchers to defend and evaluate their ML models and applications against a number of adversarial threats, such as evasion, poisoning, extraction, and inference.”

The first two attacks, evasion & poisoning highlight the incredible importance of having high quality data when training models. Evasion is false-negatives that are allowed because the model did not have a diverse enough selection of training data and poisoning can occur when the data sources are not well vetted. Data quality is probably the single biggest problem with ML models, and I wish we’d see more of a focus on it.

[ Parent | on HN ]

Re:DARPA open sources resources to aid evaluation of adversa... by orange3xchicken on Tuesday December 21, 2021 @09:54PM (#29645037)

I'm less familiar with poisoning, but at least for test-time robustness, the current benchmark for image classifiers is AutoAttack [0,1]. It's an ensemble of adaptive & parameter-free gradient-based white-box and gradient-free black-box attacks. Submitted academic work is typically considered incomplete without an evaluation on AA (and sometimes deepfool [2]). It's good to see that both are included in ART.

[0] https://arxiv.org/abs/2003.01690

[1] https://github.com/fra31/auto-attack

[2] https://arxiv.org/abs/1511.04599

[ Parent | on HN ]