Interpretability in Machine Learning: An Overview

Re:Interpretability in Machine Learning: An Overview by snovv_crash on Monday November 30, 2020 @07:21AM (#25253029)

Do you have any material you can recommend as an introduction to interpretation of NN systems, or how to design NN-based ML to be good for interpretation?

My background is more classic ML and computer vision (features design, geometry, photogrammetry) and I've been very hesitant to use NN in a number of situations precisely because of things like auditability and being able to fix bugs in critical-for-customer edge cases.

[ Parent | on HN ]

Re:Interpretability in Machine Learning: An Overview by pkage on Monday November 30, 2020 @02:50PM (#25258092)

R Guidotti et. al[0] wrote a good literature survey on black-box explainers, and contains a summary table on page 20 of the current state of the art.

In terms of designing NN-based ML, the above paper has some info and this paper by S Teso[1] is a good place to start looking further (though it is focused on XAL). SENNs are cool, but ultimately most inherenly interpretable models come down to classic ML (decision trees, linear/logreg, etc.) which is limiting compared to the power of NNs. Post-hoc explanations are basically the only option (esp. for DNNs).

[0] http://arxiv.org/abs/1802.01933

[1] https://www.semanticscholar.org/paper/Toward-Faithful-Explan...

[ Parent | on HN ]