Re:Inside Darpa’s Push to Make Artificial Intelligence Expla...
by nl on Friday August 11, 2017 @02:14AM
(#14987041)
Yes, there is plenty of work to do.
Tree-based classifiers always have the reputation of being "explainable". As you note this isn't always as simple as it should be.
In the unsupervised space I really like plotting dendrograms on hierarchical clustering. There's an excellent example in the recent DeepMoji paper[1] where they show how similar emojis cluster together AND how you can truncate the hierarchy at different depths to get capture different ranges of emotion.
It's laughable when people insist that DARPA are foolish for funding work in this area.
You get some weird outputs from these sometimes though which makes it hard to automatically show them to users. For example, a reason might be "because you liked salad restaurant X you should check out check BBQ place Y" and it's because there happens to be an overlap in the users who like both captured in your model. Yet it can cause cognitive dissonance for the users who are either strictly healthy eaters (salads only, no BBQ) or delude themselves into thinking they are (forgetting how much they actually order BBQ in addition to salads). That's the main challenge I see -- figuring out how to filter out reasons from these approaches that don't jive with common intuitions or, even harder, get people to learn to trust the reasons as counter-intuitive as they may seem.