> But not so fast. B, for instance, might be correlated with A.

Exactly. Rendering a model explainable is an active field of research at the moment. One way to tackle this is to use Shapley values from game theory.

Here is a compilation of techniques to render a model explainable: https://christophm.github.io/interpretable-ml-book/shapley.h..., and they have an elegant way to talk about these values:

> The Shapley value is the average marginal contribution of a feature value over all possible coalitions.

Now, it doesn't identify correlated variables (which you can do with other techniques), but will balance the influence in a robust manner.

----

> The story only gets more complicated for e.g. Random Forests.

Funny you should say this. Recently there was a challenge for explainable machine learning: http://explainable.ml

My proposal to the challenge was based on Random Forests. Here is the code: https://github.com/benoitparis/explainable-challenge, here is the paper: https://github.com/benoitparis/explainable-challenge/raw/mas...

I have a neat visualization for exploratory data analysis (available in the paper and in a notebook) that I'm proud of.

----

By the way if anyone is hiring here, I'm available for talking about your problem and see if my take on explainable machine learning can help you.