partial dependence plots can be used on black box models
Interpretation: if its interpretable then you can adjust an interpretable part manually part way?
Transparency of models: Sparse linear models are more transparent. Decomposition: Can each part of the model have input, output, parameters which can be interpreted? Complex feature selection means loss of this. Complex models. Boosting. Loses Can we say formal things about performance? We can for linear models (?), but not for others For agents, we can’t validate their behaviour. We can for manually defined rules. We can for interpretive models. Post-hoc interpretability
LIME Local Interpretable Model-Agnostic Explanations. Page on explainable models, h3 in that on locally explainable models
Explaining models: We may want to understand how it works. Black box algorithms are hard to understand.
This is important if the algorithm is used in high stakes cases, or where data is different to the static case used for training.
We can create explainable models (sparse linear models)
Take black box models and make them explainable.
SLAM algorithms?
Local explanation? Saliency maps?
Why do we care about transparency?