Complex models (vs. simpler models) capture complex patterns in data and perform at higher accuracy but are less interpretable. Model-agnostic interpretability promises to regain interpretability without sacrificing accuracy. I introduce LIME (Local Interpretable Model-Agnostic Explanations), conceptually and applied to a classification problem: whether couples are likely to stay together, or not.
Over the past years, models have become more complex and more accurate but less interpretable. Interpretability means you can inspect and understand a model. It means you can build a qualitative understanding of the relationship between model inputs and outputs. Interpretability matters, it can help establish that a model is safe to deploy and non-discriminatory. Interpretability helps verify a model is right for the right reasons and wrong for the right reasons which traditional measures of model performance (i.e., performance on the test set) do not capture.
One approach to ensure interpretability is to use simple models but model accuracy is likely to suffer (there is a trade off between interpretability and accuracy). Model-agnostic interpretability solutions provide the best of both worlds; they are designed to work with and provide insight into the inner workings of any existing, complex, trained model.
LIME is an interpretability solutions based on two simple ideas (perturbation, local linear approximation). LIME allows us to understand the reasons for a single, given prediction of a trained (complex, high accuracy) model. It is easy to implement, it can extend any existing machine learning pipeline, but it is not widely known (yet).
We use Stanford’s HCMST data (How Couples Meet and Stay Together) and build a random forest classifier to predict relationship success. We use LIME to understand individual predictions and demonstrate the value of LIME reasons over and above exploratory data analysis and random forest’s feature importances. Feature importances show what matters across all couples, LIME allows us to inspect and understand the fate of individuals.
We provide “algorithmic relationship advice”, taken with a grain of salt, to demonstrate that LIME reasons are not causes. Interpretability helps us understand the inner workings of models to improve these models, to help build trust in models, and to form hypotheses about phenomena in the real-world captured by models (that warrant rigorous tests). LIME picks up on patterns in the data learned by the model, it does not inform about reality. Data reflects current and past conventions and social practices (which LIME can show).
LIME is no oracle, but it allows humans to enter a more collaborative relationship with black-box machine learning models, and question them when necessary.