This article was published as a part of the Data Science Blogathon

What does Interpretability/Explainability mean in AI?

The following points encompass the explainability-

- How the internal working of the so-called black box(machine/deep learning models) can be interpreted/explained in human terms
- In addition to the outcome from our model if we know why it has arrived at such output i.e output backed by proper explanation, will be good to drive the business
- How can we build trust in such black box models?
- Scope:Local(per sample) or Global(overall) interpretation

Algorithms such as linear/logistic regression are easy to interpret based on model coefficients and Tree-based algorithms which can help us understand how it’s making decisions, with built-in support for feature importance and visualization.

One simple method is Permutation Feature Importance, It is a model inspection technique that can be used for any fitted estimator when the data is tabular. The permutation feature importance is defined to be the decrease in a model score when a single feature value is randomly shuffled. This procedure breaks the relationship between the feature and the target, thus the drop in the model score is indicative of how much the model depends on the feature.

- Train a model on a dataset
- Calculate the error metrics or score(s) for the trained model for reference
- Shuffled the validation data columns values one column(features) at a time and can be repeated K times
- Using the shuffled data to evaluate the trained model using the same error or score metrics for each iteration
- The features that affect the error metrics most are the important ones as it indicates a model dependency on that features
- We can calculate the importance score for each feature as

Where,

**i _{j} **is the feature importance of feature j

**s **is the reference score calculated from the trained model

**K **is the number of iterations(shuffling operations) performed for a feature

**s _{k,j }**is the score for

We can use the python sklearn package for build-in **permutation_importance** function

from sklearn.inspection import permutation_importance

We will be using the Xgboost algorithm with default parameters on the Boston dataset

xg = xgboost.XGBRegressor() xg.fit(Xtrain, ytrain)

Now we will use the **permutation_importance** function on the test set to calculate the feature importance, we need** **to provide a trained model and number of shuffling iteration to perform (n_repeats parameter)

r = permutation_importance(xg, Xtest, ytest, n_repeats=30, random_state=0)

Here, we have set *n_repeats=30*

The returned object, **r **will contain feature importance values for each feature** **which we can visualize using the** matplotlib** python library

plt.figure(figsize=(10,4)) plt.bar(boston.feature_names,r.importances_mean) plt.xlabel('Features') plt.ylabel('Mean Importance') plt.title('Feature importance using Feature Permutation Importance');

Source: Github

If two features are correlated and one of them is permuted then

- The model still has the other correlated feature and in that case, both features will have lower importance value even if those are actually important
- It can be biased by unrealistic data instances formed by permutation

Computationally expensive for a large number of features

A good practice is to drop one of the correlated features based on domain understanding and try to apply the Permutation Feature Importance algorithm which will provide better feature understanding.

Let’s discuss another method to interpret the black box models.

- An interpretation model trained to approximate the predictions of the black-box models
- It’s like solving black box Interpretability task using simpler and explainable models (such as linear regression, decision tree,..) i.e explaining machine learning using more machine learning

- Get the predictions(yhat) from the black-box model
- Select any simple and explainable model(linear reg., decision tree..) as per the use case
- Train the selected model on the same dataset used for training the black-box model, using predictions(yhat) as the target
- Measure the performance, as to how well the surrogate model approximates the behavior of the black-box model
- Finally, we can interpret the global surrogate model

Flexible, as a selection of surrogate model, does not depend on the black-box model. If at some time we have a better performing black-box model in place of an existing black box, we do not have to change the method of interpretation

▪️How much confidence is enough(% of variance explained) in deciding the surrogate model is close enough to the black-box model

▪️ Interpretation becomes irrelevant if the black-box model is not performing well

We have seen few model interpretation techniques for Global interpretation, what about Local Interpretation i.e when we want to understand how a model prediction was made for a particular observation.

Consider a loan approval model, what if a user request is declined, then the user has the right to question WHY? and authorities should know why the model has declined the user request and communicate the same to the user as they just can’t say that their system has rejected it instead they need to explain on what factors(features) the request is rejected.

▪️LIME can explain the predictions of any classifier or regressor in a faithful way, by approximating it locally with an interpretable model(linear reg., decision tree..)

▪️It tests what happens to the predictions when we feed variations of the data into the machine learning model

▪️Can be used on tabular, text, and image data

Source: LIME Paper

We can use python **lime **library to interpret models

import lime import lime.lime_tabular

We will be using the Xgboost algorithm with default parameters on the Boston dataset

xg = xgboost.XGBRegressor() xg.fit(Xtrain, ytrain)

Now we will create a lime explainer object, for which we have to specify the target column, features names, categorical features, and mode of the algorithm(regression or classification)

explainer = lime.lime_tabular.LimeTabularExplainer(Xtrain, feature_names=boston.feature_names, class_names=['price'], categorical_features=categorical_features, verbose=True, mode='regression')

We can use **the explain_instance **method of **the explainer** object to interpret a particular instance of data

exp = explainer.explain_instance(Xtest[i], xg.predict, num_features=5)

**i** is the index in test data that we need to interpret

we can visualize the interpretation output using **the show_in_notebook** method

exp.show_in_notebook(show_table=True)

We will get similar to below output

Source: Github

For Complete code refer to Github Repository

https://christophm.github.io/interpretable-ml-book/feature-importance.html

https://christophm.github.io/interpretable-ml-book/global.html

https://scikit-learn.org/stable/modules/permutation_importance.html

https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance_multicollinear.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-multicollinear-py

LIME Research Paper: https://arxiv.org/pdf/1602.04938.pdf

https://github.com/marcotcr/lime

I am a Machine Learning Engineer, Solving challenging business problems through data and machine learning

*The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.*

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Become a full stack data scientist
##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

##

Understanding Cost Function
Understanding Gradient Descent
Math Behind Gradient Descent
Assumptions of Linear Regression
Implement Linear Regression from Scratch
Train Linear Regression in Python
Implementing Linear Regression in R
Diagnosing Residual Plots in Linear Regression Models
Generalized Linear Models
Introduction to Logistic Regression
Odds Ratio
Implementing Logistic Regression from Scratch
Introduction to Scikit-learn in Python
Train Logistic Regression in python
Multiclass using Logistic Regression
How to use Multinomial and Ordinal Logistic Regression in R ?
Challenges with Linear Regression
Introduction to Regularisation
Implementing Regularisation
Ridge Regression
Lasso Regression

Introduction to Stacking
Implementing Stacking
Variants of Stacking
Implementing Variants of Stacking
Introduction to Blending
Bootstrap Sampling
Introduction to Random Sampling
Hyper-parameters of Random Forest
Implementing Random Forest
Out-of-Bag (OOB) Score in the Random Forest
IPL Team Win Prediction Project Using Machine Learning
Introduction to Boosting
Gradient Boosting Algorithm
Math behind GBM
Implementing GBM in python
Regularized Greedy Forests
Extreme Gradient Boosting
Implementing XGBM in python
Tuning Hyperparameters of XGBoost in Python
Implement XGBM in R/H2O
Adaptive Boosting
Implementing Adaptive Boosing
LightGBM
Implementing LightGBM in Python
Catboost
Implementing Catboost in Python

Introduction to Clustering
Applications of Clustering
Evaluation Metrics for Clustering
Understanding K-Means
Implementation of K-Means in Python
Implementation of K-Means in R
Choosing Right Value for K
Profiling Market Segments using K-Means Clustering
Hierarchical Clustering
Implementation of Hierarchial Clustering
DBSCAN
Defining Similarity between clusters
Build Better and Accurate Clusters with Gaussian Mixture Models

Introduction to Machine Learning Interpretability
Framework and Interpretable Models
model Agnostic Methods for Interpretability
Implementing Interpretable Model
Understanding SHAP
Out-of-Core ML
Introduction to Interpretable Machine Learning Models
Model Agnostic Methods for Interpretability
Game Theory & Shapley Values

Deploying Machine Learning Model using Streamlit
Deploying ML Models in Docker
Deploy Using Streamlit
Deploy on Heroku
Deploy Using Netlify
Introduction to Amazon Sagemaker
Setting up Amazon SageMaker
Using SageMaker Endpoint to Generate Inference
Deploy on Microsoft Azure Cloud
Introduction to Flask for Model
Deploying ML model using Flask