Running Low on Time? Use PyCaret to Build your Machine Learning Model in Seconds

Lakshay Arora 14 Dec, 2023
8 min read

Overview

  • PyCaret is a super useful and low-code Python library for performing multiple machine learning tasks in double-quick time
  • Learn how to rely on PyCaret for building complex machine learning models in just a few lines of code

Introduction

My first machine learning model in Python for a hackathon was quite a cumbersome block of code. I still remember the many lines of code it took to build an ensemble model – it would have taken a wizard to untangle that mess!

When it comes to building interpretable machine learning models, especially in the industry (or when we want to explain our hackathon results to the client), writing efficient code is key to success. That’s why I strongly recommend using the PyCaret library.

Pycaret

I wish PyCaret was around during my rookie machine learning days! It is a super flexible and useful library that I’ve leaned on quite a bit in recent months. I firmly believe anyone with an aspiration to succeed as a data science or analytics professional will benefit a lot from using PyCaret.

We’ll see what exactly PyCaret it, how to install it on your machine, and then we’ll dive into using PyCaret for building interpretable machine learning models, including ensemble models. A lot of learning to be done so let’s dig in.

What is PyCaret and Why Should you Use it?

PyCaret is an open-source, machine learning library in Python that helps you from data preparation to model deployment. It is easy to use and you can do almost every data science project task with just one line of code.

What is pycaret

I’ve found PyCaret extremely handy. Here are two primary reasons why:

  • PyCaret, being a low-code library, makes you more productive. You can spend less time on coding and can do more experiments
  • It is an easy to use machine learning library that will help you perform end-to-end machine learning experiments, whether that’s imputing missing values, encoding categorical data, feature engineering, hyperparameter tuning, or building ensemble models

Installing PyCaret on your Machine

This is as straightforward as it gets. You can install the first stable version of PyCaret, v1.0.0, directly using pip. Just run the below command in your Jupyter Notebook to get started:

!pip3 install pycaret

Let’s Get Familiar with PyCaret

Problem Statement and Dataset

In this article, we are going to solve a classification problem. We have a bank dataset with features like customer age, experience, income, education, and whether he/she has a credit card or not. The bank wants to build a machine learning model that will help them identify the potential customers who have a higher probability of purchasing a personal loan.

The dataset has 5000 rows and we have kept 4000 for training our model and the remaining 1000 for testing the model. You can find the complete code and dataset used in this article here.

Let’s start by reading the dataset using the Pandas library:

Dataset Dashboard

The very first step before we start our machine learning project in PyCaret is to set up the environment. It’s just a two-step process:

  1. Importing a Module: Depending upon the type of problem you are going to solve, you first need to import the module. In the first version of PyCaret, 6 different modules are available – regression, classification, clustering, natural language processing (NLP), anomaly detection, and associate mining rule. In this article, we will solve a classification problem and hence we will import the classification module
  2. Initializing the Setup: In this step, PyCaret performs some basic preprocessing tasks, like ignoring the IDs and Date Columns, imputing the missing values, encoding the categorical variables, and splitting the dataset into the train-test split for the rest of the modeling steps. When you run the setup function, it will first confirm the data types, and then if you press enter, it will create the environment for you to go ahead

We’re all set to explore PyCaret!

Training our Machine Learning Model using PyCaret

Training a Model

Training a model in PyCaret is quite simple. You just need to use the create_model function that takes just the one parameter – the model abbreviation as a string. Here, we are going to first train a decision tree model for which we have to pass “dt” and it will return a table with k-fold cross-validated scores of common evaluation metrics used for classification models.

Here’s q quick reminder of the evaluation metrics used for supervised learning:

  • Classification: Accuracy, AUC, Recall, Precision, F1, Kappa
  • Regression: MAE, MSE, RMSE, R2, RMSLE, MAPE

You can check the documentation page of PyCaret for more abbreviations.

Training model Dashboard

Similarly, for training the XGBoost model, you just need to pass the string “xgboost“:

Hyperparameter Tuning

We can tune the hyperparameters of a machine learning model by just using the tune_model function which takes one parameter – the model abbreviation string (the same as we used in the create_model function).

PyCaret provides us a lot of flexibility. For example, we can define the number of folds using the fold parameter within the tune_model function. Or we can change the number of iterations using the n_iter parameter. Increasing the n_iter parameter will obviously increase the training time but will give a much better performance.

Let’s train a tuned CatBoost model:

Hyperparameter tuning model, Pycaret

Building Ensemble Models using PyCaret

Ensemble models in machine learning combine the decisions from multiple models to improve the overall performance.

In PyCaret, we can create bagging, boosting, blending, and stacking ensemble models with just one line of code.

If you want to learn ensemble models in-depth, I would highly recommend this article: A Comprehensive Guide to Ensemble Learning.

Let’s train a boosting ensemble model here. It will also return a table with k-fold cross-validated scores of common evaluation metrics:

Building Ensemble models

Another very famous ensembling technique is blending. You just need to pass the models that you have created in a list of the blend_models function.

That’s it! You just need to write a single line of code in PyCaret to do most of the stuff.

Compare Models

This is another useful function of the PyCaret library. If you do not want to try the different models one by one, you can use the compare models function and it will train and compare common evaluation metrics for all the available models in the library of the module you have imported.

This function is only available in the pycaret.classification and pycaret.regression modules.

Let’s Analyze our Model!

Now, after training the model, the next step is to analyze the results. This especially useful from a business perspective, right? Analyzing a model in PyCaret is again very simple. Just a single line of code and you can do the following:

  1. Plot Model Results: Analyzing model performance in PyCaret is as simple as writing plot_model. You can plot decision boundaries, precision-recall curve, validation curve, residual plots, etc.. Also, for clustering models, you can plot the elbow plot and silhouette plot. For text data, you can plot word clouds, bigram and trigram frequency plots, etc.
  2. Interpret Results: Interpreting model results helps in debugging the model by analyzing the important features. This is a crucial step in industry-grade machine learning projects. In PyCaret, we can interpret the model by SHAP values and correlation plot with just one line of code (getting to be quite a theme this, isn’t it?)

Plot Model Results

You can plot model results by providing the model object as the parameter and the type of plot you want. Let’s plot the AUC-ROC curve and decision boundary:

Model Results

Let’s plot the precision-recall curve and validation curve of the trained model:

Evaluate our Model

If you do not want to plot all these visualizations individually, then the PyCaret library has another amazing function – evaluate_model. In this function, you just need to pass the model object and PyCaret will create an interactive window for you to see and analyze the model in all the possible ways:

Pretty cool!

Interpret our Model

Interpreting complex models is very important in most machine learning projects. It helps in debugging the model by analyzing what the model thinks is important. In PyCaret, this step is as simple as writing interpret_model to get the Shapley values.

You can read about the Shapley Values here: A Unique Method for Machine Learning Interpretability: Game Theory & Shapley Values.

Interpreting the Model

Let’s try to plot the correlation plot:

Time to Make Predictions!

Finally, we will make predictions on unseen data. For this, we just need to pass the model that we will use for the predictions and the dataset. Make sure it is in the same format as we provided while setting up the environment earlier. PyCaret builds a pipeline of all the steps and will pass the unseen data into the pipeline and give us the results.

Let’s see how to predict the labels on unseen data:

Unseen Data predications dashboard

Save and Load the Model

Now, once the model is built and tested, we can save this in the pickle file using the save_model function. Pass the model to be saved and the file name and that’s it:

We can load this model later on and predict labels on the unseen data:

Conclusion

It really is that easy to use. I’ve personally found PyCaret to be quite useful for generating quick results when I’m working with tight timelines.

Practice using it on different types of datasets – you’ll truly grasp it’s utility the more you leverage it! It even supports model deployment on cloud services like AWS and that too with just one line of code.

If you have any suggestions/feedback related to the article, please post them in the comments section below. I look forward to hearing about your experience using PyCaret as well. Happy learning!

Frequently Asked Questions

Q1.What is the difference between PyCaret and pandas?

PyCaret:

1:Automates machine learning tasks.
2:User-friendly for quick model building.
3:Focuses on ML workflow automation.

Pandas:

1:Manages and analyzes structured data.
2:Essential for data cleaning and exploration.
3:General-purpose data manipulation.

Q2. Can Pycaret be integrated with other data science tools?

Yes, Pycaret is compatible with popular data science tools and can be seamlessly integrated into existing workflows.

Q3. Is Pycaret suitable for large-scale projects?

Absolutely. Pycaret is scalable and can be used for projects of varying sizes, from small experiments to large-scale machine-learning applications.

Lakshay Arora 14 Dec, 2023

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers

Clear

Deepti
Deepti 19 May, 2020

How is Pycaret different from sklearn? and which one is better?

Pulkit Mehta
Pulkit Mehta 19 May, 2020

Thanks , this is very useful article and I am going to give Pycaret a try .

Amit
Amit 19 May, 2020

Yes, I am impressed. this model takes few lines of code and have integrated features for different algorithms

Pranay Mehta
Pranay Mehta 19 May, 2020

Thank you for the explanation. I have a question regarding the predictions on unseen data. Imagine you have a model M that you have trained using Pycaret and now you want to put that model into production environment. In the example, that you have shown, you saved the model as a pickle file and then load it at a later stage when you have to score unseen data. My question is - How does pycaret convert that raw loan_test_data.csv dataframe into features before passing it to predict_model function? In the sense, when we learn scikit learn models, we do FIT() and TRANSFORM() while training and we only use TRANSFORM() while scoring. Is there an equivalent of that here ? Or does it do automatically?

Jude
Jude 20 May, 2020

It seems very useful for constructing the ML models. And It can use blending to combine different models as well. I really appreciated your sharing. There are a bunch of benefits to using PyCaret. I am curious in your opinion, what are the cons of PyCaret?

Betzi Miriyam Kuriakose
Betzi Miriyam Kuriakose 25 May, 2020

Impressive

Uday Mishra
Uday Mishra 26 May, 2020

Hi Lakshya, In your dataset test and train csv was available. In other data set you need to divide into some percentage, which is not clear your explanation, can you explain how you want to divide that dataset to test at the end?

Suraj Singh
Suraj Singh 02 Jun, 2020

Hye Lakshya.. I just wanted to know, how to know what are the "Hyperparameter Tuning" abbreviations are there in PyCaret as i went through their website and did not find much of the details apart from just some examples like n-iter, How to apply and when and where to apply these things... I am new in this Field, So any help would be very much appreciated..

Philip Ekanem
Philip Ekanem 18 Jul, 2020

Thanks for this well explained Article

Sanjay Ray
Sanjay Ray 11 Sep, 2020

Very interesting and article to learn a wonderful tool for ML. Thanks.

HSIEH WEI DONG
HSIEH WEI DONG 21 Oct, 2020

How to use the saved model on a different dataset with the same structure?