Exploratory Data Analysis (EDA) in Python
Introduction
Exploratory Data Analysis is a method of evaluating or comprehending data in order to derive insights or key characteristics. EDA can be divided into two categories: graphical analysis and nongraphical analysis.
EDA is a critical component of any data science or machine learning process. You must explore the data, understand the relationships between variables, and the underlying structure of the data in order to build a reliable and valuable output based on it.
The EDA stages will be carried out in this tutorial using the Python programming language.
The Dataset
For this article, we will be doing Customer Churn Prediction. When clients stop doing business with a company, this is known as customer churn or customer attrition.
Because the cost of getting a new customer is usually higher than keeping an existing one, understanding customer churn is critical to a company’s success. As a result, churn analysis is the first step in gaining a better understanding of your clients.
To gain a deeper grasp of our data, we will go deep into Exploratory Data Analysis (EDA). The dataset is available here.
Importing the Python Libraries
First of all, we need to import all the libraries that are required for the analysis, namely Pandas for handling data, Numpy for numerical calculations, Matplotlib and Seaborn for visualization.
.
Loading the Dataset in Python
Now, load the dataset into the pandas dataframe.
Structured Based Data Exploration
This is the first part of EDA where the data frame is evaluated for structure, columns and data types. The goal of this step is to get a general understanding of the dataset.
Display the first 5 Observations
We get the output as:
Display the Last 5 Observations
The output is:
Display the Number of Variables and Observations
This can be done with df.shape which gives the output as a tuple having 2 values. The first value counts the number of data points and the second value represents the number of features in the dataset.
In this dataframe, there are 7043 rows and 21 columns.
Display the Variable Names and their Data Types
Count the number of NonMissing Values for each variable
df.count() counts the number of nonempty values. It gives the idea of missing values in our dataset.
Descriptive Statistics
Now to know more about the characteristics of the dataset we will use the df.describe() which by default gives statistical information of all numerical features in our data frame.
df.describe() gives some basic statistical details like count, percentile, mean, standard deviation, and the 5 point summary which includes minimum, first quartile, second quartile, third quartile and maximum of numerical features.
What about the categorical features?
By providing an include argument and assigning it the value βallβ, we can get the summary of all the categorical features too.
Display the Complete Summary of the Dataset
df.info() gives the summary of the dataframe including data types, shape and memory storage.
Handling Missing Values
Missing values are the unknown values in the dataset. The concept of missing values is important to understand in order to successfully manage data. The first step is to detect the missing value in the dataset and then treat them using the appropriate method.
Detecting the Missing Values

Using error = βcoerceβ will replace all nonnumeric values with NaN.

isnull().sum() returns the number of missing values in the dataset.
We have 11 missing values in the βTotal Chargesβ column. Now, we will see different methods to deal with them.
Missing Value Treatment
To treat missing values we can use the following ways:

Drop the variable

Drop the observation(s)

Mean imputation or median imputation or mode imputation
For variable βTotal Chargesβ only 11 values are missing. Since these data records are comparatively very low as compared to the total data set, we can drop them.
Done. Let’s check!
Analysis using Charts
Data Visualizations
Now, it’s time to visualize the data.Β We can see how the data appears and what sort of relation the properties of data hold with the help of data visualization. It’s the quickest approach to check if the features reflect the output.
Target Variable
Let’s visualize the target variable i.e. Churn. It has two categories Yes or No.
Display a frequency distribution of churn
The plot shows a class imbalance of the data between churners and nonchurners. To address this, resampling would be a suitable approach.
Categorical Variables
There are 17 Categorical features in the dataset. Let’s see their churning rate with respect to the target variable.
Note: I have only shown 5 graphs here which are more important according to me.
Β
Relationship between Monthly Charges and Total Charges
Total charges are the sum total of monthly charges. So, letβs visualize their relationship.
 Here we can see that Total Charges and monthly charges are highly correlated.
Customer Contract Distribution
Here we are trying to visualize the churning rate with respect to Contract.

About 75% of customers with MonthtoMonth Contracts opted to move out as compared to 13% of customers with oneyear contract and 3% with twoyear contracts.
Payment Method Distribution
This is the visualization of the payment method. It has four categories.

Β The electronic check has the highest users.
Dependents distribution
This graph shows the churning rate with respect to Dependents.

Customers without dependents are more likely to churn
Churn distribution w.r.t Partners
This graph shows the churning rate with respect to Partners.

Customers that do not have partners are likely to churn more.
Conclusion
In this article, we tried to analyze customer behaviour. First, we explored the dataset on a basic level. We looked for missing values and treated them by dropping those values. Then we used the Pandas DataFrame to do Exploratory Data Analysis on sample data by plotting different graphs like Count plot, Pie Chart, Line Plot and Histplot. From this, we got some useful insights like: “Customers with monthtomonth contracts churn the most”, “Total charges and monthly charges were highly correlated”, etc. This way, we perform EDA on the datasets to explore the data and extract all possible insights from it, which can help in model building and also better decision making.
However, this was only a basic overview of how EDA works; you can go deeper into it and attempt the stages on larger datasets.
You can reach out to me on LinkedIn.Β
2 thoughts on "Exploratory Data Analysis (EDA) in Python"
Marcel says: April 08, 2022 at 1:09 pm
this one is perfect: https://towardsdatascience.com/exploringyourdatawithjust1lineofpython4b35ce21a82dMarcel says: April 08, 2022 at 1:11 pm
Thanks for your article. It's very nice. A couple of years back I found this one, which is great too: This one is perfect: https://towardsdatascience.com/exploringyourdatawithjust1lineofpython4b35ce21a82d