Aman Preet Gulati — January 12, 2022
Beginner Data Exploration Data Visualization Database Datasets Python

This article was published as a part of the Data Science Blogathon.

Overview 

In this article, we will be predicting the income of US people based on the US census data and later we will be concluding whether that individual American have earned more or less than 50000 dollars a year. If you want to know more about the dataset visit this link.

US Census Data
Image source: NC state university

Takeaways

  1. Exploratory data analysis: Learn Exploratory data analysis on the complex dataset.
  2. Data Insights: Visualizing the data and getting the business-related insights using data visualization.
  3. Visualization Library: Learn about the powerful visualization library i.e.Plotly and Dexplot.

Import all the libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px 
import plotly.graph_objs as go
from plotly.offline import iplot

Reading the Data from the CSV file

df = pd.read_csv(r"D:Data Science projectsUS census income predictionPredicting the Income Level- US census dataadult.csv")
df.head()

Output:

 

Output | US Census Data

Let’s check what columns do this dataset has

df.columns

Output:

Index(['39', ' State-gov', ' 77516', ' Bachelors', ' 13', ' Never-married',
       ' Adm-clerical', ' Not-in-family', ' White', ' Male', ' 2174', ' 0',
       ' 40', ' United-States', ' <=50K'],
      dtype='object')

Exploratory Data Analysis (EDA)

Here we are dropping down 2174, 0, and 40 columns from the dataset which is irrelevant to the dataset.

df.drop([' 2174', ' 0', ' 40'], axis = 'columns', inplace = True)
df.head()

Output:

EDA | US Census Data

Since with the given name, we are unable to judge what the data from the US census data is indicating so, let us rename the columns name to understand the dataset more easily.

df.columns = ['Age', 'Type_of_Owner', 'id', 'Education', 'No_of_Projects_Done',
              'Marital_Status', 'Job_Designation', 'Family_Relation', 'Race', 'Gender',
              'Country', 'Salary']

Let’s have a look at our dataset now

df.head()

Output:

 

Output 2 | US Census Data

Inference: Now the data seems quite readable to us so let’s move forward now.

Finding the shape of the data

df.shape

Output:

(32560, 12)

Inference: There are 32560 rows and 12 columns in the dataset.

Let’ have a look at what information we can draw from our dataset.

df.info()

Output:

 

Output 3 | US Census Data

Describing the Data

df.describe()

Output:

 

Describing the data | US Census Data

Let’s see how many null values are there in our dataset

df.isnull().sum()

Output:

 

Output 4 | US Census Data

Inference: Bingo! there are no missing values in our dataset.

Data Visualization

Visualizing the Type of US Census Data

labels = df['Type_of_Owner'].value_counts().index
values = df['Type_of_Owner'].value_counts().values

colors = df['Type_of_Owner']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Data Visualisation| US Census Data

Inference: From the above datasets we can see that most of the US census data says that Jobs lie in the private sector and it’s around 70%.

Visualizing the Type of Age Dataset in US Census Data

labels = df['Age'].value_counts()[:10].index
values = df['Age'].value_counts()[:10].values

colors = df['Age']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Output | US Census Data

Inference: From the above graph we can see that most of the job-seeker falls in the age-group of 30-40.

Visualizing the Highest Degree of Education

labels = df['Education'].value_counts().index
values = df['Education'].value_counts().values

colors = df['Education']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Output | Data Visualization | US Census Data

Inference: Most of the Working Class people have High school graduation degrees followed by Some-College degrees and bachelor.

Visualizing the No_of_Projects_Done

labels = df['No_of_Projects_Done'].value_counts().index
values = df['No_of_Projects_Done'].value_counts().values

colors = df['No_of_Projects_Done']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Data Visualization | US Census Data

Inference: Here we can conclude that most of the People have 9 or 10 projects.

Visualizing the Marital Status of the Working Class People

labels = df['Marital_Status'].value_counts().index
values = df['Marital_Status'].value_counts().values

colors = df['Marital_Status']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Visualizing the Marital Status of the Working Class People

Inference: From the output, we can see that 46% of people are married and 32.8% of people never get married.

Visualizing the Job Descriptions of the Working Class People

labels = df['Job_Designation'].value_counts().index
values = df['Job_Designation'].value_counts().values

colors = df['Job_Designation']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Output | Data Visualization

Inference: About 50% of the people are involved in Prof-speciality, Craft-repair, Exec-managerial, and Adm-clerical.

Visualizing the Family-Relation of the Working Class People

labels = df['Family_Relation'].value_counts().index
values = df['Family_Relation'].value_counts().values

colors = df['Family_Relation']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Data Visualization Output | Visualizing the Marital Status of the Working Class People

Inference: Most of the working-class people are Husbands of someone.

Try to see the different types of the race of the Working Class

df['Race'].unique()

Output:

array([' White', ' Black', ' Asian-Pac-Islander', ' Amer-Indian-Eskimo',
       ' Other'], dtype=object)

Inference: There are 4 different races of the Working Class of the people.

Visualizing the Race of the Working Class People

labels = df['Race'].value_counts().index
values = df['Race'].value_counts().values

colors = ['#98adbf',
 '#1d4466',
 '#2678bf',
 '#2c6699']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

Output

Inference: From the above plot it is quite visible that White People have supremacy over the Black while getting jobs and Black people still face Color Discrimination.

Visualizing the Gender of the Working Class People

labels = df['Gender'].value_counts().index
values = df['Gender'].value_counts().values

colors = ['#98adbf',
 '#2c6699']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Output

Visualizing the Salary of the Working Class People

labels = df['Salary'].value_counts().index
values = df['Salary'].value_counts().values

colors = ['#98adbf',
 '#2c6699']

fig = go.Figure(data = [go.Pie()])

fig.show()

Output:

 

Salary of working class people

Inference: Only around 24% of the people get a salary above 24% and around 76% of the people get 50k or less than 50k as salary.

Importing the Dexplot

import dexplot as dxp

dxp.count(
    val="Age",
    data = df,
    split="Type_of_Owner",
    stacked = True,
    figsize=(12,12))

Output:

 

Output of working class people

Inference: From the above graph we can see how the Job Type of People of Different Age varies, Though most people are involved in Private Job Type in all the age group Private Job is predominantly occupied by the people in the age group of 17-60 years old people.

dxp.count(
    val="Age",
    data = df,
    split="Marital_Status",
    stacked = True,
    figsize=(12,12))

Output:

 

Output | Histogram

Inference: From the above data it is clear that people falling in the age group of 17-30 are unmarried and people falling in the age group of 30-65 years are predominantly married also a large portion of people in the age-group 30-55 years are divorced.

dxp.count(
    val="Age",
    data = df,
    split="Job_Designation",
    stacked = True,
    figsize=(12,12))

Output:

 

Output | Job Description

Inference: From the above graph we can see how the Age and Job-Profile of people vary.

dxp.count(
    val="Age",
    data = df,
    split="Race",
    stacked = True,
    figsize=(12,12))

Output:

 

Output | US Census Data

Inference: From the above graph it’s clear most people in any age group are predominantly white.

dxp.count(
    val="Age",
    data = df,
    split="Gender",
    stacked = True,
    figsize=(12,12))

Output:

 

Output | Females

Inference: It’s clear from the above graph that most working females fall in the age group of 17-55 and in fact they have started working at an early age while most males are in the age group of 23 onwards.

dxp.count(
    val="Age",
    data = df,
    split="Salary",
    stacked = True,
    figsize=(12,12))

Output:

 

Age trend output

Inference: It’s obvious from the graph that with the passes of age tend to get more salaries increases in general.

Conclusion

We did the entire EDA process for this dataset from looking at the head of the dataset to get the insights of each and every feature whether it is univariate analysis or the bivariate analysis and along with getting the insights from the data numerically we also have used two one of the most interactive visualization libraries i.e. Plotly and Dexplot.

Endnotes

Here’s the repo link to this article.

Note: All images/ screenshots used in the article are by the Author. If the source isn’t mentioned otherwise.

About Me

Greeting to everyone, I’m currently working in TCS and previously, I worked as a Data Science Analyst in Zorba Consulting India. Along with full-time work, I’ve got an immense interest in the same field, i.e. Data Science, along with its other subsets of Artificial Intelligence such as Computer Vision, Machine learning, and Deep learning; feel free to collaborate with me on any project on the domains mentioned above (LinkedIn).

Here you can access my other articles, which are published on Analytics Vidhya as a part of the Blogathon (link).

The media shown in this article is not owned by Analytics Vidhya and are used at the Author’s discretion. 

About the Author

Our Top Authors

Download Analytics Vidhya App for the Latest blog/Article

Leave a Reply Your email address will not be published. Required fields are marked *