Akshay Gupta — May 14, 2021
Beginner Libraries Listicle Machine Learning Python

This article was published as a part of the Data Science Blogathon.

Introduction

Welcome Readers!!

Data cleaning and Data Manipulation is one the primary step in a machine learning project. It involves many steps like removing null values, handling outliers, features encoding, and many more. Data cleaning is very time-consuming and very tedious and it requires very patience. According to a recent survey, data scientists spend almost 60% of their time in data cleaning. We can’t neglect this step because we can’t feed messy data in machine learning models otherwise we won’t able to get useful insights.

There are many tool and libraries that are useful to handle messy data and saves developers time. In this blog, I am going to cover some useful python libraries and tools that could be very handy. So let’s get started without any further delay!!

Data Cleaning Libraries image

 

Table of Contents

  1. Dora
  2. Arrow
  3. PrettyPandas
  4. DataCleaner
  5. scrubadub
  6. Beautifier
  7. Tabulate

 

Dora

Dora is a library intended to improve on exploratory data analysis which is a particularly difficult task. It tries to automate the monotonous tasks that require a lot of time. The library provides a lot of functions that are very useful for feature extraction, data cleaning, feature selection, visualization, etc. Aside from this, it is also useful for versioning transformation of data and partitioning data for model validation.

This library utilizes scikit-learn, pandas, and matplotlib. The goal of this library is to add extra highlights to the general library referenced before for exploratory data analysis. This library is created by Nathan Epstein.

Installation

Dora can be installed by using the below command:

pip install Dora

Features

  • Data Cleaning
  • Data Versioning
  • Data Visualization
  • Feature Extraction
  • Feature Selection
  • Model Validation

For more information check official documentation: Link

Arrow

As a python user/developer, you may experience confronted a great deal of difficulty managing date and time format into some other time region format. We mostly used manually built functions to handle days, hours, minutes, etc. You may wind up utilizing plenty of libraries like datetime, time, dateutil, and so on that require a ton of additional code to be composed. Imagine a scenario where you learn one single library that presents all significant library highlights and, most importantly, gives extra highlights to make you code less. Isn’t it amazing right? So, Arrow is a python library that handles dates and times. It helps you to work with dates and times by writing lesser code and fewer imports. It has an intelligent module API that handles many common scenarios.

Features

  • It works for a Python 3.6+
  • Timezone conversion
  • It is aware of timezones and it enables easy timezone conversion
  • It parses and formats string automatically
  • It supports the ISO 8601 standard
  • It support for pytz, ZoneInfo tzinfo, dateutil objects
  • It generates ranges, floors, Time spans, and ceilings for a time ranging from microsecond to years

Supported Tokens:

 

Data Cleaning Libraries arrow

 

Installation

It can be installed by using the below command:

pip install –U arrow

For more information check official documentation: Link

PrettyPandas

DataFrames are incredible, however, they don’t create the sort of tables you’d need to show your chief. PrettyPandas utilizes the Pandas Style API to change DataFrames into nice presentable look tables. Make outlines, add styling, and design numbers, sections, and columns. Special reward: strong, simple to understand documentation.

Features

  • It has chaining commands
  • It adds a summary of columns and rows
  • It provides number formatting for scientific units, currency, and percentages
  • It has a lot of pretty and customizable themes
  • It works flawlessly with Pandas Style API
prettypandas Data Cleaning Libraries

 

Installation

It can be installed by using the below command:

pip install prettypandas

For more information check official documentation: Link

DataCleaner

It is an open-source python library that is very useful to automate the process of data cleaning work ie to automate the most time-consuming task in any machine learning project. It is built on top of Pandas Dataframe and scikit-learn data preprocessing features. This library is pretty new and very underrated, but it is worth checking out. Creator of this library constantly updating new features. Some of the features are given below:

  • Helps in encoding categorical values to numerical values
  • It handles null values and replaces them with median(categorical) or mean(numerical)
  • Optionally it drops the columns that have missing values

Installation

It can be installed by using the below command:

pip install datacleaner

For more information check official documentation: Link

Scrubadub

It is a free open-source python library that removes personally identifiable information(PII) from free text. So generally speaking, in the fields like finance and healthcare, data scientists have to anonymize data. Sometimes we don’t. This package makes it simple to flawlessly clean close to personal data from free content, without compromising the security of individuals we are attempting to ensure. One of the best things is, it has very decretive and nicely arranged documents.

It currently supports removing the following information from free text:

  • Email Addresses
  • URLs
  • Names
  • Skype usernames
  • Phone number
  • Password/username combinations
  • Social security numbers

Installation

It can be installed by using the below command:

pip install scrubadub

For more information check official documentation: Link

Beautifier

It is an open-source python library that is helpful to handle URLs and email addresses. Basic library to clean up and prettify URL patterns, domains, etc. Library assists with cleaning Unicode, special characters, and unnecessary redirection designs from the URLs and give you clean data.

Beautifier

Installation

It can be installed by using the below command:

pip install beautifier

Email Cleanup API’s

from beautifier import Email
Email_add = Email([email protected])
Email_add.domain
>>> “gmail.com”
Email_add.username
>>> “gakshay1210”

For more information check official documentation: Link

Tabulate

It is a free and open-source python library that is useful to print small tables without issue by just one function call and it handles all formatting on its own. It’s convenient for making tables more readable with number formatting, headers, column alignment by a decimal, and many more.

One of the best things is, it outputs data in many formats like PHP, HTML, or Markdown Extra, so you can keep working with your tabular data in another language or tool.

Installation

It can be installed by using the below command:

pip install tabulate

For more information check official documentation: Link

Conclusion

So in this article, we have covered the top 7 Data Cleaning libraries in python for machine learning in 2021. I hope you learn something from this blog and it will turn out best for your project. Thanks for reading and your patience. Good luck!

You can check my articles here: Articles

Thanks for reading this article on python libraries for image processing and your patience. Do let me in the comment section. Share this article, it will give me the motivation to write more blogs for the data science community.

Email id: gakshay1210@gmail.com

Connect me on LinkedIn: LinkedIn

The media shown in this article on Data Cleaning Libraries are not owned by Analytics Vidhya and is used at the Author’s discretion. 

About the Author

Our Top Authors

  • Analytics Vidhya
  • Guest Blog
  • Tavish Srivastava
  • Aishwarya Singh
  • Aniruddha Bhandari
  • Abhishek Sharma
  • Aarshay Jain

Download Analytics Vidhya App for the Latest blog/Article

Leave a Reply Your email address will not be published. Required fields are marked *