Learn everything about Analytics

17 Ultimate Data Science Projects To Boost Your Knowledge and Skills (& can be accessed freely)

SHARE

Introduction

Data science projects offer you a promising way to kick-start your analytics career. Not only you get to learn data science by applying, you also get projects to showcase on your CV. Nowadays, recruiters evaluate a candidate’s potential by his/her work, not as much by certificates and resumes. It wouldn’t matter, if you just tell them how much you know, if you have nothing to show them! That’s where most people struggle and miss out!

You might have worked on several problems, but if you can’t make it presentable & explanatory, how on earth would someone know what you are capable of? That’s where these projects would help you. Think of the time spend on these projects like your training sessions. I guarantee, the more time you spend, the better you’ll become!

The data sets in the list below are handpicked. I’ve made sure to provide you a taste of variety of problems from different domains with different sizes. I believe, everyone must learn to smartly work on large data sets, hence large data sets are added. Also, I’ve made sure all the data sets are open and free to access.

17-data-science-projects for career in analytics

 

Useful Information

To help you decide your start line, I’ve divided the data set into 3 levels namely:

  1. Beginner Level: This level comprises of data sets which are fairly easy to work with, and doesn’t require complex data science techniques. You can solve them using basic regression / classification algorithms. Also, these data sets have enough open tutorials to get you going. In this list, I’ve provided tutorials also to help you get started.
  2. Intermediate Level: This level comprises of data sets which are challenging. It consists of mid & large data sets which require some serious pattern recognition skills. Also, feature engineering will make a difference here. There is no limit of use of ML techniques, everything under the sun can be put to use.
  3. Advanced Level: This level is best suited for people who understand advanced topics like neural networks, deep learning, recommender systems etc. High dimensional data are also featured here. Also, this is the time to get creative – see the creativity best data scientists bring in their work and codes.

 

Table of Contents

  1. Beginner Level
    • Iris Data
    • Titanic Data
    • Loan Prediction Data
    • Bigmart Sales Data
    • Boston Housing Data
  2. Intermediate Level
    • Human Activity Recognition Data
    • Black Friday Data
    • Siam Competition Data
    • Trip History Data
    • Million Song Data
    • Census Income Data
    • Movie Lens Data
  3. Advanced Level
    • Identify your Digits
    • Yelp Data
    • ImageNet Data
    • KDD Cup 1998
    • Chicago Crime Data

 

Beginner Level

1. Iris Data Set

iris_dataset_scatterplot-svgThis is probably the most versatile, easy and resourceful data set in pattern recognition literature. Nothing could be simpler than iris data set to learn classification techniques. If you are totally new to data science, this is your start line. The data has only 150 rows & 4 columns.

Problem: Predict the flower class based on available attributes.

Start: Get Data | Tutorial: Get Here

 

2. Titanic Data Set

titanic_sn1912This is another most quoted data set in global data science community. With several tutorials and help guides, this project should give you enough kick to pursue data science deeper. With healthy mix of variables comprising categories, numbers, text, this data set has enough scope to support crazy ideas! This is a classification problem. The data has 891 rows & 12 columns.

Problem: Predict the survival of passengers in Titanic.

Start: Get Data | Tutorial: Get Here

 

3. Loan Prediction Data Set

ssAmong all industries, insurance domain has the largest use of analytics & data science methods. This data set would provide you enough taste of working on data sets from insurance companies, what challenges are faced, what strategies are used, which variables influence the outcome etc. This is a classification problem. The data has 615 rows and 13 columns.

Problem: Predict if a loan will get approved or not.

Start: Get Data | Tutorial: Get Here

 

4. Bigmart Sales Data Set

shopping-cart-1269174_960_720Retail is another industry which extensively uses analytics to optimize business processes. Tasks like product placement, inventory management, customized offers, product bundling etc are being smartly handled using data science techniques. As the name suggests, this data comprises of transaction record of a sales store. This is a regression problem. The data has 8523 rows of 12 variables.

Problem: Predict the sales.

Start: Get Data | Tutorial: Get Here

 

5. Boston Housing Data Set

14938-illustration-of-a-yellow-house-pvThis is another popular data set used in pattern recognition literature. The data set comes from real estate industry in Boston (US). This is a regression problem. The data has 506 rows and 14 columns. Thus, it’s a fairly small data set where you can attempt any technique without worrying about your laptop’s memory issue.

Problem: Predict the median value of owner occupied homes

Start: Get Data | Tutorial: Get Here

 

Intermediate Level

1. Human Activity Recognition

asThis data set is collected from recordings of 30 human subjects captured via smartphones enabled with embedded inertial sensors. Many machine learning courses use this data for students practice. It’s your turn now. This is a multi-classification problem. The data set has 10299 rows and 561 columns.

Problem: Predict the activity category of a human

Start: Get Data

 

2. Black Friday Data Set

black-fridayThis data set comprises of sales transactions captured at a retail store. It’s a classic data set to explore your feature engineering skills and day to day understanding from your shopping experience. It’s a regression problem. The data set has 550069 rows and 12 columns.

Problem: Predict purchase amount.

Start: Get Data

 

3. Text Mining Data Set

De l'éloquence judiciaire À AthenesThis data set is originally from siam competition 2007. The data set comprises of aviation safety reports describing problem(s) which occurred in certain flights. It is a multi-classification, high dimensional problem. It has 21519 rows and 30438 columns.

Problem: Classify the documents according to their labels

Start: Get Data | Get Information

 

4. Trip History Data Set

trip-history-dataThis data set comes from a bike sharing service in US. This data set requires you to exercise your pro data munging skills. The data set is provided quarter wise from 2010 (Q4) onwards. Each file has 7 columns. It is a classification problem.

Problem: Predict the class of user

Start: Get Data

 

5. Million Song Data Set

million-songDidn’t you know analytics can be used in entertainment industry also? Do it yourself now. This data set puts forward a regression task. It consists of 515345 observations and 90 variables. However, this is just a tiny subset of original database of million song data. You should use data linked below.

Problem: Predict release year of the song

Start: Get Data

 

6. Census Income Data Set

us-censusIt’s an imbalanced classification and a classic machine learning problem. You know, machine learning is being extensively used to solve imbalanced problems such as cancer detection, fraud detection etc. It’s time to get your hand dirty. The data set has 48842 rows and 14 columns. For guidance, you can check my imbalanced data project.

Problem: Predict the income class of US population

Start: Get Data

 

7. Movie Lens Data Set

movie-lens-dataThis data set allows you to build a recommendation engine. Have you created one before?  It’s one of the most popular & quoted data set in data science industry. It is available in various dimensions. Here I’ve used a fairly small size. It has 1 million ratings from 6000 users on 4000 movies.

Problem: Recommend new movies to users

Start: Get Data

 

Advanced Level

1. Identify your Digits Data Set

identify-the-digitsThis data set allows you to study, analyze and recognize elements in the images. That’s exactly how your camera detects your face, using image recognition! It’s your turn to build and test that technique. It’s an digit recognition problem. This data set has 7000 images of 28 X 28 size, sizing 31MB.

Problem: Identify digits from an image

Start: Get Data

 

2. Yelp Data Set

yelp-data-setThis data set is a part of round 8 of The Yelp Dataset Challenge. It comprises of nearly 200,000 images, provided in 3 json files of ~2GB. These images provide information about local businesses in 10 cities across 4 countries. You are required to find insights from data using cultural trends, seasonal trends, infer categories, text mining, social graph mining etc.

Problem: Find insights from images

Start: Get Data

 

3. Image Net Data Set

laImageNet offers variety of problems which encompasses object detection, localization, classification and screen parsing. All the images are freely available. You can search for any type of image and build your project around it. As of now, this image engine has 14,197,122 images of multiple shapes sizing up to 140GB.

Problem: Problem to solve is subjected to the image type you download

Start: Get Data

 

4. KDD 1999 Data Set

kdd-datasetHow could I miss KDD Cup? Originally, KDD brought the taste of data mining competition to the world. Don’t you want to see what data set they used to offer? I assure you, it’ll be an enriching experience. This data poses a classification problem. It has 4M rows and 48 columns in a ~1.2GB file.

Problem: Classify a network intrusion detector as good or bad.

Start: Get Data

 

5. Chicago Crime Data Set

chicago-crimeThe ability of handle large data sets is expected of every data scientist these days. Companies no longer prefer to work on samples, they use full data. This data set would provide you much needed hands on experience of handling large data sets on your local machines. The problem is easy, but data management is the key!  This data set has 6M observations. It’s a multi-classification problem.

Problem: Predict the type of crime.

Start: Get Data | To download data, click on Export -> CSV

 

End Notes

Out of the 17 data sets listed above, you should start by finding the right match of your skills. Say, if you are a beginner in machine learning, avoid taking up advanced level data sets. Don’t bite more than you can chew and don’t feel overwhelmed with how much you still have to do. Instead, focus on making step wise progress.

Once you complete 2 – 3 projects, showcase them on your resume and your github profile (most important!). Lots of recruiters these days hire candidates by stalking github profiles. Your motive shouldn’t be to do all the projects, but to pick out selected ones based on data set, domain, data set size whichever excites you the most. If you want me to solve any of above problem and create a complete project like this, let me know.

Did you find this article useful ? Have you already build any project on these data sets? Do share your experience, learnings and suggestions in comments below.

You can test your skills and knowledge. Check out Live Competitions and compete with best Data Scientists from all over the world.

21 Comments

  • Demudu says:

    Fantastic

  • Mallikarjun says:

    Thank You so much… 🙂 I have been wondering, how to start with projects. This will help me out. I have done machine learning course of Prof. Andrew Ng. and I have good knowledge of statistics and R and Matlab. Please let me know, if any skill required to be a data scientist. Thank you again. 🙂

    • Analytics Vidhya Content Team says:

      Hi Mallikarjun, I received several emails and messages to help people in selecting their data science projects, which motivated me to write this post. I’d suggest you to take up any project according to your understanding and start working on it. Through the way you’ll discover topics which you are yet to pick up. All the best!

  • venugopal rao says:

    Great Collection.. All together in one place

  • Akash says:

    Hey Manish,

    Good one. Do you have the ‘R’ equivalent for Boston housing and Big mart sales ?

    Thanks,

  • Femy says:

    Do you have anything on operational risk or risk in general especially consumer credit risk

  • Sridevi says:

    Hi Manish,

    Wonderful collection. Great resource for exploring ML.

    Thanks

  • Krishna says:

    Hi Manish,

    Can you please give some insights about the Knoctober data which was conducted recently?

    • Analytics Vidhya Content Team says:

      Hi Krishna, tomorrow we’ll post a complete article on knocktober competition so that people like you can gather their learnings. Do keep a check on your in mails.

  • Tesfaye says:

    Thank you Manish. These are wonderful project ideas. Please I would love to have sample solutions if/when you have them.

  • Funso Iyaju says:

    Hi Manish. Thanks for sharing this. It will help a lot in my love for data analytics

  • Prasanna Venktesh says:

    Hi what it takes to be a Datascientist.
    I don’t have a powerful computer.
    whats the configuration I need.
    I’m a Software developer..

    Do I need to do a course or can i learn on my own.
    Does working on here help me acquire the datascientist skills.

    Can You please tell me how can i enter data scientist role without losing
    my job.

    How to start my Datascientist career and what the employee seek.

  • RAMESH MAMILLA says:

    Hello Manish,

    Thanks for sharing this information..am also want to build up my career in Analytics.. am facing lot of problems to getting a job..It would be a great article for the beginners..Thanks Bro..

  • Satya says:

    This is Great Manish. I was able to get all the theoretical material for learning . But Application of those concepts in real life scenarios was always an issue.

    But I can say not anymore . Thanks to you.

    Regards,
    Satya

  • Palanisingh says:

    How to analyze the yearly data.

  • Varun says:

    Hi Manish,

    Awesome job getting all the information together.

  • Partha sundar sahoo says:

    Hi manish sir
    I am an electrical engineer having 1.5yr experience. Iwant to switch from my sector to it. Is this big data a right choice for me.
    Please give your valuable advice.

Leave A Reply

Your email address will not be published.

Join world’s fastest growing Analytics Community
Receive awesome tips, guides, infographics and become expert at: