Karthe — May 30, 2018
Data Science Intermediate Listicle Machine Learning Project Python R

This article was originally published on October 26, 2016 and updated with new projects on 30th May, 2018.

Introduction

Data science (Machine Learning) projects offer you a promising way to kick-start your career in this field. Not only do you get to learn data science by applying it but you also get projects to showcase on your CV! Nowadays, recruiters evaluate a candidate’s potential by his/her work and don’t put a lot of emphasis on certifications. It wouldn’t matter if you just tell them how much you know if you have nothing to show them! That’s where most people struggle and miss out.

You might have worked on several problems before, but if you can’t make it presentable & easy-to-explain, how on earth would someone know what you are capable of? That’s where these projects will help you. Think of the time you’ll spend on these projects like your training sessions. The more time you spend practicing, the better you’ll become!

We’ve made sure to provide you with a taste of a variety of problems from different domains. We believe everyone must learn to smartly work with huge amounts of data, hence large datasets are included. Also, we’ve made sure all the datasets are open and free to access.

Data Science, machine learning, projects

24 Data Science Projects

Useful Information

To help you decide where to begin, we’ve divided this list into 3 levels, namely:

  1. Beginner LevelThis level comprises of data sets which are fairly easy to work with, and don’t require complex data science techniques. You can solve them using basic regression or classification algorithms. Also, these data sets have enough open tutorials to get you going. In this list, we have also provided tutorials to help you get started. You can also check out AV’s ‘Introduction to Data Science‘ course along with this!
  2. Intermediate Level: This level comprises of data sets which are more challenging in nature. It consists of mid & large data sets which require some serious pattern recognition skills. Also, feature engineering will make a difference here. There is no limit on the use of ML techniques; everything under the sun can be put to use.
  3. Advanced Level: This level is best suited for people who understand advanced topics like neural networks, deep learning, recommender systems etc. High dimensional datasets are also featured here. Also, this is the time to get creative. See the creativity best data scientists bring into their work and codes.

Do you want to Master Machine Learning and Deep Learning? Here is a comprehensive program that covers the Machine Learning and Deep Learning concepts in Detail along with 25+ real life Projects! Check out the complete list of Projects in the link below

 

Table of Contents

  1. Beginner Level
    • Iris Data
    • Loan Prediction Data
    • Bigmart Sales Data
    • Boston Housing Data
    • Time Series Analysis Data
    • Wine Quality Data
    • Turkiye Student Evaluation Data
    • Heights and Weights Data
  2. Intermediate Level
    • Black Friday Data
    • Human Activity Recognition Data
    • Siam Competition Data
    • Trip History Data
    • Million Song Data
    • Census Income Data
    • Movie Lens Data
    • Twitter Classification Data
  3. Advanced Level
    • Identify your Digits
    • Urban Sound Classification
    • Vox Celebrity Data
    • ImageNet Data
    • Chicago Crime Data
    • Age Detection of Indian Actors Data
    • Recommendation Engine Data
    • VisualQA Data

 

Beginner Level

1. Iris Data Set

iris_dataset_scatterplot-svgThis is probably the most versatile, easy and resourceful dataset in pattern recognition literature. Nothing could be simpler than the Iris dataset to learn classification techniques. If you are totally new to data science, this is your start line. The data has only 150 rows & 4 columns.

Problem: Predict the class of the flower based on available attributes.

Start: Get Data | Tutorial: Get Here

Let’s have a look at the Iris data and build a Logistic Regression Model in the Live Coding window below.

2. Loan Prediction Dataset

ssAmong all industries, the insurance domain has one of the largest uses of analytics & data science methods. This dataset provides you a taste of working on data sets from insurance companies – what challenges are faced there, what strategies are used, which variables influence the outcome, etc. This is a classification problem. The data has 615 rows and 13 columns.

Problem: Predict if a loan will get approved or not.

Start: Get Data | Tutorial: Get Here

Let’s have a look at the Loan data and build a Logistic Regression Model in the Live Coding window below.

 

3. Bigmart Sales Data Set

shopping-cart-1269174_960_720Retail is another industry which extensively uses analytics to optimize business processes. Tasks like product placement, inventory management, customized offers, product bundling, etc. are being smartly handled using data science techniques. As the name suggests, this data comprises of transaction records of a sales store. This is a regression problem. The data has 8523 rows of 12 variables.

Problem: Predict the sales of a store.

Start: Get Data | Tutorial: Get Here

Let’s have a look at the Big Mart Sales data and build a Linear Regression Model in the Live Coding window below.

4. Boston Housing Data Set

14938-illustration-of-a-yellow-house-pvThis is another popular dataset used in pattern recognition literature. The data set comes from the real estate industry in Boston (US). This is a regression problem. The data has 506 rows and 14 columns. Thus, it’s a fairly small data set where you can attempt any technique without worrying about your laptop’s memory being overused.

Problem: Predict the median value of owner occupied homes.

Start: Get Data | Tutorial: Get Here

 

5. Time Series Analysis Dataset

Time Series is one of the most commonly used techniques in data science. It has wide ranging applications – weather forecasting, predicting sales, analyzing year on year trends, etc. This dataset is specific to time series and the challenge here is to forecast traffic on a mode of transportation. The data has ** rows and ** columns.

Problem: Predict the traffic on a new mode of transport.

Start: Get Data  | Tutorial: Get Here

 

6. Wine Quality Dataset

This is one of the most popular datasets along data science beginners. It is divided into 2 datasets. You can perform both regression and classification tasks on this data. It will test your understanding in different fields – outlier detection, feature selection, and unbalanced data. There are 4898 rows and 12 columns in this dataset.

Problem: Predict the quality of the wine.

Start: Get Data | Tutorial: Get Here

 

7. Turkiye Student Evaluation Dataset

This dataset is based on an evaluation form filled out by students for different courses. It has different attributes including attendance, difficulty, score for each evaluation question, among others. This is an unsupervised learning problem. The dataset has 5820 rows and 33 columns.

Problem: Use classification and clustering techniques to deal with the data.

Start: Get Data | Tutorial: Get Here

 

8. Heights and Weights Dataset

This is a fairly straightforward problem and is ideal for people starting off with data science. It is a regression problem.  The dataset has 25,000 rows and 3 columns (index, height and weight).

Problem: Predict the height or weight of a person.

Start: Get Data | Tutorial: Get Here

 

If you’re new to the world of data science, Analytics Vidhya has curated a comprehensive course – ‘Introduction to Data Science’, aimed for beginners! We will cover the basics of Python, before moving to Statistics and finally going through various Modelling techniques.

 

Intermediate Level

1. Black Friday Dataset

black-fridayThis dataset comprises of sales transactions captured at a retail store. It’s a classic dataset to explore and expand your feature engineering skills and day to day understanding from multiple shopping experiences. This is a regression problem. The dataset has 550,069 rows and 12 columns.

Problem: Predict purchase amount.

Start: Get Data | Tutorial: Get Here

 

2. Human Activity Recognition Dataset

asThis data set is collected from recordings of 30 human subjects captured via smartphones enabled with embedded inertial sensors. Many machine learning courses use this data for teaching purposes. It’s your turn now. This is a multi-classification problem. The data set has 10,299 rows and 561 columns.

Problem: Predict the activity category of a human.

Start: Get Data | Tutorial: Get Here

 

3. Text Mining Dataset

De l'éloquence judiciaire À AthenesThis dataset is originally from the Siam Text Mining Competition held in 2007. The data comprises of aviation safety reports describing problem(s) which occurred in certain flights. It is a multi-classification and high dimensional problem. It has 21,519 rows and 30,438 columns.

Problem: Classify the documents according to their labels.

Start: Get Data | Tutorial: Get Here

 

4. Trip History Dataset

trip-history-dataThis dataset comes from a bike sharing service in the United States. This dataset requires you to exercise your pro data munging skills. The data is provided quarter-wise from 2010 (Q4) onwards. Each file has 7 columns. It is a classification problem.

Problem: Predict the class of user.

Start: Get Data | Tutorial: Get Here

 

5. Million Song Dataset

million-songDid you know data science can be used in the entertainment industry also? Do it yourself now. This data set puts forward a regression task. It consists of 5,15,345 observations and 90 variables. However, this is just a tiny subset of the original database of data about a million songs.

Problem: Predict release year of the song.

Start: Get Data | Tutorial: Get Here

 

6. Census Income Dataset

us-censusIt’s an imbalanced classification and a classic machine learning problem. You know, machine learning is being extensively used to solve imbalanced problems such as cancer detection, fraud detection etc. It’s time to get your hands dirty. The data set has 48,842 rows and 14 columns. For guidance, you can check this imbalanced data project.

Problem: Predict the income class of US population.

Start: Get Data | Tutorial: Get Here

 

7. Movie Lens Dataset

movie-lens-dataHave you built a recommendation system yet? Here’s your chance! This dataset is one of the most popular & quoted datasets in the data science industry. It is available in various dimensions. Here I’ve used a fairly small size. It has 1 million ratings from 6,000 users on 4,000 movies.

Problem: Recommend new movies to users.

Start: Get Data | Tutorial: Get Here

 

8. Twitter Classification Dataset

Mining Twitter DataWorking with Twitter data has become an integral part of sentiment analysis problems. If you want to carve a niche for yourself in this area, you will have fun working on the challenge this dataset poses. The dataset is 3MB in size and has 31,962 tweets.

Problem: Identify the tweets which are hate tweets and which are not.

Start: Get Data | Tutorial: Get Here

 

Advanced Level

1. Identify your Digits Dataset

identify-the-digitsThis dataset allows you to study, analyze and recognize elements in the images. That’s exactly how your camera detects your face, using image recognition! It’s your turn to build and test that technique. It’s a digit recognition problem. This data set has 7,000 images of 28 X 28 size, totalling 31MB.

Problem: Identify digits from an image.

Start: Get Data | Tutorial: Get Here

 

2. Urban Sound Classification

When you start your machine learning journey, you go with simple machine learning problems like titanic survival prediction. But you still don’t have enough practice when it comes to real life problems. Hence, this practice problem is meant to introduce you to audio processing in the usual classification scenario. This dataset consists of 8,732 sound excerpts of urban sounds from 10 classes.

Problem: Classify the type of sound from the audio.

Start: Get Data | Tutorial: Get Here

 

3. Vox Celebrity Dataset

Audio processing is rapidly becoming an important field in deep learning hence here’s another challenging problem. This dataset is for large-scale speaker identification and contains words spoken by celebrities, extracted from YouTube videos. It’s an intriguing use case for isolating and identifying speech recognition. The data contains 100,000 utterances spoken by 1,251 celebrities.

Problem: Figure out which celebrity the voice belongs to.

Start: Get Data | Tutorial: Get Here

 

4. ImageNet Dataset

laImageNet offers variety of problems which encompasses object detection, localization, classification and screen parsing. All the images are freely available. You can search for any type of image and build your project around it. As of now, this image engine has more than 15 million images of multiple shapes sizing up to 140GB.

Problem: Problem to solve is subjected to the image type you download.

Start: Get Data | Tutorial: Get Here

 

5. Chicago Crime Dataset

chicago-crimeThe ability to handle large datasets is expected of every data scientist these days. Companies no longer prefer to work on samples when they the computational power to work on the full dataset. This dataset provides you a much needed hands-on experience of handling large data sets on your local machines. The problem is easy, but data management is the key! This dataset has 6M observations. It’s a multi-classification problem.

Problem: Predict the type of crime.

Start: Get Data | Tutorial: Get Here

 

6. Age Detection of Indian Actors Dataset

This is a fascinating challenge for any deep learning enthusiast. The dataset contains thousands of images of Indian actors and your task is to identify their age. All the images are manually selected and cropped from the video frames resulting in a high degree of variability interms of scale, pose, expression, illumination, age, resolution, occlusion, and makeup. There are 19,906 images in the training set and 6,636 in the test set.

Problem: Predict the age of the actors.

Start: Get Data | Tutorial: Get Here

 

7. Recommendation Engine Dataset

This is an advanced recommendation system challenge. In this practice problem, you are given the data of programmers and questions that they have previously solved, along with the time that they took to solve that particular question. As a data scientist, the model you build will help online judges to decide the next level of questions to recommend to a user.

Problem: Predict the time taken to solve a problem given the current status of the user.

Start: Get Data

 

8. VisualQA Dataset

VisualQA is a dataset containing open-ended questions about images. These questions require an understanding of computer vision and language. There is an automatic evaluation metric for this problem. The dataset has 265,016 images, 3 questions per image and 10 ground truth answers per question.

Problem: Use deep learning technique to answer open-ended questions about images.

Start: Get Data | Tutorial: Get Here

 

End Notes

Out of the 24 datasets listed above, you should start by finding the one that matches your skillset. Say, if you are a beginner in machine learning, avoid taking up advanced level data sets from the get go. Don’t bite more than you can chew and don’t feel overwhelmed with how much you still have to do. Instead, focus on making step-wise progress.

Once you complete 2 – 3 projects, showcase them on your resume and your GitHub profile (very important!). Lots of recruiters these days hire candidates by checking their GitHub profiles. Your motive shouldn’t be to do all the projects, but to pick out selected ones based on the problem to be solved, domain and the dataset size. If you want to look at complete project solution, take a look at this article.

Did you find this article useful? Have you already built any projects on these datasets? Do share your experience, learnings and suggestions in the comments section below.

 

Participate in our Hackathons and compete with the best Data Scientists from all over the world!

 

About the Author

Karthe
Karthe

I work in data analytics domain with exposure in the capital markets and retail businesses. I use data as a tool to solve the business problems of our customers. I mine massive datasets to develop analytical framework that help clients to address their complex business challenges. I build predictive analytics based solutions using the statistical and machine learning algorithms to the forefront of our Client's decision making process.

Our Top Authors

Download Analytics Vidhya App for the Latest blog/Article

49 thoughts on "24 Ultimate Data Science (Machine Learning) Projects To Boost Your Knowledge and Skills (& can be accessed freely)"

Demudu
Demudu says: October 26, 2016 at 6:02 am
Fantastic Reply
Mallikarjun
Mallikarjun says: October 26, 2016 at 6:31 am
Thank You so much... :) I have been wondering, how to start with projects. This will help me out. I have done machine learning course of Prof. Andrew Ng. and I have good knowledge of statistics and R and Matlab. Please let me know, if any skill required to be a data scientist. Thank you again. :) Reply
venugopal rao
venugopal rao says: October 26, 2016 at 7:18 am
Great Collection.. All together in one place Reply
Analytics Vidhya Content Team
Analytics Vidhya Content Team says: October 26, 2016 at 8:52 am
Thanks : ) Reply
Analytics Vidhya Content Team
Analytics Vidhya Content Team says: October 26, 2016 at 8:55 am
Hi Mallikarjun, I received several emails and messages to help people in selecting their data science projects, which motivated me to write this post. I'd suggest you to take up any project according to your understanding and start working on it. Through the way you'll discover topics which you are yet to pick up. All the best! Reply
Analytics Vidhya Content Team
Analytics Vidhya Content Team says: October 26, 2016 at 8:56 am
Thank you! :) Reply
Akash
Akash says: October 26, 2016 at 10:17 am
Hey Manish, Good one. Do you have the 'R' equivalent for Boston housing and Big mart sales ? Thanks, Reply
Femy
Femy says: October 26, 2016 at 12:29 pm
Do you have anything on operational risk or risk in general especially consumer credit risk Reply
Sridevi
Sridevi says: October 26, 2016 at 12:54 pm
Hi Manish, Wonderful collection. Great resource for exploring ML. Thanks Reply
Krishna
Krishna says: October 26, 2016 at 1:02 pm
Hi Manish, Can you please give some insights about the Knoctober data which was conducted recently? Reply
Analytics Vidhya Content Team
Analytics Vidhya Content Team says: October 26, 2016 at 5:03 pm
Hi Krishna, tomorrow we'll post a complete article on knocktober competition so that people like you can gather their learnings. Do keep a check on your in mails. Reply
Analytics Vidhya Content Team
Analytics Vidhya Content Team says: October 26, 2016 at 5:06 pm
Hi Femy, there are several data sets I came across while composing this post. For your persual, I would suggest you to look at https://github.com/caesar0301/awesome-public-datasets Chances are you'll find the data you are looking for! Reply
Tesfaye
Tesfaye says: October 26, 2016 at 7:41 pm
Thank you Manish. These are wonderful project ideas. Please I would love to have sample solutions if/when you have them. Reply
Tesfaye
Tesfaye says: October 26, 2016 at 7:42 pm
Thank you Manish. These are wonderful project ideas. Please I would love to have sample solutions if/when you have them. Reply
Funso Iyaju
Funso Iyaju says: October 27, 2016 at 5:52 am
Hi Manish. Thanks for sharing this. It will help a lot in my love for data analytics Reply
Prasanna Venktesh
Prasanna Venktesh says: October 28, 2016 at 6:22 am
Hi what it takes to be a Datascientist. I don't have a powerful computer. whats the configuration I need. I'm a Software developer.. Do I need to do a course or can i learn on my own. Does working on here help me acquire the datascientist skills. Can You please tell me how can i enter data scientist role without losing my job. How to start my Datascientist career and what the employee seek. Reply
RAMESH MAMILLA
RAMESH MAMILLA says: October 29, 2016 at 4:11 pm
Hello Manish, Thanks for sharing this information..am also want to build up my career in Analytics.. am facing lot of problems to getting a job..It would be a great article for the beginners..Thanks Bro.. Reply
Satya
Satya says: October 30, 2016 at 6:09 am
This is Great Manish. I was able to get all the theoretical material for learning . But Application of those concepts in real life scenarios was always an issue. But I can say not anymore . Thanks to you. Regards, Satya Reply
Palanisingh
Palanisingh says: November 01, 2016 at 2:08 pm
How to analyze the yearly data. Reply
Varun
Varun says: November 01, 2016 at 2:30 pm
Hi Manish, Awesome job getting all the information together. Reply
Partha sundar sahoo
Partha sundar sahoo says: November 07, 2016 at 2:52 am
Hi manish sir I am an electrical engineer having 1.5yr experience. Iwant to switch from my sector to it. Is this big data a right choice for me. Please give your valuable advice. Reply
sandip
sandip says: May 21, 2017 at 6:21 am
Nice collection, great Reply
Data Science Training in Hyderabad
Data Science Training in Hyderabad says: September 25, 2017 at 3:09 pm
Nice blog.Thanks for sharing great information about the Data Science projects to boost your knowledge and skills. Reply
Anju
Anju says: September 27, 2017 at 1:22 am
Your website is so well organized, perfect for an online self learner like me. Thank you team! Reply
Owusu
Owusu says: October 09, 2017 at 2:50 pm
Thanks for sharing these valuable projects. They're much appreciated. I'm starting to work on projects to boost my knowledge and skills in Data Analytics. Reply
Shahnawaz
Shahnawaz says: November 14, 2017 at 11:37 pm
Fantastic Collection and most helpful, Appreciate your efforts Reply
Ajay Chander R.
Ajay Chander R. says: January 29, 2018 at 3:56 pm
excellent info. Reply
Shakuntala
Shakuntala says: March 13, 2018 at 9:52 pm
Is there any way to download the data that we will be working on? Reply
Akshay Kumar
Akshay Kumar says: March 16, 2018 at 4:19 pm
I have worked on the Boston housing dataset. Doing these kinds of projects is the best way to test our understanding of the subject. The dataset lets us do all kinds of preprocessing and then apply many machine learning algorithms for best accuracy. By far this is the best web-page present currently for data science. Thanks, AnalyticsVidhya. Reply
Yash
Yash says: March 26, 2018 at 8:11 pm
These projects can be done using R or do we have to learn Python. I am more comfortable towards R. Reply
pankaj
pankaj says: April 17, 2018 at 11:41 pm
Thanks for the data set. Can u share the bigdata project code also with GUI feature. So that we do the practice of coding also Reply
xq
xq says: April 24, 2018 at 10:53 pm
I am a beginner and I started with the iris data. I have a question: the data has no training and test data. how to predict the class?Do I need to separate the data into two parts (training and test, and make the class of the test data to na? ) or there is some other data source? look forward to your response. thanks. Reply
Aishwarya Singh
Aishwarya Singh says: April 25, 2018 at 2:40 pm
Hi xp, Yes you can choose to split your data into train and test set, remove target column from the test set and work accordingly. If you have any queries you can always ask on the discuss portal so that the community can help you resolve the same. Reply
Binod Jung Bogati
Binod Jung Bogati says: May 31, 2018 at 10:12 am
Great, I'm willing to work with new projects added here. Thanks for the new updates. Reply
Manish
Manish says: June 01, 2018 at 11:51 am
Simply excellent effort to summaries all details in one post . Reply
Marcel
Marcel says: June 01, 2018 at 1:00 pm
People from Analytics Vidhya, Thanks again, for a great article. I keep coming back to your site. Reply
Aishwarya Singh
Aishwarya Singh says: June 01, 2018 at 4:26 pm
Hi Shakuntala, Links to download the dataset are provided in the article itself. Reply
Valentin
Valentin says: June 04, 2018 at 8:46 pm
Thanks for the wonderful post information above very useful guide helped me a lot. Reply
Hari challa
Hari challa says: June 06, 2018 at 11:16 am
Nice article useful to all Reply
vipul vijay Dere
vipul vijay Dere says: June 14, 2018 at 4:30 pm
Hello All, Can anyone tell me how can I download this projects?? Reply
Hema Ramachandran
Hema Ramachandran says: June 16, 2018 at 7:21 am
Hi, the tutorial link for NO. 5 'Time Series Analysis' is not opening. Can you kindly check that? Reply
KEVIN
KEVIN says: June 16, 2018 at 6:54 pm
actually this gives you confident in approaching ML Reply
Aishwarya Singh
Aishwarya Singh says: June 18, 2018 at 9:46 pm
Hi Hema, The link works fine for me. You can also view the course on the training portal Reply
Pulkit Sharma
Pulkit Sharma says: June 21, 2018 at 3:21 pm
Hi vipul, You can download these datasets from the links provided after each dataset. The "Get Data" option will take you to the page from where you can download the dataset. Reply
Shalabh Nair
Shalabh Nair says: July 18, 2018 at 2:36 pm
Can anyone post the tutorial to the recommendation engine? It's missing here. Reply
Pulkit Sharma
Pulkit Sharma says: July 19, 2018 at 3:03 pm
Hi Shalabh, You can refer this article to learn about recommendation engines. Reply
Kalyani Sisodiya
Kalyani Sisodiya says: August 08, 2018 at 12:42 pm
Hi, I am going through UIC site for downloading data set but i am not able to download. can anyone tell me how to download that data set.. Thanks, Kalyani Reply
Pulkit Sharma
Pulkit Sharma says: August 08, 2018 at 12:54 pm
Hi Kalyani, Can you please specify which dataset you are unable to download? Reply
Vishwa Dadhania
Vishwa Dadhania says: December 31, 2018 at 11:36 pm
Hi, I am trying to get dataset for twitter sentiment classification using the link given: https://datahack.analyticsvidhya.com/contest/practice-problem-twitter-sentiment-analysis/. I have signed up but i am not finding any link on the page. Is it because the practice problem is already closed.?? Is there anyway I can get the data since I want to practice this problem. Thanks and Best Regards. Reply

Leave a Reply Your email address will not be published. Required fields are marked *