A Complete Guide on Getting Started with Deep Learning in Python

Faizan Shaikh , August 31, 2016 / 24

Introduction

Deep Learning, a prominent topic in Artificial Intelligence domain, has been in the spotlight for quite some time now. It is especially known for its breakthroughs in fields like Computer Vision and Game playing (Alpha GO), surpassing human ability. Since the last survey, there has been a drastic increase in the trends. (click here to check out the survey)

Here is what Google trends shows us:

If you are interested in the topic here’s an excellent non-technical introduction. If you are interested to know the recent trends, here’s a great compilation.

Here our aim is to provide a learning path to all those who are new to deep learning and also the ones who want to explore it further. So are you ready to step onto the journey of conquering Deep Learning? Let’s GO!

Step 0 : Pre-requisites

It is recommended that before jumping on to Deep Learning, you should know the basics of Machine Learning. The Learning Path on Machine Learning is a complete resource to get you started in the field.

If you want a shorter version, here it is:

Basics of Math (Resource 1: “Math | Khan academy” (Especially Calculus, Probability and Linear Algebra))
Basics of Python (Resource: “Intro to Computer Science”, edX course)
Basics of Statistics (Resource: “Intro to Stats”, Udacity course)
Basics of Machine Learning (Resource: “Intro to Machine Learning”, Udacity course)

Timeline : Suggested: 2-6 months

Step 1 : Setup your Machine

Before going on to the next step, make sure you have the supported hardware. It is generally recommended that you should have atleast

A good enough GPU (4+ GB), preferably Nvidia
An OK CPU (eg. Intel Core i3 is ok, Intel Pentium may not be)
4 GB RAM or depending upon the dataset.

If you are still unsure, go through this hardware guide.

PS: If you are a hardcore gamer (not just candy crushers obviously!), you may already have the required hardware.

If you don’t have the required specifications, you could either buy it or lease an Amazon Web Service instance. Here’s a good guide for using AWS for deep learning.

Note: Do not install any deep learning libraries at this stage, do it on step 3.

Step 2 : A Shallow Dive

Now that you have a good enough knowledge of pre-requisites, you should go on further into understanding Deep Learning.

As per your preference you could follow:

Blog Approach : (Resource 1: “Fundamentals of Deep Learning”, Resource 2: “Hacker’s guide to Neural Networks”)
Video Approach : “Deep Learning Simplified”
TextBook Approach : (“Neural networks and Deep Learning”)

Along with the pre-requisites, you should get to know the popular deep learning libraries and the languages for running them. Here’s a (non-comprehensive) list (Check the wiki page for a more comprehensive list):

Some other notable libraries include Mocha, neon, H2O, MXNet, Keras, Lasagne, Nolearn. Here’s a list of Deep Learning libraries by Language.

Check out Lecture 12 of Stanford’s CS231n course for a brief overview of some of the popular libraries.

Timeline : Suggested 1-3 weeks

Step 3 : Choose your own Adventure!

Now comes the interesting part! Deep Learning has been applied in various fields with state-of-the-art results. To get a taste of this side of the moon, you, the reader, gets to choose which path to take. This should be a hands-on experience, so that you get a proper foundation on what you have understood until now.

Note: Each path contains a primer blog, a practical project, the required deep learning library for the project and an assisting course. First go through the primer, then install the required libraries and get on with the project. If you face any difficulties along the way, use the associated course to back you up.

Deep Learning for Computer Vision
- Primer : “DL for Computer Vision” blog.
- Project : “Facial Keypoint Detection” Tutorial
- Required libraries : Nolearn
- Associated Course : “CS231n: Convolutional Neural Networks for Visual Recognition”
Deep Learning for Natural Language Processing
- Primer : “Deep Learning, NLP, and Representations” blog.
- Project : “Deep Learning for Chatbots”: “Part 1”, “Part 2”
- Required library : Tensorflow
- Associated Course : “CS224d: Deep Learning for Natural Language Processing”
Deep Learning for Speech/Audio
- Primer : “Deep Speech: Lessons from Deep Learning” news article and corresponding video.
- Project : “Music Generation using Magenta (Tensorflow)”
- Required library : Magenta
- Associated Course : “Deep Learning (Spring 2016), CILVR Lab@NYU”
Deep Learning for Reinforcement Learning
- Primer and Project : “Deep Reinforcement Learning: Pong from Pixels”
- Required Library : No deep learning library required. Although you do require openAI gym to test your model.
- Associated Course : “CS294: Deep Reinforcement Learning”

Timeline : Suggested 1-2 months

Step 4 : Deep Dive into Deep Learning

Now you are (almost) ready to make a dent in Deep Learning Hall of Fame! The path ahead is long and deep (pun intended) and mostly unexplored. Now it is upto you to make use of this newly acquired skill as efficiently as you can. Here are some tips you should do to hone your skill.

Reiterate the above step with a different adventure.
Deep Learning for none of the above! (eg. DL for trading, DL for optimizing energy efficiency)
Use your newly learned skills to build something (Remember, with great power; comes great responsibility)
Test your Deep Learning skills (eg. kaggle)
Participate in Deep Learning community. (eg. Google Group, DL Subreddit)
Follow recent researches / researchers. (eg. “RE.WORK DL Summit”)

Timeline : Suggested – Infinity!

Noteworthy Resources

Complete Deep Learning book
Stanford UFLDL Tutorial
“Deep Learning in Neural Networks: An Overview”, a survey paper on Deep Learning
Awesome Deep Learning github repository
Yann LeCun’s recommendations for Deep Learning self-study

End Notes

I hope this learning path was helpful to you. I have tried to make it as comprehensive as possible. Now, it’s time for you to practice and read as much as you can. To gain expertise in working in neural network try out our deep learning practice problem – Identify the Digits.

Once you have an understanding of Deep Learning and its associated concepts, take the Deep Learning Skill test. The way Deep learning is gaining recognition it is important to be familiar with it.

Good luck! Did you like reading this article? Do you follow a different approach / package / library to get started with Deep Learning? I’d love to interact with you in comments.