Students will learn how to gather and extract data from widely used data formats. They will learn how to assess the quality of data and explore best practices for data cleaning. We will also introduce students to MongoDB, covering the essentials of storing data and the MongoDB query language together with exploratory analysis using the MongoDB aggregation framework.
At the end of the class, students should be able to:
- Programmatically extract data stored in common formats such as csv, Microsoft Excel, JSON, XML and scrape web sites to parse data from HTML.
- Audit data for quality (validity, accuracy, completeness, consistency, and uniformity) and critically assess options for cleaning data in different contexts.
- Store, retrieve, and analyze data using MongoDB.
- Lesson 1: Data Extraction Fundamentals
- Lesson 2: Data in More Complex Formats
- Lesson 3: Data Quality
- Lesson 4: Working with MongoDB
- Lesson 5: Analyzing Data
- Lesson 6: Case Study – Open Street Map Data
Use important skills from data munging to improve OpenStreetMaps data for a part of the world that you care about and give back to the community.
Assumes 6 Hour/week (work at your own pace)
Fees: – INR 12,000/Month (assuming $ = INR 60)
Part Time/ Full time:
The ideal student should have the following skills:
- Programming experience in Python or a willingness to read a little documentation to understand examples and exercises throughout the course.
- The ability to perform rudimentary system administration on Windows or Unix
- At least some experience using a unix shell or Windows PowerShell will be helpful, but is not required.
- No prior experience with databases is needed.
- Shannon Bradshaw
- Gundega Dekena