DataHour: Big Data Architecture and the Best Practices on AWS

In this DataHour, we will simplify big data processing on AWS comprising various stages: collect, store, process, analyze, and visualize. We will discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we will provide reference architectures, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

Over the last decade, the 3Vs of data - Volume, Velocity & Variety has grown massively. The Big Data revolution has completely changed the way companies collect, analyse & store data. So, it’s important to understand and choose technology wisely.

In this DataHour, Aakash will explain the following points in detail: 

• Different services available on AWS for big data processing.
• Choosing the best solution – factoring multiple conditions and stages of data.
• Step by step approach to building an effective big data platform.
• Architectural Patterns for big data processing.
• Trends and best practices on AWS.

Prerequisites: Enthusiasm for learning Data Engineering.

Aakash Bansal is currently working as a Data Engineer with Amazon. Prior to Amazon, he was working at Gartner Inc and Tata Consultancy Services as Lead Analytics Engineer.He has 10 years of experience in the data engineering field ,with a deep focus on data warehousing and data modeling.He has worked on multiple cloud technologies  - AWS and Microsoft Azure , providing solutions for complex data problems by setting up Data Warehouse and data pipeline that caters to organizational overall data needs.