Data Engineering

Overview Here’s a quick introduction to building machine learning pipelines using PySpark The ability to build these machine learning pipelines is a must-have skill …

Data engineers are a rare breed. Without them, a machine learning project would crumble before it starts. Their knowledge and understanding of software and …

This article has a list of resources, certifications and other important links that will help you get started on your data engineering journey.

Introduction My first encounter with Docker was not to solve a Data Science problem, but to install MySQL. Yes, to install MySQL! Quite an …

Introduction I remember the initial days of my Machine Learning (ML) projects. I had put in a lot of efforts to build a really …

Introduction In my previous article, I introduced you to the basics of Apache Spark, different data representations (RDD / DataFrame / Dataset) and basics …

Introduction Industry estimates that we are creating more than 2.5 Quintillion bytes of data every year. Think of it for a moment – 1 Qunitillion …

Introduction The data researchers no longer depend only on interviews, surveys, observational studies to collect data. Instead, they have switched to the faster ways of data …

Introduction   How many of you would agree/disagree with this statement: Google knows and understands you better than what you yourself do? Do let …

In previous article, we discussed the Hadoop ecosystem ( link ). We also spoke about two most heavily used Hadoop tools i.e. PIG and …