Series

Data Engineering

This is a series of blog posts covering data engineering topics like HDFS, MapReduce, Sqoop, Hive, YARN and Spark with examples. This series is still a work in progress :)

Articles in this series

Book Review- Fundamentals of Data Engineering

Jul 31, 20235 min read206 views

Hi! Hope you’re doing well. Let me walk you through what’s going on in my head when I need to explain “What is Data Engineering? And What has been...

Book Review- Fundamentals of Data Engineering

What is Data Engineering?

Nov 12, 20223 min read142 views

Intro Hello all! Hope you are doing good. In the last couple of years, you might’ve heard a lot about Data Engineering. It surely gained a lot of buzz...

What is Data Engineering?

A Typical Data Pipeline

Nov 1, 20225 min read295 views

Introduction Hello people, Hope you are doing well. As data engineers, we build data pipelines to collect data from different source systems and...

A Typical Data Pipeline

WordCount Example with MapReduce

Aug 14, 202110 min read746 views

Hello! Hope you're doing well. In my last post I've explained about internals of Hadoop MapReduce. As promised in that post, we will write and execute...

WordCount Example with MapReduce

Hadoop MapReduce - A Programming Paradigm

Aug 1, 202111 min read1.3K views

Hello! Hope you're doing well. In my last post I've explained about the internals of HDFS in detail with hands-on examples. In this post we will...

Hadoop MapReduce - A Programming Paradigm

HDFS - Hadoop Distributed Filesystem

Jul 1, 202113 min read1.4K views

Hello! Hope you're doing well. In my last post I've shared a 10,000 feet overview of Data Engineering with Spotify as an example. From that post we...

HDFS - Hadoop Distributed Filesystem