Lecture 1: Why Linux Dominates Data Infrastructure?

Lecture 2: Filesystem, Permissions, and Processes Overview

Lecture 3: Understanding systemctl Commands for Hadoop and Spark

Lecture 4: Navigating Large Datasets in the CLI

Lecture 5: Working with CSV, JSON, and Log Files

Video: Introduction to Linux for Data Professionals

Generate Your 1 GB Practice Dataset

Hands-On activity: Analyze a 2 GB CSV File Using Linux Command-Line Tools

Assignment: Shell ETL – Filter & Aggregate Sales Data (No Python/R)

Linux Command Reference for Data Ops

Quiz: Introduction to Linux

Lecture 1: Installing Linux on VM, WSL, or Cloud (Ubuntu Server 22.04)

Lecture 2: User Management, SSH Key Setup, and Inter-Node Communication

Bonus: Difference Between useradd and adduser in Linux

Lecture 3: Basics of Networking, /etc/hosts, and Passwordless SSH

Lecture 4: Introduction to systemd Services and Daemons for Distributed Components

Video: Environment Setup & Cluster Configuration

3-Node Cluster Configuration Guide

Sample /etc/hosts File

Hands-On Activity: Create a 3-Node Linux Cluster

Assignment: Configure Passwordless SSH and Verify Node Connectivity

Quiz: Environment Setup & Cluster Configuration

Lecture 1: Understanding Hadoop Architecture

Lecture 2: Installing Java and Hadoop on Linux

Lecture 3: Starting and Testing the HDFS Cluster

Lecture 4: Running Your First MapReduce Job on Linux

Video: Hadoop & HDFS on Linux

Hadoop Deployment Automation Resources

Hands-on Activity: Deploy a Single-Node to 3-Node Hadoop Cluster

Hands-on Activity: Upload and Process a CSV in HDFS

Assignment: Bash Automation — Deploy Hadoop on Multiple Nodes

Quiz: Hadoop and HDFS on Linux

Lecture 1: Apache Spark Overview — Executors, Drivers, and YARN

Lecture 2: Installing and Running Spark Standalone on Linux

Lecture 3: Submitting Jobs via CLI and Python Scripts

Lecture 4: Integrating Spark with HDFS

Video: Spark on Linux

Spark Setup and ETL Practice

Hands-on Activity: Set Up Apache Spark Standalone and Verify via Web UI

Hands-on Activity: Run a PySpark Job to Read and Write Data in HDFS

Assignment: Build & Submit a Spark Job via Shell Script Automation

Quiz: Spark on Linux

Lecture 1: ETL Overview Using Linux Tools

Lecture 2: Building Pipelines with Shell Scripts

Lecture 3: Integrating Linux Scripts with Airflow or Luigi

Lecture 4: Using curl, wget, and jq for API and JSON Data Ingestion

Video: Linux for ETL and Automation

Hands-on Activity: Automate a Daily Data Fetch & Transform with Bash

Assignment: Build a fully automated ETL shell script pulling CSV data from API and processing it into local HDFS

Quiz: Linux for ETL and Automation

Lecture 1: CPU, Memory, and I/O Profiling Tools

Lecture 2: Linux Kernel Parameters and Tuning for Hadoop/Spark

Lecture 3: Filesystems & Storage Fundamentals for Big-Data Workloads (Disk I/O Optimization)

Lecture 4: Using cgroups and ulimit for Resource Control

Video: Performance Tuning for Data Workloads

Hands-On Activity: Measure Spark Job Performance Before and After Memory Tuning

Resources for Performance Tuning and Monitoring

Assignment: Tune and Document 3 Kernel Parameters for Better Data Performance

Quiz: Performance Tuning for Data Workloads

User and Group Management for Data Clusters

File Permissions and ACLs in HDFS and Linux

Using Sudoers and Restricting Access for Jobs

Key Management, SSH Hardening, and Basic Firewalls

Hands-On Activity: Simulating Hadoop Security — User ACLs & SSH Key Rotation (No Real Cluster Required)

Troubleshooting: Fixing Common ACL & SSH Key Rotation Issues

Quiz: Security And Access Management

Minimum Setup Options (Pick One)

How to Simulate a Multi-Node Cluster

What the Capstone Pipeline Can Look Like

Capstone Deliverables