Spark Practice
Learn Spark hands-on, from DataFrames and schemas to joins, performance tuning, and streaming pipelines.
SparkSession & DataFrame Basics
Start a configured SparkSession, build DataFrames, and transform them lazily with select, filter and derived columns.
One Session
Select, Filter, Derive
Lazy Until Asked
Price List Normaliser
Schemas, Types & Nulls
Declare explicit schemas, cast safely under ANSI rules, and handle nulls with a clear, deliberate policy.
Say What You Mean
Casting Under ANSI Rules
Nulls Are Not Values
Customer Order Typing Layer
About this path
This path starts Spark with DataFrames: creating a SparkSession and transforming data, then schemas, types and nulls. You should be comfortable with basic programming and with filtering and grouping rows of data, as you would in SQL. No earlier Spark experience is needed. The schemas module is about getting types right and ends with a typing layer for customer orders.
Frequently asked questions
4What will I practice in this path?
2 modules: SparkSession & DataFrame Basics and Schemas, Types & Nulls.
How many challenges are there, and how hard are they?
8 challenges, all easy.
What does this path run on?
Challenges run on Airflow.
Do I need to install anything?
No. Every challenge runs in a browser-based IDE with everything already set up, and your code is graded automatically against a test suite.