StaffCoder
Spark

Spark Practice Exercises

Airflow
2 modules · 8 challengesSign up
Module 1 of 2 · 4 challenges

SparkSession & DataFrame Basics

Start a configured SparkSession, build DataFrames, and transform them lazily with select, filter and derived columns.
Finish every challenge, or complete the star challenge, to complete the module.

Challenges

4 challenges

One Session

Sign up to unlock
50%1 of 2 solvedeasy
About this challenge

Provide a single SparkSession factory with a defined app name, master and shuffle partitions setting that survives a stop, and build a DataFrame from seeded in-memory rows.

Solved in a browser IDE and graded by tests. Sign up to solve

Select, Filter, Derive

Sign up to unlock
100%1 of 1 solvedeasy
About this challenge

Rewrite a driver-side price list loop as DataFrame transformations: select, filter, derive decimal columns, rename, de-duplicate and sort.

Solved in a browser IDE and graded by tests. Sign up to solve

Lazy Until Asked

Sign up to unlock
100%1 of 1 solvedeasy
About this challenge

Restructure a PySpark report whose debug row counts trigger a job after every step, so building the chain runs no job, producing it runs only the jobs of one collect, and the source DataFrame is left unchanged.

Solved in a browser IDE and graded by tests. Sign up to solve

Price List Normaliser

STAR
Solving it completes the module
100%1 of 1 solvedeasy
About this challenge

Build a supplier price list normaliser as one module: a single configured SparkSession factory that survives a stop, the input DataFrame from seeded rows, and a lazy chain that derives retail price and margin, filters, renames, de-duplicates and sorts into a catalogue feed that is collected once.

Solved in a browser IDE and graded by tests. Sign up to solve

About this path

Learn Spark hands-on, from DataFrames and schemas to joins, performance tuning, and streaming pipelines.

This path starts Spark with DataFrames: creating a SparkSession and transforming data, then schemas, types and nulls. You should be comfortable with basic programming and with filtering and grouping rows of data, as you would in SQL. No earlier Spark experience is needed. The schemas module is about getting types right and ends with a typing layer for customer orders.

Frequently asked questions

4

What will I practice in this path?

2 modules: SparkSession & DataFrame Basics and Schemas, Types & Nulls.

How many Spark exercises are there, and how hard are they?

8 exercises, all easy.

What does this path run on?

Challenges run on Airflow.

Do I need to install anything?

No. Every challenge runs in a browser-based IDE with everything already set up, and your code is graded automatically against a test suite.

We use cookies that are necessary to run Staffcoder, and optional analytics cookies to see which pages get used. Analytics stays off unless you accept. Read our Cookie Policy.