Data Engineering Practice
Move and shape data the way production pipelines do. The Kafka path covers topics and partitions, producing records, and consuming them reliably: the poll loop, seeking to a position and replaying a topic. The Spark path starts with a SparkSession and DataFrame transformations, then gets schemas, types and nulls right. On every challenge you choose whether to work with AI agents, AI chat only, or no AI.
Data Engineering
2 pathsFrequently asked questions
6Which paths are in this category?
2 paths: Kafka (12 challenges) and Spark (8 challenges).
How many challenges are there?
20 challenges across 5 modules.
Do I need to install anything?
No. Every challenge runs in a browser-based IDE with everything already set up, and your code is graded automatically against a test suite.
What should I know before starting?
For Kafka, you should be able to write a small program in Java, Node.js or Python, since you work through a Kafka client. For Spark, you should be comfortable with basic programming and with filtering and grouping rows, as you would in SQL. No earlier Kafka or Spark experience is needed.
What do I build?
For Kafka, producers and consumers that read data back reliably, including seeking and replaying a topic. For Spark, DataFrame transformations and a typing layer for customer orders that handles schemas and nulls.
Can I use AI while I solve a challenge?
Yes, if you want to. Each challenge lets you choose how to work: with AI agents, with AI chat only, or with no AI at all.
