StaffCoder
Learning paths
Sign up

Data Engineering Practice

Move and shape data the way production pipelines do. The Kafka path covers topics and partitions, producing records, and consuming them reliably: the poll loop, seeking to a position and replaying a topic. The Spark path starts with a SparkSession and DataFrame transformations, then gets schemas, types and nulls right. On every challenge you choose whether to work with AI agents, AI chat only, or no AI.

Data Engineering

2 paths

Frequently asked questions

6

Which paths are in this category?

2 paths: Kafka (12 challenges) and Spark (8 challenges).

How many challenges are there?

20 challenges across 5 modules.

Do I need to install anything?

No. Every challenge runs in a browser-based IDE with everything already set up, and your code is graded automatically against a test suite.

What should I know before starting?

For Kafka, you should be able to write a small program in Java, Node.js or Python, since you work through a Kafka client. For Spark, you should be comfortable with basic programming and with filtering and grouping rows, as you would in SQL. No earlier Kafka or Spark experience is needed.

What do I build?

For Kafka, producers and consumers that read data back reliably, including seeking and replaying a topic. For Spark, DataFrame transformations and a typing layer for customer orders that handles schemas and nulls.

Can I use AI while I solve a challenge?

Yes, if you want to. Each challenge lets you choose how to work: with AI agents, with AI chat only, or with no AI at all.

We use cookies that are necessary to run Staffcoder, and optional analytics cookies to see which pages get used. Analytics stays off unless you accept. Read our Cookie Policy.