Prompting & Evaluation Practice
Learn to design effective LLM prompts and build reliable evaluations to measure, compare, and improve model performance.
Linux CLIPrompt Engineering
Master the craft of prompting: write prompts that hit accuracy targets on test sets — classification, extraction, format control, few-shot, and chain-of-thought.
Ticket Classification Prompt
Write a prompt that makes the model label support tickets correctly.
Invoice Field Extraction Prompt
Write a prompt that extracts invoice fields into exact JSON.
House Style Enforcement Prompt
Write a prompt that rewrites text into a strict house style.
Few-Shot Uplift
Improve a prompt's accuracy by adding well chosen examples.
Reasoning Versus Direct
Compare a reasoning prompt against a direct one and make the better one pass.
Triage Desk Prompt
Write one prompt that routes, labels, and summarizes incoming tickets.
Evaluation
Learn to measure AI quality: build metrics, scorers, and calibrated LLM judges that agree with humans and catch regressions.
Exact Match and F1 Metrics
Implement exact match and F1, the two metrics every evaluation starts with.
Pairwise Comparison Scorer
Turn head to head judgments into win rates and a ranking.
Answer Quality Judge Rubric
Write a judge rubric whose verdicts agree with human labels.
Regression Hunt
Build a suite that catches a quality regression and names the slice it hit.
Judge Calibration
Calibrate a judge to keep its detections while cutting its false alarms.
Release Gate
Assemble metrics, scorer, and judge into a gate that decides ship or no ship.
About this path
This path covers two skills that go together: writing prompts that hit accuracy targets, and building evaluations that measure them. You work in a Linux terminal and should be comfortable running scripts there. No earlier experience with LLMs is needed. The evaluation module covers metrics, model-graded judging and regression checks, and ends with a release gate.
Frequently asked questions
4What will I practice in this path?
2 modules: Prompt Engineering and Evaluation.
How many challenges are there, and how hard are they?
12 challenges, all easy.
What does this path run on?
Challenges run on Linux CLI.
Do I need to install anything?
No. Every challenge runs in a browser-based IDE with everything already set up, and your code is graded automatically against a test suite.