Problem statement
Namespace slow-start runs three workloads. ledger-legacy needs about twenty-five seconds to come up before /healthz answers - that is simply how long it takes. pricing-api is up in a moment and then deadlocks ten seconds later, listener still open, and only a restart clears it. archive-worker claims it is just slow too, and takes ninety seconds. Any liveness probe tight enough for pricing-api kills ledger-legacy mid-boot, over and over; any liveness probe loose enough to let ledger-legacy boot lets pricing-api sit wedged for minutes.
About this exercise
That trade is real, and a liveness probe cannot resolve it - it has exactly one schedule, and it has to serve both the slowest boot and the fastest fault. initialDelaySeconds is the usual attempt at a fix, and it is a bad one: it is a fixed grace period that also applies after every restart, and it has to be sized for the worst boot you have ever seen.
A startup probe splits the problem in two. While it is defined and has not yet succeeded:
- the liveness and readiness probes are not run at all - the container cannot be killed for being unhealthy, because nothing is asking
- the container is not marked ready
- the startup probe runs on its own schedule, with its own failureThreshold
The moment it succeeds it never runs again, and liveness and readiness take over on their schedules. If it never succeeds - if periodSeconds x failureThreshold runs out first - the kubelet kills the container exactly as a liveness failure would, and records that the startup probe was what failed. So the startup probe is a boot budget: this much time to come up, and after that the tight rules apply.
That is what lets the liveness probe be tight. It no longer has to accommodate the boot, so it can be sized for the only thing it is actually for: how long a wedged container is allowed to keep taking traffic. With the boot covered, initialDelaySeconds on liveness stops being needed at all.
kubectl explain pod.spec.containers.startupProbe lists every field, and kubectl describe pod shows the probe lines and the events the kubelet recorded for each one. Probes are part of a container's spec, and a pod's containers cannot be rewritten in place - a pod is replaced, not edited.
How this project is graded
The grader reads both probes back off each live pod and compares every field against the specification below. Then it watches: it measures how long ledger-legacy takes to become ready and confirms it got there without a single restart and without one liveness failure on record; it waits for archive-worker to be killed and checks that the kubelet blamed the startup probe; and it reads how long the wedged incarnation of pricing-api actually lived, which is what proves the liveness timing is tight rather than merely declared. All three pods are re-seeded without probes if they are missing.
What you will build
All three pods in slow-start keep the containers, images, commands and volume they have now - manifest.yaml holds them as they run today - and each gains the same two probes.
startup probe handler HTTP GET on path /healthz, port 80 period one check every 3 seconds timeout 1 second for a single check failure threshold 15 consecutive failures (a 45 second boot budget)liveness probe handler HTTP GET on path /healthz, port 80 period one check every 3 seconds timeout 1 second for a single check failure threshold restart after 2 consecutive failures initial delay none - the boot window is the startup probe's job
No readiness probe on any of the three.
Step 1 - a budget for the boot, tight rules after it
Give every container both probes exactly as specified, with no initial delay on the liveness probe.
startup_and_liveness_probes_declared_on_every_workload and probe_budgets_match_the_required_timing should pass
Step 2 - the slow boot survives
ledger-legacy must reach ready on its own schedule, twenty-five seconds in, having never been restarted - and the events must show the startup probe was the one doing the checking while it booted, never the liveness probe.
slow_boot_completes_inside_the_startup_budget and liveness_holds_off_until_startup_succeeds should pass
Step 3 - the boot that never ends, and the wedge that comes after
archive-worker runs out of budget and must be restarted with the startup probe named as the cause. pricing-api boots instantly, wedges ten seconds later, and must be restarted on the tight liveness timing - the lifetime of the killed container is the proof.
over_budget_boot_is_restarted_with_the_startup_failure_recorded and post_startup_wedge_is_caught_on_the_tight_liveness_timing should pass
When every test passes
You have:
- A slow boot protected by a budget instead of by a guess at initialDelaySeconds
- A liveness probe sized for faults alone, because it no longer has to survive a boot
- A workload that cannot come up in its budget failing loudly, with the kubelet naming the startup probe
- A post-boot deadlock caught in seconds by the same tight probe that would have killed the slow booter without the startup gate
Verify
Click Test after every step.
