StaffCoder
Kubernetes
15 modules · 60 challengesSign up
Module 10 · Health Probes

Readiness Probes

mediumReadiness ProbesPod Ready Conditions—No data yet

Problem statement

Namespace warmup-gate runs catalog-cache. The listener is up the instant the container starts, so the pod reports itself alive immediately - but the cache behind it is cold, and /ready does not answer until it has warmed, roughly ten seconds later. About twelve seconds after that, an upstream refresh drops /ready again for another ten before it comes back on its own. Every one of those windows currently counts as healthy, and everything downstream believes it.

About this exercise

Alive and ready are not the same claim. Alive means the process should not be restarted. Ready means it can serve a request right now. A cache that is loading, a pool that has not opened, a worker replaying a journal - all alive, none ready, and a restart would only start the wait over.

A readiness probe is what separates the two. It runs on the same schedule and with the same handlers as a liveness probe (httpGet, tcpSocket, exec), but it drives a different outcome: the container's ready flag, which raises the pod's Ready condition. Two things follow from that flag, and neither of them is a restart:

  • A pod that is not ready is removed from the endpoint list of every Service that selects it, so traffic stops arriving.
  • A pod that becomes ready again is put back, with no interruption to the process.

That is why a readiness failure mid-life is not a crisis. The container keeps running, keeps its restart count, keeps whatever state it had warmed, and simply stops being sent work until it says otherwise. Wire the same warm-up condition into a liveness probe instead and you get the opposite: a container killed for the crime of still starting up.

The timing fields matter in a different direction here than for liveness. initialDelaySeconds and periodSeconds decide how quickly a warmed instance is let into rotation; failureThreshold decides how many bad checks it takes to pull it back out. kubectl explain pod.spec.containers.readinessProbe lists every field, and kubectl get pod -o wide shows the READY column that the flag feeds.

Probes are part of a container's spec, and a pod's containers cannot be rewritten in place - a pod is replaced, not edited.

How this project is graded

The grader reads the probe back off the live pod and compares every field against the specification below. Then, to see the whole life cycle from the first second, it builds the same container with your probe in a scratch namespace of its own, samples the pod's readiness every two seconds from creation until the cycle has run, and checks the sequence: not ready through the warm-up, ready afterwards, not ready again through the refresh, ready once more - with the restart count still at zero at the end. Your own pod is never touched, and the scratch namespace is removed afterwards.

What you will build

catalog-cache in warmup-gate keeps the container, image and command it has now - manifest.yaml holds it as it runs today - and gains one readiness probe:

handler             HTTP GET on path /ready, port 80
initial delay       2 seconds after the container starts
period              one check every 2 seconds
timeout             1 second for a single check
failure threshold   2 consecutive failures mark the pod not ready

That is the only probe on the container. It carries no liveness probe and no startup probe.

Step 1 - a gate on the warm-up

Give the container the probe described above, and nothing else.

readiness_probe_declared_with_the_required_check and readiness_thresholds_match_the_required_timing should pass

Step 2 - the cold start stops lying

From the moment the container starts until the cache is warm, the pod must report itself not ready - and flip to ready once /ready answers.

pod_stays_unready_until_the_warmup_completes should pass

Step 3 - a bad patch is not a crisis

When /ready drops mid-life, the pod must leave rotation and come back without the container ever being restarted. Its restart count is the evidence.

midlife_failure_unreadies_the_pod_without_restarting_it and ready_condition_returns_true_when_the_warmup_recovers should pass

When every test passes

You have:

  • A pod that admits it is not ready while it is still warming, instead of accepting traffic it cannot serve
  • A Ready condition that tracks the application's own answer rather than the fact that a process exists
  • A mid-life failure that costs rotation and nothing else - no restart, no lost warm state
  • Recovery that puts the instance back automatically

You do not have yet:

  • Any answer for a service that takes minutes to boot. Readiness keeps traffic away, but a liveness probe tight enough to be useful will still kill it before it finishes. That is the next exercise.

Verify

Click Test after every step.

More in Health Probes

See the module
Liveness ProbesCatch a deadlocked container that keeps its port open by giving it an HTTP liveness probe with tight thresholds, and prove the restart clears the wedge while a healthy service is left alone.—No data yetmediumStartup ProbesProtect a slow boot with a startup probe carrying its own budget so a tight liveness probe can take over afterwards, and prove both the over-budget kill and the post-boot wedge land on their own timing.—No data yetmediumProbed Service FleetProbe a fleet of four misbehaving services so each self-reports and self-heals honestly - tight liveness for the wedger, warm-up readiness for the cache, a startup budget in front of liveness for the slow booters - and audit conditions, restarts and probe failures across the namespace.—No data yetmedium

We use cookies that are necessary to run Staffcoder, and optional analytics cookies to see which pages get used. Analytics stays off unless you accept. Read our Cookie Policy.