StaffCoder
Kubernetes
15 modules · 60 challengesSign up
Module 2 · kubectl Inspection & Debugging

Cluster Triage Kit

easyCluster TriageRead-Only Diagnostics—No data yet

Problem statement

On-call gets one tool: given a namespace of healthy and broken workloads, produce the truth - what runs where, what each said, what is wrong and why - without touching a single workload. triage.sh prints nothing yet.

Namespace triage holds seven pods: a web server, a worker, a two-container pod, one workload that failed three times before it connected, and three that are stuck for three different reasons. You change none of them.

About this exercise

This brings together the topic's three exercises:

  • Cluster state - placement and status read with output formats and field paths (Reading Cluster State)
  • Evidence - current and previous-instance logs, and checks run inside containers (Logs & Exec)
  • Diagnosis - warning events filtered to one object, each naming a root cause (Events & Field Selectors)

One rule governs all of it: a triage sweep is read-only. It answers questions; it never applies, deletes, patches or writes anything. The grader fingerprints the namespace before and after your run and compares.

How this project is graded

The grader runs bash triage.sh, checks each section against the live cluster, verifies nothing in the namespace changed, and then runs the script a second time - so the report must be read from the cluster on every run, not printed from memory.

What you will build

triage.sh prints one report with three sections, each introduced by its name on a line of its own:

PLACEMENT
<pod> <node> <phase> <restarts>        one line per pod, sorted by name
                                       a pod that was never scheduled has no node: print -

EVIDENCE the failed mailer-crash instance's first log line ledger-worker's most recent log line the contents of /work/invoice.txt inside the renderer container of invoice-pair

DIAGNOSIS <pod> <reason> one line per stuck pod, using the reason the cluster recorded on its warning event

The tests

  • State_placement_table_matches_live_status - the table matches what the cluster reports right now
  • State_every_workload_present_exactly_once - every pod appears, none twice
  • Logs_previous_instance_evidence_captured - the evidence quotes the failed instance, not the running one
  • Logs_current_output_and_in_container_check - the worker's latest line and the file seen inside the container
  • Events_diagnosis_names_each_root_cause - each stuck workload paired with the cluster's own reason
  • Events_diagnosis_covers_only_broken_workloads - exactly three, no healthy pods
  • Composition_sweep_mutates_nothing - the namespace is byte-for-byte unchanged afterwards
  • Composition_report_reflects_a_second_run - a second run reads the cluster again

When every test passes

You have:

  • One command that tells an on-call engineer what runs where, what it said, and what is wrong
  • Evidence taken from the instance that actually failed
  • Diagnoses that quote the cluster instead of guessing
  • A sweep safe to run on a production namespace, proven by the audit
  • The next topic unlocked

Verify

Click Test.

More in kubectl Inspection & Debugging

See the module
Reading Cluster StateAnswer real questions about a running namespace with get and describe - wide placement, exact field extraction, sorted listings and the events section.—No data yeteasyLogs & ExecRead what containers actually said - current and previous instances, tail limits, the right container - and check in-container truth with exec.—No data yeteasyEvents & Field SelectorsDiagnose stuck workloads from the cluster's own record - events filtered by object, reason and type, each failure named by its cause.—No data yeteasy

We use cookies that are necessary to run Staffcoder, and optional analytics cookies to see which pages get used. Analytics stays off unless you accept. Read our Cookie Policy.