StaffCoder
Kubernetes
15 modules · 60 challengesSign up
Module 3 · Labels, Selectors & Annotations

Fleet Bookkeeping

easyFleet BookkeepingSelector-Driven Operations—No data yet

Problem statement

Eight workloads landed in fleet-audit over two years and nobody agreed on anything. One is unlabeled entirely, one spells its environment the old way, one is filed under the wrong tier, two still carry a key that was retired last spring. Every operational question - what is in the web tier, what can be cleaned up, what is still unaccounted for - is currently answered by someone reading a list of names off a wiki page. Bring the fleet under management, then prove it by running the operations against the cluster instead of the wiki.

About this exercise

This brings together the topic's three exercises:

  • Labels - one scheme carried by every object, applied at creation and by correcting live objects, with the retired key removed (Labeling Scheme)
  • Selectors - equality and set-based queries, in, notin and existence, answering questions no name list can keep up with (Selector Queries)
  • Annotations - the machine metadata that no selector can see and no label could hold (Annotations for Machinery)

One rule governs the whole thing: the operations may not know any pod's name. A selector describes a set by what its members are, so it keeps answering correctly when membership changes; a name list is a snapshot of yesterday that nothing updates. The grader proves the difference by adding an object of its own to the namespace, re-running your script, and requiring the report to have noticed.

The scheme every object must carry - exactly these three keys, no others:

app    the service's own name
tier   the role it plays        one of: web, worker, batch
env    where it runs            one of: prod, staging, dev

squad is retired. It must not survive on any object.

How this project is graded

The grader reads the labels and annotations off the live objects, runs bash fleet.sh and checks every section against the same query run against the cluster. It then runs the script a second time, and finally creates a throwaway object of its own, runs the script again, and requires the report to include it - which only happens if the sections are selector results rather than lists you typed. The grader's object is its own; it removes it afterwards and never touches yours.

What you will build

The migration. Every pod in fleet-audit ends up labeled exactly like this, with no squad key anywhere:

checkout-web       app=checkout   tier=web     env=prod
checkout-worker    app=checkout   tier=worker  env=prod
ledger-worker      app=ledger     tier=worker  env=prod
search-indexer     app=search     tier=worker  env=staging
invoice-batch      app=invoice    tier=batch   env=staging
report-batch       app=report     tier=batch   env=dev
pricing-api        app=pricing    tier=web     env=dev
mailer-relay       app=mailer     tier=worker  env=dev

The deploy record, on checkout-web, exactly these keys and values:

deploy.fleet/ticket   OPS-4502
deploy.fleet/owner    payments-platform
deploy.fleet/build    {"commit":"9f2c1ab","pipeline":41,"artifact":"checkout-web-1.8.3"}

fleet.sh, printing one report with six sections, each introduced by its name alone on a line. Every section holds matching pod names, one per line, sorted ascending, and is empty when nothing matches:

WEB          the web tier
WORKER       the worker tier
BATCH        the batch tier
CLEANUP      cleanup candidates: everything running in dev
NONPROD      everything whose env is not prod
LEGACY       everything still carrying the retired squad key

No pod name may appear anywhere in fleet.sh. Every section is a query the cluster answers.

Step 1 - bring the fleet to the scheme

Fix the unlabeled object, the wrong tier, the old environment spelling and the missing key, and retire squad wherever it survives. When you are done every pod carries exactly app, tier and env.

Labels_every_object_carries_the_scheme and Labels_retired_key_gone_from_the_fleet should pass

Step 2 - the deploy record

Attach the three deploy annotations to checkout-web. The build value is a JSON document and must be stored exactly as written - which is why it is an annotation and not a label.

Annotations_machine_metadata_reads_back_exactly should pass

Step 3 - the tier listings

Write the WEB, WORKER and BATCH sections of fleet.sh. Three sections, three selectors, no names.

Selectors_per_tier_listing_matches_each_tier should pass

Step 4 - the cleanup scope

Write CLEANUP: everything running in the dev environment, the fleet's candidates for deletion. Getting this wrong is how a cleanup job takes production with it.

Selectors_cleanup_candidates_match_the_env_scope should pass

Step 5 - the audits

Write NONPROD and LEGACY. NONPROD is everything whose env is not prod; LEGACY is everything that still carries the squad key at all, whatever its value - and after Step 1 that is nothing, which is the point of running it. Neither is an equality query, and the two use different requirements.

Selectors_audits_use_notin_and_exists_semantics and Composition_report_follows_a_grader_relabel should pass

Step 6 - the report is live

Nothing in the report may be remembered from an earlier run, and nothing you annotated may change any section: the annotations are invisible to every selector in the file.

Annotations_change_no_selector_result and Composition_second_run_reads_the_cluster_again should pass

The tests

  • Labels_every_object_carries_the_scheme - every pod carries exactly app, tier and env with the assigned values
  • Labels_retired_key_gone_from_the_fleet - the squad key survives on nothing
  • Annotations_machine_metadata_reads_back_exactly - the ticket, owner and build JSON read back character for character
  • Annotations_change_no_selector_result - a selector for an annotation key matches nothing, and the tier sections are unmoved by them
  • Selectors_per_tier_listing_matches_each_tier - WEB, WORKER and BATCH match the same queries run against the cluster
  • Selectors_cleanup_candidates_match_the_env_scope - CLEANUP is exactly the dev environment
  • Selectors_audits_use_notin_and_exists_semantics - NONPROD excludes prod and nothing else, LEGACY comes back empty
  • Composition_second_run_reads_the_cluster_again - a second run produces the same live report
  • Composition_report_follows_a_grader_relabel - an object the grader adds appears in the sections it belongs to, which no typed-out list could do

When every test passes

You have:

  • A fleet where every object is described the same way, and nothing carries the retired key
  • Machine metadata living on the objects it describes, in a form no label could hold
  • Every operational question answered by a query rather than a list of names off a wiki
  • Proof that the answers follow the cluster: the grader changed the fleet and the report changed with it

Verify

Click Test.

More in Labels, Selectors & Annotations

See the module
Labeling SchemeBring an inconsistently labeled namespace to one label scheme - applied at creation, corrected on live objects, a retired key removed, and read back from the cluster.—No data yeteasySelector QueriesAnswer seven operational questions about a labeled namespace with equality and set-based selectors - in, notin, exists - including the empty result and the objects missing the key.—No data yeteasyAnnotations for MachineryAttach deploy metadata - a ticket, an owner, a build JSON and an over-length summary - to the object it describes, update one key, retire another, and prove selectors never see any of it.—No data yeteasy

We use cookies that are necessary to run Staffcoder, and optional analytics cookies to see which pages get used. Analytics stays off unless you accept. Read our Cookie Policy.