StaffCoder
Kubernetes
15 modules · 60 challengesSign up
Module 14 · Volumes & Persistent Storage

Ephemeral Volumes

mediumemptyDir VolumesPod-Lifetime Storage—No data yet

Problem statement

Namespace frame-scratch runs frame-renderer, a two-container render pod. The renderer keeps its pass counter and its mid-job state on the container filesystem, and it aborts its first pass on purpose - so every restart begins the job from zero and it never gets past pass one. The packer next to it looks for that job state in its own filesystem and never finds it, because two containers in one pod do not share a filesystem. There are also two different kinds of throwaway data here - mid-job state and rendered thumbnails - and nothing in the pod tells them apart.

About this exercise

A volume is a directory the kubelet makes available to a pod's containers. It is declared once on the pod, under a name, and then each container says where in its filesystem that directory should appear. Two containers mounting the same volume at two different paths are looking at one directory: that is how containers in a pod hand work to each other.

An emptyDir volume is the simplest kind. It is created empty when the pod is assigned to a node and it lives exactly as long as the pod does. That lifetime is the whole point, and it is easy to get wrong in both directions:

  • A container that dies and is restarted by the pod's restart policy gets the same emptyDir back, with everything in it. The container filesystem is thrown away on restart; the volume is not.
  • The pod being deleted - or replaced, or rescheduled - takes the emptyDir with it. Nothing is written down anywhere, and a new pod of the same name starts with an empty directory again.

So an emptyDir is right for work in progress that a restart must not lose, and wrong for anything that has to outlive the pod.

An emptyDir also has a medium. By default it is backed by whatever the node uses for storage. Set the medium to memory and it becomes a tmpfs: fast, never written to disk, and counted against the pod's memory. A memory-backed emptyDir should carry a size limit, because otherwise a runaway writer fills the node's RAM.

kubectl explain pod.spec.volumes.emptyDir and kubectl explain pod.spec.containers.volumeMounts list every field. Volumes and mounts are part of a pod's spec, and a pod's spec cannot be rewritten in place - a pod is replaced, not edited.

How this project is graded

The grader reads the volumes and mounts back off the live pod and compares them against the specification below, then watches the pod do the work: the pass counter must climb across a real container restart, and each container must be able to see what the other one wrote. To find out whether the scratch space outlives the pod, the grader copies your pod spec into a namespace of its own, runs the copy until its job state has carried a restart, deletes that copy and recreates it - your pod in frame-scratch is never deleted, patched or written to. The scratch namespace is removed afterwards. If frame-renderer is missing it is re-seeded without volumes, so a half-finished replacement is graded as having none.

What you will build

frame-renderer keeps the two containers, their images and their commands exactly as they are - manifest.yaml holds the pod as it runs today - and gains two volumes:

scratch    pod-lifetime throwaway space, node-backed
           mounted at /scratch in the renderer container
           mounted at /input   in the packer container

cache pod-lifetime throwaway space, memory-backed, limited to 32Mi mounted at /cache in the renderer container not mounted in the packer container at all

The renderer writes passes, job-state.txt and its thumbnails on its own; the packer copies what it finds. You only decide where those directories come from.

Step 1 - one directory, two containers

Give the pod the scratch volume and mount it at both paths above. The packer must be able to read what the renderer wrote, and the renderer must be able to read what the packer wrote back.

scratch_volume_is_declared_and_mounted_by_both_containers and the_two_containers_share_one_scratch_directory should pass

Step 2 - the job finishes

With the mid-job state on a volume instead of the container filesystem, the aborted first pass stops costing anything: the restarted container picks the job up at pass two and the render completes.

mid_job_state_survives_the_container_restart should pass

Step 3 - scratch is not cache

Add the cache volume with the medium and limit above, mounted only where the thumbnails are written. The two kinds of throwaway data must be in two separate volumes, so the packer's listing never shows a thumbnail.

cache_volume_is_memory_backed_and_size_limited should pass

Step 4 - know what you are trusting

The scratch space survives a container restart. It does not survive the pod. Nothing in this step needs writing - it needs the spec above to be right, so that a replacement pod starts from an empty directory.

the_scratch_space_does_not_outlive_its_pod should pass

When every test passes

You have:

  • Mid-job state that a container restart no longer destroys
  • Two containers in one pod handing work to each other through a shared directory
  • Fast scratch and durable-looking cache told apart in the spec instead of by convention
  • A clear answer to what happens to all of it when the pod goes away

You do not have yet:

  • Anything that outlives the pod. Storage you can claim, and that stays behind when the pod is gone, is the next exercise.

Verify

Click Test after every step.

More in Volumes & Persistent Storage

See the module
PVCs & Dynamic ProvisioningMove a ledger off a node-local host path onto storage it actually asks for: a PersistentVolumeClaim against the default class that binds to a dynamically provisioned volume and keeps its data across pod replacement.—No data yetmediumClaim Lifecycle & ReclaimHand a bound claim from one pod to the next, retire an expired claim under a Delete reclaim policy, leave a volume behind under a Retain one, and tell a claim that is waiting apart from a claim that will never bind.—No data yetmediumStateful Scratch and StoreGive one pipeline pod shared scratch, a memory-backed cache and a claim holding the master library, retire an expired claim under its class reclaim policy, and report what each volume survives.—No data yetmedium

We use cookies that are necessary to run Staffcoder, and optional analytics cookies to see which pages get used. Analytics stays off unless you accept. Read our Cookie Policy.