Kubernetes and container orchestration

Why it matters

Many cloud services you will build or operate run on Kubernetes or something shaped like it. Command syntax matters far less than understanding the control loops underneath well enough to predict what happens when a probe is wrong, a limit is too tight or a node disappears. For cloud and SRE roles, this is where design meets operations.

Core ideas

Containers and VMs (recap). A container is a group of processes isolated by kernel namespaces and limited by cgroups, sharing the host kernel; a VM runs its own kernel on virtualized hardware, isolating more strongly at higher cost. Kubernetes schedules containers grouped into pods, which share a network identity and can share volumes.

Architecture. The API server is the only component that talks to etcd, the consistent store holding every object. The scheduler assigns pending pods to nodes. Controllers, mostly bundled in the controller manager, watch objects and act to close the gap between desired and observed state. On each node the kubelet starts containers through the container runtime and reports status, while kube-proxy (or a network plugin that replaces it) programs service routing.

Reconciliation. Each object has a spec (what you want) and a status (what exists). A controller repeatedly compares them and takes one step toward the spec. It is level-triggered: it reacts to the current state, not to the sequence of events, so a missed notification or a controller restart is harmless. Deployments create ReplicaSets, ReplicaSets create pods, and each layer is just another controller. Infrastructure as code rests on the same desired-state idea.

Scheduling uses requests, not usage. The scheduler filters nodes that can fit a pod's requests and satisfy its constraints (node selectors, affinity, taints and tolerations), then scores the remainder. A node can be full by requests while nearly idle by usage.

Limits are enforced differently per resource. A container that uses up its CPU limit is throttled: it is paused until the kernel's next CPU bandwidth period begins, which shows up as latency spikes rather than errors. A container that exceeds its memory limit is OOM-killed and restarted under its restart policy. QoS classes follow from requests and limits: Guaranteed (every container has equal CPU and memory requests and limits), Burstable (some requests or limits) and BestEffort (none). Under node memory pressure, BestEffort pods are evicted first, then Burstable pods using more than they requested.

Probes have distinct jobs.

• Liveness: restart the container when it is stuck. Check only the process itself; a liveness probe that calls a database restarts every replica when the database is slow, turning a dependency problem into a full outage.
• Readiness: remove the pod from Service endpoints while it cannot serve (warming up, overloaded, draining). No restart.
• Startup: hold off the other probes until a slow-starting container is up, instead of inflating liveness delays.

Services and rollouts. A Service selects pods by label and routes to the ready ones behind a stable virtual IP and DNS name. A rolling update adds new pods and removes old ones within surge and unavailability bounds, gated on readiness, so a readiness probe that passes too early ships a broken version straight to traffic.

Autoscaling. The HPA periodically sets desired replicas to the current count times the ratio of the observed metric to its target, ignores small deviations and smooths scale-down over a stabilization window. CPU utilization is measured relative to requests, so wrong requests produce wrong scaling. It reacts over tens of seconds to minutes and cannot add capacity faster than new pods start and become ready, so it does not replace headroom or load shedding.

Common misconceptions

• "Limits protect my latency." A tight CPU limit throttles the service even on an idle node. One widely used approach is to set CPU requests carefully and keep CPU limits generous or absent for latency-sensitive services, while keeping memory limits.
• "Liveness and readiness are interchangeable." A liveness failure restarts the container; a readiness failure only removes it from load balancing. Dependency checks belong, if anywhere, in readiness.
• "Kubernetes schedules by actual usage." It schedules by requests; usage matters for throttling, eviction and autoscaling.
• "The HPA handles traffic spikes." It reacts after the fact and is bounded by pod start time; plan headroom and admission control.
• "Controllers process events in order." They reconcile current state and must be idempotent; ordering and missed events must not matter.

How to approach it

Read the components overview and the controllers page, then the resource-management, QoS and probe pages, and the HPA algorithm section. Then do the kind exercise: deploy a toy service with a liveness probe that checks a dependency, a CPU limit far below need and a memory limit below its working set, and observe events, restarts, throttling and endpoint changes. If your cloud coursework used Kubernetes, connect what you saw there to these mechanisms. Graduates should explain pods, deployments, services and probes. Mid-level candidates should debug misconfigurations and explain requests, limits and QoS. Senior candidates should discuss controllers, multi-tenancy, disruption budgets, autoscaling dynamics and cluster upgrades.