Kubernetes Fundamentals: Orchestrating Containers at Scale

The night the stream cannot drop a frame

Disney+Hotstar has publicized concurrent viewership above 25 million for a single India cricket match, most of that load compressed into the tense last few overs when everyone opens the app at once. Behind that number sits a video-encoding and delivery pipeline running across thousands of containers, spread across hundreds of physical machines, none of which the viewer ever thinks about. You already know how to package an application into a container and run it with docker run from your Grade 10 work. That skill gets you one container on one machine. It does not answer five questions a system like this asks every second: which of the thousand available machines should run this particular container right now, given what else is already running there? If a machine dies at 9:47 PM during a run chase, who notices and who restarts the work, and where? If traffic doubles in the space of an over, who adds capacity, and who removes it twenty minutes later when the match ends? When a container's IP address changes because it moved to a different machine, how does every other service that talks to it find the new address without a human editing a config file? And when you ship a bug fix, how do you replace every running copy of the old code with the new one without the stream visibly dropping?

Kubernetes exists to answer exactly those five questions: placement, self-healing, scaling, service discovery, and update rollout. None of them are things docker run was ever designed to solve, because Docker's job stops at "run this one container correctly." Orchestration begins where that job ends.

The mechanism underneath everything: declarative state and a reconciliation loop

The single idea that makes Kubernetes different from a fleet of shell scripts calling docker run on different machines is this: you never tell Kubernetes what to do. You tell it what should exist, and a continuously running control loop is responsible for making reality match that description, forever, without further instruction from you.

Contrast the two styles directly. An imperative instruction says "start a container from image X." A declarative instruction says "there should always be 3 running, healthy copies of a pod built from image X." The imperative form is a one-time command; once it executes, nobody is watching. The declarative form is a standing commitment that something in the system keeps re-checking. That "something" is a controller, and the pattern is: observe the actual state of the cluster, diff it against the desired state stored by the user, and act to close any gap. Then repeat, indefinitely.

Two details make this efficient rather than wasteful. First, controllers do not poll the whole cluster every few seconds by brute force. They register a watch on the API server, a long-lived streaming connection that pushes an event the instant a relevant object is created, updated, or deleted, so most reconciliation is triggered by change, not by a timer. Second, as a safety net against missed events, controllers also perform a periodic full resync, so the system self-corrects even if a watch stream silently dropped a message. This architecture, and the terms "desired state" and "reconciliation," trace directly back to Google's internal cluster manager Borg; Kubernetes' own designers describe the lineage in Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, and John Wilkes, "Borg, Omega, and Kubernetes," published in ACM Queue in 2016.

Cluster anatomy: who does what

A Kubernetes cluster splits into a control plane, which holds and enforces desired state, and a set of worker nodes, which actually run your containers. Four control-plane components matter for this chapter. The API server is the single front door: every read and write, from your kubectl command to every controller's watch stream, goes through it, and it validates each object before accepting it. etcd is a distributed key-value store and the cluster's only durable memory; it holds the desired state of every object. The scheduler watches for pods that exist as objects but have not yet been assigned to a node, and picks a node for each one in two phases: a filter phase that eliminates nodes that cannot possibly host the pod (not enough free CPU or memory, a taint the pod does not tolerate, a required node label missing), and a score phase that ranks the surviving candidates by weighted criteria such as how evenly resource usage would be spread across the cluster. Filtering and scoring each run in time proportional to the number of candidate nodes for every pod being placed, which is why very large clusters bound this work rather than scoring every node for every pod. The controller manager runs the reconciliation loops described above, one per object type (Deployments, ReplicaSets, and dozens more).

On every worker node, the kubelet is the agent that actually talks to the container runtime, starts and stops containers to match the pod specs assigned to that node, and reports status back to the API server. kube-proxy, covered below, handles network routing on each node.

Pod, ReplicaSet, Deployment: three layers, not one

The smallest object Kubernetes schedules is not a container, it is a Pod: one or more containers that share a network namespace (they see each other on localhost) and can share storage volumes, always scheduled together on the same node and always started, stopped, and restarted as a unit. The reason to group containers this way, rather than schedule bare containers, is the sidecar pattern: a web server container paired with a log-shipping container that tails its output and forwards it, or a container paired with a proxy that handles TLS termination for it. Both need to reach each other instantly over localhost with no network hop, which only works if they are co-located in one Pod.

Students commonly assume a Deployment directly owns and manages Pods. It does not; it manages an intermediate object called a ReplicaSet, and the ReplicaSet is the thing that actually watches Pod counts and creates or deletes Pods to match a target number. A Deployment's job is to manage a sequence of ReplicaSets over time: when you change the container image in a Deployment, it creates a new ReplicaSet with the new Pod template, scales that one up and the old one down according to a rollout strategy, and keeps the old ReplicaSet around at zero replicas afterward. That retained, scaled-to-zero ReplicaSet is exactly what makes kubectl rollout undo fast: rolling back means scaling the old ReplicaSet back up and the new one back down, no rebuilding required. Run kubectl get replicasets after a few updates and you will see this history sitting there directly.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 10
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: registry.example.com/web:2.3.1

Service: a stable name for a moving target

Pods are disposable: a controller can delete and recreate one at any time, and each incarnation gets a new internal IP address. Nothing that depends on a backend can hard-code a Pod IP and survive. A Service solves this by giving a stable virtual IP (a ClusterIP) and DNS name to a group of Pods selected by label, independent of which specific Pods currently exist. An endpoint controller continuously watches which Pods matching that label are currently Ready and writes that live list into an Endpoints (or EndpointSlice) object. On every node, kube-proxy watches that Endpoints object and programs the node's packet-routing rules, typically via iptables or IPVS, so that any packet sent to the Service's virtual IP gets destination-NAT'd to one of the currently healthy Pod IPs. A Pod that fails its readiness check is removed from Endpoints immediately, before its replacement even exists, so traffic never gets routed to it; this is a separate, faster mechanism than the Deployment-level self-healing described next.

apiVersion: v1
kind: Service
metadata:
  name: web-svc
spec:
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 8080
  type: ClusterIP

The diagram below traces one full reconciliation cycle end to end: a Deployment declares 3 replicas, one Pod crashes, and the control loop detects and repairs the gap while the Service quietly stops routing to the dead Pod and starts routing to its replacement.

Worked example 1: tracing a rolling update batch by batch

A Deployment's rolling update is governed by two bounds: maxSurge, the most pods allowed above the desired count at any moment, and maxUnavailable, the most pods allowed to be below-ready at any moment. Both can be given as percentages of the desired replica count, and Kubernetes rounds them in opposite directions: maxSurge rounds up, maxUnavailable rounds down, so that in the worst case the system never dips below the availability guarantee you asked for.

Take the Deployment above with replicas: 10, maxSurge: 25%, maxUnavailable: 25%.

10 × 0.25 = 2.5. Rounding maxSurge up: 3. Rounding maxUnavailable down: 2. So during this rollout, the controller may create at most 10 + 3 = 13 total pods at once, and must never let ready pod count fall below 10 − 2 = 8.

The table traces one valid sequence the Deployment controller can take to replace all 10 old pods with new ones, staying inside both bounds at every step.

At no point does the ready count drop below 8 or the total exceed 13; that is precisely what the two bounds were computed to guarantee. In practice the controller reconciles continuously rather than in the discrete cycles shown here, but the constraint arithmetic is exactly this, and it is why a rollout with a tighter maxUnavailable takes visibly longer: it can only ever retire a smaller batch of old pods per step.

Worked example 2: the Horizontal Pod Autoscaler's formula

A Horizontal Pod Autoscaler (HPA) periodically reads a metric, usually average CPU utilization across a Deployment's pods, and recomputes a target replica count using one formula:

desiredReplicas = ceil( currentReplicas × ( currentMetricValue / desiredMetricValue ) )

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

Suppose the Deployment is currently at 4 replicas, and the metrics pipeline reports average CPU utilization of 80%, against the target of 50% set above.

Step 1, compute the ratio: 80 / 50 = 1.6. Step 2, multiply by current replicas: 4 × 1.6 = 6.4. Step 3, round up: ceil(6.4) = 7. The HPA sets the Deployment's replica count to 7.

Two operational details keep this from thrashing. The HPA controller re-evaluates on a periodic sync (commonly every 15 seconds), not continuously, and it applies asymmetric stabilization: by default a scale-up can take effect immediately, but a scale-down is held back for a stabilization window (5 minutes by default), during which the controller takes the highest replica recommendation seen in that window rather than the newest one. That asymmetry is deliberate: it is safe to over-provision briefly, but flapping capacity down right before a second traffic spike is the failure mode worth avoiding.

The misconception worth correcting directly

A student who has only worked with plain Docker naturally reads kubectl delete pod web-7d9f4 as an instruction that removes that piece of the running application. Run it, then immediately run kubectl get pods, and the pod count is still 3. Nothing was "removed" in any lasting sense. The command deleted one object; it did not touch the Deployment's declared desired state of 3 replicas, so within seconds the ReplicaSet controller's next reconciliation notices observed (2) ≠ desired (3) and creates a new pod to close the gap. The correct mental model is that you are never managing individual Pods in a Deployment-backed application, you are managing a declaration, and Kubernetes is the thing enforcing it against reality on a loop that never stops running. Deleting a Pod under a controller is closer to plucking one weed while gardeners keep replanting the bed to a fixed layout than it is to deleting a file.

Active recall

Attempt each question before reading its answer.

1. A Deployment shows 3/3 ready pods. You run kubectl delete pod web-abcde and immediately check again. What do you see, and which component is responsible?
2. A Deployment has replicas: 16, maxSurge: 25%, maxUnavailable: 25%. Compute the surge cap and the unavailable floor.
3. Same Deployment, but an operator changes maxUnavailable to 10% while leaving maxSurge at 25%. Recompute both bounds and describe how the rollout's behavior changes as a result.
4. An HPA targets 70% CPU utilization instead of 50%, with currentReplicas = 4 and currentMetricValue = 80%. What does the formula compute?
5. Immediately after that computation, but before the HPA's next sync, someone manually runs kubectl scale deployment web --replicas=6, and CPU utilization is still measured at 80% against the same 70% target. What does the HPA compute on its next cycle, and does the manual scale survive?
6. Why is losing etcd's data catastrophic for a cluster's ability to self-heal, while losing and restarting the scheduler process is not?

Answers

1. Still 3/3 ready, typically within a few seconds. The ReplicaSet controller watches the API server, sees the pod count drop to 2 against a desired count of 3, and creates a replacement. The Deployment itself never directly touches Pods; it manages the ReplicaSet that does.

2. 16 × 0.25 = 4 exactly, so no rounding is needed either direction: maxSurge = 4, maxUnavailable = 4. The rollout may run up to 20 total pods and must keep at least 12 ready at all times.

3. maxSurge is unchanged at 4 (still 16 × 0.25 = 4, unaffected by the edit). maxUnavailable recomputes as 16 × 0.10 = 1.6, rounded down to 1. The new window is: total pods ≤ 20 (unchanged), ready pods ≥ 15 (up from 12). Because the floor for ready pods rose from 12 to 15, the controller can retire far fewer old pods per batch before it must wait for new ones to become ready, so the rollout proceeds in smaller, more frequent steps and takes longer wall-clock time, in exchange for a much smaller availability dip at any instant.

4. Ratio = 80 / 70 = 1.142857…. 4 × 1.142857 = 4.5714…. ceil(4.5714) = 5. Raising the target from 50% to 70% makes the autoscaler less aggressive: the same 80% load now only justifies 5 replicas instead of the 7 computed at a 50% target.

5. The HPA does not remember its own last recommendation as a baseline; each cycle it reads the Deployment's actual current replica count fresh from the API server. So it uses currentReplicas = 6 (the manual scale), not 5. Ratio is still 80/70 = 1.142857…. 6 × 1.142857 = 6.857…. ceil(6.857) = 7. The manual scale to 6 does not survive as a stable value; because utilization is still above target relative to the new base, the HPA pushes replicas up further, to 7, on its very next sync. This is the general rule: a manual kubectl scale on an HPA-managed Deployment is only ever a temporary override, erased by the next reconciliation.

6. etcd is the only durable store of desired state in the cluster: every Deployment, Service, and current pod-to-node binding lives there. The API server, scheduler, and controller manager are themselves stateless; if the scheduler process crashes, its supervisor restarts it and it resumes by re-reading pending pods from etcd, with zero data loss, because it never held any state of its own. If etcd's data is lost, already-running containers keep running for a while since each node's kubelet caches the pod specs it was told to run, but the control plane has forgotten what should exist: it can no longer create replacements for failures, enforce replica counts, or reconcile anything, because the "desired" half of the observe-diff-act loop is gone. This is why etcd is backed up independently and treated as the cluster's single point of true failure.

Think About It

Think about this: How would you explain kubernetes fundamentals: orchestrating containers at scale to a friend who has never seen a computer? What real-world analogy would you use? Imagine you had to build a system using these concepts — what would be your first step? Try this: before moving on, write down three things you learned and one question you still have.

Key Takeaways — Summary and Recap

Let us recap what we covered: the core ideas behind kubernetes fundamentals: orchestrating containers at scale, how they connect to real-world applications, and why they matter for your journey in computer science. Remember these key points as you move forward. For competitive exam preparation (CBSE, JEE, BITSAT), focus on understanding the WHY behind each concept, not just the WHAT.