Definition
- Control plane is the set of components that make global decisions about the cluster: scheduling, detecting failures, and responding to events.
- Worker node is a machine (physical or virtual) that runs application containers managed by the control plane.
- API server (kube-apiserver) is the REST API that every component and user interacts with to read and modify cluster state.
- etcd is the distributed, consistent key-value store that persists all cluster data.
- Scheduler (kube-scheduler) assigns newly created Pods to suitable worker nodes.
- Controller manager runs controllers that watch for state changes and take action to reach the desired state.
- kubelet is the agent on each node that manages containers according to Pod specifications from the API server.
Think of the National Bank of India headquarters. The control plane is the head office that makes decisions: which branch handles which customer, how many tellers each branch should have, and what to do when a branch goes offline. The worker nodes are the branch offices that serve customers. The API server is the central switchboard that all communication goes through. etcd is the master ledger that records every decision and configuration. The scheduler is the HR department that assigns new employees to branches based on capacity. The controller manager is the auditor who constantly checks if each branch has the right number of tellers and sends reinforcements if not. kubelet is the branch manager who receives instructions from headquarters and makes sure employees are working.
Understanding the Concepts
What is a Kubernetes cluster?
A Kubernetes cluster is a set of machines (nodes) that run containerized applications. The cluster has two types of nodes:
- Control plane nodes run the components that manage the cluster.
- Worker nodes run the application containers (Pods).
You interact with the cluster through kubectl, which sends requests to the API server. The API server stores state in etcd, the scheduler places Pods on nodes, and the kubelet on each node ensures the containers are running.
# View cluster information
kubectl cluster-info
# List all nodes in the cluster
kubectl get nodes
# Describe a specific node
kubectl describe node worker-1
NAME STATUS ROLES AGE VERSION
control-1 Ready control-plane 30d v1.30.0
worker-1 Ready <none> 30d v1.30.0
worker-2 Ready <none> 30d v1.30.0
worker-3 Ready <none> 30d v1.30.0
package com.bank.k8s;
public class ClusterOverviewDemo {
public static void main(String[] args) {
System.out.println("National Bank of India Kubernetes Cluster:");
System.out.println();
System.out.println("Control Plane (1 or 3 nodes for HA):");
System.out.println(" kube-apiserver -> REST API gateway");
System.out.println(" etcd -> cluster state store");
System.out.println(" kube-scheduler -> Pod placement");
System.out.println(" kube-controller-manager -> state reconciliation");
System.out.println();
System.out.println("Worker Nodes (3 nodes):");
System.out.println(" kubelet -> manages containers on node");
System.out.println(" kube-proxy -> networking rules");
System.out.println(" container runtime -> runs containers (containerd)");
System.out.println();
System.out.println("Running workloads:");
System.out.println(" transfer-service -> 3 replicas across 3 nodes");
System.out.println(" account-service -> 2 replicas");
System.out.println(" notification-svc -> 2 replicas");
}
}
National Bank of India Kubernetes Cluster:
Control Plane (1 or 3 nodes for HA):
kube-apiserver -> REST API gateway
etcd -> cluster state store
kube-scheduler -> Pod placement
kube-controller-manager -> state reconciliation
Worker Nodes (3 nodes):
kubelet -> manages containers on node
kube-proxy -> networking rules
container runtime -> runs containers (containerd)
Running workloads:
transfer-service -> 3 replicas across 3 nodes
account-service -> 2 replicas
notification-svc -> 2 replicas
How does the API server work?
The API server (kube-apiserver) is the central hub of the cluster. Every component communicates through it. When you run kubectl apply -f deployment.yaml, kubectl sends the request to the API server. The API server validates it, stores it in etcd, and notifies the scheduler and controllers.
The API server is the only component that directly reads and writes to etcd. All other components go through the API server.
# All kubectl commands talk to the API server
kubectl get pods -n banking # API server reads from etcd
kubectl apply -f deployment.yaml # API server writes to etcd
kubectl delete pod transfer-pod-abc # API server deletes from etcd
# You can also call the API server directly
kubectl proxy &
curl http://localhost:8001/api/v1/namespaces/banking/pods
package com.bank.k8s;
public class ApiServerDemo {
public static void main(String[] args) {
System.out.println("API Server request flow:");
System.out.println();
System.out.println("1. kubectl apply -f transfer-deployment.yaml");
System.out.println("2. API server authenticates the request (RBAC)");
System.out.println("3. API server validates the YAML structure");
System.out.println("4. Admission controllers run (quotas, policies)");
System.out.println("5. API server stores the object in etcd");
System.out.println("6. Scheduler watches for new unscheduled Pods");
System.out.println("7. Scheduler assigns Pod to worker-2");
System.out.println("8. kubelet on worker-2 pulls image and starts container");
}
}
API Server request flow:
1. kubectl apply -f transfer-deployment.yaml
2. API server authenticates the request (RBAC)
3. API server validates the YAML structure
4. Admission controllers run (quotas, policies)
5. API server stores the object in etcd
6. Scheduler watches for new unscheduled Pods
7. Scheduler assigns Pod to worker-2
8. kubelet on worker-2 pulls image and starts container
What does etcd store?
etcd is a distributed, strongly consistent key-value store. It stores the entire cluster state: every Pod, Service, ConfigMap, Secret, Deployment, and node status.
etcd is the source of truth. If etcd data is lost and there is no backup, the entire cluster state is lost.
# etcd stores cluster state as key-value pairs
# Key: /registry/deployments/banking/transfer-service
# Value: the Deployment object in protobuf format
# Backup etcd (critical for disaster recovery)
ETCDCTL_API=3 etcdctl snapshot save /backup/etcd-snapshot.db
# Verify backup
ETCDCTL_API=3 etcdctl snapshot status /backup/etcd-snapshot.db
package com.bank.k8s;
public class EtcdDemo {
public static void main(String[] args) {
System.out.println("What etcd stores:");
System.out.println(" /registry/pods/banking/transfer-pod-abc");
System.out.println(" /registry/deployments/banking/transfer-service");
System.out.println(" /registry/services/banking/transfer-svc");
System.out.println(" /registry/configmaps/banking/app-config");
System.out.println(" /registry/secrets/banking/db-credentials");
System.out.println();
System.out.println("Critical rules:");
System.out.println(" - Back up etcd regularly (hourly for banking)");
System.out.println(" - Encrypt etcd at rest (sensitive data)");
System.out.println(" - Run 3 or 5 etcd nodes for high availability");
System.out.println(" - Only API server talks to etcd directly");
}
}
What etcd stores:
/registry/pods/banking/transfer-pod-abc
/registry/deployments/banking/transfer-service
/registry/services/banking/transfer-svc
/registry/configmaps/banking/app-config
/registry/secrets/banking/db-credentials
Critical rules:
- Back up etcd regularly (hourly for banking)
- Encrypt etcd at rest (sensitive data)
- Run 3 or 5 etcd nodes for high availability
- Only API server talks to etcd directly
How does the scheduler place Pods on nodes?
The scheduler watches for newly created Pods that have no node assigned. For each unscheduled Pod, it:
- Filters nodes that cannot run the Pod (not enough CPU, memory, or disk; taints that the Pod does not tolerate).
- Scores the remaining nodes based on resource availability, affinity rules, and spreading.
- Assigns the Pod to the highest-scoring node.
# Pod with resource requests that the scheduler uses
kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: transfer-pod
namespace: banking
spec:
containers:
- name: transfer
image: national-bank/transfer-service:1.0.0
resources:
requests:
cpu: "250m"
memory: "512Mi"
limits:
cpu: "500m"
memory: "1Gi"
EOF
package com.bank.k8s;
public class SchedulerDemo {
public static void main(String[] args) {
System.out.println("Scheduler decision for transfer-pod:");
System.out.println();
System.out.println("Step 1: Filter");
System.out.println(" worker-1: 2 CPU free, 4 GB free -> eligible");
System.out.println(" worker-2: 0.2 CPU free, 1 GB free -> not enough CPU");
System.out.println(" worker-3: 1 CPU free, 3 GB free -> eligible");
System.out.println();
System.out.println("Step 2: Score");
System.out.println(" worker-1: score 80 (most resources available)");
System.out.println(" worker-3: score 60");
System.out.println();
System.out.println("Step 3: Assign");
System.out.println(" transfer-pod -> worker-1");
}
}
Scheduler decision for transfer-pod:
Step 1: Filter
worker-1: 2 CPU free, 4 GB free -> eligible
worker-2: 0.2 CPU free, 1 GB free -> not enough CPU
worker-3: 1 CPU free, 3 GB free -> eligible
Step 2: Score
worker-1: score 80 (most resources available)
worker-3: score 60
Step 3: Assign
transfer-pod -> worker-1
What does the controller manager do?
The controller manager runs control loops (controllers) that watch the cluster state and take action to make the actual state match the desired state. Each controller is a separate loop.
- Deployment controller: ensures the right number of Pods are running for each Deployment.
- ReplicaSet controller: maintains the desired number of Pod replicas.
- Node controller: monitors node health and evicts Pods from unhealthy nodes.
- Job controller: creates Pods for batch jobs and tracks completion.
This is the declarative model: you declare what you want (3 replicas), and the controller continuously works to make it so.
package com.bank.k8s;
public class ControllerManagerDemo {
public static void main(String[] args) {
System.out.println("Controller reconciliation loop:");
System.out.println();
System.out.println("Desired state: 3 replicas of transfer-service");
System.out.println("Actual state: 2 replicas running (one pod crashed)");
System.out.println();
System.out.println("Deployment controller detects: desired=3, actual=2");
System.out.println("Action: create 1 new Pod");
System.out.println("Scheduler: assign new Pod to worker-3");
System.out.println("kubelet on worker-3: pull image, start container");
System.out.println();
System.out.println("Actual state: 3 replicas running");
System.out.println("Desired = Actual. No further action.");
}
}
Controller reconciliation loop:
Desired state: 3 replicas of transfer-service
Actual state: 2 replicas running (one pod crashed)
Deployment controller detects: desired=3, actual=2
Action: create 1 new Pod
Scheduler: assign new Pod to worker-3
kubelet on worker-3: pull image, start container
Actual state: 3 replicas running
Desired = Actual. No further action.
What do kubelet and kube-proxy do on worker nodes?
kubelet is the agent on every worker node. It receives Pod specifications from the API server and ensures the containers described in those specs are running and healthy. If a container crashes, kubelet restarts it.
kube-proxy runs on every node and manages network rules. It enables Services by forwarding traffic to the correct Pod, regardless of which node the Pod is running on.
package com.bank.k8s;
public class WorkerNodeDemo {
public static void main(String[] args) {
System.out.println("Worker node components:");
System.out.println();
System.out.println("kubelet:");
System.out.println(" - Watches API server for Pod assignments");
System.out.println(" - Pulls container images");
System.out.println(" - Starts and monitors containers");
System.out.println(" - Reports node status and Pod status to API server");
System.out.println(" - Runs liveness and readiness probes");
System.out.println();
System.out.println("kube-proxy:");
System.out.println(" - Maintains iptables or IPVS rules");
System.out.println(" - Routes Service traffic to healthy Pods");
System.out.println(" - Enables ClusterIP, NodePort, LoadBalancer services");
System.out.println();
System.out.println("Container runtime (containerd):");
System.out.println(" - Pulls and runs OCI container images");
System.out.println(" - Manages container lifecycle");
}
}
Worker node components:
kubelet:
- Watches API server for Pod assignments
- Pulls container images
- Starts and monitors containers
- Reports node status and Pod status to API server
- Runs liveness and readiness probes
kube-proxy:
- Maintains iptables or IPVS rules
- Routes Service traffic to healthy Pods
- Enables ClusterIP, NodePort, LoadBalancer services
Container runtime (containerd):
- Pulls and runs OCI container images
- Manages container lifecycle
What are common mistakes with Kubernetes architecture?
Mistake 1: Not setting resource requests
Wrong: Pods without resource requests can be scheduled on any node, leading to over-provisioning and out-of-memory kills.
# WRONG: no resource requests
apiVersion: v1
kind: Pod
metadata:
name: transfer-pod
spec:
containers:
- name: transfer
image: national-bank/transfer-service:1.0.0
# No resources section - scheduler cannot make informed decisions
Correct: always set resource requests and limits.
# CORRECT: resource requests guide the scheduler
apiVersion: v1
kind: Pod
metadata:
name: transfer-pod
spec:
containers:
- name: transfer
image: national-bank/transfer-service:1.0.0
resources:
requests:
cpu: "250m"
memory: "512Mi"
limits:
cpu: "500m"
memory: "1Gi"
What should a senior engineer know?
- High availability control plane. Production clusters run 3 control plane nodes with a load balancer in front of the API servers and a 3 or 5 node etcd cluster. This ensures the cluster continues operating if one control plane node fails.
- etcd backup and restore. etcd snapshots should be taken hourly for banking clusters. The restore process involves stopping all API servers, restoring the snapshot, and restarting. Without a backup, losing etcd means losing the entire cluster configuration.
- Admission controllers. Webhooks that intercept API server requests before they are persisted. Use them to enforce policies: no Pods without resource limits, no containers running as root, all images must come from an approved registry.
- Custom Resource Definitions (CRDs). Extend the Kubernetes API with your own resource types. Banking platforms use CRDs for application-specific concepts like BankingService or CompliancePolicy.
- Node affinity and taints. Control which Pods run on which nodes. Taint GPU nodes so only ML workloads schedule there. Use affinity to spread transfer-service Pods across availability zones.
What are common interview follow-up questions?
- What happens when a worker node goes down?
- How does Kubernetes handle rolling updates?
- What is the difference between a DaemonSet and a Deployment?
- How do you secure the Kubernetes API server?
- What is the role of admission controllers?
Visual Overview
Kubernetes Cluster Architecture
================================
Control Plane Node(s)
┌──────────────────────────────────────────┐
│ │
│ ┌──────────────┐ ┌────────────┐ │
│ │ kube-apiserver│<─>│ etcd │ │
│ │ (REST API) │ │ (state DB) │ │
│ └──────┬───────┘ └────────────┘ │
│ │ │
│ ┌──────┴────────┐ ┌──────────────────┐ │
│ │ kube-scheduler│ │ controller-mgr │ │
│ │ (Pod -> Node) │ │ (reconcile loop) │ │
│ └───────────────┘ └──────────────────┘ │
└──────────────────────────────────────────┘
│ API calls
────────────┼──────────────────────────────
│
Worker Node 1 Worker Node 2
┌──────────────────┐ ┌──────────────────┐
│ kubelet │ │ kubelet │
│ kube-proxy │ │ kube-proxy │
│ containerd │ │ containerd │
│ │ │ │
│ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │ transfer-pod │ │ │ │ transfer-pod │ │
│ │ account-pod │ │ │ │ notif-pod │ │
│ └──────────────┘ │ │ └──────────────┘ │
└──────────────────┘ └──────────────────┘
Request Flow: kubectl -> API server -> etcd
scheduler -> assigns Pod to node
kubelet -> starts container
Key Takeaway
A Kubernetes cluster is divided into two layers: the control plane that makes decisions and the worker nodes that run your banking applications. The control plane consists of the API server (the front door), etcd (the state store), the scheduler (assigns Pods to nodes), and the controller manager (ensures actual state matches desired state).
Every interaction goes through the API server. When you run kubectl apply -f deployment.yaml, the API server validates the request, stores it in etcd, and notifies the scheduler. The scheduler finds the best node for the Pod based on resource availability. The kubelet on that node pulls the image and starts the container.
The declarative model is what makes Kubernetes powerful. You do not tell Kubernetes to start a container. You tell it you want 3 replicas of the transfer service. If a Pod crashes, the controller notices the actual state (2 Pods) does not match the desired state (3 Pods) and creates a replacement. This self-healing loop runs continuously without human intervention.
etcd is the most critical component. It stores every object in the cluster: Pods, Services, ConfigMaps, Secrets. If etcd is lost without a backup, the entire cluster configuration is gone. For banking systems, back up etcd hourly and encrypt it at rest.
One sentence to remember: the API server is the front door, etcd is the source of truth, the scheduler places Pods, the controller manager heals the cluster, and kubelet runs the containers.
---