Definition
- EC2 metrics cover host-level resources: CPU utilization, memory (requires CloudWatch agent), disk usage and IO, network throughput, and status checks
- Kubernetes metrics span node health, pod status and restart counts, CPU/memory requests vs limits vs actual, pending pods, and control-plane health
- JVM metrics include heap used vs max, garbage-collection frequency and pause time, thread counts, and application golden signals (rate, errors, latency)
- High availability combines multi-AZ redundancy, automated health checks and failover, and monitoring with alerting on user-facing golden signals
- Availability is measured as the fraction of requests served successfully within latency targets, not mere process uptime
Running the National Bank of India's systems is like running the building, the departments, and the individual clerks. The EC2 instance is the building: you watch its electricity, water, and floor space. The Kubernetes cluster is all the departments: you watch which departments are open and whether anyone is waiting outside. The JVM app is an individual clerk: you watch whether their desk is full of paper (heap), how often they stop to tidy up (garbage collection), and how fast they serve Rajesh and Priya. High availability means you never rely on one clerk, one department, or one building.
Understanding the Concepts
What metrics do you monitor for an EC2 instance?
For an EC2 instance, the host-level metrics matter first. CPU utilization tells you if the box is saturated. Memory usage, which EC2 does not report by default, requires the CloudWatch agent or a node exporter.
# EC2 / host metrics via node_exporter in Prometheus
# CPU utilization (percentage busy)
100 - (avg by (instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100)
# Memory usage (percentage used)
(1 - node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes) * 100
# Disk usage (percentage full)
(1 - node_filesystem_avail_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100
# Network throughput (bytes per second)
rate(node_network_receive_bytes_total[5m])
rate(node_network_transmit_bytes_total[5m])
| EC2 Metric | Why It Matters | Alert Threshold |
|---|---|---|
| CPU utilization | Saturation | > 85% for 10m |
| Memory usage | OOM risk | > 90% for 5m |
| Disk usage | Full disk crashes apps | > 85% |
| Disk IO (IOPS) | Database bottleneck | Approaching provisioned limit |
| Network throughput | Bandwidth saturation | > 80% of limit |
| Status checks | Hardware/instance health | Any failure |
What metrics do you monitor for a Kubernetes cluster?
Kubernetes monitoring spans several layers: node health, workload status, and control plane.
# Node readiness (0 = not ready)
kube_node_status_condition{condition="Ready", status="true"}
# Pod restart counts (climbing = crash loop)
rate(kube_pod_container_status_restarts_total[15m]) > 0
# Pending pods (scheduler cannot place them)
kube_pod_status_phase{phase="Pending"} > 0
# OOM-killed containers
kube_pod_container_status_last_terminated_reason{reason="OOMKilled"}
# CPU throttling (pod hitting its CPU limit)
rate(container_cpu_cfs_throttled_seconds_total[5m])
# Memory usage vs limits (approaching OOM)
container_memory_working_set_bytes / kube_pod_container_resource_limits{resource="memory"}
# Kubernetes alert rules
groups:
- name: kubernetes-alerts
rules:
- alert: PodCrashLooping
expr: rate(kube_pod_container_status_restarts_total[15m]) > 0.05
for: 5m
annotations:
summary: "Pod {{ $labels.pod }} is crash-looping"
- alert: PendingPods
expr: kube_pod_status_phase{phase="Pending"} > 0
for: 10m
annotations:
summary: "Pod {{ $labels.pod }} has been pending for 10 minutes"
What metrics do you monitor for a JVM application?
The JVM's memory and garbage collector dominate. You also watch the application golden signals: request rate, error rate, and latency.
# Heap used vs max (percentage)
jvm_memory_used_bytes{area="heap"} / jvm_memory_max_bytes{area="heap"} * 100
# Post-GC heap trend (rising floor = memory leak)
jvm_gc_live_data_size_bytes / jvm_memory_max_bytes{area="heap"} * 100
# GC pause time (stop-the-world events)
rate(jvm_gc_pause_seconds_sum[5m])
# GC frequency
rate(jvm_gc_pause_seconds_count[5m])
# Live thread count
jvm_threads_live_threads
# Application golden signals (Micrometer)
rate(http_server_requests_seconds_count[5m]) # request rate
rate(http_server_requests_seconds_count{status=~"5.."}[5m]) # error rate
histogram_quantile(0.99, rate(http_server_requests_seconds_bucket[5m])) # p99
// Spring Boot: expose JVM and app metrics via Micrometer + Prometheus
// application.yml
// management:
// endpoints:
// web:
// exposure:
// include: prometheus, health
// metrics:
// tags:
// application: transfer-service
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Timer;
public class TransferService {
private final Timer transferTimer;
public TransferService(MeterRegistry registry) {
this.transferTimer = Timer.builder("transfer.duration")
.tag("service", "transfer")
.publishPercentileHistogram(true)
.register(registry);
}
public void processTransfer(TransferRequest request) {
transferTimer.record(() -> {
// process the transfer for Rajesh and Priya
});
}
}
How do you achieve a high-availability SLA?
High availability ties all the monitoring together through three components: redundancy, automated failover, and alerting.
# Kubernetes Deployment for HA: 3 replicas across 3 AZs
apiVersion: apps/v1
kind: Deployment
metadata:
name: transfer-service
spec:
replicas: 3
template:
spec:
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
containers:
- name: transfer-service
image: nbi/transfer-service:2.1
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
periodSeconds: 5
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
periodSeconds: 10
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1Gi
How is availability measured from the user's perspective?
Availability is not just uptime of a process but the fraction of requests served successfully within latency targets.
# Availability SLI: successful requests within latency target
(
sum(rate(http_server_requests_seconds_count{status!~"5.."}[28d]))
-
sum(rate(http_server_requests_seconds_count{
status!~"5..",
le="0.3" # requests slower than 300ms count as failed
}[28d]))
)
/
sum(rate(http_server_requests_seconds_count[28d]))
# 99.95% availability SLA over 28 days:
# Total requests: 10,000,000
# Allowed failures: 5,000 (0.05%)
# Measured from the load balancer, close to the user
What are common mistakes in monitoring?
The most common mistake is alerting on resource metrics (CPU, memory) instead of user-facing signals.
# WRONG: Alert on cause, not symptom
- alert: HighCPU
expr: node_cpu_usage > 0.85
# CPU at 85% might be perfectly fine under load
# CORRECT: Alert on what users feel
- alert: HighTransferLatency
expr: histogram_quantile(0.99, rate(transfer_duration_bucket[5m])) > 0.5
for: 5m
# Users are actually experiencing slow transfers
What should a senior engineer know?
A senior engineer designs so that no single component is a point of failure: not a database, not an availability zone, not a dependency. They measure availability from the user's perspective at the load balancer, not from inside the application. They test with failure drills and keep capacity headroom so a single instance failure does not overload the survivors.
# HA checklist for NBI Transfer Service:
# [ ] 3+ replicas across 3 AZs
# [ ] Readiness and liveness probes configured
# [ ] Auto-scaling on CPU and request rate
# [ ] Database: Multi-AZ RDS with automatic failover
# [ ] Circuit breakers on downstream dependencies
# [ ] Alerts on golden signals with sustained duration
# [ ] Error budget tracking and burn-rate alerts
# [ ] Regular failure drills (chaos engineering)
What are common interview follow-up questions?
- Why does EC2 not report memory by default? EC2 status checks run at the hypervisor level; memory is inside the OS, so you need the CloudWatch agent or a node exporter.
- What is the difference between requests, limits, and actual usage in Kubernetes? Requests are the guaranteed minimum; limits are the hard ceiling; actual is what the container is using right now.
- How do you know if a JVM has a memory leak? The post-GC heap keeps rising: after each full GC, more memory remains in use than before.
Visual Overview
Three Monitoring Layers
┌────────────────────────────────────────────────┐
│ EC2 / HOST LAYER │
│ CPU, Memory, Disk, Network, Status Checks │
├────────────────────────────────────────────────┤
│ KUBERNETES LAYER │
│ Node health, Pod status, Restarts, OOM, │
│ Pending pods, CPU/Memory requests vs limits │
├────────────────────────────────────────────────┤
│ JVM / APPLICATION LAYER │
│ Heap, GC pauses, Threads, │
│ Request rate, Error rate, p99 Latency │
└────────────────────────────────────────────────┘
High Availability Architecture
┌─────────┐ ┌─────────┐ ┌─────────┐
│ AZ-1 │ │ AZ-2 │ │ AZ-3 │
│ replica │ │ replica │ │ replica │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────┬───────┴─────────┬───────┘
│ │
┌──────┴──────┐ ┌──────┴──────┐
│ Load │ │ Multi-AZ │
│ Balancer │ │ RDS │
└─────────────┘ └─────────────┘
Key Takeaway
For an EC2 instance you monitor CPU, memory (which needs the CloudWatch agent because EC2 does not report it by default), disk usage and IO, network throughput, and status checks. For a Kubernetes cluster you monitor node readiness, pod phase, container restart counts which reveal crash loops, OOM-kills, pending pods, and CPU/memory requests versus limits versus actual usage. For a JVM application you watch heap used versus max and especially the post-GC trend, because a rising floor after each full GC signals a memory leak, plus GC frequency and pause times which spike request latency, thread counts, and the application golden signals of rate, errors, and latency.
High availability ties all of this together through redundancy, automated failover, and alerting. Run multiple replicas across at least three availability zones behind a load balancer so losing one instance or one AZ does not take the service down. Kubernetes readiness and liveness probes restart failed pods and remove unhealthy ones from the service. Alert on the user-facing golden signals with a sustained for duration.
In the National Bank of India project, the transfer service had a 99.95% availability SLA and ran three replicas across three AZs. During one incident a host hit 95% disk, a pod entered a crash loop from a JVM heap leak that was OOM-killing it, and the post-GC heap sat at 88%. Because two of the three replicas stayed healthy across their zones, transfers for Rajesh and Priya continued. The team evicted the pod off the full-disk node, rolled the leaking pod, and auto-scaling replaced the bad host.
Remember one sentence: high availability comes from designing so no single component is a point of failure, and from alerting on what the user actually experiences so you remediate before the error budget is spent.
---