Definition
- Data plane – lightweight sidecar proxies (Envoy) next to each service instance that intercept all inbound and outbound traffic.
- Control plane – a central component (istiod in Istio) that configures all sidecars, distributes certificates, and enforces policies.
- Mutual TLS – traffic is encrypted and both ends verify identity automatically, without application code changes.
- Traffic management – load balancing, request routing, canary releases, traffic mirroring, and fault injection.
- Resilience – retries, timeouts, circuit breaking, and outlier detection applied uniformly.
- Observability – consistent golden-signal metrics, access logs, and distributed tracing for every call.
- Adoption trigger – many polyglot services where in-process libraries cannot keep concerns consistent.
Imagine National Bank of India has dozens of departments that constantly send documents to each other. Rather than training every clerk to encrypt mail, retry lost deliveries, check IDs, and keep delivery logs, the bank assigns a dedicated personal assistant who sits beside each clerk and handles all of that for every letter. A central operations office tells all the assistants the current rules. The clerks just write letters; the assistants silently handle security, reliability, and record-keeping.
Understanding the Concepts
What problem does a service mesh solve?
In a microservices system, every service needs encrypting traffic, retrying failed calls, timing out slow ones, breaking circuits, load balancing, enforcing access control, and emitting metrics and traces. If each team implements these in application code with libraries, you get inconsistency, duplicated effort, and drift, especially across multiple languages. A mesh moves these concerns into the infrastructure.
Cross-cutting concerns every service needs:
Security: mutual TLS, authorization
Resilience: retries, timeouts, circuit breaking
Traffic: load balancing, canary routing
Observability: metrics, traces, logs
Without mesh: each team implements differently, in different languages
With mesh: implemented once in the sidecar, configured centrally
How does the data plane work?
In Kubernetes, each service pod runs alongside a small Envoy proxy (the sidecar). All traffic flows through the sidecar. The application makes normal calls, but the sidecar intercepts and applies configured behavior.
# Kubernetes pod with Istio sidecar injection
apiVersion: apps/v1
kind: Deployment
metadata:
name: account-service
labels:
app: account-service
spec:
template:
metadata:
labels:
app: account-service
# Istio automatically injects the Envoy sidecar
spec:
containers:
- name: account-service
image: bank/account-service:1.0
ports:
- containerPort: 8080
# Envoy sidecar injected automatically by Istio
# All inbound/outbound traffic flows through it
Traffic flow with sidecar:
Caller app -> Caller's Envoy sidecar (encrypt, retry, trace)
-> Network
-> Callee's Envoy sidecar (decrypt, verify, log)
-> Callee app
Neither app knows the mesh exists. Zero code changes.
How does the control plane work?
The control plane (istiod in Istio) distributes configuration and certificates to every sidecar. An operator declares intent and the control plane pushes matching rules fleet-wide.
# Istio VirtualService for canary routing
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: transfer-service
spec:
hosts:
- transfer-service
http:
- route:
- destination:
host: transfer-service
subset: stable
weight: 95
- destination:
host: transfer-service
subset: canary
weight: 5
# Istio DestinationRule for subsets and circuit breaking
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: transfer-service
spec:
host: transfer-service
trafficPolicy:
connectionPool:
tcp:
maxConnections: 100
http:
h2UpgradePolicy: UPGRADE
outlierDetection:
consecutive5xxErrors: 5
interval: 30s
baseEjectionTime: 60s
subsets:
- name: stable
labels:
version: v1
- name: canary
labels:
version: v2
Control plane pushes to all sidecars:
"Route 5% of transfer-service traffic to v2 (canary)"
"Use mutual TLS for all east-west traffic"
"Retry idempotent calls twice"
"Eject any instance with 5+ consecutive 5xx errors"
No application code changes. No redeployment.
What security features does the mesh provide?
Automatic mutual TLS between all services, plus authorization policies controlling which services may call which.
# Istio PeerAuthentication: require mTLS everywhere
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
name: default
namespace: banking
spec:
mtls:
mode: STRICT
# Istio AuthorizationPolicy: only transfer-service can call account-service
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: account-service-policy
namespace: banking
spec:
selector:
matchLabels:
app: account-service
action: ALLOW
rules:
- from:
- source:
principals:
- cluster.local/ns/banking/sa/transfer-service
Mesh security:
All traffic encrypted with mutual TLS (automatic)
Certificates rotated automatically by istiod
Authorization: only approved services can call each other
Zero code changes in the application
What observability features does the mesh provide?
Consistent golden-signal metrics (request rate, error rate, latency), access logs, and distributed tracing spans for every call regardless of language.
Mesh-generated metrics (Prometheus format):
istio_requests_total{source="transfer-service",
destination="account-service",
response_code="200"}
istio_request_duration_milliseconds_bucket{...}
istio_tcp_connections_opened_total{...}
These metrics exist for EVERY service-to-service call
without any application instrumentation code.
When should you use a service mesh vs in-process libraries?
| Factor | In-Process Libraries | Service Mesh |
|---|---|---|
| Services count | Few (< 10) | Many (10+) |
| Languages | Single language | Polyglot |
| Operational cost | Low | Higher (run the mesh) |
| Latency overhead | None | ~1ms per hop |
| Consistency | Depends on teams | Uniform |
| Code changes | Required | None |
| Example | Resilience4j + Spring Security | Istio + Envoy |
Decision guide:
5 Java services? -> Resilience4j + Spring Security (simpler)
30 services in Java, Go, Python? -> Service mesh (consistent)
Need mTLS everywhere? -> Mesh (automatic rotation)
Need canary routing? -> Mesh (traffic splitting is built in)
What are the trade-offs?
A mesh adds operational complexity, a small per-hop latency and resource overhead, and a learning curve.
Mesh trade-offs:
+ Automatic mTLS, metrics, tracing for ALL services
+ Canary routing without code changes
+ Consistent resilience policies
+ Language-agnostic
- Extra sidecar container per pod (~50MB memory)
- ~1ms added latency per hop (two extra proxies)
- Operational complexity (run and upgrade the mesh)
- Learning curve for mesh configuration
- Debugging through proxies can be harder
How does a mesh complement an API gateway?
The gateway handles north-south (external) traffic. The mesh handles east-west (internal) traffic. They are complementary, not replacements.
External Client
│
▼ north-south
┌──────────────────────────┐
│ API Gateway │ Rate limiting, auth, TLS termination
│ (Kong / Spring Gateway) │
└──────────┬───────────────┘
│
▼ east-west
┌──────────────────────────┐
│ Service Mesh │ mTLS, retries, tracing, canary
│ (Istio / Envoy) │
│ │
│ [Svc A] <-> [Svc B] │
│ [Svc B] <-> [Svc C] │
└──────────────────────────┘
What should a senior engineer know?
Adopt incrementally, mesh a subset first. Even with mesh mTLS, services still enforce their own authorization under zero trust.
package com.bank.security;
import org.springframework.security.config.annotation.web.builders.HttpSecurity;
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
@EnableWebSecurity
public class ZeroTrustConfig {
// Even with mesh mTLS, the SERVICE still enforces authorization
// Mesh secures the channel; service owns the business decision
protected void configure(HttpSecurity http) throws Exception {
http.authorizeHttpRequests(auth -> auth
.requestMatchers("/api/transfers/**")
.hasRole("TRANSFER_SERVICE")
.anyRequest().authenticated()
);
}
}
Visual Overview
Service Mesh Architecture:
┌──────────────────────────────────────────┐
│ Control Plane (istiod) │
│ Config distribution, certificate mgmt │
│ Policy enforcement, service discovery │
└──────┬──────────────┬──────────────┬─────┘
│ │ │
config push config push config push
│ │ │
┌──────▼─────┐ ┌──────▼─────┐ ┌──────▼─────┐
│ Pod A │ │ Pod B │ │ Pod C │
│ ┌────────┐ │ │ ┌────────┐ │ │ ┌────────┐ │
│ │App │ │ │ │App │ │ │ │App │ │
│ │Container│ │ │ │Container│ │ │ │Container│ │
│ └────┬───┘ │ │ └────┬───┘ │ │ └────┬───┘ │
│ │ │ │ │ │ │ │ │
│ ┌────▼───┐ │ │ ┌────▼───┐ │ │ ┌────▼───┐ │
│ │Envoy │ │ │ │Envoy │ │ │ │Envoy │ │
│ │Sidecar │◄┼─┼►│Sidecar │◄┼─┼►│Sidecar │ │
│ └────────┘ │ │ └────────┘ │ │ └────────┘ │
└────────────┘ └────────────┘ └────────────┘
mTLS mTLS mTLS
+ metrics + metrics + metrics
+ tracing + tracing + tracing
Key Takeaway
A service mesh is a dedicated infrastructure layer that manages service-to-service communication transparently. The data plane is a set of sidecar proxies (Envoy) deployed next to each service, intercepting all traffic. The control plane (istiod) configures every sidecar, distributes certificates, and translates high-level policies into proxy rules. An operator declares intent once, and the control plane pushes it fleet-wide without touching any application code.
The features group into security (automatic mutual TLS and authorization), traffic management (load balancing, canary routing, fault injection), resilience (retries, timeouts, circuit breaking), and observability (golden-signal metrics, distributed tracing). All of this is applied identically to every service regardless of language. The trade-offs are real: a mesh adds operational complexity, a small per-hop latency overhead, and a learning curve.
At National Bank of India, with dozens of services in Java, Go, and Python, inconsistent TLS and patchy metrics triggered the adoption. The mesh gave automatic mTLS everywhere, uniform golden-signal metrics, and five-percent canary routing for a new transfer version, all through control-plane configuration with zero caller changes. The gateway handles north-south external traffic while the mesh handles east-west internal traffic.
Remember one sentence: a service mesh is a sidecar-proxy data plane plus a control plane that gives every service mutual TLS, traffic control, resilience, and observability without code changes, and you adopt it when you have too many polyglot services to keep those concerns consistent by hand.
---