Scenarios & Design Questions

Recover from a worker failure in layers: retry first because most failures are transient, then a coordinator detects a dead node by missed heartbeats and redistributes its work to surviving workers, and long-running jobs resume from their last checkpoint instead of restarting.

Cache-Aside puts the load-on-miss logic in the application, Read-Through delegates it to the cache, Write-Through writes synchronously for consistency, Write-Behind writes asynchronously for speed, and you move from Caffeine to Redis the moment multiple instances must agree.

A payment and order system is a Saga of local transactions with compensating undo steps, using synchronous calls for immediate answers and Kafka events for the rest, made safe by idempotent order creation so retries never duplicate or double-charge.

Guarantee no loss and no double-debit with a per-partner bulkhead, a circuit breaker to fail fast, transaction-ID idempotency so replays return the stored result, and a Saga with compensations plus reconciliation, because external banks are REST endpoints, not XA resources.

A double-entry ledger records every event as immutable, append-only entries that sum to zero, enforced by database constraints, with balances derived by summing entries and corrections made by reversing entries, so money is only moved and the books can never silently go wrong.

Fixed window is simplest but allows a boundary double-burst, sliding window smooths that at memory cost, token bucket allows controlled bursts, leaky bucket enforces constant outflow, and across instances you must share the counter in Redis atomically.

The parent forks independent child tasks to run concurrently, waits for all with a timeout, and combines their results, so total time equals the slowest child, not the sum.

More application instances cannot relieve a shared downstream bottleneck -- fix it at the point of contention with per-tenant fairness, sharding, a connection proxy, external quotas, and hot-key spreading.

Persist first then deliver, use WebSockets for live full-duplex delivery with a Redis pub-sub fan-out across servers, order per conversation with a sequence number, and fall back to queue plus push for offline users.

The hard problem is merging concurrent edits so all replicas converge; OT transforms ops with a central server, CRDTs merge coordination-free, both backed by an op log, WebSocket sync, and offline replay.

Start with popularity and rules, add collaborative and content-based filtering, precompute candidates offline and rank in real time, close the loop with event feedback, and handle cold-start with popularity and attributes.

Design Uber as focused microservices around a real-time geospatial location pipeline and a low-latency matching engine, with event-driven communication, a fast in-memory geo-index, a safe idempotent payment service, and full observability, all scaled horizontally.

Send notifications at scale by decoupling generation from delivery with a queue, autoscaling workers by queue depth, rate-limiting and batching to the provider, and making each notification idempotent by unique ID.

A tamper-evident audit log chains each record's hash to the previous one so any edit or deletion breaks the chain, backed by WORM storage, periodic signed checkpoints, and tiered cold storage with an index.

A URL shortener is a write-rarely, read-constantly key-value system where you generate unique short codes with a counter in base62 to avoid collisions, serve redirects from cache, choose 301 versus 302 based on analytics needs, and scale the key space with enough code length and sharding.

Accept uploads fast and transcode asynchronously into multiple renditions, store in object storage, stream adaptively with HLS/DASH, and serve segments from a CDN edge, not the origin.

Survive an OTP spike by making Resend re-send the existing valid OTP, then add multi-level rate limiting, a TTL-and-priority queue, multi-provider failover, and TOTP/email as provider-independent fallbacks.

Isolate priorities onto separate topics and pipelines with their own dedicated consumer groups and provider capacity, so the critical OTP pipeline keeps low latency even when the marketing pipeline is saturated or paused.

Decompose a "Design X" question by user journeys into components, and for each one reason Requirement then Responsibility then Data then Technology, introducing Redis, Kafka, a CDN, or sharding only where a named bottleneck proves it is needed, never technology-first.

Start simple, measure to find the one real bottleneck, relieve exactly that, and repeat, evolving through indexes, stateless app plus load balancer, caching and replicas, async queues, microservices and sharding, and auto-scaling with multi-region.

You cannot exceed a synchronous dependency's availability -- to beat it, remove it from the critical path via caching, replication, async, or degradation, and express the target as concrete behavior, not just an SLA number.

"It still handles the load" is only one constraint; replace an architecture when today's constraints -- change cost, ownership, operability, infrastructure cost, compliance, or relevance -- are the ones it fails, not the scaling problem it was built to solve.

Do not scan the primary DB with LIKE; keep a dedicated inverted-index engine updated asynchronously from events, so search is millisecond-fast with ranking and typo tolerance while the DB stays the source of truth.

Pre-rank the top completions under each prefix in a trie or edge-ngram index, debounce keystrokes, cache hot prefixes, and keep the whole lookup under a few milliseconds.

When a dependency is down, fail fast with a timeout, retry transient errors with backoff, trip a circuit breaker to stop the bleeding, and serve a fallback so the user still gets a useful answer.

A cascading failure is a small fault that the system's own retries, shared pools, synchronous chains, and autoscaling amplify into a total outage, and you contain it with bounded concurrency, deadlines, retry budgets, bulkheads, load shedding, backpressure, and graceful degradation.

Multi-level caching puts a per-process Caffeine L1 in front of a shared Redis L2, so when a L2 key expires each instance keeps serving from L1 and only one request per instance reloads, with the L1 TTL tuning the trade-off between freshness and database protection.

Make the consumer idempotent with an inbox of processed event IDs, a unique constraint, or no-op state transitions, because the real goal is that reprocessing never corrupts state, not literal exactly-once delivery.

Naive dedup double-processes because the check and the mark are separate, racing under concurrency and losing the mark on crash, so you make the mark an atomic unique insert and commit it in the same transaction as the business change.

Give every service its own private database and choose that database by the service's workload: relational for correctness, NoSQL for scale and flexibility, Redis for speed.

A shared database couples everyone through one schema and one failure point; database-per-service gives each service a private store it fully owns and others reach only through its API.

Independent retries at each layer multiply -- three layers of three retries is twenty-seven attempts, not nine -- and because retries only fire on failure, this amplification stays hidden until degradation, when it becomes the flood that finishes the system off.

"Transient" tells you the condition will change, never whether your request succeeded, so it cannot by itself decide whether retrying is safe.

Retries can hide overload by doing amplified hidden work; removing them exposes failures honestly and frees the capacity that actually keeps the system reliable.

Treat a huge migration as four separable phases -- compatible schema, dual-writing code, throttled restartable backfill with continuous verification, then staged cutover and cleanup -- never as one deployment.

Never force fifteen teams to deploy at one instant -- evolve additively or dual-publish a versioned event through a controlled deprecation, and treat the change as an organizational migration, not a schema edit.

Put a router in front of the legacy service and move one capability or slice at a time to the new one -- validating in parallel and keeping an instant path back -- until the legacy is strangled and retired.

A trustworthy architecture comes with its own falsification conditions -- the designer can state the measurable evidence that would make each choice wrong, which proves it was reasoned, not fashionable.

A crash after commit but before the response is indistinguishable from a failure to the client, so the client must retry, and only server-side idempotency keyed to the operation can make that retry safe.

Never confirm before the durable commit, because a rollback turns your confirmation into a lie; confirm only on the real committed outcome, and for slow work return 202 with a pending state.

Events are contracts like APIs, so publish deliberate, versioned domain facts that consumers need, not a raw changelog of every table mutation.

Separate instances share no memory, so a change becomes visible only by externalizing it: through a shared database, a distributed cache, or an event that other instances consume.

Do not move big bytes through your server; mint a presigned URL so the client uploads straight to object storage, and handle only lightweight metadata asynchronously on a non-blocking stack.

The app server mints a short-lived signed URL and steps aside; bytes go client-to-storage directly, in parts if huge, and you trust the file's actual magic bytes, never its label.

Defaults are ~1 MB per file and ~10 MB per request; raise them with properties for modestly larger files, but for truly large files stream the InputStream or go direct-to-storage, because getBytes() buffers the whole file in the heap.

Debug slowness by confirming and scoping with metrics first, then narrowing outside-in through CPU, memory and GC, database and connection pool, external APIs, and thread pools using logs, thread dumps, heap analysis, and traces.

Debug high CPU by first proving which threads and methods are burning it with CPU metrics, GC separation, and an async profiler, then fix the proven cause, usually a tight loop, expensive serialization, lock contention, regex backtracking, or GC pressure.

Adding servers multiplies a thread defect instead of fixing it, so you take thread dumps to see what the threads are doing and decide whether the app is spawning unbounded threads per request or a badly sized pool has all its threads blocked on a slow dependency.

Synchronous REST ties up a thread for the entire job and exhausts the pool under load, so for long work we accept-then-process, return 202 with a handle, and finish asynchronously.

Version only for breaking changes, keep every change within a version additive and optional, run old and new in parallel, and deprecate before you delete.

Every node is both client and server talking directly to peers with no central authority, which you choose only when decentralization itself is the requirement, as in blockchain, file sharing, and mesh networks.

Refresh-ahead recomputes a hot value in the background just before its TTL expires so readers never miss, and probabilistic early recompute spreads refreshes over time so keys do not all reload at the same instant and stampede the database.

Put the hot data in a replicated in-memory grid so requests hit memory instead of a database, scale by adding grid nodes, and flush to the database asynchronously – use it only when the database truly cannot keep up with extreme bursty load.

Availability is 'did it respond'; reliability is 'was the response correct' -- a bank must measure whether the transfer actually settled, because a cheerful HTTP 200 over wrong financial state is worse than an honest error.

Keep a small stable core that defines a plugin contract and runs whatever plugins are registered, so new features arrive as new plugins without ever touching the core.

The noisy neighbor is one tenant hogging shared resources and starving the rest; isolate with the cheapest mechanism -- quotas, bounded concurrency, separate pools, or dedicated infra -- that meets the product's fairness and SLA needs.

Over an unreliable network you can never get guaranteed instantaneous agreement because the last ack can always be lost, so you design for it with retries, idempotency, and reconciliation, and accept eventual consistency.

Pick round robin for uniform equal instances, least connections for variable request lengths, least response time for uneven instances, sticky or IP hash when local state ties a client to an instance, weighted for unequal capacity, and consistent hashing when instances come and go and you must minimize remapping.

When a service is down, try another instance, retry transient blips with backoff, open a circuit breaker to fail fast, and return a cached or default fallback so the client always gets a bounded, graceful response.

A service mesh is a sidecar-proxy data plane plus a central control plane that transparently gives every service mutual TLS, traffic control, resilience, and observability without code changes.

An unbounded queue turns overload into unbounded latency and memory growth while pretending to be healthy; a bounded queue with load shedding keeps accepted work fast and makes overload honest.

Physical clocks drift and lie about order, so you order distributed events by causality using logical clocks: Lamport for order, vector for detecting concurrency, and HLC for monotonic, causally-correct, human-readable timestamps.

computeIfAbsent holds the bin lock for the whole mapping function, so a slow lambda serializes every thread hashing to that bin into a thundering herd that burns CPU on park/unpark, and the fix is an AsyncLoadingCache that returns the current value instantly while one background thread recomputes.

A state machine allows only defined states and legal transitions so impossible jumps are rejected, pairing with Sagas for multi-step workflows and idempotency keys so retries never create duplicates.

One million users is a marketing number, not an engineering one: you must convert it into peak requests per second, read-to-write ratio, data size, and latency and consistency targets before you can design anything.

Keep request handling stateless and fast, add instances automatically, cache hot reads, offload slow work asynchronously, scale the database for reads, and watch the system so you can shed load before it breaks.

A service is known to be up through layered signals: liveness and readiness endpoints, registry heartbeats, Kubernetes probes, gateway health checks, and monitoring alerts, not from the fact that it once started.