Data Management & Consistency

Sharding splits rows across many independent database nodes by a shard key so no node holds everything, giving horizontal scale and fault isolation at the cost of cross-shard complexity.

A good shard key spreads data and traffic evenly AND keeps common queries on one shard; a bad one creates a hot shard that becomes the bottleneck capping the whole system.

On a shard key a query is a cheap single-node hit, but a query spanning shards is an expensive scatter-gather, and cross-shard transactions lose single-node ACID and need distributed coordination, so sharded designs co-locate related data and avoid global operations.

With hash modulo N, changing N remaps almost every key so rebalancing moves nearly all data, while consistent hashing on a ring moves only about a 1/N slice when a shard is added or removed, and virtual nodes keep that movement even.

Extract, transform, load, reconcile, and parallel-run, treating the migration as idempotent, secure, observable, and automated rather than a one-shot copy.

Keep money strongly consistent inside one service, and stitch multi-service operations together with Sagas, reliable events via the outbox, and eventual consistency: not with blocking two-phase commit.

Use idempotency keys to replay a stored payment response, unique constraints to block duplicate rows, state checks to skip already-done transitions, and distributed locks to guard a single scarce resource under concurrency.

Idempotency means repeating a request gives the same end state as doing it once, GET/PUT/DELETE/HEAD/OPTIONS are idempotent while POST/PATCH are not, and it matters because unreliable networks cause retries that must not double-apply.

Microservices stay stateless by having the client carry a signed JWT in the Authorization header on every request, and each service validates it locally, so there is no shared session and any instance can serve any request.

Accept data loss where the data is reconstructable or low-value enough that durability would cost more than the loss, and write down the acceptable loss window per data class so the cheap data never taxes the critical data.