Consumer: Commits, Polling, Deserialization

Adding partitions cannot relieve a single hot key because hash(key) mod partitions is deterministic; split the hot key with a bucket suffix so it spans multiple partitions, and accept that consumers must read and merge across those buckets, losing strict per-logical-key ordering.

enable.auto.commit controls safe manual commit for at-least-once, auto.offset.reset picks backlog versus skip for a fresh group, max.poll.records sizes each poll batch, and max.poll.interval.ms evicts a consumer that processes a batch too slowly.

Unconsumed-but-no-errors is usually already-committed offsets, a latest reset skipping the backlog, more consumers than partitions leaving some idle, or a stuck rebalance, and consumer-groups describe shows the offsets, lag, and assignment that tell you which.

Heartbeat interval and session timeout detect a dead or unreachable consumer via a background thread, while max poll interval detects a live but stalled consumer via the application thread, so a slow handler triggers a rebalance even with heartbeats still flowing.

Lag tells you the per-partition backlog in records, but not the time delay, the throughput, or the root cause, and an aggregate lag can hide a single stuck partition.

Order is guaranteed per partition because records get sequential offsets and exactly one consumer per group reads each partition.

A DLQ consumer tries to process, retries transient failures, and on a poison message publishes the original plus failure details to a dead-letter topic and commits, so one bad record never blocks or is lost.