Batch vs Stream Processing

A Flink delivery-tracking pipeline reads Kafka partitioned by user id, forwards through a stateless enrichment map, redistributes via keyBy, aggregates in 30-min tumbling windows, and sinks to PostgreSQL, with checkpoints for exactly-once.

Stateful streaming remembers per-key running state inside time windows, snapshotted by checkpoints, so you can compute things like the top five symbols by volume each minute over an infinite stream.

Ingest via a durable log, score each event with stateful streaming plus feature lookups, split OLTP serving from OLAP analytics, wrap it in validation, dead-letter queues, and monitoring, and prefer safer-but-slower in a bank.

Batch processes a finished pile of data on a schedule; streaming processes each event the instant it arrives.

Batch processes finished chunks on a schedule, micro-batch processes tiny continuous stacks with a small latency floor, and record-at-a-time processes each event instantly for the lowest latency.