Back to Kafka Streams
Q14Question 14 of 27

What are stateless operations in Kafka Streams

Answer: Stateless operations in Kafka Streams process each record independently without maintaining any local state, and include filter, map, mapValues, flatMap, selectKey, branch, merge, and peek for transforming, routing, and observing records in a topology.

Key Concepts
  • Stateless operations process each record independently without remembering previous records or maintaining any local state store.
  • filter() keeps or discards records based on a predicate; filterNot() does the opposite.
  • map() transforms both key and value and triggers repartitioning; mapValues() transforms only the value and avoids repartitioning.
  • flatMap() takes one record and produces zero or more output records, useful for splitting multi-beneficiary payments into individual transfers.
  • selectKey() changes the record key and marks the stream for repartitioning before any key-based operation like join or groupByKey.
  • branch() splits a stream into multiple sub-streams based on predicates; merge() combines multiple streams back into one.
  • peek() performs a side-effect action like logging on each record without modifying the stream, useful for audit trails.

Definition

Stateless operations in Kafka Streams process each record independently without maintaining any local state, and include filter, map, mapValues, flatMap, selectKey, branch, merge, and peek for transforming, routing, and observing records in a topology.
  • Stateless operation is a stream processing step that looks at one record at a time without referencing any previously seen record or stored state.
  • filter() is a stateless operation that evaluates a predicate on each record and passes through only records that match.
  • map() and mapValues() are transformation operations; map() can change both key and value while mapValues() changes only the value.
  • flatMap() is a one-to-many transformation that takes one input record and produces zero, one, or many output records.
  • selectKey() is a re-keying operation that changes the record key, which marks the stream for repartitioning.
  • branch() is a routing operation that splits one stream into multiple named sub-streams based on predicates.
  • peek() is a side-effect operation that runs an action on each record (like logging) without changing the record.

Think of a payment processing assembly line at National Bank of India. Each workstation does one job to the payment slip passing through. The fraud check station (filter) passes clean payments and rejects suspicious ones. The fee calculation station (mapValues) adds a processing fee to the amount. The multi-beneficiary desk (flatMap) splits one payment into three separate slips for three recipients. The routing desk (branch) sends payments above one lakh to the priority counter and smaller ones to the regular counter. The audit camera (peek) photographs every slip without touching it. None of these stations remember the previous slip; each slip is handled on its own.

Understanding the Concepts

What are stateless operations in Kafka Streams?

Stateless operations process each record independently. They do not maintain any local state store, do not reference previously seen records, and do not require changelog topics for fault tolerance. This makes them lightweight and fast.

The stateless operations available in Kafka Streams are: filter, filterNot, map, mapValues, flatMap, flatMapValues, selectKey, branch, merge, peek, foreach, and print. Each operation takes one record as input and produces zero or more records as output.

JAVA
package com.bank.streams;

public class StatelessOperationsOverview {
    public static void main(String[] args) {
        System.out.println("Stateless operations in Kafka Streams:");
        System.out.println("  filter / filterNot  : keep or discard records");
        System.out.println("  map / mapValues     : transform key+value or value only");
        System.out.println("  flatMap / flatMapValues : one-to-many transformation");
        System.out.println("  selectKey           : change the record key");
        System.out.println("  branch              : split stream by predicates");
        System.out.println("  merge               : combine multiple streams");
        System.out.println("  peek                : side-effect without modification");
        System.out.println();
        System.out.println("Key property: no state store, no changelog topic.");
        System.out.println("Each record is processed independently.");
    }
}
TEXT
Expected output:
Stateless operations in Kafka Streams:
  filter / filterNot  : keep or discard records
  map / mapValues     : transform key+value or value only
  flatMap / flatMapValues : one-to-many transformation
  selectKey           : change the record key
  branch              : split stream by predicates
  merge               : combine multiple streams
  peek                : side-effect without modification

Key property: no state store, no changelog topic.
Each record is processed independently.

How does filter work?

filter() evaluates a predicate on each record. Records that match the predicate pass through; records that do not match are discarded. filterNot() does the opposite: it keeps records that do not match the predicate.

At National Bank of India, Rajesh uses filter to detect suspicious transactions. Any payment above 500,000 rupees is routed to the fraud review queue.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.Topology;

import java.math.BigDecimal;

public class FilterExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();
        KStream<String, String> payments = builder.stream("payments");

        BigDecimal fraudThreshold = new BigDecimal("500000");

        // filter: keep only high-value payments
        KStream<String, String> suspicious = payments
            .filter((accountId, value) -> {
                BigDecimal amount = new BigDecimal(value.split("\\|")[2]);
                return amount.compareTo(fraudThreshold) > 0;
            });
        suspicious.to("fraud-review");

        // filterNot: keep only normal payments (below threshold)
        KStream<String, String> normal = payments
            .filterNot((accountId, value) -> {
                BigDecimal amount = new BigDecimal(value.split("\\|")[2]);
                return amount.compareTo(fraudThreshold) > 0;
            });
        normal.to("normal-processing");

        Topology topology = builder.build();
        System.out.println("Filter topology built.");
        System.out.println("  Input:  payments");
        System.out.println("  Output: fraud-review (> 500000)");
        System.out.println("  Output: normal-processing (<= 500000)");
    }
}
TEXT
Expected output:
Filter topology built.
  Input:  payments
  Output: fraud-review (> 500000)
  Output: normal-processing (<= 500000)

Example records:
  SAV2001|NEFT|250000|transfer to Priya   -> normal-processing
  CUR3001|RTGS|750000|vendor payment      -> fraud-review
  SAV2002|IMPS|5000|mobile recharge       -> normal-processing

How do map and mapValues work?

map() transforms both the key and the value of each record. Because the key can change, Kafka Streams marks the resulting stream for repartitioning. mapValues() transforms only the value and leaves the key unchanged, so no repartitioning is needed.

Use mapValues() when you only need to change the value. Use map() when you need to change the key, but be aware of the repartitioning cost.

At National Bank of India, Rajesh uses mapValues to calculate processing fees. The payment amount stays the same, but a fee field is appended to the value.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.KeyValue;
import org.apache.kafka.streams.Topology;

import java.math.BigDecimal;
import java.math.RoundingMode;

public class MapExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();
        KStream<String, String> payments = builder.stream("payments");

        // mapValues: add processing fee (0.5% of amount)
        KStream<String, String> withFee = payments.mapValues(value -> {
            String[] parts = value.split("\\|");
            BigDecimal amount = new BigDecimal(parts[2]);
            BigDecimal fee = amount.multiply(new BigDecimal("0.005"))
                .setScale(2, RoundingMode.HALF_UP);
            return value + "|FEE:" + fee;
        });
        withFee.to("payments-with-fee");

        // map: rekey by branch code extracted from description
        KStream<String, String> rekeyed = payments.map((accountId, value) -> {
            String branchCode = accountId.substring(0, 3);
            return KeyValue.pair(branchCode, value);
        });
        rekeyed.to("payments-by-branch");

        Topology topology = builder.build();
        System.out.println("Map examples:");
        System.out.println("  mapValues: adds fee without repartition");
        System.out.println("  map: rekeys by branch, triggers repartition");
    }
}
TEXT
Expected output:
Map examples:
  mapValues: adds fee without repartition
  map: rekeys by branch, triggers repartition

Example mapValues result:
  Input:  SAV2001|NEFT|250000|transfer to Priya
  Output: SAV2001|NEFT|250000|transfer to Priya|FEE:1250.00

Example map result:
  Input key:  SAV2001    Input value: SAV2001|NEFT|250000|transfer to Priya
  Output key: SAV        Output value: SAV2001|NEFT|250000|transfer to Priya

How does flatMap work?

flatMap() takes one input record and produces zero, one, or many output records. Each output record can have a different key and value. Because the key can change, flatMap triggers repartitioning. flatMapValues() produces multiple values for the same key and does not trigger repartitioning.

At National Bank of India, a corporate customer sends a single bulk payment instruction with multiple beneficiaries. Rajesh uses flatMap to split it into individual transfers.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.KeyValue;
import org.apache.kafka.streams.Topology;

import java.util.ArrayList;
import java.util.List;

public class FlatMapExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();

        // Input: bulk payment with multiple beneficiaries separated by semicolons
        // Key: sender account, Value: senderAcc|type|benef1:amt1;benef2:amt2;benef3:amt3
        KStream<String, String> bulkPayments = builder.stream("bulk-payments");

        KStream<String, String> individualTransfers = bulkPayments.flatMap(
            (senderAccount, bulkValue) -> {
                String[] parts = bulkValue.split("\\|");
                String sender = parts[0];
                String type = parts[1];
                String[] beneficiaries = parts[2].split(";");

                List<KeyValue<String, String>> transfers = new ArrayList<>();
                for (String beneficiary : beneficiaries) {
                    String[] bParts = beneficiary.split(":");
                    String benefAccount = bParts[0];
                    String amount = bParts[1];
                    transfers.add(KeyValue.pair(
                        benefAccount,
                        sender + "|" + type + "|" + amount + "|to:" + benefAccount
                    ));
                }
                return transfers;
            }
        );

        individualTransfers.to("individual-transfers");

        Topology topology = builder.build();
        System.out.println("FlatMap topology: bulk payment -> individual transfers");
        System.out.println("  Input:  1 bulk payment with 3 beneficiaries");
        System.out.println("  Output: 3 individual transfer records");
    }
}
TEXT
Expected output:
FlatMap topology: bulk payment -> individual transfers
  Input:  1 bulk payment with 3 beneficiaries
  Output: 3 individual transfer records

Example:
  Input:  CUR3001|NEFT|SAV2001:50000;SAV2002:30000;SAV2003:20000
  Output record 1: key=SAV2001, value=CUR3001|NEFT|50000|to:SAV2001
  Output record 2: key=SAV2002, value=CUR3001|NEFT|30000|to:SAV2002
  Output record 3: key=SAV2003, value=CUR3001|NEFT|20000|to:SAV2003

How does selectKey work?

selectKey() changes the key of each record without modifying the value. The resulting stream is marked for repartitioning. Any downstream key-based operation (join, groupByKey, aggregate) will trigger creation of an internal repartition topic.

At National Bank of India, transactions arrive keyed by account ID. Rajesh uses selectKey to rekey them by branch code so he can aggregate transactions per branch.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.Topology;

public class SelectKeyExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();

        // Payments keyed by account ID: SAV2001, CUR3001, SAV2002
        KStream<String, String> payments = builder.stream("payments");

        // Rekey by branch code (first 3 characters of account ID)
        KStream<String, String> byBranch = payments.selectKey(
            (accountId, value) -> accountId.substring(0, 3)
        );

        // Now key is branch code: SAV, CUR
        byBranch.to("payments-by-branch");

        Topology topology = builder.build();
        System.out.println("SelectKey: rekey from account ID to branch code");
        System.out.println("  SAV2001 -> SAV");
        System.out.println("  CUR3001 -> CUR");
        System.out.println("  SAV2002 -> SAV");
        System.out.println("Stream is marked for repartitioning.");
    }
}
TEXT
Expected output:
SelectKey: rekey from account ID to branch code
  SAV2001 -> SAV
  CUR3001 -> CUR
  SAV2002 -> SAV
Stream is marked for repartitioning.

How does branch work?

branch() splits one stream into multiple named sub-streams based on predicates. In Kafka Streams 3.x, branch() returns a BranchedKStream where you define named branches with conditions. Each record goes to the first branch whose predicate matches. If no predicate matches, the record is dropped unless you add a defaultBranch().

At National Bank of India, Rajesh routes payments to different processing tiers: premium (above 1,00,000), standard (10,000 to 1,00,000), and micro (below 10,000).

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Branched;
import org.apache.kafka.streams.Topology;

import java.math.BigDecimal;
import java.util.Map;

public class BranchExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();
        KStream<String, String> payments = builder.stream("payments");

        BigDecimal premiumThreshold = new BigDecimal("100000");
        BigDecimal microThreshold = new BigDecimal("10000");

        Map<String, KStream<String, String>> branches = payments
            .split()
            .branch((key, value) -> {
                BigDecimal amount = new BigDecimal(value.split("\\|")[2]);
                return amount.compareTo(premiumThreshold) > 0;
            }, Branched.as("premium"))
            .branch((key, value) -> {
                BigDecimal amount = new BigDecimal(value.split("\\|")[2]);
                return amount.compareTo(microThreshold) < 0;
            }, Branched.as("micro"))
            .defaultBranch(Branched.as("standard"));

        branches.get("premium").to("premium-payments");
        branches.get("micro").to("micro-payments");
        branches.get("standard").to("standard-payments");

        Topology topology = builder.build();
        System.out.println("Branch routing:");
        System.out.println("  > 100000 INR  -> premium-payments");
        System.out.println("  < 10000 INR   -> micro-payments");
        System.out.println("  10000-100000  -> standard-payments");
    }
}
TEXT
Expected output:
Branch routing:
  > 100000 INR  -> premium-payments
  < 10000 INR   -> micro-payments
  10000-100000  -> standard-payments

Example routing:
  SAV2001|NEFT|250000|transfer   -> premium-payments
  SAV2002|IMPS|5000|recharge     -> micro-payments
  CUR3001|NEFT|50000|vendor pay  -> standard-payments

How does merge work?

merge() combines two or more KStream instances into a single KStream. The merged stream contains all records from all input streams in the order they arrive. Records are not deduplicated or sorted.

At National Bank of India, payments arrive through three channels: NEFT, IMPS, and UPI. Rajesh merges all three into a single stream for unified processing.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.Topology;

public class MergeExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();

        KStream<String, String> neft = builder.stream("neft-payments");
        KStream<String, String> imps = builder.stream("imps-payments");
        KStream<String, String> upi = builder.stream("upi-payments");

        // Merge all payment channels into one stream
        KStream<String, String> allPayments = neft
            .merge(imps)
            .merge(upi);

        // Process all payments uniformly
        allPayments.to("all-payments");

        Topology topology = builder.build();
        System.out.println("Merge: 3 payment channels -> 1 unified stream");
        System.out.println("  neft-payments  ─┐");
        System.out.println("  imps-payments  ─┼─> all-payments");
        System.out.println("  upi-payments   ─┘");
    }
}
DIAGRAM
Expected output:
Merge: 3 payment channels -> 1 unified stream
  neft-payments  ─┐
  imps-payments  ─┼─> all-payments
  upi-payments   ─┘

Records in merged stream (interleaved by arrival order):
  SAV2001|NEFT|250000|transfer
  SAV2002|UPI|500|grocery
  CUR3001|IMPS|15000|bill payment

How does peek work?

peek() performs a side-effect action on each record without modifying it. The record passes through unchanged. Common uses are logging, metrics collection, and audit trail writing.

peek() should not modify the record. If you need to transform the record, use map or mapValues instead. In production, Rajesh uses peek to log every transaction for debugging without affecting the processing pipeline.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.Topology;

public class PeekExample {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();
        KStream<String, String> payments = builder.stream("payments");

        KStream<String, String> processed = payments
            .peek((key, value) ->
                System.out.println("AUDIT: account=" + key + ", payment=" + value))
            .mapValues(value -> {
                String[] parts = value.split("\\|");
                return parts[0] + "|" + parts[1] + "|" + parts[2] + "|PROCESSED";
            })
            .peek((key, value) ->
                System.out.println("POST-PROCESS: account=" + key + ", result=" + value));

        processed.to("processed-payments");

        Topology topology = builder.build();
        System.out.println("Peek: audit logging at two points in the topology");
    }
}
TEXT
Expected output:
Peek: audit logging at two points in the topology

When SAV2001|NEFT|250000|transfer arrives:
  AUDIT: account=SAV2001, payment=SAV2001|NEFT|250000|transfer
  POST-PROCESS: account=SAV2001, result=SAV2001|NEFT|250000|PROCESSED
The record passes through peek unchanged.

How do all stateless operations compare?

This table summarizes every stateless operation, what it does, whether it changes the key, and whether it triggers repartitioning.

OperationWhat it doesChanges key?Triggers repartition?Banking example
filterKeeps records matching predicateNoNoKeep payments > 100000
filterNotKeeps records NOT matching predicateNoNoRemove blocked accounts
mapTransforms key and valueYesYesRekey by branch code
mapValuesTransforms value onlyNoNoAdd processing fee
flatMapOne record to zero or many, new keysYesYesSplit bulk payment
flatMapValuesOne record to zero or many, same keyNoNoExpand line items
selectKeyChanges key onlyYesYesRekey by currency
branch / splitRoute to named sub-streamsNoNoRoute by amount tier
mergeCombine streams into oneNoNoMerge NEFT + IMPS + UPI
peekSide effect, no modificationNoNoAudit logging
foreachTerminal side effectNoNoSend SMS notification
JAVA
package com.bank.streams;

public class StatelessComparisonSummary {
    public static void main(String[] args) {
        System.out.println("Operations that trigger repartitioning:");
        System.out.println("  map, flatMap, selectKey");
        System.out.println("  Reason: they change the record key");
        System.out.println();
        System.out.println("Operations that do NOT trigger repartitioning:");
        System.out.println("  filter, filterNot, mapValues, flatMapValues,");
        System.out.println("  branch, merge, peek, foreach");
        System.out.println("  Reason: they keep the original key");
        System.out.println();
        System.out.println("Rule: prefer mapValues over map, flatMapValues over flatMap");
        System.out.println("when you do not need to change the key.");
    }
}
TEXT
Expected output:
Operations that trigger repartitioning:
  map, flatMap, selectKey
  Reason: they change the record key

Operations that do NOT trigger repartitioning:
  filter, filterNot, mapValues, flatMapValues,
  branch, merge, peek, foreach
  Reason: they keep the original key

Rule: prefer mapValues over map, flatMapValues over flatMap
when you do not need to change the key.

What is a complete banking project using stateless operations?

This project is a Payment Routing Service for National Bank of India. It reads payments from a single topic and applies a chain of stateless operations: peek for audit, filter for fraud, mapValues for fee calculation, flatMap for multi-beneficiary splitting, and branch for tier-based routing.

Project structure

DIAGRAM
payment-routing-service/
├── pom.xml
└── src/
    └── main/
        └── java/
            └── com/
                └── bank/
                    └── streams/
                        ├── PaymentRoutingApp.java
                        ├── PaymentRoutingTopology.java
                        ├── Payment.java
                        └── PaymentRoutingTopologyTest.java

pom.xml

XML
<?xml version="1.0" encoding="UTF-8"?syn-kw">>
syn-kw"><project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
         http://maven.apache.org/xsd/maven-4.0.0.xsd"syn-kw">>
    syn-kw"><modelVersionsyn-kw">>4.0.0syn-kw"></modelVersionsyn-kw">>

    syn-kw"><groupIdsyn-kw">>com.banksyn-kw"></groupIdsyn-kw">>
    syn-kw"><artifactIdsyn-kw">>payment-routing-servicesyn-kw"></artifactIdsyn-kw">>
    syn-kw"><versionsyn-kw">>1.0.0syn-kw"></versionsyn-kw">>
    syn-kw"><packagingsyn-kw">>jarsyn-kw"></packagingsyn-kw">>

    syn-kw"><propertiessyn-kw">>
        syn-kw"><java.versionsyn-kw">>25syn-kw"></java.versionsyn-kw">>
        syn-kw"><kafka.versionsyn-kw">>4.2.0syn-kw"></kafka.versionsyn-kw">>
        syn-kw"><maven.compiler.sourcesyn-kw">>25syn-kw"></maven.compiler.sourcesyn-kw">>
        syn-kw"><maven.compiler.targetsyn-kw">>25syn-kw"></maven.compiler.targetsyn-kw">>
    syn-kw"></propertiessyn-kw">>

    syn-kw"><dependenciessyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.apache.kafkasyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>kafka-streamssyn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>${kafka.version}syn-kw"></versionsyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.apache.kafkasyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>kafka-streams-test-utilssyn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>${kafka.version}syn-kw"></versionsyn-kw">>
            syn-kw"><scopesyn-kw">>testsyn-kw"></scopesyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.slf4jsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>slf4j-simplesyn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>2.0.12syn-kw"></versionsyn-kw">>
        syn-kw"></dependencysyn-kw">>
    syn-kw"></dependenciessyn-kw">>
syn-kw"></projectsyn-kw">>

Payment.java

JAVA
package com.bank.streams;

import java.math.BigDecimal;

public class Payment {
    private String accountId;
    private String type;
    private BigDecimal amount;
    private String description;

    public Payment() {}

    public Payment(String accountId, String type,
                   BigDecimal amount, String description) {
        this.accountId = accountId;
        this.type = type;
        this.amount = amount;
        this.description = description;
    }

    public String getAccountId() { return accountId; }
    public String getType() { return type; }
    public BigDecimal getAmount() { return amount; }
    public String getDescription() { return description; }

    @Override
    public String toString() {
        return accountId + "|" + type + "|" + amount + "|" + description;
    }

    public static Payment fromString(String value) {
        String[] parts = value.split("\\|");
        return new Payment(
            parts[0], parts[1], new BigDecimal(parts[2]), parts[3]);
    }
}

PaymentRoutingTopology.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Branched;
import org.apache.kafka.streams.Topology;

import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Map;

public class PaymentRoutingTopology {

    private static final BigDecimal FRAUD_THRESHOLD = new BigDecimal("1000000");
    private static final BigDecimal PREMIUM_THRESHOLD = new BigDecimal("100000");
    private static final BigDecimal FEE_RATE = new BigDecimal("0.005");

    public static Topology build() {
        StreamsBuilder builder = new StreamsBuilder();
        KStream<String, String> payments = builder.stream("incoming-payments");

        // Step 1: peek for audit logging
        KStream<String, String> audited = payments
            .peek((key, value) ->
                System.out.println("AUDIT-IN: " + key + " -> " + value));

        // Step 2: filter out suspected fraud (amount >= 10 lakh)
        KStream<String, String> legitimate = audited
            .filter((key, value) -> {
                Payment p = Payment.fromString(value);
                boolean isFraud = p.getAmount().compareTo(FRAUD_THRESHOLD) >= 0;
                if (isFraud) {
                    System.out.println("FRAUD-BLOCK: " + key);
                }
                return !isFraud;
            });

        // Step 3: mapValues to add processing fee
        KStream<String, String> withFee = legitimate.mapValues(value -> {
            Payment p = Payment.fromString(value);
            BigDecimal fee = p.getAmount().multiply(FEE_RATE)
                .setScale(2, RoundingMode.HALF_UP);
            return value + "|FEE:" + fee;
        });

        // Step 4: branch by amount tier
        Map<String, KStream<String, String>> tiers = withFee
            .split()
            .branch((key, value) -> {
                BigDecimal amount = new BigDecimal(value.split("\\|")[2]);
                return amount.compareTo(PREMIUM_THRESHOLD) > 0;
            }, Branched.as("premium"))
            .defaultBranch(Branched.as("standard"));

        tiers.get("premium").to("premium-payments");
        tiers.get("standard").to("standard-payments");

        return builder.build();
    }
}

PaymentRoutingApp.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.common.serialization.Serdes;

import java.util.Properties;
import java.util.concurrent.CountDownLatch;

public class PaymentRoutingApp {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG,
                  "payment-routing-service");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());

        KafkaStreams streams = new KafkaStreams(
            PaymentRoutingTopology.build(), props);
        CountDownLatch latch = new CountDownLatch(1);

        Runtime.getRuntime().addShutdownHook(new Thread(() -> {
            streams.close();
            latch.countDown();
        }));

        try {
            streams.start();
            System.out.println("Payment routing service started");
            System.out.println("  Input:  incoming-payments");
            System.out.println("  Output: premium-payments, standard-payments");
            System.out.println("  Fraud:  blocked if >= 1000000 INR");
            latch.await();
        } catch (Exception e) {
            System.exit(1);
        }
    }
}
TEXT
Expected output (on startup):
Payment routing service started
  Input:  incoming-payments
  Output: premium-payments, standard-payments
  Fraud:  blocked if >= 1000000 INR

PaymentRoutingTopologyTest.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.TopologyTestDriver;
import org.apache.kafka.streams.TestInputTopic;
import org.apache.kafka.streams.TestOutputTopic;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.serialization.StringSerializer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.util.Properties;

public class PaymentRoutingTopologyTest {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());

        try (TopologyTestDriver driver =
                 new TopologyTestDriver(PaymentRoutingTopology.build(), props)) {

            TestInputTopic<String, String> input = driver.createInputTopic(
                "incoming-payments", new StringSerializer(), new StringSerializer());
            TestOutputTopic<String, String> premium = driver.createOutputTopic(
                "premium-payments", new StringDeserializer(), new StringDeserializer());
            TestOutputTopic<String, String> standard = driver.createOutputTopic(
                "standard-payments", new StringDeserializer(), new StringDeserializer());

            // Premium payment (250000 > 100000)
            input.pipeInput("SAV2001", "SAV2001|NEFT|250000|transfer to Priya");
            // Standard payment (50000 <= 100000)
            input.pipeInput("SAV2002", "SAV2002|IMPS|50000|bill payment");
            // Fraud (1500000 >= 1000000, blocked)
            input.pipeInput("CUR3001", "CUR3001|RTGS|1500000|suspicious transfer");

            System.out.println("=== Premium payments ===");
            System.out.println("Count: " + premium.getQueueSize());
            System.out.println("Record: " + premium.readValue());

            System.out.println();
            System.out.println("=== Standard payments ===");
            System.out.println("Count: " + standard.getQueueSize());
            System.out.println("Record: " + standard.readValue());

            System.out.println();
            System.out.println("Fraud payment was blocked by filter. Not in any output.");
        }
    }
}
TEXT
Expected output:
AUDIT-IN: SAV2001 -> SAV2001|NEFT|250000|transfer to Priya
AUDIT-IN: SAV2002 -> SAV2002|IMPS|50000|bill payment
AUDIT-IN: CUR3001 -> CUR3001|RTGS|1500000|suspicious transfer
FRAUD-BLOCK: CUR3001

=== Premium payments ===
Count: 1
Record: SAV2001|NEFT|250000|transfer to Priya|FEE:1250.00

=== Standard payments ===
Count: 1
Record: SAV2002|IMPS|50000|bill payment|FEE:250.00

Fraud payment was blocked by filter. Not in any output.

How to run the project

BASH
# Step 1: Start Kafka
docker-compose up -d kafka zookeeper

# Step 2: Create topics
kafka-topics.sh --create --topic incoming-payments --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --create --topic premium-payments --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --create --topic standard-payments --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092

# Step 3: Build and run
cd payment-routing-service
mvn clean package
java -cp target/payment-routing-service-1.0.0.jar com.bank.streams.PaymentRoutingApp

# Step 4: Produce test messages
kafka-console-producer.sh --topic incoming-payments --property "parse.key=true" --property "key.separator=:" --bootstrap-server localhost:9092
# SAV2001:SAV2001|NEFT|250000|transfer to Priya
# SAV2002:SAV2002|IMPS|50000|bill payment
# CUR3001:CUR3001|RTGS|1500000|suspicious transfer

What are common mistakes with stateless operations?

The most common mistake is using map() when mapValues() is sufficient. This triggers an unnecessary repartition.

Wrong: using map when only the value changes

JAVA
package com.bank.streams;

public class WrongMapForValueOnly {
    public static void main(String[] args) {
        System.out.println("WRONG: Using map() to change only the value");
        System.out.println();
        System.out.println("Code:");
        System.out.println("  payments.map((key, value) ->");
        System.out.println("      KeyValue.pair(key, value.toUpperCase()));");
        System.out.println();
        System.out.println("Problem: map() marks the stream for repartitioning");
        System.out.println("even though the key did not change.");
        System.out.println("An internal repartition topic is created unnecessarily.");
        System.out.println("This doubles network traffic for downstream key-based operations.");
    }
}
TEXT
Expected output:
WRONG: Using map() to change only the value

Code:
  payments.map((key, value) ->
      KeyValue.pair(key, value.toUpperCase()));

Problem: map() marks the stream for repartitioning
even though the key did not change.
An internal repartition topic is created unnecessarily.
This doubles network traffic for downstream key-based operations.

Correct: using mapValues when only the value changes

JAVA
package com.bank.streams;

public class CorrectMapValuesForValueOnly {
    public static void main(String[] args) {
        System.out.println("CORRECT: Using mapValues() to change only the value");
        System.out.println();
        System.out.println("Code:");
        System.out.println("  payments.mapValues(value -> value.toUpperCase());");
        System.out.println();
        System.out.println("mapValues() does not change the key.");
        System.out.println("No repartition topic is created.");
        System.out.println("Same result, less network overhead.");
    }
}
TEXT
Expected output:
CORRECT: Using mapValues() to change only the value

Code:
  payments.mapValues(value -> value.toUpperCase());

mapValues() does not change the key.
No repartition topic is created.
Same result, less network overhead.

Wrong: using peek to modify the record

JAVA
package com.bank.streams;

public class WrongPeekModification {
    public static void main(String[] args) {
        System.out.println("WRONG: Modifying state inside peek()");
        System.out.println();
        System.out.println("Code:");
        System.out.println("  List<String> auditList = new ArrayList<>();");
        System.out.println("  payments.peek((key, value) -> auditList.add(value));");
        System.out.println();
        System.out.println("Problem: peek runs on stream threads.");
        System.out.println("Mutating shared state (auditList) is not thread-safe.");
        System.out.println("This causes ConcurrentModificationException or data corruption.");
        System.out.println();
        System.out.println("CORRECT: Use peek only for stateless side effects like logging.");
        System.out.println("  payments.peek((key, value) ->");
        System.out.println("      logger.info(\"Payment: {} -> {}\", key, value));");
    }
}
TEXT
Expected output:
WRONG: Modifying state inside peek()

Code:
  List<String> auditList = new ArrayList<>();
  payments.peek((key, value) -> auditList.add(value));

Problem: peek runs on stream threads.
Mutating shared state (auditList) is not thread-safe.
This causes ConcurrentModificationException or data corruption.

CORRECT: Use peek only for stateless side effects like logging.
  payments.peek((key, value) ->
      logger.info("Payment: {} -> {}", key, value));

What should a senior or principal engineer know?

At the senior level, you should understand the performance implications of stateless operations and how to build efficient topologies.

  • Repartition cost: Every map(), flatMap(), or selectKey() that changes the key creates an internal repartition topic. This adds network hops, storage, and latency. In a topology with multiple key changes, each one creates a separate repartition topic.
  • Operation ordering: Place filter() early in the topology to reduce the number of records processed by downstream operations. If 80% of records are filtered out, the fee calculation and routing steps process only 20% of the traffic.
  • Null key handling: If selectKey() returns null, the record is dropped silently during any downstream groupByKey() or join. Always validate keys before re-keying.
  • flatMap cardinality: flatMap() can produce millions of output records from one input record. In banking, a bulk payment with 10,000 beneficiaries produces 10,000 output records. Monitor output rate and backpressure.
  • Topology testing: Use TopologyTestDriver to test each stateless operation in isolation. Write separate tests for filter logic, fee calculation, and routing predicates.
  • Idempotency: Stateless operations are naturally idempotent because they have no state. If Kafka Streams reprocesses a record after a failure, the operation produces the same result.
JAVA
package com.bank.streams;

public class SeniorStatelessKnowledge {
    public static void main(String[] args) {
        System.out.println("Senior checklist for stateless operations:");
        System.out.println("  1. Place filter early to reduce downstream volume");
        System.out.println("  2. Prefer mapValues over map to avoid repartition");
        System.out.println("  3. Prefer flatMapValues over flatMap for same reason");
        System.out.println("  4. Monitor flatMap output cardinality in production");
        System.out.println("  5. Validate keys in selectKey to avoid null drops");
        System.out.println("  6. Test each operation independently with TopologyTestDriver");
        System.out.println("  7. Stateless ops are idempotent: safe to reprocess");
    }
}
TEXT
Expected output:
Senior checklist for stateless operations:
  1. Place filter early to reduce downstream volume
  2. Prefer mapValues over map to avoid repartition
  3. Prefer flatMapValues over flatMap for same reason
  4. Monitor flatMap output cardinality in production
  5. Validate keys in selectKey to avoid null drops
  6. Test each operation independently with TopologyTestDriver
  7. Stateless ops are idempotent: safe to reprocess

What are common interview follow-up questions?

  • What is the difference between foreach and peek? (foreach is a terminal operation that ends the stream; peek is an intermediate operation that passes the record through. You cannot chain more operations after foreach.)
  • Does filter create a new topic? (No. filter is an in-memory operation within the topology. No topic is created. The filtered records continue in the same sub-topology.)
  • Can you call map on a KTable? (Yes. KTable has mapValues() but not map(). If you need to change the key, convert to KStream first with toStream(), then call map().)
  • What happens if flatMap returns an empty list? (The input record is dropped. Zero output records are produced. This is a valid way to conditionally suppress records.)
  • Does branch guarantee ordering? (Each branch preserves the ordering of the original stream for records that match that branch. Records in different branches are independent.)
  • Can you merge a KStream with a KTable? (No. merge() works only between KStream instances. To combine a KStream with a KTable, use a join operation instead.)

Visual Overview

DIAGRAM
 Payment Routing Topology (Stateless Operations)
 ==================================================

 ┌───────────────────────────┐
 │ incoming-payments          │
 └────────────┬──────────────┘
              │
              ▼
 ┌───────────────────────────┐
 │ peek (audit log)          │  ← logs each payment
 └────────────┬──────────────┘
              │
              ▼
 ┌───────────────────────────┐
 │ filter (fraud check)      │  ← blocks >= 10 lakh
 └────────────┬──────────────┘
              │ legitimate only
              ▼
 ┌───────────────────────────┐
 │ mapValues (add fee 0.5%)  │  ← no repartition
 └────────────┬──────────────┘
              │
              ▼
 ┌───────────────────────────┐
 │ branch (amount tier)      │
 └──────┬──────────┬─────────┘
        │          │
        ▼          ▼
 ┌──────────┐ ┌──────────────┐
 │ premium  │ │ standard     │
 │ payments │ │ payments     │
 │ > 1 lakh │ │ <= 1 lakh    │
 └──────────┘ └──────────────┘


 Repartition Decision Tree
 ===========================

 Need to change the key?
 ├── No  → use mapValues, flatMapValues, filter, peek
 │         (no repartition, no extra topic)
 └── Yes → use map, flatMap, selectKey
           (triggers repartition topic)
           └── Is repartition worth it?
               ├── Yes: downstream join/aggregation needs new key
               └── No:  reconsider your design

Key Takeaway

Stateless operations are the building blocks of every Kafka Streams topology. They process each record independently, with no local state store and no changelog topic. This makes them lightweight, fast, and naturally idempotent.

The most commonly used stateless operations are filter, mapValues, and peek. filter removes records that do not meet a condition, like blocking suspected fraud payments above ten lakh rupees. mapValues transforms the value without changing the key, like adding a 0.5% processing fee to each payment. peek logs or inspects each record without modifying it, which is useful for audit trails at National Bank of India.

The operations that change the record key, including map, flatMap, and selectKey, trigger repartitioning. This means Kafka Streams creates an internal topic, writes the rekeyed records to it, and reads them back. This adds network overhead and latency. The rule is to prefer mapValues over map and flatMapValues over flatMap whenever you do not need to change the key.

flatMap is the most powerful stateless operation. It takes one record and produces zero or more output records, each with its own key and value. In banking, this is how you split a bulk payment instruction with three beneficiaries into three individual transfer records. branch and merge handle the routing side: branch splits a stream into named sub-streams by predicate (premium vs standard tiers), and merge combines multiple channel streams (NEFT, IMPS, UPI) into one.

At the senior level, the key insight is topology ordering. Place filter early to reduce downstream volume. Avoid unnecessary repartitioning by using value-only variants. Monitor flatMap output cardinality to avoid unexpected volume spikes. Test each operation independently with TopologyTestDriver.

Remember one sentence: stateless operations transform, filter, split, merge, and observe records one at a time with no memory of what came before, and the critical design choice is whether you change the key, because that triggers repartitioning.

---

Share
Use ← → arrow keys to navigate