Back to Kafka Streams
Q22Question 22 of 27

How do you handle errors and deserialization exceptions in Kafka St

Answer: Kafka Streams provides three handler interfaces for errors: DeserializationExceptionHandler for bad input records, ProductionExceptionHandler for write failures, and StreamsUncaughtExceptionHandler for unexpected processing exceptions.

Key Concepts
  • DeserializationExceptionHandler handles records that cannot be deserialized from the input topic.
  • ProductionExceptionHandler handles errors when writing records to an output or changelog topic.
  • StreamsUncaughtExceptionHandler handles unexpected exceptions thrown by the processing logic.
  • LogAndContinueExceptionHandler is a built-in handler that logs the bad record and skips it.
  • LogAndFailExceptionHandler is the default handler that logs and shuts down the stream thread.
  • A dead letter topic pattern forwards bad records to a separate topic for manual review instead of losing them.
  • In production banking systems, combine all three handlers with monitoring and alerting for complete error coverage.

Definition

Kafka Streams provides three handler interfaces for errors: DeserializationExceptionHandler for bad input records, ProductionExceptionHandler for write failures, and StreamsUncaughtExceptionHandler for unexpected processing exceptions.
  • DeserializationExceptionHandler is called when a record from an input topic cannot be deserialized; you choose to skip the record (CONTINUE) or shut down the thread (FAIL).
  • ProductionExceptionHandler is called when Kafka Streams fails to write a record to an output or changelog topic; you choose to continue or fail.
  • StreamsUncaughtExceptionHandler is called when any unhandled exception escapes user processing code; you choose to replace the thread, shut down the client, or shut down the application.
  • LogAndContinueExceptionHandler is a built-in DeserializationExceptionHandler that logs the error and skips the bad record.
  • Dead letter topic is a pattern where bad records are forwarded to a separate Kafka topic for later inspection and reprocessing.

Think of National Bank of India's cheque processing system. A DeserializationExceptionHandler is like the machine that scans cheques: when it encounters a torn or unreadable cheque, it can either set it aside in a reject pile (CONTINUE) or stop the entire machine (FAIL). A ProductionExceptionHandler is like the printer that stamps processed cheques: if the printer jams, the system decides whether to skip that cheque or halt. The dead letter topic is the reject pile where unreadable cheques are stored for manual review by Rajesh the developer.

Understanding the Concepts

What errors can occur in a Kafka Streams application?

Kafka Streams applications face three categories of errors:

  1. Deserialization errors: the input record cannot be converted from bytes to the expected type. This happens when a producer writes a String but the consumer expects JSON, or when the schema changes.
  2. Production errors: the application cannot write output records to Kafka. This happens during broker outages, topic authorization failures, or record-too-large errors.
  3. Processing errors: the user's processing logic throws an unexpected exception, such as NullPointerException, NumberFormatException, or a timeout calling an external service.
JAVA
package com.bank.streams;

public class ErrorCategories {
    public static void main(String[] args) {
        System.out.println("Error categories in Kafka Streams:");
        System.out.println();
        System.out.println("1. Deserialization error:");
        System.out.println("   Cause: bad data on input topic");
        System.out.println("   Handler: DeserializationExceptionHandler");
        System.out.println("   Config: default.deserialization.exception.handler");
        System.out.println();
        System.out.println("2. Production error:");
        System.out.println("   Cause: cannot write to output/changelog topic");
        System.out.println("   Handler: ProductionExceptionHandler");
        System.out.println("   Config: default.production.exception.handler");
        System.out.println();
        System.out.println("3. Processing error:");
        System.out.println("   Cause: unhandled exception in user code");
        System.out.println("   Handler: StreamsUncaughtExceptionHandler");
        System.out.println("   Config: streams.setUncaughtExceptionHandler()");
    }
}
TEXT
Expected output:
Error categories in Kafka Streams:

1. Deserialization error:
   Cause: bad data on input topic
   Handler: DeserializationExceptionHandler
   Config: default.deserialization.exception.handler

2. Production error:
   Cause: cannot write to output/changelog topic
   Handler: ProductionExceptionHandler
   Config: default.production.exception.handler

3. Processing error:
   Cause: unhandled exception in user code
   Handler: StreamsUncaughtExceptionHandler
   Config: streams.setUncaughtExceptionHandler()

How does DeserializationExceptionHandler work?

DeserializationExceptionHandler is called when Kafka Streams cannot deserialize a record from an input topic. You implement the handler and return either DeserializationHandlerResponse.CONTINUE to skip the bad record or DeserializationHandlerResponse.FAIL to shut down the stream thread.

  • Kafka Streams provides two built-in implementations:
  • LogAndContinueExceptionHandler: logs the error and skips the record.
  • LogAndFailExceptionHandler: logs the error and shuts down (this is the default).
JAVA
package com.bank.streams;

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.streams.errors.DeserializationExceptionHandler;
import org.apache.kafka.streams.processor.ProcessorContext;

import java.util.Map;

public class BankDeserializationHandler implements DeserializationExceptionHandler {

    @Override
    public DeserializationHandlerResponse handle(ProcessorContext context,
                                                  ConsumerRecord<byte[], byte[]> record,
                                                  Exception exception) {
        System.out.println("Deserialization error on topic: " + record.topic()
            + ", partition: " + record.partition()
            + ", offset: " + record.offset());
        System.out.println("  Exception: " + exception.getMessage());
        System.out.println("  Action: CONTINUE (skip bad record)");

        // In production: send to dead letter topic and increment metric
        return DeserializationHandlerResponse.CONTINUE;
    }

    @Override
    public void configure(Map<String, ?> configs) {
        // No configuration needed
    }

    public static void main(String[] args) {
        System.out.println("Custom DeserializationExceptionHandler:");
        System.out.println("  Logs the error details");
        System.out.println("  Returns CONTINUE to skip bad records");
        System.out.println("  In production: forward to dead letter topic");
        System.out.println();
        System.out.println("Configure in properties:");
        System.out.println("  props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,");
        System.out.println("    BankDeserializationHandler.class);");
    }
}
TEXT
Expected output:
Custom DeserializationExceptionHandler:
  Logs the error details
  Returns CONTINUE to skip bad records
  In production: forward to dead letter topic

Configure in properties:
  props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
    BankDeserializationHandler.class);

How does ProductionExceptionHandler work?

ProductionExceptionHandler is called when Kafka Streams fails to write a record to an output topic or a changelog topic. You return either ProductionExceptionHandlerResponse.CONTINUE to skip the failed write or ProductionExceptionHandlerResponse.FAIL to shut down.

JAVA
package com.bank.streams;

import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.streams.errors.ProductionExceptionHandler;

import java.util.Map;

public class BankProductionHandler implements ProductionExceptionHandler {

    @Override
    public ProductionExceptionHandlerResponse handle(ProducerRecord<byte[], byte[]> record,
                                                      Exception exception) {
        System.out.println("Production error on topic: " + record.topic());
        System.out.println("  Exception: " + exception.getMessage());

        if (exception instanceof org.apache.kafka.common.errors.RecordTooLargeException) {
            System.out.println("  Action: CONTINUE (skip oversized record)");
            return ProductionExceptionHandlerResponse.CONTINUE;
        }

        System.out.println("  Action: FAIL (unknown error, shut down)");
        return ProductionExceptionHandlerResponse.FAIL;
    }

    @Override
    public void configure(Map<String, ?> configs) {
        // No configuration needed
    }

    public static void main(String[] args) {
        System.out.println("Custom ProductionExceptionHandler:");
        System.out.println("  RecordTooLargeException -> CONTINUE (skip)");
        System.out.println("  Other exceptions -> FAIL (shut down)");
        System.out.println();
        System.out.println("Configure in properties:");
        System.out.println("  props.put(StreamsConfig.DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG,");
        System.out.println("    BankProductionHandler.class);");
    }
}
TEXT
Expected output:
Custom ProductionExceptionHandler:
  RecordTooLargeException -> CONTINUE (skip)
  Other exceptions -> FAIL (shut down)

Configure in properties:
  props.put(StreamsConfig.DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG,
    BankProductionHandler.class);

How does StreamsUncaughtExceptionHandler work?

StreamsUncaughtExceptionHandler catches any unhandled exception that escapes user processing code. You set it on the KafkaStreams instance before calling start(). The handler returns one of three responses:

  • REPLACE_THREAD: replace the failed stream thread with a new one.
  • SHUTDOWN_CLIENT: shut down this KafkaStreams instance but leave other instances in the group running.
  • SHUTDOWN_APPLICATION: shut down all instances in the application (by encoding the error in the consumer group protocol).
JAVA
package com.bank.streams;

import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.errors.StreamsUncaughtExceptionHandler;
import org.apache.kafka.common.serialization.Serdes;

import java.util.Properties;

public class UncaughtExceptionHandlerExample {
    public static void main(String[] args) {
        System.out.println("StreamsUncaughtExceptionHandler responses:");
        System.out.println("  REPLACE_THREAD:      restart the failed thread");
        System.out.println("  SHUTDOWN_CLIENT:      stop this instance only");
        System.out.println("  SHUTDOWN_APPLICATION: stop all instances");
        System.out.println();
        System.out.println("Example configuration:");
        System.out.println("  streams.setUncaughtExceptionHandler(exception -> {");
        System.out.println("    if (exception instanceof TransientException) {");
        System.out.println("      return REPLACE_THREAD;");
        System.out.println("    }");
        System.out.println("    return SHUTDOWN_CLIENT;");
        System.out.println("  });");
        System.out.println();
        System.out.println("In banking: use REPLACE_THREAD for transient errors");
        System.out.println("(network timeout), SHUTDOWN_CLIENT for data corruption.");
    }
}
TEXT
Expected output:
StreamsUncaughtExceptionHandler responses:
  REPLACE_THREAD:      restart the failed thread
  SHUTDOWN_CLIENT:      stop this instance only
  SHUTDOWN_APPLICATION: stop all instances

Example configuration:
  streams.setUncaughtExceptionHandler(exception -> {
    if (exception instanceof TransientException) {
      return REPLACE_THREAD;
    }
    return SHUTDOWN_CLIENT;
  });

In banking: use REPLACE_THREAD for transient errors
(network timeout), SHUTDOWN_CLIENT for data corruption.

How do the three error handlers compare?

FeatureDeserializationExceptionHandlerProductionExceptionHandlerStreamsUncaughtExceptionHandler
When calledCannot deserialize input recordCannot write to output/changelogUnhandled exception in user code
ConfigurationStreamsConfig propertyStreamsConfig propertystreams.setUncaughtExceptionHandler()
OptionsCONTINUE, FAILCONTINUE, FAILREPLACE_THREAD, SHUTDOWN_CLIENT, SHUTDOWN_APPLICATION
Default behaviorLogAndFail (shuts down)LogAndFail (shuts down)Shuts down client
Access to recordYes (ConsumerRecord)Yes (ProducerRecord)No (only exception)
Can send to DLTYes (in handler)Yes (in handler)Difficult (no record context)
JAVA
package com.bank.streams;

public class ErrorHandlerComparison {
    public static void main(String[] args) {
        System.out.println("Error handler comparison:");
        System.out.println();
        System.out.println("DeserializationExceptionHandler:");
        System.out.println("  Trigger: bad input data");
        System.out.println("  Example: producer sends XML, deserializer expects JSON");
        System.out.println("  Best action: CONTINUE + send to dead letter topic");
        System.out.println();
        System.out.println("ProductionExceptionHandler:");
        System.out.println("  Trigger: cannot write output");
        System.out.println("  Example: record exceeds max.message.bytes");
        System.out.println("  Best action: CONTINUE for size errors, FAIL for auth errors");
        System.out.println();
        System.out.println("StreamsUncaughtExceptionHandler:");
        System.out.println("  Trigger: bug in processing code");
        System.out.println("  Example: NullPointerException in map function");
        System.out.println("  Best action: REPLACE_THREAD for transient, SHUTDOWN_CLIENT for permanent");
    }
}
TEXT
Expected output:
Error handler comparison:

DeserializationExceptionHandler:
  Trigger: bad input data
  Example: producer sends XML, deserializer expects JSON
  Best action: CONTINUE + send to dead letter topic

ProductionExceptionHandler:
  Trigger: cannot write output
  Example: record exceeds max.message.bytes
  Best action: CONTINUE for size errors, FAIL for auth errors

StreamsUncaughtExceptionHandler:
  Trigger: bug in processing code
  Example: NullPointerException in map function
  Best action: REPLACE_THREAD for transient, SHUTDOWN_CLIENT for permanent

What is the dead letter topic pattern?

A dead letter topic (DLT) is a separate Kafka topic where you forward records that cannot be processed. Instead of dropping bad records, you write them to the DLT with metadata about the error. A separate consumer can read the DLT for manual review or reprocessing.

This is critical in banking: you cannot silently lose transactions. Every failed record must be traceable.

JAVA
package com.bank.streams;

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;

public class DeadLetterTopicPattern {

    static KafkaProducer<String, String> createDltProducer() {
        Properties props = new Properties();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        return new KafkaProducer<>(props);
    }

    static void sendToDeadLetter(KafkaProducer<String, String> producer,
                                  String originalTopic, String key,
                                  String value, String error) {
        String dltValue = "ERROR|topic=" + originalTopic
            + "|key=" + key
            + "|value=" + value
            + "|reason=" + error;
        producer.send(new ProducerRecord<>(originalTopic + "-dlt", key, dltValue));
    }

    public static void main(String[] args) {
        System.out.println("Dead letter topic pattern:");
        System.out.println("  Input topic:  bank-transactions");
        System.out.println("  DLT topic:    bank-transactions-dlt");
        System.out.println();
        System.out.println("  Record: SAV2001|INVALID_AMOUNT|abc");
        System.out.println("  Error: NumberFormatException on amount field");
        System.out.println();
        System.out.println("  DLT record:");
        System.out.println("    ERROR|topic=bank-transactions|key=SAV2001");
        System.out.println("    |value=SAV2001|INVALID_AMOUNT|abc");
        System.out.println("    |reason=NumberFormatException: For input string abc");
    }
}
TEXT
Expected output:
Dead letter topic pattern:
  Input topic:  bank-transactions
  DLT topic:    bank-transactions-dlt

  Record: SAV2001|INVALID_AMOUNT|abc
  Error: NumberFormatException on amount field

  DLT record:
    ERROR|topic=bank-transactions|key=SAV2001
    |value=SAV2001|INVALID_AMOUNT|abc
    |reason=NumberFormatException: For input string abc

What is a complete banking project for resilient transaction processing?

This project reads bank transactions, parses them, and routes valid transactions to an output topic. Invalid records that fail parsing are sent to a dead letter topic. The application uses all three error handlers.

Project structure

DIAGRAM
resilient-processor/
├── pom.xml
└── src/
    ├── main/
    │   └── java/
    │       └── com/
    │           └── bank/
    │               └── streams/
    │                   ├── ResilientProcessorApp.java
    │                   ├── ResilientProcessorTopology.java
    │                   ├── TransactionParser.java
    │                   └── BankDeserializationHandler.java
    └── test/
        └── java/
            └── com/
                └── bank/
                    └── streams/
                        └── ResilientProcessorTopologyTest.java

pom.xml

XML
<?xml version="1.0" encoding="UTF-8"?syn-kw">>
syn-kw"><project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
         http://maven.apache.org/xsd/maven-4.0.0.xsd"syn-kw">>
    syn-kw"><modelVersionsyn-kw">>4.0.0syn-kw"></modelVersionsyn-kw">>
    syn-kw"><groupIdsyn-kw">>com.banksyn-kw"></groupIdsyn-kw">>
    syn-kw"><artifactIdsyn-kw">>resilient-processorsyn-kw"></artifactIdsyn-kw">>
    syn-kw"><versionsyn-kw">>1.0.0syn-kw"></versionsyn-kw">>
    syn-kw"><packagingsyn-kw">>jarsyn-kw"></packagingsyn-kw">>
    syn-kw"><propertiessyn-kw">>
        syn-kw"><java.versionsyn-kw">>25syn-kw"></java.versionsyn-kw">>
        syn-kw"><kafka.versionsyn-kw">>4.2.0syn-kw"></kafka.versionsyn-kw">>
        syn-kw"><maven.compiler.sourcesyn-kw">>25syn-kw"></maven.compiler.sourcesyn-kw">>
        syn-kw"><maven.compiler.targetsyn-kw">>25syn-kw"></maven.compiler.targetsyn-kw">>
    syn-kw"></propertiessyn-kw">>
    syn-kw"><dependenciessyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.apache.kafkasyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>kafka-streamssyn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>${kafka.version}syn-kw"></versionsyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.apache.kafkasyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>kafka-streams-test-utilssyn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>${kafka.version}syn-kw"></versionsyn-kw">>
            syn-kw"><scopesyn-kw">>testsyn-kw"></scopesyn-kw">>
        syn-kw"></dependencysyn-kw">>
        syn-kw"><dependencysyn-kw">>
            syn-kw"><groupIdsyn-kw">>org.slf4jsyn-kw"></groupIdsyn-kw">>
            syn-kw"><artifactIdsyn-kw">>slf4j-simplesyn-kw"></artifactIdsyn-kw">>
            syn-kw"><versionsyn-kw">>2.0.12syn-kw"></versionsyn-kw">>
        syn-kw"></dependencysyn-kw">>
    syn-kw"></dependenciessyn-kw">>
syn-kw"></projectsyn-kw">>

TransactionParser.java

JAVA
package com.bank.streams;

import java.math.BigDecimal;

public class TransactionParser {

    public static String parseAndValidate(String key, String value) {
        try {
            String[] parts = value.split("\\|");
            if (parts.length < 4) {
                return "DLT|" + key + "|" + value + "|INVALID_FORMAT";
            }
            String accountId = parts[0];
            String type = parts[1];
            BigDecimal amount = new BigDecimal(parts[2]);
            String status = parts[3];

            if (amount.compareTo(BigDecimal.ZERO) <= 0) {
                return "DLT|" + key + "|" + value + "|NEGATIVE_AMOUNT";
            }

            return "VALID|" + accountId + "|" + type + "|" + amount + "|" + status;
        } catch (NumberFormatException e) {
            return "DLT|" + key + "|" + value + "|" + e.getMessage();
        }
    }

    public static void main(String[] args) {
        System.out.println("Valid:   " + parseAndValidate("SAV2001", "SAV2001|NEFT|50000|SUCCESS"));
        System.out.println("Invalid: " + parseAndValidate("SAV2002", "SAV2002|NEFT|abc|SUCCESS"));
        System.out.println("Short:   " + parseAndValidate("SAV2003", "SAV2003|NEFT"));
        System.out.println("Neg:     " + parseAndValidate("SAV2004", "SAV2004|NEFT|-500|SUCCESS"));
    }
}
TEXT
Expected output:
Valid:   VALID|SAV2001|NEFT|50000|SUCCESS
Invalid: DLT|SAV2002|SAV2002|NEFT|abc|SUCCESS|For input string: "abc"
Short:   DLT|SAV2003|SAV2003|NEFT|INVALID_FORMAT
Neg:     DLT|SAV2004|SAV2004|NEFT|-500|SUCCESS|NEGATIVE_AMOUNT

ResilientProcessorTopology.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.Topology;

public class ResilientProcessorTopology {

    public static Topology build() {
        StreamsBuilder builder = new StreamsBuilder();

        KStream<String, String> transactions = builder.stream("bank-transactions");

        // Parse and validate each record
        KStream<String, String> parsed = transactions.mapValues(
            (key, value) -> TransactionParser.parseAndValidate(key, value));

        // Branch: valid records go to output, invalid to DLT
        KStream<String, String>[] branches = parsed.branch(
            (key, value) -> value.startsWith("VALID|"),
            (key, value) -> value.startsWith("DLT|")
        );

        branches[0].to("processed-transactions");
        branches[1].to("bank-transactions-dlt");

        return builder.build();
    }
}

ResilientProcessorApp.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.errors.StreamsUncaughtExceptionHandler;
import org.apache.kafka.common.serialization.Serdes;

import java.util.Properties;
import java.util.concurrent.CountDownLatch;

public class ResilientProcessorApp {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "resilient-processor");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
                  "org.apache.kafka.streams.errors.LogAndContinueExceptionHandler");

        KafkaStreams streams = new KafkaStreams(
            ResilientProcessorTopology.build(), props);

        // Handle uncaught exceptions: replace thread for transient errors
        streams.setUncaughtExceptionHandler(exception -> {
            System.out.println("Uncaught exception: " + exception.getMessage());
            return StreamsUncaughtExceptionHandler
                .StreamThreadExceptionResponse.REPLACE_THREAD;
        });

        CountDownLatch latch = new CountDownLatch(1);
        Runtime.getRuntime().addShutdownHook(new Thread(() -> {
            streams.close();
            latch.countDown();
        }));

        try {
            streams.start();
            System.out.println("Resilient processor started");
            System.out.println("  Input:  bank-transactions");
            System.out.println("  Output: processed-transactions");
            System.out.println("  DLT:    bank-transactions-dlt");
            latch.await();
        } catch (Exception e) {
            System.exit(1);
        }
    }
}

ResilientProcessorTopologyTest.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.TopologyTestDriver;
import org.apache.kafka.streams.TestInputTopic;
import org.apache.kafka.streams.TestOutputTopic;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.serialization.StringSerializer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.util.Properties;

public class ResilientProcessorTopologyTest {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());

        try (TopologyTestDriver driver =
                 new TopologyTestDriver(ResilientProcessorTopology.build(), props)) {

            TestInputTopic<String, String> input = driver.createInputTopic(
                "bank-transactions", new StringSerializer(), new StringSerializer());
            TestOutputTopic<String, String> validOutput = driver.createOutputTopic(
                "processed-transactions",
                new StringDeserializer(), new StringDeserializer());
            TestOutputTopic<String, String> dltOutput = driver.createOutputTopic(
                "bank-transactions-dlt",
                new StringDeserializer(), new StringDeserializer());

            // Valid record
            input.pipeInput("SAV2001", "SAV2001|NEFT|50000|SUCCESS");
            // Invalid: bad amount
            input.pipeInput("SAV2002", "SAV2002|NEFT|abc|SUCCESS");
            // Invalid: too few fields
            input.pipeInput("SAV2003", "SAV2003|NEFT");
            // Valid record
            input.pipeInput("CUR3001", "CUR3001|RTGS|75000|SUCCESS");

            System.out.println("Valid output:");
            while (!validOutput.isEmpty()) {
                System.out.println("  " + validOutput.readValue());
            }
            System.out.println();
            System.out.println("Dead letter topic:");
            while (!dltOutput.isEmpty()) {
                System.out.println("  " + dltOutput.readValue());
            }
        }
    }
}
TEXT
Expected output:
Valid output:
  VALID|SAV2001|NEFT|50000|SUCCESS
  VALID|CUR3001|RTGS|75000|SUCCESS

Dead letter topic:
  DLT|SAV2002|SAV2002|NEFT|abc|SUCCESS|For input string: "abc"
  DLT|SAV2003|SAV2003|NEFT|INVALID_FORMAT

How to run the project

BASH
# Step 1: Start Kafka
docker-compose up -d kafka zookeeper

# Step 2: Create topics
kafka-topics.sh --create --topic bank-transactions --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --create --topic processed-transactions --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --create --topic bank-transactions-dlt --partitions 3 --replication-factor 1 --bootstrap-server localhost:9092

# Step 3: Build
cd resilient-processor
mvn clean package

# Step 4: Run
java -cp target/resilient-processor-1.0.0.jar com.bank.streams.ResilientProcessorApp

# Step 5: Produce test messages
kafka-console-producer.sh --topic bank-transactions --property "parse.key=true" --property "key.separator=:" --bootstrap-server localhost:9092
# SAV2001:SAV2001|NEFT|50000|SUCCESS
# SAV2002:SAV2002|NEFT|abc|SUCCESS

# Step 6: Check valid output
kafka-console-consumer.sh --topic processed-transactions --from-beginning --bootstrap-server localhost:9092

# Step 7: Check dead letter topic
kafka-console-consumer.sh --topic bank-transactions-dlt --from-beginning --bootstrap-server localhost:9092

What are common mistakes with error handling in Kafka Streams?

The most common mistake is using the default handler (LogAndFail) in production, which shuts down the entire application on a single bad record.

Wrong: default handler shuts down on one bad record

JAVA
package com.bank.streams;

public class WrongDefaultHandler {
    public static void main(String[] args) {
        System.out.println("WRONG: Using default LogAndFailExceptionHandler in production");
        System.out.println();
        System.out.println("Scenario:");
        System.out.println("  1. Application processes 10,000 transactions per second");
        System.out.println("  2. One producer sends a malformed record");
        System.out.println("  3. Default handler: LogAndFailExceptionHandler");
        System.out.println("  4. Result: ENTIRE APPLICATION SHUTS DOWN");
        System.out.println();
        System.out.println("Impact: All 10,000 transactions/second stop processing.");
        System.out.println("One bad record from one producer halts the entire bank.");
    }
}
TEXT
Expected output:
WRONG: Using default LogAndFailExceptionHandler in production

Scenario:
  1. Application processes 10,000 transactions per second
  2. One producer sends a malformed record
  3. Default handler: LogAndFailExceptionHandler
  4. Result: ENTIRE APPLICATION SHUTS DOWN

Impact: All 10,000 transactions/second stop processing.
One bad record from one producer halts the entire bank.

Correct: custom handler with dead letter topic and alerting

JAVA
package com.bank.streams;

public class CorrectErrorHandling {
    public static void main(String[] args) {
        System.out.println("CORRECT: Custom handler with DLT and alerting");
        System.out.println();
        System.out.println("Scenario:");
        System.out.println("  1. Application processes 10,000 transactions per second");
        System.out.println("  2. One producer sends a malformed record");
        System.out.println("  3. Custom handler: log, send to DLT, increment metric, CONTINUE");
        System.out.println("  4. Result: bad record in DLT, processing continues");
        System.out.println();
        System.out.println("Monitoring:");
        System.out.println("  Alert on DLT consumer lag > 0 (new bad records)");
        System.out.println("  Dashboard shows bad record count by producer");
        System.out.println("  Manual review team processes DLT records");
    }
}
TEXT
Expected output:
CORRECT: Custom handler with DLT and alerting

Scenario:
  1. Application processes 10,000 transactions per second
  2. One producer sends a malformed record
  3. Custom handler: log, send to DLT, increment metric, CONTINUE
  4. Result: bad record in DLT, processing continues

Monitoring:
  Alert on DLT consumer lag > 0 (new bad records)
  Dashboard shows bad record count by producer
  Manual review team processes DLT records

What should a senior or principal engineer know?

At the senior level, error handling strategy is a production readiness requirement.

  • Handler combination: use all three handlers together. DeserializationExceptionHandler for bad input, ProductionExceptionHandler for write failures, StreamsUncaughtExceptionHandler for bugs in processing code.
  • DLT schema: include the original topic, partition, offset, timestamp, key, value, and exception message in the DLT record. This makes reprocessing possible.
  • Retry vs skip: for transient errors (network timeout), REPLACE_THREAD retries from the last committed offset. For permanent errors (bad data), CONTINUE skips the record.
  • Exactly-once and errors: when using exactly_once_v2, a failed production handler that returns CONTINUE can leave the transaction in an inconsistent state. Test this scenario carefully.
  • Monitoring: export custom metrics for deserialization errors, production errors, and DLT record counts. Alert on thresholds (e.g., more than 10 DLT records per minute).
  • Circuit breaker: if the DLT rate exceeds a threshold, it may indicate a systematic problem (schema change, broken producer). Consider shutting down to prevent cascading data loss.
JAVA
package com.bank.streams;

public class SeniorErrorHandlingKnowledge {
    public static void main(String[] args) {
        System.out.println("Senior/Principal error handling checklist:");
        System.out.println("  1. Use all three handlers: deser, production, uncaught");
        System.out.println("  2. DLT records must include: topic, partition, offset, error");
        System.out.println("  3. REPLACE_THREAD for transient, CONTINUE for permanent");
        System.out.println("  4. Test error handlers with exactly_once_v2");
        System.out.println("  5. Export custom metrics for each error type");
        System.out.println("  6. Alert on DLT rate > threshold (schema change?)");
        System.out.println("  7. Circuit breaker: shut down if DLT rate too high");
        System.out.println("  8. Reprocessing pipeline: DLT -> fix -> replay");
    }
}
TEXT
Expected output:
Senior/Principal error handling checklist:
  1. Use all three handlers: deser, production, uncaught
  2. DLT records must include: topic, partition, offset, error
  3. REPLACE_THREAD for transient, CONTINUE for permanent
  4. Test error handlers with exactly_once_v2
  5. Export custom metrics for each error type
  6. Alert on DLT rate > threshold (schema change?)
  7. Circuit breaker: shut down if DLT rate too high
  8. Reprocessing pipeline: DLT -> fix -> replay

What are common interview follow-up questions?

  • What is the default DeserializationExceptionHandler? (LogAndFailExceptionHandler. It logs the error and shuts down the stream thread. This is safe for development but dangerous in production.)
  • Can you use different handlers for different topics? (No. The handler is set globally via StreamsConfig. You can check the topic name inside the handler to apply different logic per topic.)
  • What happens to uncommitted records when a thread is replaced? (Processing restarts from the last committed offset. Records processed but not yet committed are reprocessed.)
  • How do you reprocess DLT records? (Read from the DLT topic, fix the records, and write them back to the original input topic. Use a separate consumer group for the DLT reader.)
  • Does the DeserializationExceptionHandler catch schema registry errors? (Yes. If the Avro deserializer fails because the schema is not in the registry, the handler catches it.)

Visual Overview

DIAGRAM
 Error Handling Flow in Kafka Streams
 =====================================

 ┌────────────────┐
 │ Input Topic    │
 │ (raw records)  │
 └───────┬────────┘
         │
         ▼
 ┌───────────────────────────────┐
 │ Deserialize                  │
 │  Success? ──────────────────────> Process
 │  Fail? ──> DeserExHandler    │
 │            ├─ CONTINUE (skip)│
 │            └─ FAIL (shutdown)│
 └───────────────────────────────┘
         │
         ▼
 ┌───────────────────────────────┐
 │ Process (user code)          │
 │  Success? ──────────────────────> Produce
 │  Exception? ──> Uncaught     │
 │                 Handler      │
 │   ├─ REPLACE_THREAD          │
 │   ├─ SHUTDOWN_CLIENT         │
 │   └─ SHUTDOWN_APPLICATION    │
 └───────────────────────────────┘
         │
         ▼
 ┌───────────────────────────────┐
 │ Produce to output topic      │
 │  Success? ──────────────────────> Done
 │  Fail? ──> ProdExHandler     │
 │            ├─ CONTINUE (skip)│
 │            └─ FAIL (shutdown)│
 └───────────────────────────────┘


 Dead Letter Topic Pattern
 ==========================

 ┌──────────┐    ┌──────────┐    ┌──────────┐
 │ Input    │───>│ Process  │───>│ Output   │
 │ Topic    │    │          │    │ Topic    │
 └──────────┘    └────┬─────┘    └──────────┘
                      │ error
                      ▼
                ┌──────────┐
                │ Dead     │
                │ Letter   │
                │ Topic    │
                └──────────┘

Key Takeaway

Kafka Streams provides three layers of error handling. DeserializationExceptionHandler catches bad input records before processing starts. ProductionExceptionHandler catches failures when writing to output topics. StreamsUncaughtExceptionHandler catches any unhandled exception from your processing code. Together, they form a complete error handling strategy.

The default behavior for all handlers is to shut down on error. This is safe for development but dangerous in production. A single malformed record from one producer should not halt the entire transaction processing system at National Bank of India.

In our resilient processor project, we used the branch pattern to route valid records to the output topic and invalid records to a dead letter topic. The DLT record includes the original data and the error reason, making it possible for the operations team to review, fix, and reprocess failed records.

At the senior level, combine all three handlers, include full metadata in DLT records (topic, partition, offset, error), export custom metrics for each error type, and set up alerting when the DLT rate exceeds a threshold. A spike in DLT records often indicates a schema change or a broken producer, not just bad data.

Remember one sentence: use LogAndContinue for deserialization errors, a dead letter topic for traceability, and REPLACE_THREAD for transient processing failures to keep the banking pipeline running while capturing every failed record.

---

Share
Use ← → arrow keys to navigate