Back to Kafka Streams
Q12Question 12 of 27

How do you convert between KStream and KTable

Answer: KStream to KTable converts events into state using groupByKey() followed by aggregate(), reduce(), or count(), while KTable to KStream converts state changes back into events using toStream().

Key Concepts
  • KStream to KTable: call groupByKey() then aggregate(), reduce(), or count() to accumulate events into a state table.
  • KTable to KStream: call toStream() to emit every state change as a separate event in the stream.
  • KStream to KTable is a stateful operation: it creates a state store backed by a changelog topic.
  • KTable to KStream is stateless: toStream() forwards each KTable update as a KStream record.
  • Use KStream to KTable when you need running totals, counts, or latest-value-per-key from a stream of events.
  • Use KTable to KStream when you need to write KTable changes to an output topic or chain further stateless processing.
  • An alternative KStream to KTable path is writing the stream to a compacted topic and reading it back with builder.table().

Definition

KStream to KTable converts events into state using groupByKey() followed by aggregate(), reduce(), or count(), while KTable to KStream converts state changes back into events using toStream().
  • KStream to KTable requires a stateful operation: you group the stream by key (groupByKey() or groupBy()), then apply aggregate(), reduce(), or count() to produce a KTable holding accumulated results.
  • KTable to KStream uses toStream(), which takes every state change (insert, update, delete) in the KTable and emits it as a record in a KStream.
  • groupByKey() groups records by their existing key without repartitioning; groupBy() groups by a new key and triggers repartitioning.
  • Materialized names the state store and configures serdes for the resulting KTable.
  • Alternative path: write the KStream to a compacted topic with .to("topic"), then read it as a KTable with builder.table("topic"); this is useful when you want the topic to serve as a source of truth.

At National Bank of India, the transaction register (KStream) lists every deposit and withdrawal. When the end-of-day process adds up all transactions per account, it produces the balance sheet (KTable). That is KStream to KTable: aggregating events into state. The next morning, the balance sheet is published as a daily report, and each balance becomes a line item in the broadcast (KStream). That is KTable to KStream: turning current state into a sequence of events for downstream consumption.

Understanding the Concepts

How do you convert KStream to KTable?

You convert a KStream to a KTable by grouping records and then applying a stateful operation. The three options are count(), reduce(), and aggregate().

count() counts the number of records per key. reduce() combines values using a binary function. aggregate() applies a custom aggregation with an initializer and adder.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.TopologyTestDriver;
import org.apache.kafka.streams.TestInputTopic;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.KTable;
import org.apache.kafka.streams.kstream.Materialized;
import org.apache.kafka.streams.state.KeyValueStore;
import org.apache.kafka.streams.state.ValueAndTimestamp;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.serialization.StringSerializer;
import org.apache.kafka.common.utils.Bytes;

import java.math.BigDecimal;
import java.util.Properties;

public class KStreamToKTableDemo {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();

        KStream<String, String> transactions = builder.stream("bank-transactions");

        // Method 1: count() -> number of transactions per account
        KTable<String, Long> txnCount = transactions
            .groupByKey()
            .count(Materialized.as("txn-count-store"));

        // Method 2: aggregate() -> running balance per account
        KTable<String, String> balance = transactions
            .groupByKey()
            .aggregate(
                () -> "0",
                (accountId, txnValue, currentBalance) -> {
                    String[] parts = txnValue.split("\\|");
                    String type = parts[0];
                    BigDecimal amount = new BigDecimal(parts[1]);
                    BigDecimal bal = new BigDecimal(currentBalance);
                    if ("CREDIT".equals(type)) {
                        return bal.add(amount).toPlainString();
                    }
                    return bal.subtract(amount).toPlainString();
                },
                Materialized.<String, String, KeyValueStore<Bytes, byte[]>>as(
                    "balance-store")
                    .withKeySerde(Serdes.String())
                    .withValueSerde(Serdes.String())
            );

        txnCount.toStream().to("transaction-counts");
        balance.toStream().to("account-balances");

        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());

        try (TopologyTestDriver driver =
                 new TopologyTestDriver(builder.build(), props)) {

            TestInputTopic<String, String> input = driver.createInputTopic(
                "bank-transactions",
                new StringSerializer(), new StringSerializer());

            input.pipeInput("SAV2001", "CREDIT|50000");
            input.pipeInput("SAV2001", "CREDIT|30000");
            input.pipeInput("SAV2001", "DEBIT|10000");
            input.pipeInput("CUR3001", "CREDIT|100000");

            KeyValueStore<String, ValueAndTimestamp<Long>> countStore =
                driver.getTimestampedKeyValueStore("txn-count-store");
            KeyValueStore<String, ValueAndTimestamp<String>> balStore =
                driver.getTimestampedKeyValueStore("balance-store");

            System.out.println("KStream -> KTable conversions:");
            System.out.println();
            System.out.println("count() store:");
            System.out.println("  SAV2001: " + countStore.get("SAV2001").value() + " transactions");
            System.out.println("  CUR3001: " + countStore.get("CUR3001").value() + " transaction");
            System.out.println();
            System.out.println("aggregate() store:");
            System.out.println("  SAV2001: INR " + balStore.get("SAV2001").value());
            System.out.println("  CUR3001: INR " + balStore.get("CUR3001").value());
        }
    }
}
TEXT
Expected output:
KStream -> KTable conversions:

count() store:
  SAV2001: 3 transactions
  CUR3001: 1 transaction

aggregate() store:
  SAV2001: INR 70000
  CUR3001: INR 100000

How do you convert KTable to KStream?

You call toStream() on a KTable. Every state change in the KTable (insert, update, delete) becomes a record in the resulting KStream. This is useful for writing KTable changes to an output topic or chaining stateless operations.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.TopologyTestDriver;
import org.apache.kafka.streams.TestInputTopic;
import org.apache.kafka.streams.TestOutputTopic;
import org.apache.kafka.streams.kstream.KTable;
import org.apache.kafka.streams.kstream.Materialized;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.serialization.StringSerializer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.util.Properties;

public class KTableToKStreamDemo {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();

        KTable<String, String> accountStatus = builder.table(
            "account-status", Materialized.as("status-store"));

        // toStream(): every KTable change becomes a KStream event
        accountStatus.toStream()
            .mapValues(status -> "NOTIFICATION: Account status changed to " + status)
            .to("status-notifications");

        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());

        try (TopologyTestDriver driver =
                 new TopologyTestDriver(builder.build(), props)) {

            TestInputTopic<String, String> input = driver.createInputTopic(
                "account-status",
                new StringSerializer(), new StringSerializer());
            TestOutputTopic<String, String> output = driver.createOutputTopic(
                "status-notifications",
                new StringDeserializer(), new StringDeserializer());

            // Status changes for SAV2001
            input.pipeInput("SAV2001", "ACTIVE");
            input.pipeInput("SAV2001", "FROZEN");
            input.pipeInput("SAV2001", "ACTIVE");

            System.out.println("KTable -> KStream (toStream) output:");
            while (!output.isEmpty()) {
                var record = output.readKeyValue();
                System.out.println("  " + record.key + ": " + record.value);
            }
            System.out.println();
            System.out.println("Every KTable state change emitted as a KStream event");
        }
    }
}
TEXT
Expected output:
KTable -> KStream (toStream) output:
  SAV2001: NOTIFICATION: Account status changed to ACTIVE
  SAV2001: NOTIFICATION: Account status changed to FROZEN
  SAV2001: NOTIFICATION: Account status changed to ACTIVE

Every KTable state change emitted as a KStream event

What are all the conversion paths between KStream and KTable?

ConversionMethodStateful?State store?Use case
KStream to KTablegroupByKey().count()YesYes (RocksDB)Count events per key
KStream to KTablegroupByKey().reduce()YesYes (RocksDB)Combine values (e.g., sum)
KStream to KTablegroupByKey().aggregate()YesYes (RocksDB)Custom aggregation (e.g., balance)
KStream to topic to KTable.to("topic") then builder.table("topic")No (separate steps)YesPersist as source of truth
KTable to KStream.toStream()NoNoEmit changes for output or chaining
JAVA
package com.bank.streams;

public class ConversionPaths {
    public static void main(String[] args) {
        System.out.println("KStream -> KTable paths:");
        System.out.println("  1. groupByKey().count()     -> count per key");
        System.out.println("  2. groupByKey().reduce()    -> combine values");
        System.out.println("  3. groupByKey().aggregate() -> custom logic");
        System.out.println("  4. .to('topic') + builder.table('topic') -> via compacted topic");
        System.out.println();
        System.out.println("KTable -> KStream paths:");
        System.out.println("  1. .toStream() -> emit all changes as events");
        System.out.println();
        System.out.println("Common pattern:");
        System.out.println("  KStream (events)");
        System.out.println("    -> groupByKey().aggregate() -> KTable (state)");
        System.out.println("    -> toStream() -> KStream (state changes)");
        System.out.println("    -> .to('output-topic')");
    }
}
TEXT
Expected output:
KStream -> KTable paths:
  1. groupByKey().count()     -> count per key
  2. groupByKey().reduce()    -> combine values
  3. groupByKey().aggregate() -> custom logic
  4. .to('topic') + builder.table('topic') -> via compacted topic

KTable -> KStream paths:
  1. .toStream() -> emit all changes as events

Common pattern:
  KStream (events)
    -> groupByKey().aggregate() -> KTable (state)
    -> toStream() -> KStream (state changes)
    -> .to('output-topic')

What is a complete banking project demonstrating both conversions?

This project builds an account summary service. It reads transaction events (KStream), aggregates them into running balances (KTable), converts balance changes back to events (KStream), and publishes balance notifications.

Project structure

DIAGRAM
account-summary-service/
├── pom.xml
└── src/
    ├── main/
    │   └── java/
    │       └── com/
    │           └── bank/
    │               └── streams/
    │                   ├── AccountSummaryApp.java
    │                   └── AccountSummaryTopology.java
    └── test/
        └── java/
            └── com/
                └── bank/
                    └── streams/
                        └── AccountSummaryTest.java

AccountSummaryTopology.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.Topology;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.KTable;
import org.apache.kafka.streams.kstream.Materialized;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.utils.Bytes;
import org.apache.kafka.streams.state.KeyValueStore;

import java.math.BigDecimal;

public class AccountSummaryTopology {
    public static Topology build() {
        StreamsBuilder builder = new StreamsBuilder();

        // Step 1: Read transaction events as KStream
        KStream<String, String> transactions = builder.stream("bank-transactions");

        // Step 2: KStream -> KTable (aggregate running balance)
        KTable<String, String> balances = transactions
            .groupByKey()
            .aggregate(
                () -> "0|0",  // balance|transactionCount
                (accountId, txnValue, current) -> {
                    String[] txnParts = txnValue.split("\\|");
                    String type = txnParts[0];
                    BigDecimal amount = new BigDecimal(txnParts[1]);

                    String[] curParts = current.split("\\|");
                    BigDecimal balance = new BigDecimal(curParts[0]);
                    int count = Integer.parseInt(curParts[1]);

                    if ("CREDIT".equals(type)) {
                        balance = balance.add(amount);
                    } else {
                        balance = balance.subtract(amount);
                    }
                    return balance.toPlainString() + "|" + (count + 1);
                },
                Materialized.<String, String, KeyValueStore<Bytes, byte[]>>as(
                    "account-summary-store")
                    .withKeySerde(Serdes.String())
                    .withValueSerde(Serdes.String())
            );

        // Step 3: KTable -> KStream (emit balance changes)
        KStream<String, String> balanceEvents = balances.toStream();

        // Step 4: Enrich and write to output
        balanceEvents
            .mapValues(value -> {
                String[] parts = value.split("\\|");
                return "BALANCE:" + parts[0] + "|TOTAL_TXN:" + parts[1];
            })
            .to("account-summaries");

        return builder.build();
    }
}

AccountSummaryTest.java

JAVA
package com.bank.streams;

import org.apache.kafka.streams.TopologyTestDriver;
import org.apache.kafka.streams.TestInputTopic;
import org.apache.kafka.streams.TestOutputTopic;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.state.KeyValueStore;
import org.apache.kafka.streams.state.ValueAndTimestamp;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.serialization.StringSerializer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.util.Properties;

public class AccountSummaryTest {
    public static void main(String[] args) {
        Properties props = new Properties();
        props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
        props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
        props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());
        props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG,
                  Serdes.String().getClass());

        try (TopologyTestDriver driver =
                 new TopologyTestDriver(AccountSummaryTopology.build(), props)) {

            TestInputTopic<String, String> input = driver.createInputTopic(
                "bank-transactions",
                new StringSerializer(), new StringSerializer());
            TestOutputTopic<String, String> output = driver.createOutputTopic(
                "account-summaries",
                new StringDeserializer(), new StringDeserializer());

            // Rajesh: 3 transactions
            input.pipeInput("SAV2001", "CREDIT|50000");
            input.pipeInput("SAV2001", "CREDIT|30000");
            input.pipeInput("SAV2001", "DEBIT|10000");

            // Priya: 1 transaction
            input.pipeInput("CUR3001", "CREDIT|100000");

            System.out.println("Output (KTable -> KStream -> output topic):");
            while (!output.isEmpty()) {
                var record = output.readKeyValue();
                System.out.println("  " + record.key + ": " + record.value);
            }

            KeyValueStore<String, ValueAndTimestamp<String>> store =
                driver.getTimestampedKeyValueStore("account-summary-store");

            System.out.println();
            System.out.println("KTable state store (final values):");
            System.out.println("  SAV2001: " + store.get("SAV2001").value());
            System.out.println("  CUR3001: " + store.get("CUR3001").value());
            System.out.println();
            System.out.println("Flow: KStream -> aggregate -> KTable -> toStream -> KStream -> output");
        }
    }
}
TEXT
Expected output:
Output (KTable -> KStream -> output topic):
  SAV2001: BALANCE:50000|TOTAL_TXN:1
  SAV2001: BALANCE:80000|TOTAL_TXN:2
  SAV2001: BALANCE:70000|TOTAL_TXN:3
  CUR3001: BALANCE:100000|TOTAL_TXN:1

KTable state store (final values):
  SAV2001: 70000|3
  CUR3001: 100000|1

Flow: KStream -> aggregate -> KTable -> toStream -> KStream -> output

How to run

BASH
# Step 1: Create topics
kafka-topics.sh --create --topic bank-transactions --partitions 4 --replication-factor 1 --bootstrap-server localhost:9092
kafka-topics.sh --create --topic account-summaries --partitions 4 --replication-factor 1 --bootstrap-server localhost:9092

# Step 2: Build and test
cd account-summary-service
mvn clean package
java -cp target/account-summary-service-1.0.0.jar com.bank.streams.AccountSummaryTest

# Step 3: Run and produce transactions
java -cp target/account-summary-service-1.0.0.jar com.bank.streams.AccountSummaryApp

What are common mistakes with KStream-KTable conversions?

Wrong: using groupBy instead of groupByKey when the key is already correct

JAVA
package com.bank.streams;

public class WrongGroupByUnnecessary {
    public static void main(String[] args) {
        System.out.println("WRONG: transactions.groupBy((key, value) -> key)");
        System.out.println();
        System.out.println("  The key is already the account ID");
        System.out.println("  groupBy() thinks the key changed and triggers repartitioning");
        System.out.println("  Kafka Streams creates an internal repartition topic");
        System.out.println("  Records are written and re-read (2x I/O for nothing)");
        System.out.println();
        System.out.println("CORRECT: transactions.groupByKey()");
        System.out.println("  groupByKey() keeps the existing key");
        System.out.println("  No repartition needed");
        System.out.println("  No extra I/O");
    }
}
TEXT
Expected output:
WRONG: transactions.groupBy((key, value) -> key)

  The key is already the account ID
  groupBy() thinks the key changed and triggers repartitioning
  Kafka Streams creates an internal repartition topic
  Records are written and re-read (2x I/O for nothing)

CORRECT: transactions.groupByKey()
  groupByKey() keeps the existing key
  No repartition needed
  No extra I/O

Correct: use groupByKey when the key is already the aggregation key

JAVA
package com.bank.streams;

public class CorrectGroupByKey {
    public static void main(String[] args) {
        System.out.println("CORRECT approach:");
        System.out.println();
        System.out.println("  Key = account ID, aggregate by account ID:");
        System.out.println("    transactions.groupByKey().aggregate(...)");
        System.out.println("    No repartition");
        System.out.println();
        System.out.println("  Key = account ID, aggregate by branch code:");
        System.out.println("    transactions.groupBy((key, val) -> extractBranch(val)).aggregate(...)");
        System.out.println("    Repartition needed (different key)");
        System.out.println();
        System.out.println("  Rule: groupByKey() = same key = no repartition");
        System.out.println("        groupBy()    = new key  = repartition");
    }
}
TEXT
Expected output:
CORRECT approach:

  Key = account ID, aggregate by account ID:
    transactions.groupByKey().aggregate(...)
    No repartition

  Key = account ID, aggregate by branch code:
    transactions.groupBy((key, val) -> extractBranch(val)).aggregate(...)
    Repartition needed (different key)

  Rule: groupByKey() = same key = no repartition
        groupBy()    = new key  = repartition

What should a senior or principal engineer know?

At the senior level, know the difference between groupByKey (no repartition) and groupBy (repartition). Know that KStream to KTable always creates a state store and a changelog topic.

At the principal level, consider:

  • Changelog overhead: every KTable created by aggregation has a changelog topic in Kafka. For N aggregations, there are N changelog topics. Monitor their size and retention.
  • Suppress: by default, KTable emits an update for every input record. If you only need the final result (e.g., end-of-window count), use .suppress(Suppressed.untilWindowCloses(...)) to buffer intermediate results.
  • toStream() with key mapper: toStream((key, value) -> newKey) lets you change the key during conversion, useful for routing state changes by a different key for downstream consumers.
  • Idempotent consumers: since toStream() emits every state change, downstream consumers may see intermediate values. Design consumers to handle replayed or duplicate state changes.
  • State store compaction: the internal changelog topic for KTable aggregations is automatically compacted. Do not change its retention policy manually.
JAVA
package com.bank.streams;

public class ConversionPrincipalKnowledge {
    public static void main(String[] args) {
        System.out.println("Principal-level conversion knowledge:");
        System.out.println();
        System.out.println("1. groupByKey vs groupBy:");
        System.out.println("   groupByKey = no repartition (same key)");
        System.out.println("   groupBy    = repartition (new key, 2x I/O)");
        System.out.println();
        System.out.println("2. Changelog topics per aggregation:");
        System.out.println("   Each groupByKey().aggregate() creates 1 changelog topic");
        System.out.println("   3 aggregations = 3 changelog topics to monitor");
        System.out.println();
        System.out.println("3. Suppress for windowed results:");
        System.out.println("   Without suppress: emit after every record");
        System.out.println("   With suppress:    emit only when window closes");
        System.out.println();
        System.out.println("4. toStream() key mapper:");
        System.out.println("   balances.toStream((accountId, balance) -> branchCode)");
        System.out.println("   Rekeys the stream for downstream routing");
    }
}
TEXT
Expected output:
Principal-level conversion knowledge:

1. groupByKey vs groupBy:
   groupByKey = no repartition (same key)
   groupBy    = repartition (new key, 2x I/O)

2. Changelog topics per aggregation:
   Each groupByKey().aggregate() creates 1 changelog topic
   3 aggregations = 3 changelog topics to monitor

3. Suppress for windowed results:
   Without suppress: emit after every record
   With suppress:    emit only when window closes

4. toStream() key mapper:
   balances.toStream((accountId, balance) -> branchCode)
   Rekeys the stream for downstream routing

What is the alternative KStream to KTable path via a compacted topic?

Instead of aggregating in memory, you can write the KStream to a compacted Kafka topic and then read it back as a KTable. This is useful when you want the topic to serve as a durable source of truth for other applications.

JAVA
package com.bank.streams;

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.Topology;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.KTable;

public class TopicBasedConversion {
    public static void main(String[] args) {
        StreamsBuilder builder = new StreamsBuilder();

        // Read events
        KStream<String, String> transactions = builder.stream("bank-transactions");

        // Write latest status to a compacted topic
        transactions
            .mapValues(value -> {
                String[] parts = value.split("\\|");
                return parts[0] + "|PROCESSED";
            })
            .to("account-latest-status");

        // Read the compacted topic as KTable
        KTable<String, String> latestStatus = builder.table("account-latest-status");

        latestStatus.toStream().to("status-changes");

        System.out.println("Topic-based KStream to KTable:");
        System.out.println("  1. KStream writes to compacted topic");
        System.out.println("  2. builder.table() reads the compacted topic as KTable");
        System.out.println("  3. Other apps can also read this topic directly");
        System.out.println();
        System.out.println("  Advantage: topic is durable, shared source of truth");
        System.out.println("  Disadvantage: extra topic, extra I/O, slight delay");
    }
}
TEXT
Expected output:
Topic-based KStream to KTable:
  1. KStream writes to compacted topic
  2. builder.table() reads the compacted topic as KTable
  3. Other apps can also read this topic directly

  Advantage: topic is durable, shared source of truth
  Disadvantage: extra topic, extra I/O, slight delay

What are common interview follow-up questions?

  • Is KStream to KTable conversion reversible without data loss? (The aggregation loses individual events. You cannot recover the original transactions from the balance KTable. If you need both, keep the original KStream and the aggregated KTable as separate paths.)
  • Does toStream() consume extra resources? (No. toStream() is a simple passthrough that converts the KTable's internal change log into a KStream. It does not create a new state store or topic.)
  • Can you chain multiple aggregations? (Yes. You can convert a KTable back to a KStream with toStream(), group by a different key, and aggregate again. Each aggregation creates its own state store.)
  • What is the difference between reduce() and aggregate()? (reduce() combines two values of the same type: (V, V) -> V. aggregate() can produce a different type: initializer returns VR, adder takes (K, V, VR) -> VR.)
  • When should you use the topic-based KStream to KTable path? (When you want the topic to serve as a durable source of truth that other applications can also read. The compacted topic acts as a snapshot of the latest state per key.)
  • What happens to the KTable state store when you restart the app? (It is rebuilt from the changelog topic. This can take minutes for large state stores. Configure standby replicas to reduce recovery time.)

Visual Overview

DIAGRAM
 KStream <-> KTable Conversions
 ================================

 KStream -> KTable (aggregation)
 ────────────────────────────────

 KStream (bank-transactions)
   │
   │ SAV2001|CREDIT|50000
   │ SAV2001|CREDIT|30000
   │ SAV2001|DEBIT|10000
   │
   ▼ groupByKey()
 KGroupedStream
   │
   ▼ aggregate()
 KTable (account-summary-store)
   ┌─────────┬──────────────┐
   │   Key   │    Value     │
   ├─────────┼──────────────┤
   │ SAV2001 │ 70000|3      │
   └─────────┴──────────────┘


 KTable -> KStream (toStream)
 ─────────────────────────────

 KTable change: SAV2001 -> 70000|3
   │
   ▼ toStream()
 KStream event: SAV2001 -> BALANCE:70000|TOTAL_TXN:3
   │
   ▼ .to('account-summaries')
 Output topic


 Full Flow
 ──────────
 KStream ──> groupByKey ──> aggregate ──> KTable
                                           │
                                      toStream()
                                           │
                                           ▼
                                        KStream
                                           │
                                         .to()
                                           │
                                           ▼
                                      Output topic

Key Takeaway

Converting between KStream and KTable is one of the most common patterns in Kafka Streams. KStream to KTable uses groupByKey() followed by aggregate(), reduce(), or count() to turn a stream of events into a table of accumulated state. KTable to KStream uses toStream() to emit every state change as an event for downstream consumers.

In our National Bank of India account summary project, transaction events flowed in as a KStream. The aggregate() function computed a running balance and transaction count per account, storing the result in a KTable's state store. Then toStream() converted those state changes back to a KStream, enriched them with labels, and wrote them to the account-summaries output topic.

The critical optimization is using groupByKey() when the key is already the aggregation key. Using groupBy() with the same key wastes resources by creating an unnecessary repartition topic. Reserve groupBy() for cases where you genuinely need to change the key.

At the principal level, monitor changelog topic sizes for each aggregation, use suppress() when you only need final windowed results, and configure standby replicas to speed up state store recovery on restart.

Remember one sentence: groupByKey().aggregate() turns events into state (KStream to KTable), and toStream() turns state changes back into events (KTable to KStream).

---

Share
Use ← → arrow keys to navigate