Definition
- Interactive queries is the feature that exposes Kafka Streams state stores for read-only access through the KafkaStreams.store() method.
- ReadOnlyKeyValueStore is the interface for querying key-value state stores; it provides get(key), range(from, to), and all() methods.
- StoreQueryParameters specifies the store name and type when calling store(); types include keyValue, window, and session.
- queryMetadataForKey() returns metadata about which Kafka Streams instance holds the partition for a given key, enabling query routing in multi-instance deployments.
- application.server is the config that advertises each instance's host and port so other instances can discover and route queries to the correct owner.
Think of National Bank of India with 5 branch offices, each handling accounts starting with a different prefix. Interactive queries are like calling any branch and asking "What is the balance for SAV2001?" If you call the wrong branch, the clerk checks a directory to find which branch holds that account and redirects your call. The directory is queryMetadataForKey(), each branch is a Kafka Streams instance, and the clerk's phone number is application.server.
Understanding the Concepts
What are interactive queries?
Interactive queries let you read from Kafka Streams state stores directly, without consuming from a Kafka topic. After you build a topology with a stateful operation (count, aggregate, reduce), the results are stored in a local state store. Instead of reading the output topic to get the latest value, you call KafkaStreams.store() and query the store in-memory.
This turns your Kafka Streams application into a queryable microservice. You can build a REST API on top of the state store to serve read requests.
package com.bank.streams;
public class InteractiveQueriesOverview {
public static void main(String[] args) {
System.out.println("Interactive queries:");
System.out.println(" Without: topology -> output topic -> consumer -> serve");
System.out.println(" With: topology -> state store -> REST API -> serve");
System.out.println();
System.out.println("Benefits:");
System.out.println(" 1. No separate database needed");
System.out.println(" 2. State is always up to date (no replication lag)");
System.out.println(" 3. Reads are local and fast (RocksDB)");
System.out.println();
System.out.println("Use case: real-time account balance lookup at National Bank of India");
}
}
Expected output:
Interactive queries:
Without: topology -> output topic -> consumer -> serve
With: topology -> state store -> REST API -> serve
Benefits:
1. No separate database needed
2. State is always up to date (no replication lag)
3. Reads are local and fast (RocksDB)
Use case: real-time account balance lookup at National Bank of India
How do you query a key-value state store?
After creating a state store with Materialized.as("store-name"), you query it using KafkaStreams.store() with StoreQueryParameters. The returned ReadOnlyKeyValueStore provides get(), range(), and all() methods.
package com.bank.streams;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StoreQueryParameters;
import org.apache.kafka.streams.state.QueryableStoreTypes;
import org.apache.kafka.streams.state.ReadOnlyKeyValueStore;
import org.apache.kafka.streams.state.KeyValueIterator;
import org.apache.kafka.streams.KeyValue;
public class QueryKeyValueStore {
public static void main(String[] args) {
System.out.println("Querying a key-value state store:");
System.out.println();
System.out.println(" // Get the store");
System.out.println(" ReadOnlyKeyValueStore<String, Long> store = streams.store(");
System.out.println(" StoreQueryParameters.fromNameAndType(");
System.out.println(" \"account-balances\",");
System.out.println(" QueryableStoreTypes.keyValueStore()));");
System.out.println();
System.out.println(" // Get single key");
System.out.println(" Long balance = store.get(\"SAV2001\");");
System.out.println();
System.out.println(" // Range query (SAV2001 to SAV2099)");
System.out.println(" KeyValueIterator<String, Long> range = store.range(\"SAV2001\", \"SAV2099\");");
System.out.println();
System.out.println(" // Iterate all entries");
System.out.println(" KeyValueIterator<String, Long> all = store.all();");
}
}
Expected output:
Querying a key-value state store:
// Get the store
ReadOnlyKeyValueStore<String, Long> store = streams.store(
StoreQueryParameters.fromNameAndType(
"account-balances",
QueryableStoreTypes.keyValueStore()));
// Get single key
Long balance = store.get("SAV2001");
// Range query (SAV2001 to SAV2099)
KeyValueIterator<String, Long> range = store.range("SAV2001", "SAV2099");
// Iterate all entries
KeyValueIterator<String, Long> all = store.all();
Why is state partitioned and how do distributed queries work?
Each Kafka Streams instance owns a subset of the input partitions and holds state only for the keys in those partitions. If you have 6 partitions and 3 instances, each instance holds state for 2 partitions. Querying instance A for a key owned by instance B returns null.
To handle this, Kafka Streams provides queryMetadataForKey() that tells you which instance owns a given key. You then route the query to that instance's REST API.
package com.bank.streams;
public class DistributedQueryExplained {
public static void main(String[] args) {
System.out.println("Distributed query routing:");
System.out.println();
System.out.println("Setup: 3 instances, 6 partitions");
System.out.println(" Instance A (host-a:8080): partitions 0, 1");
System.out.println(" Instance B (host-b:8080): partitions 2, 3");
System.out.println(" Instance C (host-c:8080): partitions 4, 5");
System.out.println();
System.out.println("Query: GET /api/balance/SAV2001");
System.out.println(" SAV2001 hashes to partition 3 (owned by Instance B)");
System.out.println();
System.out.println("If query hits Instance A:");
System.out.println(" 1. A calls queryMetadataForKey(\"SAV2001\")");
System.out.println(" 2. Metadata says: host-b:8080");
System.out.println(" 3. A forwards request to http://host-b:8080/api/balance/SAV2001");
System.out.println(" 4. B queries its local store and returns the balance");
}
}
Expected output:
Distributed query routing:
Setup: 3 instances, 6 partitions
Instance A (host-a:8080): partitions 0, 1
Instance B (host-b:8080): partitions 2, 3
Instance C (host-c:8080): partitions 4, 5
Query: GET /api/balance/SAV2001
SAV2001 hashes to partition 3 (owned by Instance B)
If query hits Instance A:
1. A calls queryMetadataForKey("SAV2001")
2. Metadata says: host-b:8080
3. A forwards request to http://host-b:8080/api/balance/SAV2001
4. B queries its local store and returns the balance
How do you configure application.server for discovery?
Each Kafka Streams instance must advertise its host and port using the application.server config. Other instances use this information to route queries.
package com.bank.streams;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.common.serialization.Serdes;
import java.util.Properties;
public class ApplicationServerConfig {
public static void main(String[] args) {
Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "balance-service");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
// Advertise this instance's REST API endpoint
props.put(StreamsConfig.APPLICATION_SERVER_CONFIG, "host-a:8080");
System.out.println("application.server configuration:");
System.out.println(" Instance A: host-a:8080");
System.out.println(" Instance B: host-b:8080");
System.out.println(" Instance C: host-c:8080");
System.out.println();
System.out.println("Each instance sets its own host:port.");
System.out.println("Kafka Streams shares this metadata via the consumer group.");
System.out.println("queryMetadataForKey() returns the host:port for any key.");
}
}
Expected output:
application.server configuration:
Instance A: host-a:8080
Instance B: host-b:8080
Instance C: host-c:8080
Each instance sets its own host:port.
Kafka Streams shares this metadata via the consumer group.
queryMetadataForKey() returns the host:port for any key.
What is a complete banking project for account balance REST API?
This project maintains running balances per account and exposes them via a REST endpoint. When queried, it checks if the key is owned locally. If not, it returns the redirect information.
Project structure
balance-query-service/
├── pom.xml
└── src/
├── main/
│ └── java/
│ └── com/
│ └── bank/
│ └── streams/
│ ├── BalanceQueryApp.java
│ ├── BalanceQueryTopology.java
│ ├── BalanceRestService.java
│ └── BalanceEvent.java
└── test/
└── java/
└── com/
└── bank/
└── streams/
└── BalanceQueryTopologyTest.java
pom.xml
<?xml version="1.0" encoding="UTF-8"?syn-kw">>
syn-kw"><project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd"syn-kw">>
syn-kw"><modelVersionsyn-kw">>4.0.0syn-kw"></modelVersionsyn-kw">>
syn-kw"><groupIdsyn-kw">>com.banksyn-kw"></groupIdsyn-kw">>
syn-kw"><artifactIdsyn-kw">>balance-query-servicesyn-kw"></artifactIdsyn-kw">>
syn-kw"><versionsyn-kw">>1.0.0syn-kw"></versionsyn-kw">>
syn-kw"><packagingsyn-kw">>jarsyn-kw"></packagingsyn-kw">>
syn-kw"><propertiessyn-kw">>
syn-kw"><java.versionsyn-kw">>25syn-kw"></java.versionsyn-kw">>
syn-kw"><kafka.versionsyn-kw">>4.2.0syn-kw"></kafka.versionsyn-kw">>
syn-kw"><maven.compiler.sourcesyn-kw">>25syn-kw"></maven.compiler.sourcesyn-kw">>
syn-kw"><maven.compiler.targetsyn-kw">>25syn-kw"></maven.compiler.targetsyn-kw">>
syn-kw"></propertiessyn-kw">>
syn-kw"><dependenciessyn-kw">>
syn-kw"><dependencysyn-kw">>
syn-kw"><groupIdsyn-kw">>org.apache.kafkasyn-kw"></groupIdsyn-kw">>
syn-kw"><artifactIdsyn-kw">>kafka-streamssyn-kw"></artifactIdsyn-kw">>
syn-kw"><versionsyn-kw">>${kafka.version}syn-kw"></versionsyn-kw">>
syn-kw"></dependencysyn-kw">>
syn-kw"><dependencysyn-kw">>
syn-kw"><groupIdsyn-kw">>org.apache.kafkasyn-kw"></groupIdsyn-kw">>
syn-kw"><artifactIdsyn-kw">>kafka-streams-test-utilssyn-kw"></artifactIdsyn-kw">>
syn-kw"><versionsyn-kw">>${kafka.version}syn-kw"></versionsyn-kw">>
syn-kw"><scopesyn-kw">>testsyn-kw"></scopesyn-kw">>
syn-kw"></dependencysyn-kw">>
syn-kw"><dependencysyn-kw">>
syn-kw"><groupIdsyn-kw">>com.sun.net.httpserversyn-kw"></groupIdsyn-kw">>
syn-kw"><artifactIdsyn-kw">>httpsyn-kw"></artifactIdsyn-kw">>
syn-kw"><versionsyn-kw">>20070405syn-kw"></versionsyn-kw">>
syn-kw"><scopesyn-kw">>systemsyn-kw"></scopesyn-kw">>
syn-kw"><systemPathsyn-kw">>${java.home}/lib/modulessyn-kw"></systemPathsyn-kw">>
syn-kw"></dependencysyn-kw">>
syn-kw"><dependencysyn-kw">>
syn-kw"><groupIdsyn-kw">>org.slf4jsyn-kw"></groupIdsyn-kw">>
syn-kw"><artifactIdsyn-kw">>slf4j-simplesyn-kw"></artifactIdsyn-kw">>
syn-kw"><versionsyn-kw">>2.0.12syn-kw"></versionsyn-kw">>
syn-kw"></dependencysyn-kw">>
syn-kw"></dependenciessyn-kw">>
syn-kw"></projectsyn-kw">>
BalanceEvent.java
package com.bank.streams;
import java.math.BigDecimal;
public class BalanceEvent {
private String accountId;
private String type;
private BigDecimal amount;
public BalanceEvent() {}
public BalanceEvent(String accountId, String type, BigDecimal amount) {
this.accountId = accountId;
this.type = type;
this.amount = amount;
}
public String getAccountId() { return accountId; }
public String getType() { return type; }
public BigDecimal getAmount() { return amount; }
public static BalanceEvent fromString(String value) {
String[] parts = value.split("\\|");
return new BalanceEvent(parts[0], parts[1], new BigDecimal(parts[2]));
}
}
BalanceQueryTopology.java
package com.bank.streams;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Materialized;
import org.apache.kafka.streams.Topology;
import java.math.BigDecimal;
public class BalanceQueryTopology {
public static final String STORE_NAME = "account-balances";
public static Topology build() {
StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> events = builder.stream("balance-events");
events.groupByKey()
.aggregate(
() -> "0",
(accountId, eventStr, currentBalance) -> {
BalanceEvent event = BalanceEvent.fromString(eventStr);
BigDecimal balance = new BigDecimal(currentBalance);
if ("DEPOSIT".equals(event.getType())) {
balance = balance.add(event.getAmount());
} else if ("WITHDRAWAL".equals(event.getType())) {
balance = balance.subtract(event.getAmount());
}
return balance.toPlainString();
},
Materialized.as(STORE_NAME)
);
return builder.build();
}
}
BalanceRestService.java
package com.bank.streams;
import com.sun.net.httpserver.HttpServer;
import com.sun.net.httpserver.HttpExchange;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.KeyQueryMetadata;
import org.apache.kafka.streams.StoreQueryParameters;
import org.apache.kafka.streams.state.QueryableStoreTypes;
import org.apache.kafka.streams.state.ReadOnlyKeyValueStore;
import org.apache.kafka.common.serialization.Serdes;
import java.io.IOException;
import java.io.OutputStream;
import java.net.InetSocketAddress;
public class BalanceRestService {
private final KafkaStreams streams;
private final String hostPort;
public BalanceRestService(KafkaStreams streams, String hostPort) {
this.streams = streams;
this.hostPort = hostPort;
}
public void start(int port) throws IOException {
HttpServer server = HttpServer.create(new InetSocketAddress(port), 0);
server.createContext("/api/balance/", this::handleBalance);
server.start();
System.out.println("REST API started on port " + port);
}
private void handleBalance(HttpExchange exchange) throws IOException {
String path = exchange.getRequestURI().getPath();
String accountId = path.substring("/api/balance/".length());
// Check if this instance owns the key
KeyQueryMetadata metadata = streams.queryMetadataForKey(
BalanceQueryTopology.STORE_NAME, accountId, Serdes.String().serializer());
String activeHost = metadata.activeHost().host() + ":" + metadata.activeHost().port();
String response;
if (activeHost.equals(hostPort)) {
// Local query
ReadOnlyKeyValueStore<String, String> store = streams.store(
StoreQueryParameters.fromNameAndType(
BalanceQueryTopology.STORE_NAME,
QueryableStoreTypes.keyValueStore()));
String balance = store.get(accountId);
response = "{\"account\": \"" + accountId + "\", \"balance\": "
+ (balance != null ? balance : "0") + "}";
} else {
// Redirect to owning instance
response = "{\"redirect\": \"http://" + activeHost
+ "/api/balance/" + accountId + "\"}";
}
byte[] bytes = response.getBytes();
exchange.sendResponseHeaders(200, bytes.length);
try (OutputStream os = exchange.getResponseBody()) {
os.write(bytes);
}
}
}
BalanceQueryApp.java
package com.bank.streams;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.common.serialization.Serdes;
import java.util.Properties;
import java.util.concurrent.CountDownLatch;
public class BalanceQueryApp {
public static void main(String[] args) throws Exception {
String host = args.length > 0 ? args[0] : "localhost";
int port = args.length > 1 ? Integer.parseInt(args[1]) : 8080;
String hostPort = host + ":" + port;
Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "balance-query-service");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.APPLICATION_SERVER_CONFIG, hostPort);
KafkaStreams streams = new KafkaStreams(BalanceQueryTopology.build(), props);
CountDownLatch latch = new CountDownLatch(1);
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
streams.close();
latch.countDown();
}));
streams.start();
new BalanceRestService(streams, hostPort).start(port);
System.out.println("Balance query service started at " + hostPort);
System.out.println(" Query: curl http://" + hostPort + "/api/balance/SAV2001");
latch.await();
}
}
BalanceQueryTopologyTest.java
package com.bank.streams;
import org.apache.kafka.streams.TopologyTestDriver;
import org.apache.kafka.streams.TestInputTopic;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.state.KeyValueStore;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.serialization.StringSerializer;
import java.util.Properties;
public class BalanceQueryTopologyTest {
public static void main(String[] args) {
Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "test");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "dummy:1234");
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
try (TopologyTestDriver driver = new TopologyTestDriver(BalanceQueryTopology.build(), props)) {
TestInputTopic<String, String> input = driver.createInputTopic(
"balance-events", new StringSerializer(), new StringSerializer());
input.pipeInput("SAV2001", "SAV2001|DEPOSIT|50000");
input.pipeInput("SAV2001", "SAV2001|DEPOSIT|30000");
input.pipeInput("SAV2001", "SAV2001|WITHDRAWAL|10000");
input.pipeInput("CUR3001", "CUR3001|DEPOSIT|200000");
KeyValueStore<String, String> store =
driver.getKeyValueStore("account-balances");
System.out.println("Interactive query results (from state store):");
System.out.println(" SAV2001 balance: " + store.get("SAV2001"));
System.out.println(" CUR3001 balance: " + store.get("CUR3001"));
System.out.println(" SAV9999 balance: " + store.get("SAV9999"));
}
}
}
Expected output:
Interactive query results (from state store):
SAV2001 balance: 70000
CUR3001 balance: 200000
SAV9999 balance: null
How to run the project
# Step 1: Start Kafka
docker-compose up -d kafka zookeeper
# Step 2: Create topic
kafka-topics.sh --create --topic balance-events --partitions 6 --replication-factor 1 --bootstrap-server localhost:9092
# Step 3: Build
cd balance-query-service
mvn clean package
# Step 4: Start 2 instances (different ports)
java -cp target/balance-query-service-1.0.0.jar com.bank.streams.BalanceQueryApp localhost 8080 &
java -cp target/balance-query-service-1.0.0.jar com.bank.streams.BalanceQueryApp localhost 8081 &
# Step 5: Produce events
kafka-console-producer.sh --topic balance-events --property "parse.key=true" --property "key.separator=:" --bootstrap-server localhost:9092
# SAV2001:SAV2001|DEPOSIT|50000
# CUR3001:CUR3001|DEPOSIT|200000
# Step 6: Query balance (may hit either instance)
curl http://localhost:8080/api/balance/SAV2001
curl http://localhost:8081/api/balance/CUR3001
What are common mistakes with interactive queries?
The most common mistake is querying the wrong instance and getting null.
Wrong: query without checking ownership
package com.bank.streams;
public class WrongQueryWithoutOwnershipCheck {
public static void main(String[] args) {
System.out.println("WRONG: Query state store without checking ownership");
System.out.println();
System.out.println(" ReadOnlyKeyValueStore<String, String> store = streams.store(...)");
System.out.println(" String balance = store.get(\"SAV2001\");");
System.out.println(" // Returns null even though SAV2001 has a balance!");
System.out.println();
System.out.println("Why: SAV2001 is owned by partition 3, assigned to Instance B.");
System.out.println("This instance (A) owns partitions 0 and 1.");
System.out.println("store.get() only queries local partitions.");
}
}
Expected output:
WRONG: Query state store without checking ownership
ReadOnlyKeyValueStore<String, String> store = streams.store(...)
String balance = store.get("SAV2001");
// Returns null even though SAV2001 has a balance!
Why: SAV2001 is owned by partition 3, assigned to Instance B.
This instance (A) owns partitions 0 and 1.
store.get() only queries local partitions.
Correct: check ownership and route to correct instance
package com.bank.streams;
public class CorrectQueryWithRouting {
public static void main(String[] args) {
System.out.println("CORRECT: Check ownership before querying");
System.out.println();
System.out.println(" KeyQueryMetadata meta = streams.queryMetadataForKey(");
System.out.println(" \"account-balances\", \"SAV2001\", serializer);");
System.out.println(" if (meta.activeHost().equals(thisHost)) {");
System.out.println(" // Query locally");
System.out.println(" store.get(\"SAV2001\");");
System.out.println(" } else {");
System.out.println(" // Forward to owning instance");
System.out.println(" httpClient.get(meta.activeHost() + \"/api/balance/SAV2001\");");
System.out.println(" }");
System.out.println();
System.out.println("Every query first checks who owns the key.");
}
}
Expected output:
CORRECT: Check ownership before querying
KeyQueryMetadata meta = streams.queryMetadataForKey(
"account-balances", "SAV2001", serializer);
if (meta.activeHost().equals(thisHost)) {
// Query locally
store.get("SAV2001");
} else {
// Forward to owning instance
httpClient.get(meta.activeHost() + "/api/balance/SAV2001");
}
Every query first checks who owns the key.
What should a senior or principal engineer know?
At the senior level, interactive queries require careful design for production.
- Rebalancing interruptions: during a rebalance, state stores are temporarily unavailable. Return HTTP 503 and let the client retry.
- Stale reads: state stores are eventually consistent. The latest record may not be in the store yet if commit has not happened. For financial reads, accept that the balance may be a few hundred milliseconds behind.
- Load balancing: put a load balancer (Nginx, HAProxy) in front of all instances. The query routing logic handles forwarding internally.
- Standby queries: with StoreQueryParameters.enableStaleStores(), you can query standby replicas for read-only access, reducing load on the active instance.
- Global stores: a GlobalKTable is replicated to every instance. Interactive queries on global stores do not need routing because every instance has the full data.
package com.bank.streams;
public class SeniorInteractiveQueryKnowledge {
public static void main(String[] args) {
System.out.println("Senior/Principal interactive query checklist:");
System.out.println(" 1. Return 503 during rebalance, client retries");
System.out.println(" 2. State stores are eventually consistent");
System.out.println(" 3. Use load balancer, routing happens internally");
System.out.println(" 4. enableStaleStores() for standby reads");
System.out.println(" 5. GlobalKTable: no routing needed (full data everywhere)");
System.out.println(" 6. Set application.server on every instance");
System.out.println(" 7. Close KeyValueIterators to avoid resource leaks");
System.out.println(" 8. Monitor query latency per instance via metrics");
}
}
Expected output:
Senior/Principal interactive query checklist:
1. Return 503 during rebalance, client retries
2. State stores are eventually consistent
3. Use load balancer, routing happens internally
4. enableStaleStores() for standby reads
5. GlobalKTable: no routing needed (full data everywhere)
6. Set application.server on every instance
7. Close KeyValueIterators to avoid resource leaks
8. Monitor query latency per instance via metrics
What are common interview follow-up questions?
- Can you write to a state store via interactive queries? (No. Interactive queries provide read-only access. Writes happen only through the topology processing pipeline.)
- What happens to queries during a rebalance? (The store is temporarily unavailable. store() throws InvalidStateStoreException. Catch it and return HTTP 503.)
- Can you query windowed state stores? (Yes. Use QueryableStoreTypes.windowStore() and call fetch(key, timeFrom, timeTo) to query a specific window range.)
- How do you handle schema changes in state stores? (Use a version prefix in the store name, e.g., "account-balances-v2". Old and new topologies use different stores.)
- Is interactive queries the same as ksqlDB? (No. Interactive queries are a Java API for reading state stores from within the application. ksqlDB is a separate SQL engine built on top of Kafka Streams.)
Visual Overview
Interactive Query: Local vs Remote
====================================
Client: GET /api/balance/SAV2001
│
▼
┌──────────────────────┐
│ Instance A (:8080) │
│ │
│ queryMetadataForKey() │
│ SAV2001 -> Instance B │
│ │
│ Forward to B │──────────────────┐
└──────────────────────┘ │
▼
┌──────────────────────┐
│ Instance B (:8081) │
│ │
│ store.get("SAV2001") │
│ = 70000 │
│ │
│ Return balance │
└──────────────────────┘
State Distribution (6 partitions, 3 instances)
================================================
┌──────────────────┬──────────────────┬──────────────────┐
│ Instance A │ Instance B │ Instance C │
│ Partitions: 0, 1 │ Partitions: 2, 3 │ Partitions: 4, 5 │
│ Keys: CUR30xx │ Keys: SAV20xx │ Keys: SAV90xx │
└──────────────────┴──────────────────┴──────────────────┘
Key Takeaway
Interactive queries let you read directly from Kafka Streams state stores, turning your stream processing application into a queryable microservice. Instead of consuming an output topic and loading data into a database, you expose the state store through a REST API and serve reads from local RocksDB.
The key challenge is that state is partitioned. Each instance holds state only for the partitions it owns. To query a key owned by another instance, you call queryMetadataForKey() to discover the owner and forward the request. The application.server config advertises each instance's REST endpoint so the metadata knows where to route.
In our balance query service at National Bank of India, we maintained running balances per account in a state store. The REST endpoint at /api/balance/{accountId} checks ownership, queries locally if possible, or returns redirect information. The TopologyTestDriver confirmed that deposits and withdrawals produce correct balances.
At the senior level, handle rebalancing by returning HTTP 503, accept that reads are eventually consistent, use GlobalKTable for small lookup tables that need no routing, and close KeyValueIterators to prevent resource leaks.
Remember one sentence: interactive queries turn Kafka Streams into a queryable service by reading from local state stores, with queryMetadataForKey() handling the routing for keys owned by other instances.
---