Definition
- Processing Unit – an application instance that contains both the business logic and an in-memory data grid holding the data it needs, so reads and writes hit RAM with no database round trip.
- In-Memory Data Grid – a distributed, partitioned, replicated store (Hazelcast, Apache Ignite, Coherence) that spans all processing units and keeps data consistent across them.
- Data Writer (Data Pump) – a separate asynchronous process that reads changes from the grid and persists them to a backing database, taking the database off the critical path.
- Messaging Grid – distributes incoming requests across processing units so each handles a share of the load.
- Scaling Model – capacity grows by adding processing units, each bringing its own memory and CPU, giving roughly linear throughput growth with no single database to scale.
On a normal day at National Bank of India the single ledger clerk at the back (the central database) can handle every request. But on market-open morning, thousands of trade orders arrive in seconds, and that one clerk becomes a hopeless bottleneck. Space-Based architecture is like giving every counter its own fast in-memory notepad holding the data it needs, so counters serve customers instantly without waiting for the back clerk. The notepads sync with each other so they agree, and a separate quiet clerk copies everything from the notepads into the permanent ledger in the background, off the critical path.
Understanding the Concepts
What is Space-Based architecture?
Space-Based architecture removes the central database as the bottleneck by keeping application state in an in-memory data grid (the "space") that is replicated and partitioned across many processing units. Each processing unit holds a slice of the data in memory and serves requests against it. Changes are propagated across the grid and asynchronously flushed to a backing database by a separate data-writer process.
The architecture has several components: the processing unit, the in-memory data grid, the messaging grid that distributes incoming requests, the processing grid that coordinates work spanning units, and the data pump or data writer that persists changes off the critical path.
package com.bank.grid;
import java.math.BigDecimal;
import java.util.concurrent.ConcurrentHashMap;
import java.util.Map;
/**
* Simplified in-memory grid that each processing unit uses.
* In production this would be Hazelcast IMap or Apache Ignite IgniteCache.
*/
public class InMemoryAccountGrid {
private final Map<String, BigDecimal> balances = new ConcurrentHashMap<>();
public void put(String accountId, BigDecimal balance) {
balances.put(accountId, balance);
}
public BigDecimal get(String accountId) {
return balances.getOrDefault(accountId, BigDecimal.ZERO);
}
public Map<String, BigDecimal> snapshot() {
return Map.copyOf(balances);
}
}
Expected: each put/get operates in microseconds from RAM,
no database round trip on the hot path.
Why does Space-Based architecture exist?
The central problem is that under extreme load the database becomes the single bottleneck. You can add more application servers, but they all funnel into one database. Once the database saturates on connections, locks, or disk I/O, latency spikes and requests queue no matter how many app servers you have.
Space-Based architecture attacks this by taking the database off the hot path entirely. Instead of jdbcTemplate.update("UPDATE orders ...") on every order, you do ordersMap.put(orderId, order) where ordersMap is a distributed in-memory map, returning in microseconds.
package com.bank.service;
import com.bank.grid.InMemoryAccountGrid;
import java.math.BigDecimal;
public class TradeOrderService {
private final InMemoryAccountGrid grid;
public TradeOrderService(InMemoryAccountGrid grid) {
this.grid = grid;
}
public void placeOrder(String accountId, BigDecimal amount) {
// Hot path: write to in-memory grid, NOT the database
BigDecimal current = grid.get(accountId);
grid.put(accountId, current.subtract(amount));
// Data writer flushes to DB asynchronously in background
}
}
Expected: placeOrder completes in microseconds.
The database is updated later by a background data writer.
How does the data writer flush to the database?
The data writer is a separate process that reads changes from the grid and persists them to the database in batches. Because it runs off the critical path, the database only needs to keep up on average, not at peak.
package com.bank.grid;
import java.math.BigDecimal;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.util.Map;
public class AsyncDataWriter implements Runnable {
private final InMemoryAccountGrid grid;
private final Connection dbConnection;
public AsyncDataWriter(InMemoryAccountGrid grid, Connection dbConnection) {
this.grid = grid;
this.dbConnection = dbConnection;
}
@Override
public void run() {
Map<String, BigDecimal> snapshot = grid.snapshot();
try (PreparedStatement ps = dbConnection.prepareStatement(
"UPDATE accounts SET balance = ? WHERE account_id = ?")) {
for (Map.Entry<String, BigDecimal> entry : snapshot.entrySet()) {
ps.setBigDecimal(1, entry.getValue());
ps.setString(2, entry.getKey());
ps.addBatch();
}
ps.executeBatch();
System.out.println("Flushed " + snapshot.size() + " entries to database");
} catch (Exception e) {
System.err.println("Flush failed, will retry: " + e.getMessage());
}
}
}
Expected: Flushed 5000 entries to database
Runs every few seconds in the background.
How does scaling work?
Scaling is the key benefit. You add capacity by adding processing units, each bringing its own memory and CPU, and the grid rebalances partitions across them. There is no single database to scale, so throughput grows roughly linearly with units.
For example, if one processing unit handles 10,000 trades per second from memory, adding five units gives approximately 50,000 trades per second. This is why the architecture suits extreme bursty load: you can elastically add units for the burst and remove them after.
package com.bank.grid;
public class ProcessingUnitScaler {
private int activeUnits = 1;
private static final int TRADES_PER_UNIT = 10_000;
public void scaleUp(int additionalUnits) {
activeUnits += additionalUnits;
System.out.println("Scaled to " + activeUnits + " units, capacity: "
+ (activeUnits * TRADES_PER_UNIT) + " trades/sec");
}
public void scaleDown(int removeUnits) {
activeUnits = Math.max(1, activeUnits - removeUnits);
System.out.println("Scaled down to " + activeUnits + " units");
}
}
Scaled to 5 units, capacity: 50000 trades/sec
Scaled down to 2 units
When is Space-Based architecture justified?
It is justified only when the database is the proven bottleneck and the load is extreme and bursty. The signature is a workload where write and read rates far exceed what any single database can sustain, and where a brief window of the database lagging behind memory is acceptable.
Common use cases include stock and derivatives trading at market open, flash sales and ticketing where hundreds of thousands buy in seconds, real-time bidding, and online gaming leaderboards.
| Scenario | DB Bottleneck? | Bursty? | Justified? |
|---|---|---|---|
| Market-open trading (10K orders/sec) | Yes | Yes | Yes |
| Flash sale ticketing (100K buys/min) | Yes | Yes | Yes |
| Normal CRUD banking app (100 req/sec) | No | No | No |
| Read-heavy catalog with caching | No | No | No |
What are the trade-offs and limitations?
The trade-offs are significant and must be stated honestly. First, since writes go to memory and are flushed asynchronously, there is a window where the database lags the grid. If a processing unit and its replica both fail before the flush, you can lose recent writes.
Second, this is eventual consistency between the grid and the database. Third, in-memory grids are operationally complex: partitioning, rebalancing, split-brain handling, and memory sizing are hard. Fourth, holding large datasets in RAM across many nodes is expensive.
package com.bank.grid;
// WRONG: assuming grid and DB are always in sync
public class WrongAssumption {
void checkBalance(String accountId) {
// Grid says balance is 50000, DB still shows 80000
// because the flush has not happened yet
// Do NOT use DB balance for authoritative decisions during burst
}
}
// CORRECT: understand the eventual consistency window
public class CorrectApproach {
void checkBalance(String accountId, InMemoryAccountGrid grid) {
// During burst, the grid IS the source of truth
// The DB is eventually consistent and catches up after the burst
System.out.println("Grid balance (authoritative during burst): "
+ grid.get(accountId));
}
}
What are common mistakes?
The most common mistake is reaching for Space-Based architecture before exhausting simpler options. Caching, read replicas, and database sharding solve the majority of scaling problems at far lower complexity and cost.
package com.bank.service;
// WRONG: jumping to space-based architecture for a 200 req/sec service
// This is massive over-engineering
public class OverEngineered {
// Setting up Hazelcast cluster, data writers, messaging grid
// for a service that a single PostgreSQL instance handles easily
}
// CORRECT: exhaust simpler options first
public class RightApproach {
// Step 1: Add Redis cache for read-heavy data
// Step 2: Add read replicas for read scaling
// Step 3: Shard the database for write scaling
// Step 4: ONLY if all three fail, consider space-based architecture
}
What should a senior engineer know?
A senior engineer should know that adequate replication and sometimes a durable write-behind log are needed to prevent data loss during the async flush window. Products like Hazelcast, Apache Ignite, GigaSpaces, and Coherence implement the in-memory data grid with built-in partitioning and replication.
The grid and database represent eventual consistency, which is fine for many use cases but not for invariants that need the database to be the instant source of truth. The mature engineering judgment is to not reach for space-based architecture by default, and to use it only after simpler options have failed.
package com.bank.grid;
import java.math.BigDecimal;
public class DurabilityConfig {
// Replication factor: how many copies of each partition
private static final int REPLICATION_FACTOR = 2;
// Write-behind log for crash recovery
private static final boolean WRITE_BEHIND_LOG = true;
// Flush interval in milliseconds
private static final long FLUSH_INTERVAL_MS = 2000;
public static void main(String[] args) {
System.out.println("Replication factor: " + REPLICATION_FACTOR);
System.out.println("Write-behind log enabled: " + WRITE_BEHIND_LOG);
System.out.println("Flush interval: " + FLUSH_INTERVAL_MS + "ms");
System.out.println("Data loss window: up to " + FLUSH_INTERVAL_MS + "ms of writes");
}
}
Replication factor: 2
Write-behind log enabled: true
Flush interval: 2000ms
Data loss window: up to 2000ms of writes
What are common interview follow-up questions?
- How do you handle split-brain in the data grid?
- What happens if both a primary and its replica fail before the async flush?
- How do you size the memory for each processing unit?
- How does this compare to database sharding?
- When would you choose CQRS with event sourcing instead of space-based?
Visual Overview
┌──────────────────────────────────────────────────┐
│ Messaging Grid (Router) │
│ distributes requests across processing units │
└──────────┬─────────────┬─────────────┬───────────┘
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Processing │ │ Processing │ │ Processing │
│ Unit 1 │ │ Unit 2 │ │ Unit 3 │
│ │ │ │ │ │
│ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │
│ │ Business │ │ │ │ Business │ │ │ │ Business │ │
│ │ Logic │ │ │ │ Logic │ │ │ │ Logic │ │
│ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │
│ ┌──────────┐ │ │ ┌──────────┐ │ │ ┌──────────┐ │
│ │ In-Memory│ │ │ │ In-Memory│ │ │ │ In-Memory│ │
│ │ Grid │◄├─┤►│ Grid │◄├─┤►│ Grid │ │
│ │ (Slice) │ │ │ │ (Slice) │ │ │ │ (Slice) │ │
│ └──────────┘ │ │ └──────────┘ │ │ └──────────┘ │
└──────────────┘ └──────────────┘ └──────────────┘
│ │ │
└─────────────┼─────────────┘
│
▼ (async flush)
┌─────────────────┐
│ Data Writer │
│ (batches to DB) │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Database │
│ (off hot path) │
└─────────────────┘
Key Takeaway
Space-Based architecture exists to remove the central database as the bottleneck under extreme load. The insight is that you can add as many application servers as you like, but if they all funnel into one database, that database saturates on connections, locks, and disk I/O, and latency spikes no matter what. Space-Based architecture takes the database off the hot path by keeping application state in an in-memory data grid, replicated and partitioned across processing units, so reads and writes hit memory instead of disk.
The architecture has processing units that combine business logic with an in-memory slice of the data, an in-memory data grid like Hazelcast or Apache Ignite that is distributed across those units, a messaging grid that spreads requests, and a separate data writer that asynchronously flushes changes from the grid to the backing database. Because you scale by adding processing units, each bringing its own memory and CPU, throughput grows roughly linearly and there is no single database to scale, which is exactly why it absorbs bursts.
The trade-offs are serious. Because writes are flushed asynchronously, there is a window where the database lags the grid, so if a unit and its replica both fail before the flush you can lose recent writes. The grid and database are eventually consistent, which is unacceptable for invariants that need the database as the instant source of truth. In-memory grids are operationally complex, and holding large datasets in RAM across many nodes is expensive. The mature judgment is to reach for this only after caching, read replicas, and sharding have failed to meet a genuinely extreme bursty load.
Remember one sentence: keep hot data in a replicated in-memory grid so requests hit memory instead of the database, scale by adding grid nodes, and flush to the database asynchronously, but reserve it for extreme bursty load because the cost and complexity are substantial.
---