What are memory-mapped files, how do they work, and when are they useful?

• Memory-mapped file – a region of a file mapped directly into process memory so reading and writing the file becomes reading and writing memory, with no explicit read or write system calls.
• FileChannel.map(mode, position, size) – the Java API that creates the mapping and returns a MappedByteBuffer.
• Demand paging – the OS loads file pages into physical memory only when first accessed, and writes dirty pages back lazily.
• Zero-copy – no data is copied between a kernel buffer and a user buffer, because the application accesses the file pages directly.
• Modes – READ_ONLY, READ_WRITE (shared, writes go to disk), and PRIVATE (copy-on-write, writes stay local).

What is a memory-mapped file?

A memory-mapped file maps a region of a file directly into the process virtual address space. After mapping, reading and writing the file becomes reading and writing memory. There is no explicit read() or write() system call per access, and there is no copy between a kernel buffer and a user-space buffer.

In Java you create a mapping with FileChannel.map(mode, position, size), which returns a MappedByteBuffer. You then use the standard ByteBuffer methods such as get, put, getLong, and putInt to access the file contents.

package com.bank.io;

import java.io.RandomAccessFile;
import java.math.BigDecimal;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class LedgerMapper {
    public static void main(String[] args) throws Exception {
        try (RandomAccessFile raf = new RandomAccessFile("ledger.dat", "rw");
             FileChannel channel = raf.getChannel()) {

            // Map the first 1024 bytes for read-write
            MappedByteBuffer buffer = channel.map(
                FileChannel.MapMode.READ_WRITE, 0, 1024);

            // Write Rajesh's balance at offset 0 (stored as long paise)
            long balancePaise = new BigDecimal("50000.75")
                .movePointRight(2).longValue();
            buffer.putLong(0, balancePaise);

            // Read it back
            long read = buffer.getLong(0);
            BigDecimal balance = BigDecimal.valueOf(read, 2);
            System.out.println("Rajesh balance: " + balance);
        }
    }
}

Rajesh balance: 50000.75

Why do memory-mapped files exist?

Normal file I/O uses system calls. Each read() copies bytes from the OS page cache into a user buffer, and each write() copies the other way. For large files or high request rates, these system calls and copies add up.

Memory mapping removes the per-access system call and the copy. The file pages live directly in the process address space, so accessing a byte is like accessing an array element. This is the zero-copy benefit. It matters in three situations:

• Very large files accessed randomly, such as a database file or a ledger index.
• High-throughput I/O where avoiding system-call overhead matters, which is why Kafka and Lucene map their segment files.
• Inter-process shared memory, where two processes mapping the same file see each other's writes through the same physical pages.

How does memory mapping work internally?

The mechanism is demand paging. When you call FileChannel.map, the OS does not read the whole file. It sets up page-table entries that point at the file on disk. The first time you touch a byte in a page, the CPU raises a page fault, the OS reads that page from disk into physical memory, and the access continues. Pages you never touch are never loaded.

When you write to a mapped page in READ_WRITE mode, the page becomes dirty. The OS writes it back to disk later, either when memory pressure evicts it or when you call force(). This lazy write-back is why mapped I/O is fast but also why durability needs explicit care.

package com.bank.io;

import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class DurableWrite {
    public static void main(String[] args) throws Exception {
        try (RandomAccessFile raf = new RandomAccessFile("ledger.dat", "rw");
             FileChannel channel = raf.getChannel()) {

            MappedByteBuffer buffer = channel.map(
                FileChannel.MapMode.READ_WRITE, 0, 1024);

            // Write a committed ledger record
            buffer.putLong(0, 999900L); // 9999.00 rupees in paise

            // Flush dirty pages to disk for durability
            buffer.force();
            System.out.println("Ledger record flushed to disk.");
        }
    }
}

Ledger record flushed to disk.

What are the three mapping modes?

FileChannel.MapMode defines three modes:

| Mode | Behavior | Writes propagate to file? | Other processes see writes? |
|------|----------|---------------------------|-----------------------------|
| READ_ONLY | Read-only access; put throws ReadOnlyBufferException | No | No |
| READ_WRITE | Read-write access; writes go to the file and are visible to other processes mapping the same file | Yes | Yes |
| PRIVATE | Copy-on-write; writes are visible only to this process and never written to the file | No | No |

package com.bank.io;

import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class MappingModes {
    public static void main(String[] args) throws Exception {
        try (RandomAccessFile raf = new RandomAccessFile("ledger.dat", "rw");
             FileChannel channel = raf.getChannel()) {

            // READ_ONLY: safe for reporting
            MappedByteBuffer readOnly = channel.map(
                FileChannel.MapMode.READ_ONLY, 0, 512);
            long balance = readOnly.getLong(0);
            System.out.println("Read-only balance: " + balance);

            // PRIVATE: copy-on-write for what-if simulation
            MappedByteBuffer pvt = channel.map(
                FileChannel.MapMode.PRIVATE, 0, 512);
            pvt.putLong(0, 0L); // Simulation: zero out
            System.out.println("Private write does not affect file.");
        }
    }
}

Read-only balance: 999900
Private write does not affect file.

What is the size limitation and how does the Foreign Memory API help?

The size argument of FileChannel.map is a long, but MappedByteBuffer extends ByteBuffer, which indexes with int. This means a single mapping is limited to about two gigabytes. For files larger than that, you must create multiple mappings over different regions.

The Foreign Function and Memory API (finalized in Java 22) removes this ceiling. You can map a file as a MemorySegment through FileChannel.map(mode, offset, size, arena), which supports the full long range and provides deterministic unmapping when the arena closes.

package com.bank.io;

import java.io.RandomAccessFile;
import java.lang.foreign.Arena;
import java.lang.foreign.MemorySegment;
import java.lang.foreign.ValueLayout;
import java.nio.channels.FileChannel;

public class ForeignMemoryMap {
    public static void main(String[] args) throws Exception {
        try (RandomAccessFile raf = new RandomAccessFile("ledger.dat", "rw");
             FileChannel channel = raf.getChannel();
             Arena arena = Arena.ofConfined()) {

            // Map the full file as a MemorySegment (no 2 GB limit)
            MemorySegment segment = channel.map(
                FileChannel.MapMode.READ_WRITE, 0,
                channel.size(), arena);

            segment.set(ValueLayout.JAVA_LONG, 0, 750000L);
            long val = segment.get(ValueLayout.JAVA_LONG, 0);
            System.out.println("Segment value: " + val);
        } // Arena close unmaps immediately
    }
}

Segment value: 750000

Why is unmapping a problem with the old API?

There is no public unmap() method on MappedByteBuffer. The mapping is released only when the buffer is garbage-collected and its internal Cleaner runs, which is non-deterministic. This means:

• The file may stay locked longer than expected.
• On Windows, you may not be able to delete the file until the buffer is collected.
• Disk space can remain pinned.

The Foreign Memory API solves this cleanly. Closing the Arena unmaps the region immediately and deterministically.

// Old API: no way to unmap explicitly
MappedByteBuffer buf = channel.map(FileChannel.MapMode.READ_ONLY, 0, 1024);
// buf is released only when GC collects it

// New API: deterministic unmap
try (Arena arena = Arena.ofConfined()) {
    MemorySegment seg = channel.map(
        FileChannel.MapMode.READ_ONLY, 0, 1024, arena);
    // use seg...
} // unmap happens here, immediately

How does durability work with memory-mapped files?

Because write-back is lazy, data you wrote to a MappedByteBuffer may still be only in memory, not on disk. A crash before the OS flushes would lose that data. In banking, this is unacceptable for committed records.

The solution is calling buffer.force(), which asks the OS to flush all dirty pages in the mapped region to disk. You must call force() before you can claim data is durably persisted.

package com.bank.io;

import java.io.RandomAccessFile;
import java.math.BigDecimal;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class TransactionDurability {
    public static void main(String[] args) throws Exception {
        try (RandomAccessFile raf = new RandomAccessFile("txn-log.dat", "rw");
             FileChannel channel = raf.getChannel()) {

            MappedByteBuffer buffer = channel.map(
                FileChannel.MapMode.READ_WRITE, 0, 4096);

            // Write committed transfer: Rajesh to Priya, 25000.00
            long amountPaise = new BigDecimal("25000.00")
                .movePointRight(2).longValue();
            buffer.putLong(0, amountPaise);

            // MUST force before reporting success
            buffer.force();
            System.out.println("Transfer committed and flushed.");
        }
    }
}

Transfer committed and flushed.

When should you use memory-mapped files?

Use memory mapping in three situations:

• Very large files accessed randomly. A database file, an index, or a large ledger where you access records by offset. Only the pages you touch are loaded.
• High-throughput I/O. Systems like Kafka and Lucene map their segment files to avoid per-read system calls and copies.
• Inter-process shared memory. Two processes mapping the same file in READ_WRITE mode see each other's writes through the same physical pages, giving fast IPC without sockets.

When should you NOT use memory-mapped files?

Avoid mapping when:

• The file is small and fits comfortably in a byte[]. The overhead of setting up mappings is not worth it.
• You need strict durability guarantees on every write. The lazy write-back model requires explicit force() calls, which are expensive.
• You need deterministic resource cleanup and are stuck on older Java without the Foreign Memory API.
• The file changes size frequently. Mapping a region that grows requires remapping.

What about thread safety?

A MappedByteBuffer is not thread-safe. Concurrent reads and writes to the same buffer require external synchronization. However, multiple threads can each map their own non-overlapping regions of the same file safely. For a concurrent banking service, guard access with a lock or use separate mappings per thread.

package com.bank.io;

import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.util.concurrent.locks.ReentrantLock;

public class ThreadSafeLedger {
    private final MappedByteBuffer buffer;
    private final ReentrantLock lock = new ReentrantLock();

    public ThreadSafeLedger(FileChannel channel) throws Exception {
        this.buffer = channel.map(
            FileChannel.MapMode.READ_WRITE, 0, 8192);
    }

    public void updateBalance(int offset, long paise) {
        lock.lock();
        try {
            buffer.putLong(offset, paise);
            buffer.force();
        } finally {
            lock.unlock();
        }
    }

    public long readBalance(int offset) {
        lock.lock();
        try {
            return buffer.getLong(offset);
        } finally {
            lock.unlock();
        }
    }
}

What are common interview follow-up questions?

• How does Kafka use memory-mapped files for its log segments?
• What is the difference between MappedByteBuffer.force() and FileChannel.force()?
• How does the Foreign Memory API improve over the old mapping API?
• What happens if the file is truncated while a mapping is active?
• How do you handle files larger than 2 GB with the old API?