Back to Indexing
Q16Question 16 of 18

Why can random UUIDv4 primary keys cause index fragmentation and slower inserts, and how do time-ordered IDs fix this

Key Concepts
  • Random UUIDv4 keys scatter inserts across the B-tree, causing frequent page splits, index fragmentation, and poor cache locality.
  • Auto-increment keys always append to the rightmost page, keeping the tree dense, sequential, and insert-friendly.
  • In clustered engines like InnoDB, random primary keys scatter both the index and the physical row data, and the 16-byte key bloats every secondary index.
  • Time-ordered UUIDv7 and ULID embed a leading timestamp so new values are approximately monotonically increasing, restoring insert locality.
  • UUIDv7 gives three properties at once: global uniqueness, approximate time ordering, and no coordination.
  • Always store UUIDs as a compact 16-byte binary type, not a 36-character string, to avoid tripling the key size.

Definition

Random UUIDv4 scatters inserts across the B-tree causing splits and fragmentation; time-ordered UUIDv7 or ULID keeps inserts near the right edge like auto-increment, while staying globally unique.
  • UUIDv4 – A fully random 128-bit identifier; consecutive inserts land at random positions in the B-tree, causing page splits and fragmentation.
  • Auto-increment – A monotonically increasing integer from a database sequence; every new key appends to the rightmost page, keeping the tree dense.
  • Page split – When a B-tree leaf page is full and a new key must be inserted in the middle, the engine allocates a new page and moves half the entries, leaving both pages half-full.
  • UUIDv7 – A 128-bit identifier with a 48-bit Unix millisecond timestamp prefix, giving approximately monotonic ordering and insert locality while remaining globally unique.
  • ULID – Similar to UUIDv7: 48-bit timestamp prefix + random suffix, sortable and coordination-free.

Imagine the National Bank of India files new account folders into a cabinet that must stay in sorted order. With auto-increment numbers, every new folder goes neatly at the very back – fast and tidy. With random UUIDv4 numbers, each new folder must be jammed into a random spot in the middle of the cabinet, forcing the clerk to shove existing folders aside to make room (a page split), and the cabinet ends up full of half-empty, messy drawers (fragmentation). A time-ordered UUIDv7 is like a number that starts with today's timestamp, so new folders again cluster near the back in time order, keeping the cabinet tidy, while still being globally unique.

Understanding the Concepts

Why does auto-increment work well for B-tree inserts?

A B-tree primary key index is kept in sorted key order. With auto-increment, each new key is larger than all previous ones, so it always lands on the rightmost leaf page. That page fills up and a fresh page is appended – a cheap, sequential operation.

SQL
-- Auto-increment: every INSERT appends to the rightmost page
CREATE TABLE transactions_auto (
    txn_id     BIGSERIAL PRIMARY KEY,  -- auto-increment
    account_id VARCHAR(20) NOT NULL,
    amount     DECIMAL(15,2) NOT NULL,
    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);

-- Insert 1: txn_id = 1 -> rightmost page
-- Insert 2: txn_id = 2 -> same rightmost page
-- Insert 3: txn_id = 3 -> same rightmost page
-- Page fills up -> new page appended at the end
-- Result: dense, sequential, no fragmentation

Why does random UUIDv4 cause page splits?

A UUIDv4 is random across its whole range. Each new key is equally likely to belong anywhere in the tree. When the target leaf page is full, the engine performs a page split.

SQL
-- Random UUIDv4: every INSERT targets a random leaf page
CREATE TABLE transactions_uuid (
    txn_id     UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    account_id VARCHAR(20) NOT NULL,
    amount     DECIMAL(15,2) NOT NULL,
    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);

-- Insert 1: txn_id = 'f47ac10b-...' -> middle of tree
-- Insert 2: txn_id = '1a2b3c4d-...' -> left side of tree
-- Insert 3: txn_id = '9x8y7z6w-...' -> different spot
-- Each insert may cause a page split in a different location
TEXT
Page split process:
  1. New key 'f47ac10b' targets leaf page #42
  2. Page #42 is full
  3. Engine allocates new page #500
  4. Moves half of page #42's entries to page #500
  5. Both pages are now ~50% full
  6. Wasted space + scattered pages = fragmentation

What is the impact of fragmentation?

Random inserts cause several problems.

| Problem | Auto-increment | Random UUIDv4 |
|---|---|---|
| Insert pattern | Sequential (rightmost page) | Random (any page) |
| Page splits | Rare (only at the end) | Frequent (everywhere) |
| Page fill factor | ~90-100% | ~50-70% |
| Cache locality | Excellent (hot pages stay in memory) | Poor (working set is scattered) |
| Index size | Compact | Bloated (half-full pages) |
| Write throughput | High | Lower |

How does InnoDB make it worse?

In InnoDB the primary key is the clustered index, so random primary keys scatter the actual row data too. And every secondary index stores the primary key as its row pointer, so a 16-byte UUID bloats every secondary index.

SQL
-- Bad: 36-byte UUID string as PK (common mistake)
CREATE TABLE payments_bad (
    payment_id CHAR(36) PRIMARY KEY,  -- 36 bytes!
    account_id VARCHAR(20),
    amount     DECIMAL(15,2)
);
-- Every secondary index entry carries 36 bytes for the PK pointer
-- Index is 4.5x larger than with BIGINT (8 bytes)

-- Better: native UUID type (16 bytes)
CREATE TABLE payments_better (
    payment_id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    account_id VARCHAR(20),
    amount     DECIMAL(15,2)
);
-- Still random, but at least 16 bytes instead of 36

How do time-ordered IDs fix the problem?

UUIDv7 places a 48-bit Unix-millisecond timestamp in the high bits. Because the leading bits increase with real time, new keys land near the right edge of the tree, restoring insert locality.

SQL
-- UUIDv7: timestamp prefix makes inserts approximately sequential
CREATE TABLE transactions_v7 (
    txn_id     UUID PRIMARY KEY,  -- UUIDv7 generated in app
    account_id VARCHAR(20) NOT NULL,
    amount     DECIMAL(15,2) NOT NULL,
    created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
);

-- UUIDv7 values generated over time:
-- 2026-10-25 14:30:00 -> '0192a3b4-...' (timestamp prefix)
-- 2026-10-25 14:30:01 -> '0192a3b5-...' (slightly larger)
-- 2026-10-25 14:30:02 -> '0192a3b6-...' (slightly larger)
-- All land near the rightmost page, like auto-increment
JAVA
// com.bank.service.PaymentIdGenerator
package com.bank.service;

import java.util.UUID;

public class PaymentIdGenerator {

    /**
     * Generate a UUIDv7 with timestamp prefix.
     * Java 17+ does not have built-in UUIDv7,
     * so use a library like java-uuid-generator or uuid-creator.
     *
     * The key property: new IDs are approximately monotonically
     * increasing, so B-tree inserts stay near the right edge.
     */
    public static UUID generateV7() {
        // Using com.github.f4b6a3:uuid-creator library
        // return UuidCreator.getTimeOrderedEpoch();

        // Manual implementation for illustration:
        long timestamp = System.currentTimeMillis();
        long msb = (timestamp << 16) | 0x7000L; // version 7
        long lsb = (0x8000000000000000L)         // variant
                  | (long)(Math.random() * Long.MAX_VALUE);
        return new UUID(msb, lsb);
    }
}

How does UUIDv7 compare to auto-increment and UUIDv4?

| Feature | Auto-increment | UUIDv4 | UUIDv7 |
|---|---|---|---|
| Insert locality | Perfect | Random | Near-sequential |
| Page splits | Rare | Frequent | Rare |
| Global uniqueness | No (needs DB) | Yes | Yes |
| Coordination needed | Yes (sequence) | No | No |
| Time ordering | Yes | No | Approximate |
| Size | 4-8 bytes | 16 bytes | 16 bytes |
| Guessable | Yes (enumerable) | No | Partially (timestamp) |

What about ULID?

ULID is similar to UUIDv7: 48-bit timestamp prefix plus random suffix, lexicographically sortable, and coordination-free.

TEXT
ULID format:
  01ARZ3NDEKTSV4RRFFQ69G5FAV
  |-----------||-----------|
  Timestamp     Randomness
  (48 bits)     (80 bits)

ULID is string-sortable, which is convenient for systems that store IDs as text. UUIDv7 is the standardized equivalent (RFC 9562).

What are common mistakes?

The most common mistake is storing UUIDs as 36-character strings instead of native 16-byte binary.

SQL
-- Wrong: 36-byte string, wastes space, bloats secondary indexes
CREATE TABLE payments_wrong (
    id CHAR(36) PRIMARY KEY  -- '550e8400-e29b-41d4-a716-446655440000'
);

-- Correct: native UUID type (16 bytes)
CREATE TABLE payments_correct (
    id UUID PRIMARY KEY  -- stored as 16-byte binary
);

What should a senior engineer know?

  • UUIDv7 gives approximate ordering, not a strict global sequence, because clocks on different nodes can skew.
  • The timestamp prefix means UUIDv7 leaks creation time – it is an identifier, not a secret or authentication token.
  • Monitor index bloat with pg_stat_user_tables and pgstattuple to detect fragmentation from random inserts.
  • After switching from UUIDv4 to UUIDv7, measure insert throughput and index size to confirm the improvement.

What are common interview follow-up questions?

  • What is the UUIDv7 binary structure (48-bit timestamp, version, variant, random)?
  • When should you choose a UUID key versus an auto-increment key?
  • How does InnoDB embed the primary key in secondary indexes?
  • What is the difference between UUIDv7 and Snowflake IDs?

Visual Overview

DIAGRAM
┌─────────────────────────────────────────────────┐
│   Insert Patterns: Auto-inc vs UUIDv4 vs UUIDv7 │
├─────────────────────────────────────────────────┤
│                                                 │
│  Auto-increment (sequential):                   │
│  B-tree pages: [1-100][101-200][201-300]  →     │
│  New key: 301 → appends to rightmost page       │
│  Pages: ~100% full, dense, no splits            │
│                                                 │
│  UUIDv4 (random):                               │
│  B-tree pages: [a..f][g..m][n..z]               │
│  New key: 'h3x...' → lands in middle page       │
│  Page split! → two ~50% full pages              │
│  Result: fragmented, half-empty, scattered      │
│                                                 │
│  UUIDv7 (time-ordered):                         │
│  B-tree pages: [0190..][0191..][0192..]  →      │
│  New key: 0192a3b7... → near rightmost page     │
│  Pages: ~90%+ full, near-sequential             │
│  Result: dense like auto-inc, unique like UUID   │
└─────────────────────────────────────────────────┘

Key Takeaway

Random UUIDv4 primary keys cause problems because a primary-key index is a B-tree kept in sorted order, and a fully random key is equally likely to belong anywhere in that tree. So every insert targets a random leaf page, and when that page is full the engine performs a page split, moving half the entries to a new page. Random inserts trigger frequent splits all over the tree, which leaves it fragmented with many half-empty pages, wastes space, and ruins cache locality. In a clustered engine like InnoDB it is worse, because the primary key also dictates the physical row order, so the actual data scatters too, and the 16-byte random key bloats every secondary index that embeds it.

Auto-increment avoids all of this because each new key is larger than the last, so it always appends to the rightmost page – dense, sequential, and cheap. The catch is that auto-increment needs a central database sequence, which is awkward in distributed systems. Time-ordered identifiers like UUIDv7 and ULID solve both sides: they put a 48-bit millisecond timestamp in the high bits, so new values are approximately monotonically increasing and land near the right edge of the tree just like auto-increment, while still being globally unique and generated with no central coordinator.

At the National Bank of India the payments team moved transaction keys to random UUIDv4 for distributed generation and saw insert throughput fall and the index bloat badly. Rajesh switched them to UUIDv7, which kept the global uniqueness and coordination-free generation they needed but restored insert locality, so page splits dropped and fragmentation largely disappeared. Crucially he also made sure the id was stored as a native 16-byte uuid rather than a 36-character string, because an early attempt stored it as CHAR(36) and tripled the key size, which had defeated most of the benefit.

The nuance Pankaj always closes with is that the timestamp prefix means a UUIDv7 leaks its creation time, so it is an identifier, never a secret or an authentication token, and the team always verifies the improvement by measuring index size and insert latency before and after the change.

Remember one sentence: random UUIDv4 scatters inserts across the B-tree causing splits and fragmentation, while time-ordered UUIDv7 keeps inserts near the right edge like auto-increment, staying globally unique.

---

Share
Use ← → arrow keys to navigate