Back to Indexing
Q7Question 7 of 18

When does a database actually use an index: WHERE, JOIN, ORDER BY, GROUP BY

Key Concepts
  • A cost-based optimizer uses an index when the index provides a cheaper path than a full table scan, based on table size, selectivity, and statistics.
  • In a WHERE clause the index is used when the predicate is selective and sargable, meaning the indexed column is compared directly to a constant or range.
  • In a JOIN the index on the join column lets the engine use an index nested-loop join instead of scanning the inner table for each outer row.
  • In an ORDER BY the index is used when its sort order matches the requested order, skipping an explicit sort step entirely.
  • In a GROUP BY a sorted index places equal keys adjacently so the engine groups them in a single pass, and MIN/MAX read just the first or last entry.
  • Stale statistics after a large data load can cause the optimizer to ignore a perfect index, so running ANALYZE matters.

Definition

The database uses an index whenever it makes filtering, joining, sorting, or grouping cheaper than a full table scan – and only then.
  • Cost-based optimizer – The query planner estimates the cost of different access paths and picks the cheapest one, using table size, selectivity, and current statistics.
  • WHERE usage – An index is used when the predicate is selective and sargable: the indexed column is compared directly to a constant or range.
  • JOIN usage – An index on the join column lets the engine do an index nested-loop join, seeking matching rows instead of scanning.
  • ORDER BY usage – If the index is sorted in the requested order, the engine reads rows straight from the index and skips the explicit sort step.
  • GROUP BY usage – A sorted index places equal keys adjacently so the engine groups them in one pass, and MIN/MAX read just the first or last entry.
  • Statistics dependency – The decision is statistical, based on up-to-date statistics; stale statistics can cause the optimizer to ignore a perfect index.

The National Bank of India query planner behaves like a smart clerk deciding how to find records. If you ask for one specific account (WHERE SAV2001), the clerk uses the index thumb-tabs because flipping to one page beats reading the whole ledger. If you ask to match the accounts ledger against the transactions ledger (JOIN), the clerk uses the index on the join column to pair them quickly. If you ask for records already in date order (ORDER BY created_at), the clerk hands over the index which is already sorted, skipping the sorting step. If you ask "total per branch" (GROUP BY branch), the clerk walks the branch index so all same-branch rows are already next to each other.

Understanding the Concepts

When does the optimizer choose an index?

The query planner is cost-based. It estimates the number of rows a predicate will match (using column statistics), calculates the cost of an index scan versus a sequential scan, and picks the cheaper path. The decision is not a rule – it is a calculation based on the current data distribution.

SQL
-- National Bank of India: accounts table with 10 million rows
-- Index on branch_code exists

-- High selectivity: only 50 accounts in branch 'AND001'
-- Optimizer chooses Index Scan (cheaper than reading 10M rows)
EXPLAIN ANALYZE
SELECT * FROM accounts WHERE branch_code = 'AND001';
TEXT
Index Scan using ix_accounts_branch on accounts
  Index Cond: (branch_code = 'AND001')
  Rows: 50  Cost: 12.5
SQL
-- Low selectivity: 9.5 million accounts are 'ACTIVE'
-- Optimizer chooses Seq Scan (cheaper than 9.5M random lookups)
EXPLAIN ANALYZE
SELECT * FROM accounts WHERE status = 'ACTIVE';
TEXT
Seq Scan on accounts
  Filter: (status = 'ACTIVE')
  Rows: 9,500,000  Cost: 180000

How does an index help a WHERE clause?

An index helps a WHERE clause when the predicate is sargable – the indexed column appears bare on one side and is compared to a constant or range.

SQL
-- Sargable: direct comparison on indexed column
SELECT txn_id, amount
FROM   transactions
WHERE  account_id = 'SAV2001';  -- Index Scan

-- Sargable: range on indexed column
SELECT txn_id, amount
FROM   transactions
WHERE  created_at >= '2026-01-01'
  AND  created_at <  '2026-04-01';  -- Index Range Scan

-- NOT sargable: function on indexed column
SELECT txn_id, amount
FROM   transactions
WHERE  YEAR(created_at) = 2026;  -- Seq Scan (index defeated)

The optimizer estimates how many rows match. If the fraction is small, it seeks the index. If the fraction is large, it may prefer a sequential scan because random index lookups cost more than one sequential pass.

How does an index help a JOIN?

An index on the join column of the inner table lets the engine use an index nested-loop join. For each row in the outer table, the engine seeks the matching rows in the inner table through the index.

SQL
-- National Bank of India: join accounts to transactions
-- Index on transactions(account_id) enables index nested-loop join
CREATE INDEX ix_txn_account ON transactions(account_id);

EXPLAIN ANALYZE
SELECT a.customer_name, t.amount, t.created_at
FROM   accounts a
JOIN   transactions t ON t.account_id = a.account_id
WHERE  a.branch_code = 'MUM001';
TEXT
Nested Loop
  -> Index Scan using ix_accounts_branch on accounts a
       Index Cond: (branch_code = 'MUM001')
  -> Index Scan using ix_txn_account on transactions t
       Index Cond: (account_id = a.account_id)

Without the index on transactions.account_id, the engine would scan the entire transactions table for every matching account – far more expensive.

How does an index help ORDER BY?

If the index is sorted in the requested order, the engine reads rows straight from the index and skips the explicit sort step entirely. This is a big win on large result sets.

SQL
-- Composite index on (account_id, created_at)
CREATE INDEX ix_txn_acct_date
    ON transactions(account_id, created_at);

-- The index is already sorted by created_at within each account
-- No sort step needed
EXPLAIN ANALYZE
SELECT txn_id, amount, created_at
FROM   transactions
WHERE  account_id = 'SAV2001'
ORDER BY created_at DESC
LIMIT 10;
TEXT
Limit
  -> Index Scan Backward using ix_txn_acct_date on transactions
       Index Cond: (account_id = 'SAV2001')
  -- No Sort node in the plan

If the sort order does not match the index, the engine must sort in memory or spill to disk.

SQL
-- Index is on (account_id, created_at)
-- But query orders by amount – no match
SELECT txn_id, amount, created_at
FROM   transactions
WHERE  account_id = 'SAV2001'
ORDER BY amount DESC;
-- Plan includes: Sort (cost = extra)

How does an index help GROUP BY and aggregates?

A sorted index places equal keys adjacently, so the engine can group them in a single pass without hashing or sorting.

SQL
-- Index on branch_code
CREATE INDEX ix_accounts_branch ON accounts(branch_code);

-- GROUP BY uses the sorted index
EXPLAIN ANALYZE
SELECT branch_code, COUNT(*), SUM(balance)
FROM   accounts
GROUP BY branch_code;
TEXT
GroupAggregate
  -> Index Scan using ix_accounts_branch on accounts
  -- All rows for each branch are contiguous in the index

MIN and MAX on an indexed column read just the first or last entry.

SQL
-- MIN reads the first leaf entry
SELECT MIN(created_at) FROM transactions;
-- Plan: Index Scan (reads 1 entry, not 200M rows)

-- MAX reads the last leaf entry
SELECT MAX(balance) FROM accounts;
-- Plan: Index Scan Backward (reads 1 entry)

What is an index-only scan?

When all the columns the query needs are already in the index, the engine never touches the table heap at all. This is the fastest access path.

SQL
-- Covering index: includes amount in INCLUDE clause
CREATE INDEX ix_txn_cover ON transactions(account_id, created_at)
    INCLUDE (amount);

-- All needed columns are in the index
EXPLAIN ANALYZE
SELECT amount, created_at
FROM   transactions
WHERE  account_id = 'SAV2001'
  AND  created_at >= '2026-01-01';
TEXT
Index Only Scan using ix_txn_cover on transactions
  Index Cond: (account_id = 'SAV2001'
    AND created_at >= '2026-01-01')
  Heap Fetches: 0

What happens when statistics are stale?

The optimizer relies on statistics to estimate row counts and selectivity. After a large data load, statistics can be stale, causing the optimizer to make wrong decisions.

SQL
-- After loading 5 million new transactions
-- Statistics still say the table has 1 million rows
-- Optimizer underestimates cost and may choose wrong plan

-- Fix: update statistics
ANALYZE transactions;

-- PostgreSQL auto-analyzes, but a manual ANALYZE after bulk loads
-- ensures the optimizer has fresh data immediately
JAVA
// com.bank.service.DataLoadService
package com.bank.service;

import java.sql.*;

public class DataLoadService {

    public void refreshStatisticsAfterBulkLoad(Connection conn)
            throws SQLException {
        // Always ANALYZE after a large data load
        // so the optimizer has fresh statistics
        try (Statement stmt = conn.createStatement()) {
            stmt.execute("ANALYZE transactions");
            stmt.execute("ANALYZE accounts");
        }
    }
}

What should a senior engineer know?

  • The optimizer is cost-based and probabilistic. It does not guarantee index usage – it picks the cheapest estimated plan.
  • An index-only scan is the ideal: all needed columns in the index, zero heap fetches.
  • After large data loads, manually run ANALYZE to refresh statistics.
  • Use EXPLAIN ANALYZE (not just EXPLAIN) to see actual execution times and row counts, not just estimates.
  • In PostgreSQL, SET enable_seqscan = off; forces index usage for testing, but never use this in production.
SQL
-- Debugging: force index usage to compare costs
SET enable_seqscan = off;
EXPLAIN ANALYZE
SELECT * FROM accounts WHERE status = 'ACTIVE';
-- Shows the index scan cost (likely higher than seq scan)
RESET enable_seqscan;

What are common interview follow-up questions?

  • What makes a predicate sargable versus non-sargable?
  • When does the optimizer ignore an existing index?
  • What is the left-prefix rule for composite indexes?
  • How does a covering index eliminate bookmark lookups?

Visual Overview

DIAGRAM
┌─────────────────────────────────────────────────┐
│     When Does the Optimizer Use an Index?        │
├─────────────────────────────────────────────────┤
│                                                 │
│  Query arrives                                  │
│       │                                         │
│       ▼                                         │
│  ┌──────────────────────┐                       │
│  │ Parse & analyze      │                       │
│  │ predicates           │                       │
│  └──────────┬───────────┘                       │
│             ▼                                   │
│  ┌──────────────────────┐                       │
│  │ Check available      │                       │
│  │ indexes & statistics │                       │
│  └──────────┬───────────┘                       │
│             ▼                                   │
│  ┌──────────────────────┐                       │
│  │ Estimate cost of     │                       │
│  │ each access path     │                       │
│  └──────────┬───────────┘                       │
│             ▼                                   │
│  ┌────────────────────────────────────┐         │
│  │ Selectivity < threshold?           │         │
│  │   YES ──► Index Scan (few rows)    │         │
│  │   NO  ──► Seq Scan (most rows)     │         │
│  └────────────────────────────────────┘         │
│                                                 │
│  WHERE ──► Index Scan if sargable + selective   │
│  JOIN  ──► Index Nested Loop on join column     │
│  ORDER BY ──► Index Scan if sort matches        │
│  GROUP BY ──► GroupAggregate on sorted index     │
│  MIN/MAX  ──► Read first/last index entry        │
└─────────────────────────────────────────────────┘

Key Takeaway

A database uses an index only when its cost-based optimizer decides the index is a cheaper way to satisfy a clause than scanning the whole table, and that decision is statistical rather than guaranteed. The four classic triggers are a selective, sargable predicate in a WHERE clause, a join condition on an indexed column, a sort in ORDER BY that matches the index order, and grouping or aggregates in GROUP BY, MIN, and MAX that can be answered straight from the sorted index.

In a WHERE clause the planner estimates how many rows match. If the fraction is small it seeks the index, and if it is large it may prefer a scan. In a join it uses the index on the join column to look up matching rows per outer row instead of scanning. In an ORDER BY it reads straight from a matching index and skips the explicit sort, which is a big win on large result sets. In a GROUP BY a sorted index places equal keys next to each other so the engine groups them in one pass, and MIN or MAX can be answered just by reading the first or last index entry.

At the National Bank of India Rajesh used this knowledge to design indexes around actual report queries. The fraud screen filters by account and amount, so he indexed amount. The statement screen orders by date within an account, so he used a composite index on account and date that removes the sort entirely. The branch dashboard groups totals by branch, so he indexed the branch column. Each report then showed an index scan in the plan instead of a sequential scan.

The nuance Pankaj always mentions is that the optimizer depends on fresh statistics. After a large data load, run ANALYZE, because stale statistics can make the planner wrongly estimate selectivity and fall back to a full scan even when a perfect index exists. He verifies the final plan with EXPLAIN ANALYZE rather than trusting that the index will be used.

Remember one sentence: the database uses an index whenever it makes filtering, joining, sorting, or grouping cheaper than a full table scan – and only then.

---

Share
Use ← → arrow keys to navigate