Back to Technical STAR Scenarios
Q4Question 4 of 4

A Mistake That Affected Production: how did you fix and prevent it?

Key Concepts
  • Own the mistake plainly without excuses or blame-shifting.
  • Follow the incident order: contain, fix, communicate, prevent.
  • Contain the blast radius first, then correct the affected data.
  • Communicate honestly to stakeholders at every stage.
  • Build a systemic guardrail (dry-run, second approval) not just a personal promise to be careful.
  • Show that hiding a production mistake is far worse than the mistake itself.
  • The preventive control is the most important part of the answer.

Definition

In production, contain first, fix second, communicate throughout, and prevent recurrence last : and own every step.
  • Full ownership – Admit the mistake plainly; do not blame the requirements, the deadline, or a teammate.
  • Contain first – Stop the bad process, disable the feature, or roll back before fixing.
  • Communicate honestly – Keep stakeholders informed at every stage; hiding is worse than the mistake.
  • Systemic prevention – Build a guardrail (dry-run, second approval, automated rollback) that prevents the class of mistake, not just a promise to be careful.
  • Blameless mindset – Focus on fixing the system so humans cannot make the same error again.

Imagine a developer at National Bank of India accidentally ran a data update that set the wrong interest flag on a batch of accounts including Priya's. Fixing a production mistake is like a bank catching a wrong charge: you stop the bleeding first, correct the affected accounts, communicate honestly, and then add a control so that kind of update can never run unguarded again.

Understanding the Concepts

What is this question really testing?

The interviewer wants accountability, composure, and a systemic fix. Everyone makes mistakes; the question is whether you own it, contain it calmly, and build a guardrail so it cannot repeat.

JAVA
package com.bank.interview;

public class MistakeTest {
    enum Signal { MATURE, DEFENSIVE }

    public static Signal evaluate(boolean ownedIt, boolean contained, boolean builtGuardrail) {
        return (ownedIt && contained && builtGuardrail) ? Signal.MATURE : Signal.DEFENSIVE;
    }

    public static void main(String[] args) {
        System.out.println(evaluate(true, true, true)); // MATURE
    }
}

How do you contain the blast radius?

Stop any further processing, identify exactly which accounts were affected, and revert them using audit records.

JAVA
package com.bank.service;

import java.util.logging.Logger;

public class BlastRadiusContainment {
    private static final Logger log = Logger.getLogger(BlastRadiusContainment.class.getName());

    public static void main(String[] args) {
        log.info("Step 1: Stop further processing of bulk update");
        log.info("Step 2: Identify affected accounts from audit log");
        log.info("Step 3: Revert wrong interest flag using audit records");
        log.info("Blast radius contained: 150 accounts corrected");
    }
}

What was the actual mistake?

A bulk update to correct an interest flag had a slightly too broad filter, touching more accounts than intended at National Bank of India. The update went straight to production with no preview and no second check.

JAVA
package com.bank.service;

public class TheMistake {
    public static void main(String[] args) {
        // The filter was too broad
        String intendedFilter = "WHERE account_type = 'SAVINGS' AND region = 'NORTH'";
        String actualFilter   = "WHERE account_type = 'SAVINGS'"; // missing region!
        System.out.println("Intended: " + intendedFilter);
        System.out.println("Actual:   " + actualFilter);
        System.out.println("Result: touched ALL savings accounts, not just North region");
    }
}

Why is a systemic guardrail better than a personal promise?

"I will double-check next time" is weak because it depends on human memory. "Every bulk update now requires a dry run and a second approval" is strong because it is a system.

JAVA
package com.bank.service;

import java.util.List;
import java.util.logging.Logger;

public class DryRunGuardrail {
    private static final Logger log = Logger.getLogger(DryRunGuardrail.class.getName());

    record AccountChange(String accountId, String before, String after) {}

    public static void main(String[] args) {
        // Dry run: show what WOULD change without applying
        List<AccountChange> preview = List.of(
            new AccountChange("SAV2001", "REGULAR", "PREMIUM"),
            new AccountChange("SAV2002", "REGULAR", "PREMIUM"),
            new AccountChange("SAV9999", "REGULAR", "PREMIUM") // unexpected!
        );
        log.info("DRY RUN: " + preview.size() + " accounts would change:");
        preview.forEach(c -> log.info("  " + c.accountId() + ": " + c.before() + " -> " + c.after()));
        log.warning("SAV9999 is unexpected! Filter too broad. Aborting.");
    }
}
TEXT
DRY RUN: 3 accounts would change:
  SAV2001: REGULAR -> PREMIUM
  SAV2002: REGULAR -> PREMIUM
  SAV9999: REGULAR -> PREMIUM
SAV9999 is unexpected! Filter too broad. Aborting.

How do you communicate the mistake honestly?

Keep your manager and the operations team informed at every step. In a bank, hiding a production mistake is far worse than the mistake itself.

JAVA
package com.bank.interview;

public class HonestCommunication {
    public static void main(String[] args) {
        System.out.println("[10:05] Discovered: bulk update filter was too broad");
        System.out.println("[10:10] Contained: stopped processing, identified 150 affected accounts");
        System.out.println("[10:25] Corrected: all 150 accounts reverted to correct flag");
        System.out.println("[10:30] Informed manager and ops with full timeline");
    }
}

What is the permanent fix?

Every bulk update now runs a mandatory dry run first. The actual change only applies after a second person explicitly confirms the preview.

JAVA
package com.bank.service;

public class BulkUpdateProcess {
    public static void main(String[] args) {
        System.out.println("New process for bulk data updates:");
        System.out.println("  1. Write the update query");
        System.out.println("  2. Run DRY RUN: see exactly which accounts change");
        System.out.println("  3. SECOND PERSON reviews and confirms the preview");
        System.out.println("  4. Only then: apply the change");
        System.out.println("  A wrong filter gets caught in step 2, not in production.");
    }
}

How do you show ownership without being self-flagellating?

State the mistake factually, own it plainly, and spend most of the answer on the fix and prevention. Do not dwell on guilt; the interviewer wants to see competence, not suffering.

JAVA
package com.bank.interview;

public class OwnershipBalance {
    public static void main(String[] args) {
        System.out.println("Time allocation in answer:");
        System.out.println("  20% - What happened (factual, owned)");
        System.out.println("  30% - How I contained and fixed it");
        System.out.println("  50% - The systemic prevention I built");
        System.out.println("Focus: competence and learning, not guilt");
    }
}

What are common interview follow-up questions?

Expect: "What if you had not caught it?" (describe the impact and why the guardrail prevents that), "Did the team adopt the process?" (yes, for all bulk operations), and "What was your manager's reaction?" (appreciated honesty and the systemic fix).

JAVA
package com.bank.interview;

public class MistakeFollowUps {
    public static void main(String[] args) {
        System.out.println("Not caught? -> Wrong flags on 10K accounts, regulatory risk");
        System.out.println("Team adopted? -> Yes, dry-run-first is now standard for all bulk ops");
        System.out.println("Manager reaction? -> Appreciated honesty + the systemic fix");
    }
}

Visual Overview

DIAGRAM
┌────────────────────────────────────────────────────────┐
│         Production Mistake Recovery Flow                │
├────────────────────────────────────────────────────────┤
│                                                        │
│  ┌──────────────────┐                                  │
│  │ 1. OWN IT         │  "My filter was too broad.       │
│  │    (factual)      │   It was my mistake."            │
│  └────────┬─────────┘                                  │
│           │                                            │
│           ▼                                            │
│  ┌──────────────────┐                                  │
│  │ 2. CONTAIN        │  Stop processing, identify       │
│  │    (blast radius) │  affected accounts, revert       │
│  └────────┬─────────┘                                  │
│           │                                            │
│           ▼                                            │
│  ┌──────────────────┐                                  │
│  │ 3. COMMUNICATE    │  Manager + ops informed           │
│  │    (throughout)   │  with full timeline               │
│  └────────┬─────────┘                                  │
│           │                                            │
│           ▼                                            │
│  ┌──────────────────┐                                  │
│  │ 4. PREVENT        │  Dry-run-first process            │
│  │    (systemic)     │  + second-person confirmation     │
│  │                   │  Wrong filter caught in preview   │
│  └──────────────────┘                                  │
└────────────────────────────────────────────────────────┘

Key Takeaway

At National Bank of India, I once ran a bulk update to correct an interest flag on a set of accounts, and my filter was slightly too broad, so it touched more accounts than intended, including some it should never have affected like Priya's. The change went straight to production with no preview and no second check. It was my error, plain and simple.

My first priority was to contain the damage. I stopped further processing, identified exactly which accounts had been wrongly changed using the audit records, and reverted them to their correct state. Throughout, I kept my manager and the operations team informed, because in a bank hiding a production mistake is far worse than the mistake itself.

Once the accounts were corrected, I fixed the real root cause. The problem was not simply that I made a filter mistake : humans will always make filter mistakes. The problem was that a bulk operation on customer data could run with no preview and no confirmation. So I built a guardrail: every bulk update now runs a mandatory dry run first that reports exactly which accounts would change, and the actual change only applies after a second person explicitly confirms it.

After that, a wrong filter gets caught harmlessly in the dry run instead of in production. What I learned is that owning a production mistake means containing it calmly, communicating honestly, and then turning the lesson into a systemic control so the same class of mistake simply cannot happen again.

Remember one sentence: Own production mistakes plainly, contain first, communicate honestly, and build a systemic guardrail so the same class of error is caught before production.

---

Share
Use ← → arrow keys to navigate