Your architecture worked well at 10× the original traffic: why might you still replace it? | Microservices

Definition

"It still handles the load" is only one constraint; replace an architecture when today's constraints -- change cost, ownership, operability, infrastructure cost, compliance, or relevance -- are the ones it fails, not the scaling problem it was built to solve.

Change Velocity – how quickly the organization can ship new features; a rigid architecture costs the business its ability to compete.
Ownership Fragmentation – critical parts owned by nobody, creating latent outages and fear of change.
Operability / Specialist Knowledge – if incident recovery depends on one or two people, the system is fragile in the moments it must not be.
Disproportionate Cost – infrastructure cost growing faster than traffic, imposing a tax that a different design would remove.
Regulatory Compliance – evolving rules like data residency, audit retention, or right to erasure that the current design cannot satisfy.
Relevance – a component solving a problem the company no longer has, adding pure carrying cost.

National Bank of India built a specialized high-speed cash-sorting hall years ago to handle a flood of coin deposits, and it still sorts coins beautifully at ten times the volume. But the bank barely takes coins anymore: customers moved to digital payments. The hall solves a problem the bank does not have, costs a fortune to run, and only one retired engineer knows how to repair it. The hall "works," but keeping it is a mistake.

Understanding the Concepts

Why is throughput not the only constraint?

Scaling well is necessary but not sufficient. Throughput is the most visible constraint, so teams fixate on it. But an organization is constrained by many things at once, and an architecture that aces throughput can fail every other constraint that matters now.

JAVA

package com.bank.architecture;

import java.util.Map;

public class ConstraintEvaluation {
    public static void main(String[] args) {
        Map<String, String> constraints = Map.of(
            "Throughput",    "PASS: handles 10x traffic",
            "Change velocity", "FAIL: new feature takes 4 months",
            "Ownership",     "FAIL: no team owns the settlement engine",
            "Operability",   "FAIL: only Pankaj can recover from incidents",
            "Cost",          "FAIL: 10x traffic costs 50x money",
            "Compliance",    "FAIL: cannot satisfy right-to-erasure regulation",
            "Relevance",     "FAIL: batch pipeline for reports nobody reads"
        );

        constraints.forEach((k, v) -> System.out.println(k + ": " + v));
        System.out.println();
        System.out.println("Verdict: 1 pass, 6 fails. Architecture must change.");
    }
}

TEXT

Throughput: PASS: handles 10x traffic
Change velocity: FAIL: new feature takes 4 months
Ownership: FAIL: no team owns the settlement engine
Operability: FAIL: only Pankaj can recover from incidents
Cost: FAIL: 10x traffic costs 50x money
Compliance: FAIL: cannot satisfy right-to-erasure regulation
Relevance: FAIL: batch pipeline for reports nobody reads

Verdict: 1 pass, 6 fails. Architecture must change.

What is the change velocity problem?

If every new feature takes months because the architecture is rigid, tightly coupled, or full of special cases, the architecture costs the business its ability to compete. The metric that matters is lead time for change, not requests per second.

JAVA

package com.bank.architecture;

public class ChangeVelocity {
    public static void main(String[] args) {
        int featureLeadTimeDays = 120; // 4 months
        int competitorLeadTimeDays = 14;

        System.out.println("Our lead time: " + featureLeadTimeDays + " days");
        System.out.println("Competitor lead time: " + competitorLeadTimeDays + " days");
        System.out.println("Ratio: " + (featureLeadTimeDays / competitorLeadTimeDays) + "x slower");
        System.out.println("The architecture scales requests, but not feature delivery");
    }
}

TEXT

Our lead time: 120 days
Competitor lead time: 14 days
Ratio: 8x slower
The architecture scales requests, but not feature delivery

What is the ownership fragmentation problem?

Over years, teams reorganize, people leave, and parts of the system end up owned by nobody. An orphaned critical component is a latent outage.

JAVA

package com.bank.architecture;

public class OwnershipFragmentation {
    public static void main(String[] args) {
        System.out.println("Settlement engine: owner = NOBODY");
        System.out.println("  Last meaningful change: 2 years ago");
        System.out.println("  Documentation: outdated");
        System.out.println("  On-call response when it breaks: confusion, finger-pointing");
        System.out.println("  Changes feared because no one fully understands it");
        System.out.println();
        System.out.println("A replacement that re-establishes clear ownership is worth it");
        System.out.println("even if the current design performs well");
    }
}

TEXT

Settlement engine: owner = NOBODY
  Last meaningful change: 2 years ago
  Documentation: outdated
  On-call response when it breaks: confusion, finger-pointing
  Changes feared because no one fully understands it

A replacement that re-establishes clear ownership is worth it
even if the current design performs well

What is the specialist knowledge risk?

If recovering from an incident requires deep, rare expertise that only one person holds, the system is fragile in exactly the moments it must not be.

JAVA

package com.bank.architecture;

public class SpecialistKnowledgeRisk {
    public static void main(String[] args) {
        int mttrWithExpertMinutes = 15;
        int mttrWithoutExpertMinutes = 480; // 8 hours

        System.out.println("MTTR with Pankaj available: " + mttrWithExpertMinutes + " min");
        System.out.println("MTTR without Pankaj: " + mttrWithoutExpertMinutes + " min");
        System.out.println("Ratio: " + (mttrWithoutExpertMinutes / mttrWithExpertMinutes) + "x worse");
        System.out.println();
        System.out.println("An architecture that a normal on-call engineer can operate");
        System.out.println("with a runbook is more valuable than a faster one only a guru can revive");
    }
}

TEXT

MTTR with Pankaj available: 15 min
MTTR without Pankaj: 480 min
Ratio: 32x worse

An architecture that a normal on-call engineer can operate
with a runbook is more valuable than a faster one only a guru can revive

What about disproportionate cost growth?

An architecture can scale technically while its cost per transaction climbs. If serving ten times the traffic costs fifty times the money, the architecture is scaling but not economically.

JAVA

package com.bank.architecture;

import java.math.BigDecimal;
import java.math.RoundingMode;

public class CostGrowth {
    public static void main(String[] args) {
        BigDecimal originalCost = new BigDecimal("100000"); // per month
        BigDecimal currentCost = new BigDecimal("5000000");
        int trafficMultiplier = 10;
        BigDecimal costMultiplier = currentCost.divide(originalCost, 1, RoundingMode.HALF_UP);

        System.out.println("Traffic grew: " + trafficMultiplier + "x");
        System.out.println("Cost grew: " + costMultiplier + "x");
        System.out.println("Cost per transaction: " + costMultiplier.divide(
                BigDecimal.valueOf(trafficMultiplier), 1, RoundingMode.HALF_UP) + "x higher");
        System.out.println("The architecture scales requests but not economics");
    }
}

TEXT

Traffic grew: 10x
Cost grew: 50.0x
Cost per transaction: 5.0x higher
The architecture scales requests but not economics

What about regulatory change?

Banks operate under evolving rules. An architecture built before a rule existed may be unable to comply.

JAVA

package com.bank.architecture;

public class RegulatoryChange {
    public static void main(String[] args) {
        System.out.println("New regulation: Right to Erasure (data deletion on request)");
        System.out.println("Current architecture: append-only event log");
        System.out.println("Problem: append-only CANNOT delete a customer's data");
        System.out.println("Compliance is non-negotiable");
        System.out.println("Architecture MUST change regardless of how well it scales");
    }
}

TEXT

New regulation: Right to Erasure (data deletion on request)
Current architecture: append-only event log
Problem: append-only CANNOT delete a customer's data
Compliance is non-negotiable
Architecture MUST change regardless of how well it scales

How do you evaluate architecture against today's constraints?

The reasoning that justified the design years ago is historical and sunk. List the current binding constraints, score the architecture against each, and recognize that "it still handles the load" answers only the throughput constraint.

JAVA

package com.bank.architecture;

public class TodayVsYesterday {
    public static void main(String[] args) {
        System.out.println("Yesterday's justification: 'We needed to handle the coin flood'");
        System.out.println("Today's binding constraints:");
        System.out.println("  1. Feature lead time (competitive pressure)");
        System.out.println("  2. Ownership clarity (operational risk)");
        System.out.println("  3. Cost efficiency (margin pressure)");
        System.out.println("  4. Compliance (regulatory mandate)");
        System.out.println();
        System.out.println("The sunk cost of the old architecture is not a reason to keep it.");
        System.out.println("Evaluate against TODAY's constraints, not yesterday's justification.");
    }
}

TEXT

Yesterday's justification: 'We needed to handle the coin flood'
Today's binding constraints:
  1. Feature lead time (competitive pressure)
  2. Ownership clarity (operational risk)
  3. Cost efficiency (margin pressure)
  4. Compliance (regulatory mandate)

The sunk cost of the old architecture is not a reason to keep it.
Evaluate against TODAY's constraints, not yesterday's justification.

What are common interview follow-up questions?

How do you weigh the cost and risk of replacement versus keeping the current architecture?
What is the difference between replacing for genuine constraint failure versus aesthetic preference for newer technology?
How do you plan a migration from a working-but-constrained architecture to a new one?
Which constraint failures justify the highest urgency (hint: compliance is non-negotiable)?

Visual Overview

DIAGRAM

  Architecture Constraint Evaluation
  ──────────────────────────────────

  ┌───────────────┐     ┌──────────────────┐
  │  Yesterday's   │     │  Today's Binding  │
  │  Justification │     │  Constraints      │
  │  (sunk, done)  │     │  (active, real)   │
  └───────────────┘     └──────┬───────────┘
                               │
         ┌─────────────────────┼────────────────────┐
         │                     │                    │
    Throughput           Change Velocity        Compliance
      PASS                  FAIL                  FAIL
                               │
         ┌─────────────────────┼────────────────────┐
         │                     │                    │
    Ownership             Operability             Cost
      FAIL                  FAIL                  FAIL

  "It still handles the load" answers ONE constraint.
  Architecture must be judged against ALL of today's.

Key Takeaway

The fact that an architecture handles ten times its original traffic tells you it passes the throughput constraint, and nothing more. An organization is constrained by many things at once: change velocity, ownership clarity, operability, cost, compliance, and relevance. A design that aces scaling can fail every other constraint that matters now.

The senior skill is noticing that the binding constraint has moved. An architecture may need replacing because new features take months (change velocity), critical parts are orphaned (ownership), only one person can recover it (operability), cost grew 50x for 10x traffic (economics), regulations changed (compliance), or a component solves a problem the company no longer has (relevance).

The framing is to evaluate against today's constraints, not yesterday's justification. The reasoning that justified the design years ago is historical and sunk. The replacement must solve a constraint that is genuinely hurting the business today, not satisfy an aesthetic preference for newer technology.

Remember one sentence: "it still handles the load" answers only the throughput question, and architecture must be judged against the constraints the organization has now.

---