Why did the industry move beyond synchronous REST for long-running workflows | Microservices

Q: What should a senior engineer know?

A senior engineer uses the acceptthenprocess pattern not just for performance but for operational safety. A durable PENDING record means failed jobs can be retried from the queue without asking the customer to resubmit. The idempotency key prevents financial doubleprocessing. The queue enables indep

Definition

Synchronous REST ties up a thread for the entire job and exhausts the pool under load, so for long work we accept-then-process, return 202 with a handle, and finish asynchronously.

Thread blocking – synchronous REST holds a thread from request arrival until response, even though the thread is mostly idle waiting on I/O.
Thread-pool exhaustion – when all threads are blocked on slow work, new requests queue and time out even at low CPU.
Accept-then-process – validate, persist a PENDING record, enqueue the job, and return HTTP 202 Accepted immediately.
Tracking handle – the 202 response includes a status URL or job ID the client can poll or subscribe to.
Shock absorber – the queue absorbs bursts while consumers drain at a steady rate.
Idempotency – the client attaches a key so retrying the accept call does not create duplicate work.

Imagine Rajesh asks the bank teller to process a complex international transfer that takes ten minutes of back-office checks. In the synchronous model the teller stands frozen at the counter for ten minutes, and the queue behind Rajesh grows until the whole branch jams. In the asynchronous model the teller takes the form, hands Rajesh a token number, says "we will notify you", and immediately serves the next customer while the back office does the slow work. The counter never freezes.

Understanding the Concepts

Why does synchronous REST fail for long-running work?

A server has a bounded thread pool. In synchronous REST, one request occupies one thread from arrival until response. For a workflow that takes seconds or minutes, that thread is held the whole time, and it is mostly idle waiting on I/O. You can exhaust all threads while the CPU sits at five percent.

JAVA

package com.bank.transfer;

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import java.math.BigDecimal;

@RestController
public class SyncTransferController {

    // WRONG: holds thread for entire international transfer
    @PostMapping("/api/transfers/international")
    public TransferResult transfer(@RequestBody TransferRequest req) {
        // Step 1: validate (10ms)
        validate(req);
        // Step 2: fraud check (2s)
        fraudService.check(req.getFrom(), req.getAmount());
        // Step 3: SWIFT messaging (5-30s)
        swiftGateway.send(req);
        // Step 4: correspondent bank (10-60s)
        correspondentBank.settle(req);
        // Thread blocked for up to 90 seconds!
        return TransferResult.success();
    }
}

TEXT

Thread pool exhaustion:
  Pool size: 200 threads
  Each international transfer: 30-90 seconds
  200 concurrent transfers -> all threads blocked
  Simple balance check arrives -> no thread available -> timeout
  CPU utilization: 5% (threads idle, just waiting on I/O)
  Service appears DOWN even though machine is idle!

What is the accept-then-process pattern?

The endpoint does only the fast part: validate, persist a PENDING record, enqueue, and return immediately with HTTP 202 Accepted and a tracking handle.

JAVA

package com.bank.transfer;

import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RestController;
import java.net.URI;

@RestController
public class AsyncTransferController {

    private final TransferJobService jobService;

    public AsyncTransferController(TransferJobService jobService) {
        this.jobService = jobService;
    }

    @PostMapping("/api/transfers/international")
    public ResponseEntity<TransferJob> transfer(
            @RequestHeader("Idempotency-Key") String idempotencyKey,
            @RequestBody TransferRequest req) {

        // Fast path: validate + persist + enqueue (~10ms)
        TransferJob job = jobService.accept(idempotencyKey, req);

        // Return 202 Accepted with tracking URL
        return ResponseEntity
            .status(HttpStatus.ACCEPTED)
            .location(URI.create("/api/transfers/" + job.getId() +
                "/status"))
            .body(job);
    }
}

JAVA

package com.bank.transfer;

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class TransferJobService {

    private final TransferJobRepository jobRepo;
    private final KafkaTemplate<String, TransferJob> kafka;

    public TransferJobService(TransferJobRepository jobRepo,
                               KafkaTemplate<String, TransferJob> kafka) {
        this.jobRepo = jobRepo;
        this.kafka = kafka;
    }

    @Transactional
    public TransferJob accept(String idempotencyKey,
                               TransferRequest req) {
        // Idempotency: return existing job if key already used
        TransferJob existing = jobRepo.findByIdempotencyKey(idempotencyKey);
        if (existing != null) {
            return existing;
        }

        // Create PENDING job
        TransferJob job = new TransferJob(
            idempotencyKey, req, TransferStatus.PENDING);
        jobRepo.save(job);

        // Enqueue for background processing
        kafka.send("transfer-jobs", job.getId(), job);

        return job;
    }
}

TEXT

HTTP Response:
  HTTP/1.1 202 Accepted
  Location: /api/transfers/TXF-12345/status
  Content-Type: application/json

  {
    "id": "TXF-12345",
    "status": "PENDING",
    "message": "Transfer accepted for processing"
  }

  Thread released in ~10ms (not 30-90 seconds!)

How does the client track progress?

The client either polls the status endpoint or receives a webhook/push notification when the work completes.

JAVA

package com.bank.transfer;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class TransferStatusController {

    private final TransferJobRepository jobRepo;

    public TransferStatusController(TransferJobRepository jobRepo) {
        this.jobRepo = jobRepo;
    }

    @GetMapping("/api/transfers/{jobId}/status")
    public TransferJob getStatus(@PathVariable String jobId) {
        return jobRepo.findById(jobId)
            .orElseThrow(() -> new NotFoundException("Job not found"));
    }
}

TEXT

Client polling:
  t=0s:   POST /transfers/international -> 202 PENDING
  t=5s:   GET /transfers/TXF-12345/status -> PROCESSING
  t=30s:  GET /transfers/TXF-12345/status -> COMPLETED
  Or:
  Webhook: POST https://client.bank.com/callbacks/TXF-12345
           {"status": "COMPLETED", "settlementRef": "SWIFT-789"}

What does async actually improve?

Async improves four things: throughput, decoupling, failure isolation, and user experience.

DIAGRAM

| Aspect | Synchronous REST | Accept-then-Process |
|---|---|---|
| Thread held | Entire duration | ~10ms |
| Throughput | Limited by thread pool | Limited by queue capacity |
| Failure handling | Client must retry whole flow | Failed job retried from queue |
| Scaling | API and workers coupled | API and workers scale independently |
| User experience | Spinner for 30-90 seconds | Immediate acknowledgment |
| Duplicate risk | Client timeout -> resubmit | Idempotency key -> safe retry |

How does the queue act as a shock absorber?

The queue absorbs bursts while consumers drain at a steady rate. This smooths spikes instead of forcing every request to wait.

TEXT

Queue as shock absorber:
  Morning spike: 500 international transfers submitted in 1 minute
  Producer rate: 500/min (all accepted, all get 202)
  Consumer rate: 100/min (5 workers processing steadily)
  Queue depth: 400 pending (drains in ~4 minutes)

  Without queue: 500 threads blocked -> pool exhausted -> everything fails
  With queue: all 500 accepted in seconds, processed over 5 minutes

Why is idempotency critical for the accept endpoint?

The client may retry the accept call if the 202 response is lost. The idempotency key ensures a retry does not create a duplicate job.

JAVA

package com.bank.transfer;

// WRONG: no idempotency key
// Client timeout -> retry -> two jobs created -> double transfer!
public TransferJob acceptWithoutIdempotency(TransferRequest req) {
    TransferJob job = new TransferJob(req, TransferStatus.PENDING);
    jobRepo.save(job);
    kafka.send("transfer-jobs", job);
    return job;
}

// CORRECT: idempotency key
// Client timeout -> retry with same key -> returns existing job
public TransferJob acceptWithIdempotency(String idempotencyKey,
                                          TransferRequest req) {
    TransferJob existing = jobRepo.findByIdempotencyKey(idempotencyKey);
    if (existing != null) return existing; // Safe retry
    TransferJob job = new TransferJob(idempotencyKey, req);
    jobRepo.save(job);
    kafka.send("transfer-jobs", job);
    return job;
}

How is this different from reactive/non-blocking REST?

Reactive frameworks free the thread during I/O waits and help with throughput, but they do not solve the semantics of genuinely long-running, multi-minute workflows where accept-then-process with a durable job record is still the right model.

TEXT

Reactive REST:
  Thread freed during I/O wait -> handles more concurrent requests
  But: if SWIFT takes 30 seconds, the CLIENT still waits 30 seconds
  Good for: short I/O-bound calls (milliseconds)
  Not enough for: multi-minute workflows

Accept-then-process:
  Client gets 202 in 10ms and goes about its business
  Work happens in background at its own pace
  Right model for: anything > 1-2 seconds

What should a senior engineer know?

A senior engineer uses the accept-then-process pattern not just for performance but for operational safety. A durable PENDING record means failed jobs can be retried from the queue without asking the customer to resubmit. The idempotency key prevents financial double-processing. The queue enables independent scaling of the API and workers.

JAVA

package com.bank.transfer;

public class SeniorGuidelines {
    public static void main(String[] args) {
        System.out.println("Senior guidelines for async workflows:");
        System.out.println("  1. Any work > 2 seconds: accept-then-process");
        System.out.println("  2. Always return a tracking handle (job ID or status URL)");
        System.out.println("  3. Always require an idempotency key for financial operations");
        System.out.println("  4. Persist PENDING record BEFORE enqueuing (crash-safe)");
        System.out.println("  5. Workers are independently scalable from the API");
        System.out.println("  6. Dead-letter queue for jobs that fail after N retries");
        System.out.println("  7. Monitor: queue depth, processing latency, DLQ size");
    }
}

TEXT

Expected output:
Senior guidelines for async workflows:
  1. Any work > 2 seconds: accept-then-process
  2. Always return a tracking handle (job ID or status URL)
  3. Always require an idempotency key for financial operations
  4. Persist PENDING record BEFORE enqueuing (crash-safe)
  5. Workers are independently scalable from the API
  6. Dead-letter queue for jobs that fail after N retries
  7. Monitor: queue depth, processing latency, DLQ size

Visual Overview

DIAGRAM

  Synchronous REST (WRONG for long work):
  Client ──POST──> Thread blocked 30-90s ──> Response
                   (pool exhausted)

  Accept-then-Process (CORRECT):
  Client ──POST──> Validate + Persist + Enqueue ──> 202 Accepted
                   (10ms, thread freed)              + status URL
                              │
                              ▼
                   ┌─────────────────────┐
                   │    Kafka Queue      │
                   │  (shock absorber)   │
                   └──────────┬──────────┘
                              │
                   ┌──────────▼──────────┐
                   │  Background Workers  │
                   │  SWIFT, fraud, etc.  │
                   │  (scale independently)│
                   └──────────┬──────────┘
                              │
                              ▼
                   Update status: COMPLETED
                   Notify client (webhook or poll)

Key Takeaway

The industry moved beyond synchronous REST for long-running workflows because of thread-pool exhaustion. A server has a bounded pool of worker threads, and synchronous REST holds one thread per request until the response is written. For a workflow taking seconds or minutes, the thread is mostly idle waiting on I/O, so you can exhaust all threads while the CPU sits nearly idle, and simple balance checks start failing even though the machine is not overloaded.

The fix is the accept-then-process pattern: validate the input, persist a durable PENDING record with an idempotency key, enqueue the job, and return HTTP 202 Accepted with a status URL. The request thread is freed in milliseconds. The queue acts as a shock absorber, absorbing bursts while background workers drain at a steady rate. The client either polls the status endpoint or receives a webhook when the work completes.

At National Bank of India, international transfers involving SWIFT and correspondent banks took 30 to 90 seconds. The synchronous endpoint held threads for the whole duration, and during a morning spike the pool filled while simple balance checks started failing. We rebuilt it as accept-then-process with an idempotency key, and the API stayed fast, duplicates disappeared, and operations could retry failed jobs from the durable record.

Remember one sentence: synchronous REST exhausts the thread pool under long-running load even at low CPU, so for long work return 202 with a tracking handle and complete asynchronously, which gives throughput, decoupling, and a responsive API.

---