Definition
- Git serialisation – Git accepts pushes one at a time; each commit lands at a unique position with a unique SHA.
- Independent runs – the CI system triggers a separate pipeline per commit, each on its own fresh agent, building in parallel.
- Concurrency control – shared stateful resources (deploy slots, licenses) are protected so only one run uses them at a time.
- Parallel stages – independent stages within one pipeline run at the same time in a fan-out/fan-in pattern.
- Matrix build – a strategy that auto-generates one job per combination of declared parameters (e.g., Java 17 x 21 cross transfer x fraud = 4 jobs).
Imagine two tellers at the National Bank of India each submitting a batch of cheques at the same moment. The clearing system gives each batch its own serial number and processes them on separate machines at once. But there is only one armoured van to the vault, so they take turns. A matrix build is testing a new cheque scanner against every combination of paper type and ink colour at once: you list the combinations and the system runs them all in parallel.
Understanding the Concepts
How does Git handle concurrent pushes?
Git on the server accepts pushes one at a time. If Rajesh and Priya push to the same branch simultaneously, one wins and the other is rejected with a non-fast-forward error. If they push to different branches, both succeed independently.
# Rajesh pushes to main
git push origin main
# Success: commit abc1234
# Priya pushes to main at the same time
git push origin main
# Error: non-fast-forward. Priya must pull and rebase.
git pull --rebase origin main
git push origin main
# Success: commit def5678
Every commit ends up at a unique position with a unique SHA, so there is never ambiguity about what each pipeline builds.
How does the CI system handle concurrent runs?
The CI system triggers one independent run per triggering event. Two concurrent pushes produce two independent runs, each on its own fresh agent.
# GitHub Actions: each push triggers its own run
on:
push:
branches: [main]
jobs:
build:
runs-on: ubuntu-latest # Each run gets its own fresh runner
steps:
- uses: actions/checkout@v4
- run: mvn clean verify
On Jenkins-on-Kubernetes, each run gets its own pod. The two runs build in parallel with no shared workspace and cannot corrupt each other.
How do you protect shared resources?
The build and test phases are naturally isolated, but some resources are singular: a shared QA deploy slot, a limited license count, or an external system that cannot handle two simultaneous writers.
# GitHub Actions: concurrency group ensures only one deploy at a time
jobs:
deploy-qa:
concurrency:
group: deploy-qa # Only one run in this group at a time
cancel-in-progress: true # Cancel older run if a newer one starts
steps:
- run: kubectl apply -f qa/
// Jenkins: lock step serialises access to a shared resource
pipeline {
agent any
stages {
stage('Build') { steps { sh 'mvn package' } } // Runs in parallel
stage('Deploy QA') {
steps {
lock('qa-deploy-slot') { // Only one job at a time
sh 'kubectl apply -f qa/'
}
}
}
}
}
How do parallel stages work inside one pipeline?
Independent stages like unit tests, integration tests, and a security scan can run at the same time because they do not depend on each other's output.
// Jenkins: parallel stages cut wall-clock time
pipeline {
agent any
stages {
stage('Build') { steps { sh 'mvn clean compile' } }
stage('Parallel Checks') {
parallel {
stage('Unit Tests') { steps { sh 'mvn test' } }
stage('Integration Tests') { steps { sh 'mvn verify -Pintegration' } }
stage('Security Scan') { steps { sh 'mvn sonar:sonar' } }
}
}
stage('Deploy') { steps { sh 'kubectl apply -f deploy/' } }
}
}
# GitHub Actions: parallel jobs with no needs dependency
jobs:
unit-tests:
runs-on: ubuntu-latest
steps: [{run: 'mvn test'}]
integration-tests:
runs-on: ubuntu-latest
steps: [{run: 'mvn verify -Pintegration'}]
security-scan:
runs-on: ubuntu-latest
steps: [{run: 'mvn sonar:sonar'}]
deploy:
needs: [unit-tests, integration-tests, security-scan] # Fan-in
runs-on: ubuntu-latest
steps: [{run: 'kubectl apply -f deploy/'}]
How does a matrix build work?
A matrix declares dimensions and the CI system generates one job per combination automatically.
# GitHub Actions: matrix across Java versions and services
jobs:
build:
strategy:
matrix:
java: [17, 21]
service: [transfer-service, fraud-service]
fail-fast: false # Do not cancel others on one failure
max-parallel: 4 # Cap concurrent jobs
runs-on: ubuntu-latest
steps:
- uses: actions/setup-java@v4
with: { java-version: '${{ matrix.java }}', distribution: temurin }
- run: mvn clean verify -pl ${{ matrix.service }}
This generates four jobs: transfer-service on Java 17, transfer-service on Java 21, fraud-service on Java 17, fraud-service on Java 21. All four run in parallel.
What are common mistakes with concurrency?
# WRONG: no concurrency group on deploy.
# Two merges to main both try to deploy at the same time.
# The environment may end up on the older commit if it finishes last.
jobs:
deploy:
steps:
- run: kubectl apply -f prod/
# CORRECT: concurrency group ensures the newer commit deploys last.
jobs:
deploy:
concurrency:
group: deploy-prod
cancel-in-progress: false # Let the newer one finish
steps:
- run: kubectl apply -f prod/
# WRONG: unbounded matrix swamps the runner fleet.
strategy:
matrix:
service: [s1, s2, s3, s4, s5, s6, s7, s8, s9, s10] # 10 services
java: [17, 21] # x 2 = 20 jobs!
# CORRECT: cap with max-parallel.
strategy:
matrix:
service: [s1, s2, s3, s4, s5, s6, s7, s8, s9, s10]
java: [17, 21]
max-parallel: 6 # Only 6 run at once
How do parallel tests expose shared state bugs?
Running tests in parallel can surface tests that share state, like a static field, a fixed port, or the same database row.
// WRONG: tests share a static counter.
public class TransferServiceTest {
private static int transferCount = 0;
@Test
void testTransferA() {
transferCount++;
assertEquals(1, transferCount); // Fails if another test ran first!
}
}
// CORRECT: each test has its own state.
public class TransferServiceTest {
@Test
void testTransferA() {
int localCount = 0;
localCount++;
assertEquals(1, localCount); // Always passes.
}
}
What should a senior engineer know?
- Concurrent deploys to the same environment must be ordered so the newest commit wins and the environment ends on the latest code.
- Cap matrix expansion with
max-paralleland sensible dimension sizes to avoid swamping a shared CI fleet. - Parallel tests expose any test that secretly shares state, which must be made independent.
- The
cancel-in-progressoption on concurrency groups can cancel older runs, which is useful for CI but dangerous for CD where you might cancel a production deploy.
Visual Overview
Concurrent pushes:
Rajesh push ──▶ commit abc1234 ──▶ Pipeline Run 1 (Pod A)
Priya push ──▶ commit def5678 ──▶ Pipeline Run 2 (Pod B)
│
Both run in parallel │
on separate agents │
▼
Shared QA slot ──▶ Lock: one at a time
Parallel stages (fan-out / fan-in):
┌─── Unit Tests ────────┐
Build ────▶ ├─── Integration Tests ─┤ ────▶ Deploy
└─── Security Scan ─────┘
Matrix build:
java: [17, 21] x service: [transfer, fraud]
▼
┌────────────────┐ ┌────────────────┐
│ transfer + J17 │ │ transfer + J21 │
└────────────────┘ └────────────────┘
┌────────────────┐ ┌────────────────┐
│ fraud + J17 │ │ fraud + J21 │
└────────────────┘ └────────────────┘
All 4 run in parallel
Key Takeaway
When two developers push at the same time, nothing gets mixed up because Git serialises the writes and the CI system triggers one independent run per commit, each on its own fresh agent. The two runs build in parallel with no shared workspace. The only thing that needs care is shared stateful resources like a QA deploy slot, which I protect with a concurrency group or lock so only one run uses it at a time.
Parallelism inside a single run speeds up the critical path. Independent stages like unit tests, integration tests, and a security scan run simultaneously in a fan-out, and the pipeline continues only when all finish (fan-in). A matrix build is parameterised parallelism: I declare dimensions like Java versions crossed with services, and the CI system generates one parallel job per combination automatically.
The senior concerns are ordering and cost. Concurrent deploys to main must be ordered so the newest commit deploys last and the environment ends on the latest code. I cap matrix expansion with max-parallel so a huge matrix does not swamp the CI fleet, and I make tests independent so parallel execution does not surface shared-state bugs.
Remember one sentence: concurrent pushes get independent parallel runs, shared resources get locks, and a matrix auto-expands one job across every parameter combination.
---