(Scenario) A Playwright suite passes locally but fails in CI

• Root principle – Identical code behaving differently means the environments differ. Debug by comparison, not by guess-and-edit.
• CI evidence – Gather logs, on-failure screenshots, video, and the Playwright trace showing DOM, network, and console at the failure point.
• Comparison dimensions – Node/Playwright/browser versions, environment variables, OS, test data, network access, authentication, time zone, file paths, CPU/memory.
• Reproduce locally – Match CI by running in a Linux container with the same versions, variables, and time zone.
• Never randomly edit – Adding sleeps, loosening assertions, or retrying hides real defects, is fragile, and destroys coverage.

What is the key debugging principle?

Identical test code behaving differently in two places means the environments differ. This is a comparison exercise, not a guess-and-edit exercise.

  Local: macOS, Chrome 121, IST, secrets in .env, 16GB RAM
  CI:    Linux, Chromium 121, UTC, secrets from vault, 4GB RAM

  Same test code. Different result.
  -> The difference IS the bug.

How do you gather CI evidence?

Playwright produces rich diagnostics. Start with the trace, which usually reveals the failure's nature.

// Enable diagnostics in config
export default defineConfig({
  use: {
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

# CI: upload artifacts for diagnosis
- name: Upload test artifacts
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: test-results
    path: test-results/

# Open the trace locally
npx playwright show-trace test-results/transfer-test/trace.zip
# The trace shows DOM snapshots, network, and console at each step

What dimensions should you compare?

Check each dimension systematically.

| Dimension | Local | CI | How to Fix |
|---|---|---|---|
| Node version | 20.10 | 18.17 | Pin in .nvmrc or CI config |
| Playwright version | 1.42 | 1.40 | Use same lockfile |
| Browser | Chrome 121 | Chromium 121 | Use Playwright's bundled browsers |
| OS | macOS | Linux | Account for font, rendering differences |
| Env vars | .env file | CI vault | Ensure all vars are injected |
| Base URL | localhost:3000 | staging.bank.in | Parameterize via config |
| Time zone | IST (UTC+5:30) | UTC | Set TZ explicitly |
| Auth secrets | .env | CI secrets | Verify injection |
| Test data | Seeded locally | Empty DB in CI | Create data per test |
| Network | Direct access | Behind firewall | Check connectivity |
| CPU/Memory | 16GB, 8 cores | 4GB, 2 cores | Right-size workers |

How do you fix common culprits?

Time zone: set it explicitly in CI.

# Fix time zone in CI
env:
  TZ: Asia/Kolkata

// Or write timezone-aware assertions
const expected = new Date().toLocaleDateString('en-IN', {
  timeZone: 'Asia/Kolkata',
});
await expect(page.getByTestId('date')).toContainText(expected);

Missing env vars: verify they are injected.

# Ensure all secrets are available
env:
  BASE_URL: ${{ vars.STAGING_URL }}
  TEST_USER: ${{ secrets.TEST_USER }}
  TEST_PASS: ${{ secrets.TEST_PASS }}

Version mismatch: pin versions.

// package.json: pin Playwright version
{
  "devDependencies": {
    "@playwright/test": "1.42.1"
  }
}

How do you reproduce the CI environment locally?

Run your tests in a container that matches CI.

# Run in a Playwright Docker container matching CI
docker run --rm -it \
  -v $(pwd):/work -w /work \
  -e TZ=UTC \
  -e BASE_URL=https://staging.nationalbank.in \
  -e TEST_USER=$TEST_USER \
  -e TEST_PASS=$TEST_PASS \
  mcr.microsoft.com/playwright:v1.42.1 \
  npx playwright test

Why is randomly editing the test a bad fix?

Three bad outcomes from random edits.

1. HIDES REAL DEFECTS
   A genuine timezone bug also affects users.
   Adding a sleep hides it from the test but not from customers.

2. IS FRAGILE
   A sleep that happens to work today breaks under different load.
   A loosened assertion passes on a broken page.

3. DESTROYS COVERAGE
   Loosening "expect exact amount" to "expect page loaded"
   removes the financial-correctness check that gave the test value.

What is the systematic debugging process?

1. Gather CI evidence: logs, screenshot, video, trace.
2. Open the trace viewer: identify what failed and how.
3. Compare environments dimension by dimension.
4. Reproduce locally in a matching container.
5. Fix the real environmental difference.
6. Verify the fix in CI.

What are common interview follow-up questions?

• "What is the most common cause?" Missing environment variables or time zone differences.
• "How do you prevent this from happening?" Pin versions, use containers, inject all vars, set time zone explicitly.
• "What if you cannot reproduce locally?" Add extra logging keyed by a correlation ID and wait for the next CI failure with full diagnostics.