Case study · Platform Audit
A CI pipeline the team stopped fighting.
Quarantined flaky tests and parallelized the suite so green finally meant green.
The challenge
A 47-minute pipeline wasn’t the real problem — the real problem was nobody believed it.
Flaky tests had piled up for over a year, to the point where a red build triggered an automatic rerun, not an investigation. Engineers batched PRs to avoid the wait, which meant reviews queued up and merges slowed down even further.
What we did
Made the pipeline fast enough, and honest enough, to trust again.
- Quarantined the worst offenders. Identified the top 20 flakiest tests by rerun frequency and pulled them into a non-blocking suite while they got fixed properly.
- Parallelized everything parallelizable. Split the suite across workers instead of running it as one long serial job.
- Cached aggressively. Dependency and build caching cut redundant work on every run, not just the fast path.
- Made red mean red. Once flakiness dropped, a failing build became something worth stopping for again — and the team started treating it that way.
The result
Pipeline time dropped from 47 minutes to 9. Daily reruns from flaky tests fell from 40-plus to a handful. With reviews no longer queued behind a slow, untrusted pipeline, the team nearly quadrupled the number of PRs merged per day — without lowering the bar on what got merged.