What 'regression' means here
A fix regresses if a previously-passing acceptance or canary test fails inside 30 days of merge, attributable to the fix's blast radius. Reverts, follow-on patches, and roll-backs all count. Tests added after merge to lock in the fix do not.
2,840
Scored fixes
Across all three difficulty bands
11.9%
Overall regression rate
30-day post-merge window
3.4×
Hard vs Easy band
Hard tier regresses 3.4× more often
−47%
After replay coaching
Bottom-tercile, 8-week intervention
The full grid
The heatmap is the headline. Regression rate climbs both diagonally (harder bands × harder categories) and along distinct hot stripes — security tasks regress more often at every band, not just the hardest.
| Debug-single | Debug-dist | Sec-authz | Sec-inj | Refactor | Feature | Review | |
|---|---|---|---|---|---|---|---|
| Easy | 2.1% | 4.8% | 6.9% | 5.4% | 3.2% | 4.1% | 1.8% |
| Medium | 4.6% | 9.2% | 13.1% | 11.7% | 8.4% | 9.9% | 4.2% |
| Hard | 9.4% | 17.8% | 24.6% | 22.1% | 15.7% | 19.3% | 8.1% |
The feedback loop, charted
A platform regression is not a punishment — it is the signal that closes the loop on rubric scoring. The diagram shows how a single shipped fix flows back to the candidate's cohort drift dashboard.
Movement over 16 weeks
Coaching against AI failure patterns (taxonomy here) closes most of the band gap by week 8. The bottom-tercile curve drops fastest because they have the most room to move.
What this tells hiring teams
Band is a starting point, not a verdict. The slope matters more than the intercept — a candidate who closes the regression gap in 8 weeks outperforms one who started lower but never moved. The replay record makes the slope visible.