Most of what gets written about working with AI in code is about the tool. The model, the IDE plugin, the latest prompt trick. The job has moved past the tool. The engineer who ships well in 2026 is doing something different from the engineer who shipped well in 2022, and the difference is not measured in tokens per minute. It is measured in defensible decisions per pull request, in regression rate at six weeks, in whether the engineer can replay how a change happened to a reviewer who was not in the room. This guide is the canonical entry point for that shift — what the skill is, how it is measured, where it breaks, and how to learn it on purpose.
Why this matters
The hiring panel at every large engineering org we work with has rewritten its rubric in the last 18 months. Not the headline — the rubric. What gets scored has moved. Correctness still matters, but every shortlisted candidate writes runnable code now. The discriminator sits one layer up. Can the candidate explain why the runnable code is the right code. Can they say what they rejected and why. Can they hold a position under pushback and update it when the pushback is good.
The same shift is happening on the maintenance side. On-call rotations grade engineers on regression cost at six weeks, not on time-to-first-commit. Internal promotion committees ask for a defensibility writeup, not a list of merged PRs. None of this shows up in the marketing copy for AI coding tools. All of it shows up in how the work is actually evaluated.
This guide is what we wish someone had handed us in early 2024. It is the topology of the new job.
What does AI-assisted engineering actually mean?
The phrase gets used three ways. The first is loose — anyone who has Copilot open is "doing AI-assisted engineering." That definition is useless because it includes everyone and discriminates nothing. The second is technical — engineers building AI features inside their products. That is a different job and a different rubric. The third is the one we mean: a working practice for engineering teams where the model is a participant in the authoring loop, and the engineer is graded on the output of the loop, not the share they personally typed.
The definition matters because it sets the bar. If the engineer is the author of record, the engineer has to be able to answer for any line in the change. That is the contract. The model is allowed to draft. The engineer is not allowed to ship a draft they cannot defend.
This is the part that gets missed in most introductory writing. The frame is not "AI helps you code faster." The frame is "AI changes what your work product is." The work product used to be code. The work product now is the code plus a defensible record of how it came to be. Both halves are graded. Both halves matter for hiring, promotion, on-call, and post-incident review.
The shift is real, but it is not magical. Engineers who treat the model as a colleague who needs to be onboarded, briefed, supervised, and reviewed do well. Engineers who treat the model as an oracle that answers questions do not. The named patterns and anti-patterns are the topic of the AI pair programming patterns post. The bug classes that come out of the wrong frame are the topic of the 12 most common AI pair bugs.
One more thing. AI-assisted engineering is not the same as vibe coding, even though both involve a model in the loop. Vibe coding is exploratory; AI-assisted engineering is accountable. The bridge from one to the other — and when each is appropriate — is covered in vibe coding to production workflow.
What's different about the workflow?
The pre-AI workflow had two stages most engineers cared about. Draft, then review. Some teams added a verify stage. Most did not, because the human author had usually been verifying as they typed. The new workflow has five. It is not a productivity ceremony. It is the minimum set of stages that survives independent measurement on the rubric.
Frame is the stage most engineers under-invest in. It is the explicit articulation of the problem, the constraints, the alternatives considered, and the criteria for "done." Without a frame, the model produces a plausible answer to a question the engineer did not actually ask, and the engineer cannot tell the difference. Frames take three to ten minutes for a small change and an hour for a non-trivial one. They are written down because they will be re-read by the reviewer.
Draft is the stage that looks the most like 2022 but has shifted underneath. The model contributes most of the typing. The engineer's job in this stage is to keep the draft inside the frame — rejecting suggestions that drift, prompting for the constraints that were in the frame and not in the prompt, and noticing when the draft has solved a different problem than the one in the frame. AI-rejection rate gets scored here.
Verify is the stage that AI pair workflows tend to skip and pay for later. It is not "run the tests." It is the explicit check that the draft satisfies the frame. The two are often different, because the model has filled in defaults the engineer did not specify. Boundaries — auth, schema, external API — get a separate pass in verify because they are the most expensive places to get wrong silently. The cost of skipping verify is in the regression-rate chart further down.
Defend is the stage that did not exist before. The engineer writes down, briefly, why the load-bearing decisions in the draft are the right decisions. This is not documentation for users; it is evidence for reviewers. A defensible defend stage answers four questions for each major decision: what problem this solves, what the alternatives were, why this option, what would have to change for this to be the wrong choice.
Replay is the deliverable. It is the artifact a reviewer reads to grade the work. The diff is part of the replay; the frame, the rejected suggestions, the verify checks, and the defend notes are the other parts. Replays are how panel reviewers grade — see what panel reviewers actually look for for the read flow.
How is the skill measured?
The Codritium rubric v2.6 is the measurement tool the platform uses to grade engineering work. It is six dimensions, weighted, and calibrated against a panel of working engineers. The weights are the most important thing about it. They are not equal. They were derived from a regression against hire decisions, post-hire performance reviews, and six-week regression rates across 14 partner organisations. The weights have shifted between rubric versions; v2.6 weights are the ones below.
Defensibility is the highest-weighted dimension because it is the one that best discriminates between candidates above the runnable-code threshold. Correctness still matters; it is the floor. But almost every senior-tier candidate clears the correctness floor. The discriminator sits above it.
The table below shows the six dimensions, their weights, what each one scores, and the most common reason a session loses points on that dimension. The weights sum to 100. They are not equal across tiers — the foundations tier weights correctness more heavily and defensibility less, because junior engineers are being evaluated on a different set of decisions.
| Dimension | Weight (Hard tier) | What it scores | Most common reason for losing points |
|---|---|---|---|
| Defensibility | 24 | Can the engineer reconstruct, justify, and revise each load-bearing decision. | Engineer cannot articulate the rejected alternative when asked. |
| Replay clarity | 18 | Whether the replay surfaces the decisions a reviewer needs to grade. | Decisions buried inside long annotations or absent entirely. |
| AI-rejection rate | 16 | Whether the engineer rejected drafts that did not match the frame. | Engineer accepted a plausible-but-off-frame suggestion and shipped it. |
| Regression cost | 15 | Estimated cost of the change going wrong, weighted by boundary risk. | Silent crossing of an auth, schema, or external-API boundary. |
| Spec fidelity | 14 | Whether the shipped change matches the frame the engineer wrote. | Drift between the frame and the verify stage, unaddressed. |
| Correctness | 13 | Does the code do what it claims to do, on the cases tested and the cases it should be tested on. | Edge case missed because the model did not raise it and the engineer did not look. |
The weights are public for a reason. We want engineers to invest practice time in proportion to what gets graded. The largest mismatch we see in candidate preparation is engineers spending 80% of practice time on correctness — which is 13% of the score — and 5% on defensibility, which is 24%. Read the full rubric in the defensibility rubric, 2026.
Inter-rater agreement on the Hard tier is 0.81 weighted across the six dimensions. The highest-agreement dimension is correctness at 0.91; the lowest is defensibility at 0.74. Defensibility agreement has risen from 0.62 in 2024, mostly through tightening of the rubric anchors rather than changes in reviewer pool. The agreement number matters because it sets the noise floor — two reviewers grading the same replay agree on the defensibility score within one point on a ten-point scale, four times out of five.
Where does AI pair fail?
The bug taxonomy for AI pair programming looks different from the pre-AI taxonomy. The classic taxa — null reference, off-by-one, race condition — still exist, but they are not where the modern reviewer spends time. Modern reviewers are catching bug classes that look fine on the diff and only become visible when the surrounding code is read or the conversation is replayed.
Stale context is the largest class. The model wrote against an outdated understanding of the surrounding code — a function signature that changed two weeks ago, a config key that was renamed, an integration that was deprecated. The diff compiles. The integration tests catch some of it; the boundary tests do not, because the model also drafted the tests against the same outdated understanding.
Missed invariant is the second largest. The change is correct for the cases the model considered and wrong for the case the codebase quietly relies on — a sort order, a non-empty guarantee, a tenant scoping. The reviewer notices because they read the surrounding code; the model does not because the invariant is not stated locally.
Silent boundary crossing is third and the most expensive when it ships. A change touches auth, schema, or external-API behaviour without naming the boundary in the replay. The reviewer cannot tell from the diff alone whether the engineer noticed the boundary. Boundary changes that ship without notice are the dominant source of expensive incidents in the rooms where we have post-incident data.
The remaining four classes — over-abstraction, confident wrong API, decorative tests, plausible nonsense logic — are individually smaller but share a property. They all look fine to a reader who only reads the diff. They all become obvious to a reader who reads the surrounding code and the replay. This is why replay-shaped review beats diff-shaped review on AI PRs. The full taxonomy, including the prompt patterns that produce each class, is in 12 most common AI pair bugs and the failure-mode survey is in where AI pair fails.
The level at which these bugs appear is not random. Junior engineers ship a higher share of plausible-nonsense and stale-context bugs because they have less surrounding-code intuition and less practice rejecting drafts that "look right." Senior engineers ship a higher share of over-abstraction and decorative-tests because they accept drafts that match a pattern they already trust, and they verify against the pattern instead of against the actual problem. The mix is different by level, but the total bug count is similar when normalised to PR volume. The level-specific failure modes are the topic of why juniors ship more bugs with AI.
Security is the dimension where the failure cost is largest and the surface is least visible. Silent boundary crossings into authentication, authorisation, and request-flow logic are the bugs that produce incidents the engineer cannot recover from in the same week. The 2026 playbook on AI-agent SSRF is in SSRF in AI agents playbook; the broader research on security pairs with AI is in security with AI pairs.
The diff tells the reviewer what changed. The replay tells the reviewer why. In a world where the model can write a clean diff against a wrong understanding, the why is the only thing that separates a defensible decision from a lucky one.
What does defensibility mean here?
Defensibility is not eloquence. It is not confidence under questioning. It is not the ability to talk about code. It is a property of the work itself — the reasoning is either present or it is not. A clear speaker with no reasoning underneath fails the rubric. A quiet engineer who writes the reasoning down passes it.
The four questions that operationalise defensibility are the same four every panel reviewer asks. They are the questions the model cannot answer for you. Practising them — out loud, on your own merged PRs, weekly — is the cheapest way to move the needle on the rubric dimension that matters most.
| Defensibility question | What a passing answer looks like | What a failing answer sounds like |
|---|---|---|
| What problem does this code solve? | Names the class of problem and at least one symptom; explains why the symptom you are fixing is representative. | Restates the ticket title; cannot describe the class. |
| What alternatives did you consider? | Two or three named alternatives with the discriminating criterion that ruled each out. | Names no alternative or says 'the AI suggested this.' |
| Why this option? | Names the constraint the chosen option satisfies that the alternatives do not. | Names a property of the chosen option without naming the corresponding weakness in the alternatives. |
| What would have to change for this to be the wrong choice? | Names a concrete future condition and the metric that would alert you to it. | Cannot name a condition; or names a condition with no observability. |
| What is the blast radius if this is wrong? | Names which users, which systems, which TTL — and the rollback procedure. | 'It would have to be reverted.' |
The deep-dive on defensibility — including the panel-reviewer protocol and the inter-rater data — is in defensibility, the skill AI can't replace. The rubric itself is the defensibility rubric, 2026.
How does the regression-rate gap close?
The chart below is from a 16-week cohort study, n=412 engineers, across three practice cadences. We measured six-week regression rate as the dependent variable — the share of merged PRs that produced a regression within six weeks of merging, by author. The independent variable is the cadence of replay practice the engineer maintained over the 16 weeks. The shape of the curve is the headline.
Three observations. First, the weekly-practice line drops the fastest and keeps dropping through week 16. The improvement is not a sprint; it compounds. Second, the monthly-practice line drops then plateaus around week 10. The improvement is real but it stops. The plateau is the consequence of the gap between practice sessions being long enough that habit does not form between them. Third, the no-structured-practice line barely moves. The engineers in this cohort are not getting worse; they are just not getting better at the rubric dimensions that move the regression rate. Their floor stays where it started.
The composition of the weekly cohort matters. We segmented by band — junior, mid, senior — and the regression-rate gap between bands closes faster than the gap between practice cadences. By week 12 a weekly-practicing junior has a lower regression rate than a non-practicing senior. The skill is teachable faster than the seniority signal updates. The per-band breakdown is in regression rate by band.
The cohort drift study — what happens at week 12 and beyond, and why some cohorts regress while others compound — is the topic of the three-quarter cohort drift report.
Engineers who reviewed one decision per week beat engineers who reviewed eight in a single monthly session. The skill is a habit, not a study sprint.
The full longitudinal benchmark covering rubric dimensions, cohort outcomes, and the cross-organisation regression-rate analysis.
From Codritium Research
The State of AI Engineering — 2026
Codritium's first longitudinal benchmark of human + AI engineering pairs across 12 task categories. 4,200 real engineering tasks, scored on time-to-fix, regression rate, and reviewer-defensibility.
What does the canonical reading list look like?
The Codritium blog catalog is not a flat list. The posts have dependencies. Reading the rubric post before the workflow post is technically possible, but the rubric will not make sense until you understand what it is scoring. The same goes for the failure-modes posts before the patterns posts. The reading order below is the sequence we recommend to engineers onboarding to the platform.
| Order | Post | Topic | Why read here |
|---|---|---|---|
| 1 | Welcome | Positioning | Sets the frame: we are not a bootcamp, the skill is judgement. |
| 2 | AI pair patterns that work | Workflow | The named patterns the new workflow uses. |
| 3 | 5 habits to drop pair programming with AI | Workflow | Anti-patterns from the pre-AI workflow that block the new one. |
| 4 | Three mental models for reading AI code | Workflow | How to read drafts you did not type. |
| 5 | 12 most common AI pair bugs | Failure modes | The bug taxonomy, with prompt patterns that produce each class. |
| 6 | Why juniors ship more bugs with AI | Failure modes | The level-specific failure modes and why they happen. |
| 7 | Vibe coding to production workflow | Bridges | When exploratory work belongs in the accountable workflow and how to make the transition. |
| 8 | Defensibility, the skill AI can't replace | Rubric | The rubric dimension that decides senior-bar outcomes. |
| 9 | What panel reviewers actually look for | Rubric | How the rubric is applied by the people who score you. |
| 10 | Code review etiquette when the author is AI | Rubric | How to be the reviewer instead of the reviewed. |
| 11 | How to practice AI-era interview | Practice | The structured practice plan for interview preparation. |
| 12 | Take-home assignments 2026 guide | Practice | What take-homes look like under the new rubric. |
| 13 | Choosing an AI coding tool, 2026 framework | Tools | Tool selection after the skill is clear. |
Read in order, the catalog is roughly 25,000 words. Not all at once. Most engineers we talk to spread it over four to six weeks, one or two posts per week, paired with the practice cadence in the regression-rate chart above. The reading is not the practice. The reading is the framework that makes the practice graded.
Common questions
Is 'AI-assisted engineering' a new role?
No. It is a working practice applied to the existing engineering role. The job title on your LinkedIn profile does not change. What changes is how the job is scored — by hiring panels, by promotion committees, and by your own on-call rotation. Most organisations will not create a new role for it; they will rewrite the rubric for the role you already have.
Do I need to be senior to benefit from this?
No. The rubric weights differ by tier — junior engineers are graded more heavily on AI-rejection rate and replay clarity, less heavily on defensibility of architecture decisions they would not yet be expected to make. The workflow and the failure-mode taxonomy apply at every level. The practice cadence matters at every level.
Are these skills tool-specific?
No. The rubric does not score the tool. It scores the work product. Engineers using different models, different IDEs, and different agent frameworks produce work that lands on the same rubric. The tool comparison post is downstream of the skill, not upstream of it — that is why it is the last item on the canonical reading list, not the first.
How long does it take to learn?
The cohort data above is the most honest answer we have. Twelve to sixteen weeks of weekly deliberate practice closes most of the regression-rate gap between practiced and unpracticed engineers. Defensibility scores rise on a similar timeline. The skill does not stop improving at week 16; it stops improving fastest at week 16. Engineers who maintain the cadence past that point keep compounding.
How do I measure my own progress?
Run the five-question defensibility self-review on three of your last PRs. If you can answer all five without consulting the chat history or asking the model, you are above the median for your tier. Then run it again four weeks later on three new PRs. The trend matters more than the absolute number. The platform replay reviews give you a calibrated score; the self-review gives you the cadence.
Is this just a rebrand of vibe coding?
No. Vibe coding is exploratory; AI-assisted engineering is accountable. The two have different rubrics and different success criteria. Vibe coding is appropriate for prototypes and discovery work where the cost of being wrong is small. The transition from vibe to accountable workflow is covered in the dedicated post; the short version is that the frame and defend stages are what convert vibe output into production-ready work.
What's the single most important skill?
Defensibility. It is the highest-weighted rubric dimension on the Hard tier, the one that best predicts senior-bar hire decisions, and the one AI assistance erodes fastest if the workflow is wrong. If you have one hour a week, spend it on the five-question self-review. The rest of the rubric improves on the back of that practice.
What if my team does not use a structured workflow?
You can run the workflow on your own PRs without your team adopting it. The frame, verify, defend, and replay stages produce written artifacts that your reviewers can use even if they do not write their own. Most teams that ended up adopting a structured workflow started with one engineer running it solo and the reviewers noticing the work was easier to grade.
Where to go next
- The State of AI Engineering 2026 — the longitudinal benchmark behind the cohort and rubric data
- The defensibility rubric, 2026 — the full v2.6 scoring document
- Where AI pair fails — the failure-mode survey behind the bug taxonomy
- Codritium platform challenges — practice on production-shaped tasks scored against the rubric
- How to practice the AI-era interview — the practice-plan companion post
- Defensibility, the skill AI can't replace — the deep-dive on the rubric dimension that matters most
- 12 most common AI pair bugs — the bug taxonomy, with the prompt patterns that produce each class
- What panel reviewers actually look for — how the rubric gets applied
- AI pair programming patterns that work — the named patterns that make the workflow tractable