The Top-Down Bet Needs A Bottom-Up Audit
I still think abstraction is where human advantage moves in agentic coding. But, I also don't think abstraction gets to ignore the machine.
My bet is still on top-down AI-assisted software engineering. Durable human advantage moves up the abstraction tree while agents absorb the leaves. I’ve argued the two halves of that position already in the last two posts, and I’ve been living it for a year, watching agents take over code I used to be proud of writing.
A bet deserves an audit, though. So this finale in a 3-part series is a status check rather than a memoir: what the trend data shows as of mid-2026, why the loudest skeptics are conceding ground, and where the bet could still lose.
The Leaf Nodes Are Being Absorbed, On Schedule
The claim that agents will keep improving at implementation work doesn’t need my intuition. It has a measured curve.
In October 2023, the original SWE-bench paper reported the best model of the day resolving 1.96 percent of real GitHub issues. OpenAI’s cleaned-up SWE-bench Verified launched in August 2024 with GPT-4o at 33.2 percent. Claude 3.7 Sonnet reached 62.3 percent in February 2025. GPT-5 claimed 74.9 percent that August. By mid-2026 the benchmark is close to saturated, which is why the labs have mostly stopped bragging about it.
METR measures the same slope differently: the length of task a model can finish at a 50 percent success rate. Their March 2025 paper found that horizon doubling roughly every seven months; their January 2026 update put the frontier at over five hours of human-equivalent work and the doubling time closer to three months. Whatever else is true, “the leaf nodes will improve” stopped being an opinion sometime in 2024. It’s the most consistently measured trend in software. Believe the slope.
Two numbers complicate the slope, and they belong in the same paragraph as the cheerful ones. When Scale built SWE-bench Pro (September 2025) to resist contamination, with larger multi-file tasks from unfamiliar repos, the same class of frontier models dropped to roughly 23 percent. And practitioners are voting with their anxiety: in the 2025 Stack Overflow survey, 84 percent of developers use or plan to use AI tools while only about 3 percent highly trust the output and nearly half actively distrust it. Adoption tracks the curve. Trust tracks the gap.
Read together, the numbers say the absorption is real and fast where work resembles the training distribution, and thins where tasks get larger, stranger, and less checkable. Which raises the only question that matters for the bet: what, exactly, makes work checkable?
Why the Skeptics Are Turning
The engine under the curve is not a secret. OpenAI’s o1 described it in late 2024 and DeepSeek-R1 demonstrated it in the open in January 2025: reinforcement learning against answers a machine can check. Code is the ideal substrate for that loop, because the compiler and the failing test are free verifiers, available at whatever scale training needs.
That mechanism explains the most interesting sociological fact in the field. The skeptics are conceding. And they’re all conceding the same narrow thing. François Chollet spent years as the discipline’s most rigorous LLM skeptic, and when o3 cracked his ARC-AGI benchmark in December 2024 he called it a genuine breakthrough while founding a lab premised on a different road to intelligence. By December 2025 he was crediting “remarkable progress on LLM-driven refinement loops” in commercial frontier models. Andrej Karpathy’s sober middle position is the same concession from the other direction: agents genuinely handle the routine layer, and still bloat the codebase the moment the architecture gets custom. Nobody converted to scaling faith. They conceded that refinement loops against checkable feedback work, and that coding is where feedback is cheapest. Cheap feedback compounds.
That’s the audit’s core finding. The trend is steepest exactly where verification is cheap, and the tree is really a gradient of verification cost. A leaf has a verifier the training loop can afford: it compiles, or the test goes red. One level up, the verifier gets expensive, because a module boundary proves itself over months of change traffic. At the root, the verifier is an incident review, or the market. So the bet, restated with the precision it lacked: agents keep absorbing everything with a cheap verifier, corner cases included, because a corner case is just a test nobody wrote yet. That half I’d sign today. What stays human is the layer where feedback arrives late and entangled, and no training signal exists to climb it.
Honesty about the evidence: I looked for a controlled study isolating how much agent success depends on specification precision versus abstract intent, and I can’t find one. Both sides of this bet are quoting vibes on that question. The study is missing, and whoever runs it will move the argument more than the next benchmark will.
Correct Is Still Not Fast
One caveat survives the audit intact. PerfCodeBench (May 2026) found models still trailing expert implementations on system-level optimization, especially parallelism and GPU work, and SWE-Perf shows repository-level performance gains require understanding the system, not passing its tests. Casey Muratori’s old critique of clean abstraction still bites: a tidy interface can hide cost from the reviewer as effectively as from the machine. The agent knows the words: batch, vectorize, cache, async. Knowing the words isn’t the same as owning the cost model, and a correct function can still be the bottleneck. Unpriced abstraction remains the most expensive thing in a generated codebase.
The Data Question
The part of the trend that could bend is the training data. Research on model collapse (Shumailov et al., The Curse of Recursion) shows that recursive training on generated data can erase the tails of a distribution, and the tails are where the hard cases live. They’re also where seniors were made. The counter-evidence is real too: when synthetic data accumulates alongside real data rather than replacing it, collapse may not be inevitable, and dose and curation matter. The conclusion I trust is conditional, and it loops back into practice: a passing patch with a hidden defect may be worse than noise, while a rejected diff with a good explanation may be the most valuable training example a company produces. Judgment traces, not more code, are the scarce training input.
The most valuable training artifact you produce this year might still be the PR comment that explains why a clean diff was wrong.
Where That Leaves the Human
I’ve already argued the near-term job in this series: the review economics when code gets cheap, and the retrieval problem that decides whether an agent can operate inside a legacy system at all. I won’t restate either; they’re one click away.
What this post adds is the vertical answer. The human doesn’t get to park at the root and doesn’t need to defend the leaves. What’s defensible is movement: choose the level, descend when the situation demands it, come back up carrying what the machine couldn’t check. The two jobs that show no sign of absorbing are pricing an abstraction before production bills for it, and owning outcomes no test can certify in advance.
And the falsifiable version, since a bet that can’t lose is just a mood: if design-level feedback ever gets as cheap as a failing test, the middle of the tree goes the way of the leaves, and I’ll write the correction myself.
Until then, it is a top-down bet. Just a top-down bet with muddy boots.