Does Messy Code Cost You Money? A New Paper Says Clean vs. Messy Makes No Difference to Agent Success — But a 34% Cost Difference

Here's a question I hear constantly from developers working with AI coding agents: does code quality still matter when an LLM is doing the reading?

The answer, until this week, was mostly vibes-based. "Clean code helps models understand the intent." "Messy code is fine — the model reads tokens, not architecture." Both camps had anecdotes. Neither had data.

Now we have data. A new paper from SonarSource — the company behind SonarQube, the most widely-used code quality platform — does what no one has done before: it held the agent and task fixed, and systematically varied only the cleanliness of the codebase. The results are both reassuring and inconvenient, depending on what you wanted to hear.

The Setup: 660 Trials, 6 Minimal-Pair Repos

The paper, "Does Code Cleanliness Affect Coding Agents? A Controlled Minimal-Pair Study" by Priyansh Trivedi and Olivier Schmitt, introduces a genuinely clever methodology: minimal pairs.

For each of 6 repositories, the researchers created two versions — one clean, one messy — that are behaviourally identical. Same architecture, same dependencies, same external behavior, same tests. The only difference: SonarQube rule violations and cognitive complexity scores.

They used two automated pipelines to construct the pairs:

• Slopify (clean → messy): Takes a clean codebase and inlines helpers, duplicates logic, adds dead code. Mimics code grown without review.
• Vibeclean (messy → clean): Resolves analyzer-flagged issues — deduplicates, extracts helpers, removes dead branches.

The difference between clean and messy versions is stark. The sonar-sca private repo went from 94 to 2,825 issues. The public commons-bcel (Apache) went from 694 to 2,711 issues.

Then they ran 33 coding tasks (13 cognitive-hotspot, 14 multi-module, 6 calibration) on both sides of each pair, 10 times each, using Claude Code with Sonnet 4.6. That's 660 total trials, containerized with identical sandbox environments.

The Surprising Result: Pass Rate Is Flat

Here's the headline that clean-code evangelists won't want to hear:

Cleaner code does not improve agent pass rate. The difference is -0.9 percentage points (91.3% on cleaner vs 92.1% on messier). Statistically, that's flat.

If your boss has been pushing code quality initiatives with the promise that "it'll help the AI write better code," that particular argument doesn't have legs. Agents solve tasks equally well regardless of how messy the code is.

But — and this is a big but — pass rate isn't the only metric that matters.

The Real Story: Clean Code Is Cheaper

Here's where the data gets interesting:

Two things jump out.

First, the 7-8% token reduction is real money. If you're running Claude Code on a codebase with meaningful daily usage, a 7% reduction in both input and output tokens translates directly into API cost savings. For heavy users — teams running agent-driven CI, automated refactoring, or large-scale code review — that's thousands of dollars a year.

Second, and more interestingly: file revisitations drop by a third. The agent reads slightly more files on its first pass (+3.2%) but then commits and moves on instead of looping back. On messy code, it keeps returning to files it already edited — re-reading, re-checking, re-confirming. That 34% reduction in re-reading is the strongest signal in the entire study.

"Code cleanliness joins model choice, harness, and prompting as a factor that materially affects agent behaviours."

— Trivedi & Schmitt, the paper

The Split That Changes Everything

The aggregate numbers hide a deeper pattern. When you break down by task type, the effects are not just different — they're almost opposite.

Multi-module tasks (changes spanning module boundaries) drive the savings: -10.7% input tokens, -50.8% file revisitations. Cleaner code dramatically reduces the cost of crossing module boundaries. The agent navigates more efficiently, finds what it needs faster, and doesn't keep second-guessing itself.

Cognitive-hotspot tasks (work concentrated in a single dense function or class) tell a different story: +1.8% input tokens, -20.2% revisits. The agent reads more files (+11.2%) but edits fewer lines (-9.3%).

Why the difference? Because cleaning a hotspot often means "redistributing complexity across more methods" rather than eliminating it. The extract-to-helper refactoring that reduces cognitive complexity for humans also spreads the relevant logic across more files — so the agent reads more before making its edit.

This is a genuinely useful finding for developers. If you're asking your coding agent to make a cross-cutting change (refactor a shared utility, update an API contract, fix a bug that spans services), keeping the codebase clean has a measurable payoff. If you're asking it to fix a bug inside a single complex function, cleanliness matters less — and may even slightly increase token usage.

The Concrete Evidence: Two Case Studies

The paper includes detailed case studies that make the mechanism visible.

Cleaner wins — commons-bcel (Apache): The messy side had 250-line switch statements. The clean side had thin dispatchers delegating to named helpers. Result: 35% fewer input tokens, 32% fewer conversation turns. The agent could grep for the relevant helper by name rather than parsing a monolithic switch to find its target.

"Rather, because the agent can target (grep) precisely where the edits need to happen, it ends up navigating in a more token-efficient manner."

Messier wins — genie (Netflix): The clean side extracted helpers around the focal logic but left the core intact. The file stayed the same size but the surface area grew. Result: input tokens up 8%. Extracting structure without removing complexity made things worse, not better.

This second case is the more instructive one. Cleaning without consolidating — breaking things into more pieces without reducing overall surface area — can increase what the agent needs to read. The paper doesn't use this language, but the takeaway is clear: refactoring for human readability doesn't always optimize for agent token efficiency.

What This Means for Your Team

Three practical takeaways I'm taking from this paper:

1. Track your agent's token footprint, not just pass rate. Most teams evaluate their agent workflows on correctness alone. The paper shows that two codebases with identical pass rates can have meaningfully different operational costs. If you're spending significant money on agent tokens, the cleanliness of your codebase is a cost factor you can influence.

2. Named structure beats monolithic logic. The commons-bcel case study makes this explicit. When the agent can target (grep) a named function rather than parse a giant switch statement, it reads less, reasons less, and moves on faster. Extract-and-name is the refactoring pattern with the highest ROI for agent efficiency.

3. Be careful what you clean. The genie case study is a warning: not all refactoring improves agent outcomes. Extracting helpers around focal logic without reducing the complexity of the focal logic itself can backfire. Clean with intent, not by rote.

The Bigger Picture

The paper's most important contribution isn't the specific numbers — it's the minimal-pair methodology. For the first time, we have a way to isolate codebase quality as a variable in agent performance. This opens the door to answering harder questions: What specific code patterns cause the most agent confusion? How do different models respond to the same clean/messy split? Does the effect compound as codebases grow?

The SonarSource team has also released their evaluation infrastructure on GitHub, so anyone can replicate or extend this work. That's the kind of open-science gesture that makes the whole field better.

The practical bottom line for developers building with AI agents: keep writing clean code, but stop pretending it determines whether your agent succeeds or fails. It doesn't. What it determines is how much your agent costs to run. And as agent usage scales from occasional assistance to continuous integration, that cost argument is going to become harder to ignore.

---

Paper: Trivedi, P. & Schmitt, O. (2026). "Does Code Cleanliness Affect Coding Agents? A Controlled Minimal-Pair Study." arXiv:2605.20049 [cs.SE].