The Devil Is in the Interface: How Tool Architecture Shapes Coding Agents

What an agent can do depends on its tools. But a new study asks the question almost nobody has measured: does how those tools are organized and exposed change agent behavior — even when the underlying capabilities are identical? Across 11,700 coding-agent trajectories and six tool architectures, the answer is emphatically yes.

The Six Architectures

The researchers held the underlying information and actions roughly constant and varied only the interface. Six setups, from a bare bash shell up to richer scaffolding:

• BashOnly — one general-purpose shell tool (baseline)
• Atomic — structured low-level file/SQL operations
• NLSearch — natural-language search over the repo
• Python — the agent writes Python code instead of issuing individual tool calls (CodeAct-style)
• Scratchpad — a tool for recording intermediate reasoning
• HypoTrack — a tool for tracking hypotheses

The point of the design is the comparison it enables: same repository knowledge, same action space, different presentation. If agent behavior changed across architectures, the interface itself is causal — not the underlying capability.

What Actually Changed

Four findings matter, and they point in opposite directions:

1. Code execution is dramatically cheaper. The Python/CodeAct-style interface achieved similar task performance to bash with 41.6% fewer steps and 56.3% lower token usage:

Instead of five round trips for a sequence, the model writes one program. This is the same efficiency shift behind DeepSeek Harness's "Code" preset, and it's a strong argument for giving agents executable-code interfaces over chained tool calls.

2. Structured low-level interfaces improve consistency. Compared to the bare bash baseline, more structured interfaces (Atomic) improved repeated-attempt consistency by up to 4.7×. Same capability, more reliable behavior, run after run. If you care about reproducibility — and any team running agents in CI or production does — the interface is a consistency lever you're not pulling.

3. Natural-language search broadens exploration. NLSearch increased access to relevant files by more than 11% and pushed agents to explore more diverse parts of the repository — but with more noise. It's a discovery tool, not a precision tool. The right move is to add it for unfamiliar codebases and remove it once the agent has its bearings.

4. Cognitive scaffolding is a dud. This is the counterintuitive result. Lightweight tools that let the agent record reasoning (Scratchpad, HypoTrack) had limited effect on actor behavior — agents mostly projected their existing reasoning patterns into the scaffolds. The affordance alone doesn't change the policy. A "record your reasoning" tool is not a substitute for an actual reasoning change.

The Experimental Setup

The study's validity rests on one design decision: the architectures were implemented as interchangeable tool interfaces over the same underlying filesystem and shell. The agent — same base model, same task set — only saw different presentation of the same capabilities. When behavior changes under a purely presentational change, the interface is causal, and that's what makes the four findings above attributions rather than correlations.

Two measurement details shape how to read the results:

• Token counts include the system prompt, tool outputs, and the model's own generations. Under BashOnly, a sequence like "list directory → read file → edit file → run tests" is four or five round trips; under the CodeAct-style interface the model emits one program and the harness executes it in one step. That's where the 56.3% token saving comes from — not from the model thinking less, but from the harness talking less.
• Consistency was measured by repeated attempts of the same task. The up-to-4.7× gain for structured interfaces is a reproducibility number: same task, same model, same architecture — how often does the agent converge on the same approach? For anything running agents in CI or production, this is the metric that matters.

Choosing an Architecture

The four findings map cleanly onto goals:

How to Apply This in Real Harnesses

The paper studies interfaces; the value is that you can change these interfaces today:

• DeepSeek Harness's Code preset is CodeAct by construction — the model writes a program against a generated TypeScript SDK rather than calling tools individually. It's the most direct implementation of finding #1 in shipping software, and we cover it in the DeepSeek Harness overview.
• OpenCode and Claude Code both let you front multi-step operations with executable scripts instead of ad-hoc tool chains — writing a script that a single tool call executes is the same "collapse the round trips" move.
• Repository search (the NLSearch finding) is cheap to add to any harness and costs nothing when the agent doesn't need it — but budget for the noise it introduces.
• Skip the scaffolding. If you're tempted to add a scratchpad or hypothesis-tracking tool to an agent, this study is the evidence that you're optimizing the wrong layer.

What This Means

• If you build an agent or configure one with tool access, prefer executable-code interfaces for multi-step operations — fewer steps, 56% fewer tokens, same results. That's the single highest-leverage tool-architecture choice in the study.
• Repository exploration is where search tools earn their keep. NLSearch-style interfaces are worth the noise when agents work in unfamiliar codebases.
• Structure beats vibes. The Atomic finding — up to 4.7× better consistency from structured low-level interfaces — is the quiet win. Deterministic tool schemas make agent behavior more repeatable, which is what you actually need in production.
• Don't expect scratchpads to fix reasoning. Adding a "record your reasoning" tool is not a substitute for an actual reasoning change. The research aligns with our multi-agent collaboration guidance: patterns that change behavior are structural, not cosmetic.

The lesson for 2026: tools aren't just capability, they're interface, and the interface changes what agents do. The cheapest performance gain in coding-agent design right now isn't a better model — it's exposing the same capabilities as code the agent can write.