SWE-Explore: Why AI Coding Agents Find the File but Miss the Lines That Matter

The Problem SWE-Explore Solves

Every coding agent benchmark until now has measured one thing: can the agent fix the bug? SWE-bench, SWE-bench Verified, SWE-bench Multilingual — they all present an issue and a repository, then score whether the agent produces a passing patch. That's useful for ranking, but it conflates two entirely different capabilities:

1. Exploration — finding the right code to look at
2. Repair — writing the correct patch once the right code is in context

If an agent fails, you don't know why. Was it looking at the wrong file? Reading the right file but overlooking the critical lines? Writing bad patches?

SWE-Explore, published June 2026 by researchers from Princeton, Stanford, UIUC, and the SWE-bench team, is the first benchmark that isolates exploration from repair. The results reveal a blind spot that every developer using AI coding agents should understand.

What the Benchmark Actually Measures

SWE-Explore takes 848 real GitHub issues across 10 programming languages from SWE-bench Verified, SWE-bench-Pro, and SWE-bench Multilingual. For each issue, the benchmark defines a set of core evidence regions — specific file+line ranges that a developer (or agent) must read to understand and fix the problem.

How Ground Truth Is Constructed

The key methodological innovation is how they determine "what should be read." Rather than relying on human annotators (expensive, inconsistent) or diff-based heuristics (misses context required for understanding), the authors extract ground truth from successful agent trajectories:

• Run multiple independent agent instances on the same issue
• Capture every read action (file opens, grep results, line reads)
• Intersect the read regions across successful runs
• Refine with an LLM pass and human audit

The result is a ranked list of evidence regions — not just files, but specific line ranges — that an explorer should surface to a downstream repair agent.

Evaluation Protocol

Explorers receive a repository and an issue description. They must return a ranked list of code regions (file path + line range) within a fixed budget — typically K=5 regions. The benchmark then measures:

This is fundamentally different from SWE-bench's binary pass/fail. A single number becomes a profile of where an agent's exploration breaks down.

The Numbers That Matter

Here are the results that should concern anyone building or using AI coding agents:

Three patterns jump out:

1. File-level localization is largely solved. The top agentic explorers (AweAgent, Claude Code, OpenHands, Mini-SWE-Agent) find the right file 64-68% of the time. That's respectable and trending upward.

2. Line-level recall is abysmal. Those same agents identify only 14-18% of the critical lines. They find the correct file 2 out of 3 times, but within that file they miss ~85% of the evidence that matters.

3. CoSIL looks like a different architecture entirely. CoSIL (Context-Synthesized Implicit Localization, from a prior line-level retrieval paper) achieves 78.8% line-level recall — 4-5x better than the generalist coding agents — at the cost of lower HitFile. It's retrieving deeper in the right places but sometimes missing the entry point.

Why This Gap Exists

The gap between HitFile and line-level recall isn't a bug — it's a property of how current coding agents explore codebases.

The grep-and-scroll pattern

Most coding agents follow a similar exploration loop:

1. Search: grep/find for keywords from the issue description
2. Open: read the files that match
3. Scan: scroll through the file looking for relevant code
4. Narrow: if no match, go back to step 1

This works for finding files because files are coarse-grained — a function name, a class name, or an error message from the issue typically appears somewhere in the right file. But once the file is open, the agent faces a needle-in-haystack problem. A file might be 500 lines. The critical region might be 15 lines. The agent reads the file header, sees familiar function signatures that look correct, and moves on — missing the subtle off-by-one or missing null check 400 lines down.

Context budget mismanagement

Agents have finite context windows. When an agent opens a 600-line file and dumps the whole thing into context, it spends ~85% of its budget on irrelevant lines. The paper finds a +0.95 correlation between CtxEff (context efficiency) and downstream resolution rate — meaning the agents that pack more relevant lines into their context budget succeed far more often.

Current coding agents are optimized for coverage (find everything that might be relevant) rather than precision (find exactly what matters). They open too many files and read too broadly, diluting the signal in their context.

Static retrieval is not enough

The BM25 baseline in the table above is not just bad — it's catastrophically bad. Keyword search at the file level (what most RAG pipelines do) captures 7.9% of the right files and 2.1% of the right lines. The paper confirms what practitioners have suspected: static retrieval (BM25, TF-IDF, even dense embeddings) cannot substitute for agentic exploration in large, unfamiliar repositories. The gap is not incremental — it's an order of magnitude.

What This Means for Claude Code, Codex, and Other Agents

The implications are specific and actionable.

For Agent Builders

1. Measure line-level recall, not just resolve rate. If you're evaluating your agent on SWE-bench, you don't know whether failures are exploration failures or repair failures. Add an intermediate evaluation step that checks whether the agent read the right lines before attempting a fix. SWE-Explore's methodology is directly applicable — collect trajectories, extract read regions, compare against ground truth.

2. Build line-aware retrieval, not just file-aware retrieval. Agents that "grep" broadly and then "read" the entire matching file are leaving 80% of their value on the table. Alternative approaches:

• AST-aware navigation: Parse the repository, build a symbol table, and retrieve by function/class boundaries rather than file boundaries. AutoCodeRover uses this approach — its lower HitFile (28%) but competitive line-level metrics suggest it's reading less irrelevant code, even if it misses the entry point more often.

• Structured line ranking: After opening a file, run a second retrieval step (grep with context, embedding similarity, or LLM scoring) to identify the 30-50 most relevant lines before reading the full file.

• Iterative narrowing: Don't open the full file. Use line-range reads (read lines 100-120, then 200-230 based on what you found) to zero in on the evidence.

3. Prioritize context efficiency. The CtxEff correlation is the strongest signal in the paper. When designing an agent's exploration loop, prefer actions that produce compact, targeted context over actions that dump large files. A 100-line context with 50% relevant content outperforms a 500-line context with 10% relevant content — even though the smaller context has fewer total lines.

For Tool Designers

1. Expose line-level search in agent tool APIs. Most coding agent tools operate at the file level: read_file, view_file, open_file. Add tools like search_lines(repo, issue_text), get_function_body(symbol), or grep_with_context(pattern, file, window=10) that operate at sub-file granularity. The agents that succeed on SWE-Explore use shell tools (grep, sed, awk) precisely because those tools work at the line level.

2. Surface code structure explicitly. When an agent opens a file, provide a function/class outline before the full file content. This allows the agent to choose which function to inspect rather than scrolling through the entire file. Several IDE plugins already do this for humans — agents need the same capability.

Design Patterns for Better Agent Code Search

Based on the SWE-Explore findings and the agent architectures tested, four design patterns emerge:

Pattern 1: Three-Stage Retrieval

Instead of the monolithic "search → read → fix" loop, structure exploration as three distinct stages:

Stage 1 — File Retrieval:
  Use issue keywords + repository metadata to identify candidate files.
  Target: 5-10 files. Budget: 20% of context.
  
Stage 2 — Region Retrieval:
  For each candidate file, extract function/class boundaries.
  Score each region for relevance to the issue.
  Target: 10-20 regions. Budget: 30% of context.
  
Stage 3 — Line Retrieval:
  For top-ranked regions, read the exact lines.
  Look for variable definitions, conditionals, and calls involving issue-relevant identifiers.
  Target: 30-80 lines. Budget: 50% of context.

CoSIL's architecture approximates this three-stage approach, which explains its 78.8% line-level recall.

Pattern 2: Interactive Narrowing

Don't pre-commit to reading a file entirely. Use a query-refinement loop:

1. Agent: "I found file src/parser.ts. Let me grep for 'null' and 'tokenize' nearby."
2. Tool: "Lines 142-145: null check on tokenize result"
3. Agent: "That's relevant. Show me the function that calls tokenize."
4. Tool: "Function parseExpression at lines 120-180"
5. Agent: "Read lines 140-150 and 170-175."

This pattern requires tools that support interactive narrowing (grep with context, function boundary detection, range-based reads) — exactly what shell-savvy agents like Codex CLI and Claude Code can do, but often don't do systematically.

Pattern 3: Evidence-Score-Guided Exploration

Track a running score of how confident the agent is that it has found the relevant evidence, and use that score to drive exploration decisions:

- Start with a hypothesis from the issue description
- After each read, update confidence
- If confidence < threshold: explore more (look for related symbols, callers, callees)
- If confidence > threshold: attempt repair
- If repair fails: reduce confidence and explore more

This is essentially what human developers do — we form a hypothesis about the bug, look for evidence, and refine until we're confident enough to make a change. Current agents do this implicitly through tool choice, but a structured confidence signal would make the process more robust.

Pattern 4: Separate Explorer and Repair Modules

SWE-Explore's most important design insight is that exploration and repair should be decoupled. An agent that searches for bugs shouldn't also be responsible for fixing them. Two practical architectures:

• Pipeline: Run an explorer that collects evidence regions, then pass those to a repair agent that only sees the evidence (not the full file). This prevents the repair agent from getting distracted by irrelevant code.

• Verify loop: The repair agent generates a candidate patch. A separate agent verifies it by re-reading the evidence and checking whether the patch addresses the issue. If not, cycle back with additional exploration.

This pattern maps onto SWE-Explore's own "downstream resolve rate" experiment: when explorers provide high-quality evidence, repair agents perform better; when evidence is poor, no amount of repair engineering helps.

The Road Ahead

SWE-Explore is not the final word on agent exploration — it has limitations. The benchmark uses 848 instances (modest by ML standards), the ground truth depends on the quality of successful agent trajectories, and it doesn't yet measure exploration for feature additions or refactoring (only bug fixes). An updated version covering these scenarios would be valuable.

But as a diagnostic tool, SWE-Explore is immediately useful. The paper's core finding — that agentic explorers find the right file ~65% of the time but identify only ~15% of the critical lines — should drive product decisions at every coding agent company.

If you're building an agent: instrument your exploration loop, measure line-level recall, and invest in sub-file retrieval.

If you're buying or choosing an agent: ask how it searches, not just how it fixes. The agent that finds the right file and writes a plausible-but-wrong patch is more dangerous than the agent that admits it doesn't know.

SWE-Explore gives us a vocabulary for these conversations. Before, we had pass/fail. Now we have a profile of where exploration breaks down. That's progress.

Pitfalls

• SWE-Explore is not a replacement for SWE-bench. It measures exploration, not end-to-end task completion. Use SWE-Explore for debugging exploration failures, use SWE-bench for ranking.
• Agent results are model + scaffold combinations. Claude Code and Mini-SWE-Agent results on SWE-Explore reflect both the underlying model and the exploration scaffold. A better model in the same scaffold will improve both HitFile and line-level recall.
• CoSIL's high recall has a tradeoff. Its 78.8% line-level recall comes with category restrictions — CoSIL is designed for well-scoped, localizable bugs. For issues that require cross-file understanding, its HitFile (54.4%) tells a different story.
• The Oracle baseline (0.923/0.915/0.953) is human-derived, not human-perfect. The ground truth is constructed from successful agent trajectories, not from expert developer annotations. The ceiling may shift as ground truth methodology improves.

References

• SWE-Explore paper: arXiv:2606.07297
• SWE-bench: swebench.com
• SWE-bench Verified: OpenAI
• SWE Atlas: Scale AI
• AgentBench: Liu et al., 2023