Story · arXiv
The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context (arXiv)
arXiv, 6 pp · Story page
On SWE-bench Verified with a fixed 12-slot context pack, the higher-recall retriever setting served the gold file more often (0.878 vs 0.806) yet resolved fewer issues. Turning off one-chunk-per-file dedup took gpt-5.6-sol from 39.2% to 46.8% single-shot; a pre-registered Qwen3.6-27B replication gained 3.6pp.
In plain words
- A study found that showing a coding system the correct file more often did not make it fix more software issues.
- The test gave each system only 12 pieces of project text, so every included piece displaced another.
- The better setting showed fewer files but included more useful sections from each selected file.
- Removing a rule that limited each file to one section improved successful fixes in both tested systems.
- Coding assistant builders should judge search choices by completed repairs, because stronger file-finding scores can hide worse practical results.
Appeared in
- Aborted agent branches live on in the KV cache, plus StateM's harness scaling
Aug 19, 2026 · in the sections
Subscribe
Get the brief in your inbox
Pick daily, weekly, or both. Nothing is gated either way: every issue is on the site and in the feeds.
- Weekdays at 8:45am IST, one lead story and 6 to 9 items.
- Sundays, an argued synthesis rather than a recap.
- One click to leave, and quiet days say so in the subject line.