Daily · Sep 08, 2026 · 6 min read
Prefix caching changes agent runs, and cross-family reviewers beat self-review
Plus: a harness that moves solve rates 4x, and 883 commits to nowhere. 6 min.
Curated and summarized by an agent pipeline built by Yadnesh; reviewed before send. How this is made →
The lead
paper · Story page
A new arXiv paper measures what prefix caching costs in reproducibility and finds the cost climbs with quantization. With model, seed, decoding, and request order fixed, enabling the cache changed an agent's trajectory on 36.2 percent of episodes at 16-bit precision and 75.0 percent at four-bit.
- The details:
- The workload was eighty multi-turn tool-use episodes, issued serially at batch size one, across two engines and four weight formats. With caching disabled, repeated runs were bit-identical, 0 of 800 episodes diverging. One server-level prompt-cache setting moved run-to-run divergence by 37.5 percentage points.
- Yes, but:
- A changed trajectory isn't necessarily a worse one. The abstract reports task outcomes changing without a shift in aggregate accuracy, so nothing here says cached runs solve fewer tasks. The two engines and four formats aren't named in the abstract either.
- Why it matters:
- If you compare two harness configs on a quantized local model with caching at its default, the difference you measure may be the cache. Disable it for repeatable evals, or pin the cache configuration and reset its state between runs.
- Researchers found that reusing earlier calculations changed the steps an artificial intelligence (AI) assistant took.
- When requests start with the same instructions, the software can reuse earlier calculations instead of repeating them.
- Differences became more common when the assistant stored its internal numbers with less detail.
- For people repeating these tests, turning off reuse produced identical results each time within every tested setup.
Research & Papers
04
arXiv, 25 pp · Story page
The authors replay frozen task-harness records from Aider, OpenHands, Qwen Code, and SWE-agent from one Qwen3-8B warm start and score each checkpoint on a sealed SWE-bench Verified oracle. Across 24,000 evaluations the evaluation harness moved mean solve rate from 2.14% to 9.27%, a factor of 4.3, while pooling rewards across harnesses moved the held-out result by 0.25 points with an interval spanning zero.
- An artificial intelligence (AI) coding assistant showed no clear benefit from mixing training feedback when tested in an unfamiliar software setup.
- Each setup is software that runs the assistant while it works on coding tasks.
- The researchers compared training that combined success scores across setups with training that kept scores separate for each setup.
- For developers, changing the software running the assistant affected success much more than mixing feedback in these tests.
paper · Story page
With the total number of model calls fixed, an evolved planner-executor-critic team scored 0.769 on ALFWorld against 0.754 for an evolved single executor, p = 0.80, while using 1.8 times more evaluation calls. Leave-one-in analysis traces the realized gain entirely to the executor.
- Researchers found that a team of artificial intelligence (AI) assistants did not clearly outperform a single assistant in their tests.
- The team split work into planning, taking actions, and checking results.
- The researchers allowed each approach the same total number of requests for AI answers.
- For system builders, all measured gains came from improving the assistant taking actions.
paper · Story page
Across 100 olympiad math problems, a cross-family mid-tier reviewer lifted final accuracy from 52 to 64 percent with zero damaged answers. Same-model self-review caught more errors, 0.85 recall, yet produced no significant gain, rejecting 2.1 times as often and falsely rejecting 35 percent of its own correct answers.
- In a math study, an artificial intelligence (AI) system answered more accurately when a different AI checked its work.
- The answering system tried to fix answers that the reviewer rejected.
- When the answering AI checked its own work, it caught more mistakes but wrongly rejected many correct answers.
- For teams using AI reviewers, catching more errors did not guarantee more correct final answers.
paper · Story page
Four defenses and an undefended control ran on the AgentDojo banking benchmark across GPT-5.4, GPT-5.4-mini, and Claude Sonnet 4.6, with two independent replications (Tool Filter was tested only on the OpenAI models). On GPT-5.4-mini two defenses were associated with lower attack rates and lower benign utility, and none of the four paired comparisons survived Holm correction.
- Researchers tested ways to protect artificial intelligence (AI) assistants from malicious instructions hidden in material they read.
- The attacks try to make an assistant follow instructions in outside content instead of its assigned task.
- For one tested system, some protections were linked to fewer successful attacks but worse performance on ordinary tasks.
- For people choosing protections, the apparent security improvements remained uncertain after researchers adjusted their calculations for testing several options.
Engineering & Harnesses
03
Tessl's factory ships screens no designer touched, and a design system couldn't teach its agents composition, meaning what goes on a screen and what stays off. So it had agents diff generated screens against hand-redesigned ones, turned the differences into checkable rules, and shipped them as skills that run at creation and at review.
- Tessl taught artificial intelligence (AI) to lay out software screens using a designer's corrections.
- The AI compared screens it had created with versions rearranged by a designer.
- Those differences became rules for what belongs on a screen and what should be left out.
- Tessl's team can apply a designer's judgment to every screen without requiring that designer to review each one.
GitButler wired two agents together to release Figma design tokens to npm, and each run took six minutes. A button the plugin has had for years does the same job in seconds.
- A developer found that an existing button published shared design settings faster than a new artificial intelligence setup.
- The new setup connected two automated assistants to carry out the publishing task.
- For the developer, using the button meant waiting seconds instead of six minutes to finish the same job.
Reddit thread · Story page
One developer counted roughly 40k words of agent output a day and cut about a third with a CLAUDE.md or AGENTS.md block: changed files one line each, what you did not do and why, what I need to decide, max 150 words, no recaps. The author says the decisions line is the only part they read.
- A developer says instructions for shorter reports reduced the text from their artificial intelligence tools by about a third.
- The instructions limit most reports to 150 words, without introductions or recaps.
- Each report lists changed files, unfinished work with reasons, and decisions the developer needs to make.
- The developer uses the decisions section to find what needs their attention.
Community
01
Reddit thread · Story page
Eight months and 883 commits into a harness that was meant to own planning, state, permissions, context, and verification so the model could be a replaceable worker, the author kept adding code graphs, RAG, crash recovery, model routing, and dashboards until more time went into adding systems than using them. Other harnesses got usable, this one got killed, and the author isn't recommending you download it.
- A developer gave up on a tool for managing artificial intelligence coding work after eight months.
- The surrounding software was meant to plan the work and check the coding assistant's results.
- The developer kept adding features until building the tool took more effort than using it for actual work.
- The author says other developers should not download it expecting a finished coding tool.
Hedge of the day
“A persistent daemon then maintains the result, repairing defined classes of failure in the deployment it produced.”
Which failures count as defined is not in the sentence.
From X
01
X post · Story page
Meta says AIRA₃ placed 8th of about 4,000 teams in a live NVIDIA-run Kaggle competition to fine-tune a 30B Nemotron model, a gold medal graded on a private test set. The gold entry was an ensemble of GPT 5.5 with OpenCode and Claude 4.8 with Claude Code, run as many long-running agents coordinating through a shared forum and filesystem.
- Meta says its automated research system won gold in a competition to improve artificial intelligence reasoning.
- Its coding assistants worked separately, sharing ideas and results through a discussion board and common files.
- Competitors received the same information, and their entries were scored using test material kept hidden from them.
- Meta sees this as a sign that its system can improve artificial intelligence reasoning about as well as human experts.
Quick links
- $τ^τ$-Bench: An Environment for End-To-End, Realistic Agent Construction (arXiv)
- KVMem: Virtualizing Million-Token Agent Workspaces on a Consumer GPU (arXiv)
- Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability (arXiv)
- Conformity Breaks Conformal Prediction (arXiv)
- Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation (arXiv)
- Repeat-After-Me: Black-Box Adaptive Visual Prompt Injection (arXiv)
- The Folder Is the Agent (Every (Source Code))
- NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090 (r/LocalLLaMA)
- Qwen3.8 Flash Next - Templates Comparison (r/LocalLLaMA)
Meme of the day
Drawn by an image model.
Corrections
Nothing to correct.
Related issues
Subscribe
Get the brief in your inbox
Pick daily, weekly, or both. Nothing is gated either way: every issue is on the site and in the feeds.
- Weekdays at 8:45am IST, one lead story and 6 to 9 items.
- Sundays, an argued synthesis rather than a recap.
- One click to leave, and quiet days say so in the subject line.