Story · arXiv
Do Self-Evolving Skills Generalize to Held-Out Tasks? (arXiv)
paper · Story page
Of the 21 skills that improved on their training tasks, 5 kept all of that improvement on test tasks, 13 kept part, and 3 kept none. The ones that carried over badly often hard-coded task details such as column names and output files, or turned one fix into a rule for every task.
In plain words
- Researchers found that artificial intelligence often benefited less from its rewritten instructions on new tasks than on practice tasks.
- These systems save procedures or checklists after practice, then rewrite them as they try more tasks.
- Poorly reusable instructions sometimes keep file names or table column names that should change for each task.
- For people building these systems, better practice results do not guarantee the same improvement on unfamiliar work.
Appeared in
- Six ways an agent harness can run something other than what you approved
Oct 02, 2026 · in the sections
Subscribe
Get the brief in your inbox
Pick daily, weekly, or both. Nothing is gated either way: every issue is on the site and in the feeds.
- Weekdays at 8:45am IST, one lead story and 6 to 9 items.
- Sundays, an argued synthesis rather than a recap.
- One click to leave, and quiet days say so in the subject line.