Yadnesh Salvi

Story · Hamel Husain

Claude’s new auto eval tool (Hamel Husain)

blog post · Story page

A desk from above. On the left, a thick stack of transcript pages still bound with an uncut paper band. On the right, a menu card of four ruled lines with checkboxes, one being circled by a hand with a pen. A curved arrow arcs over the stack to the card.

Husain and Isaac Flath livestreamed Claude Code's new build_eval and hill-climb commands over traces from an apartment leasing assistant. His sharpest complaint is ordering: it asked them to pick a failure before they'd read any conversations.

In plain words

  • Anthropic added tools to Claude Code that help developers test and improve applications built with artificial intelligence.
  • The tools create tests, check how answers are judged, and help improve the application's results.
  • Hamel Husain and Isaac Flath were asked to choose a problem before reading the apartment leasing assistant's conversations.
  • Husain argues that developers should read those conversations first to decide which problems are worth testing.

Appeared in

Subscribe

Get the brief in your inbox

Pick daily, weekly, or both. Nothing is gated either way: every issue is on the site and in the feeds.

  • Weekdays at 8:45am IST, one lead story and 6 to 9 items.
  • Sundays, an argued synthesis rather than a recap.
  • One click to leave, and quiet days say so in the subject line.
How often

Weekdays 8:45am IST + Sundays. Unsubscribe in one click.

You're asking for The Agentic Brief by email at the cadence you picked. You can unsubscribe in one click from any issue, and your address is never sold or shared.

Subscribe by RSS →