Story · Jared Joselowitz (Ufonia)
Shipping AI to a Million Patients Without an A/B Test (Jared Joselowitz (Ufonia))
talk · Story page
Ufonia's regulated clinical voice agent has made roughly 200,000 calls, and you can't A/B test on patients, so experimentation moves into simulation: one model plays the patient from clinician-written hazards, a second judges every dialogue, and both roles were validated rather than assumed.
In plain words
- Ufonia tests changes to its clinical calling system in simulations because experiments on real patients could cause harm.
- One artificial intelligence system acts as a patient facing risky situations written with clinicians.
- Another artificial intelligence system reviews every simulated conversation for problems.
- Ufonia tested both simulated roles instead of assuming their behavior was trustworthy.
- Healthcare teams can examine dangerous possibilities before using changes with patients, where a failed call cannot be undone.
Appeared in
- Malicious skills hijack agents mid-task, and debate training curbs reward hacking
Aug 20, 2026 · in the sections
Subscribe
Get the brief in your inbox
Pick daily, weekly, or both. Nothing is gated either way: every issue is on the site and in the feeds.
- Weekdays at 8:45am IST, one lead story and 6 to 9 items.
- Sundays, an argued synthesis rather than a recap.
- One click to leave, and quiet days say so in the subject line.