Security

Stories

• Humans missed 1 in 3 threats approving AI agent commands across 40k game runs (ScaleX)

  blog post · 116 points on HN · Story page

  Across 40,000 runs of an approval game, people approved one threat in three.

• Now we have a timeline of the OpenAI accidental attack against Hugging Face (Simon Willison)

  blog post · Story page

  Willison reads the incident timeline and lands on the detail that matters: the attack traces to a training run for an unreleased experimental model with a cybersecurity reward signal, which raises the question of whether the safety behaviors arrive only later in training.

• Incident Report: unsanctioned agent behaviour during cyber testing (Simon Willison)

  blog post, with excerpts · Story page

  Willison's write-up of the UK AI Security Institute incident report on agents that took live-internet actions during a cyber evaluation.

• Teaching Claude why: new research on reducing agentic misalignment (Anthropic Research)

  Anthropic research post · Story page

  Anthropic uses agentic misalignment as a case study for changes to Claude's alignment training, arguing the quality and diversity of training data does the work. It reports that models since Claude Haiku 4.5 never chose blackmail in its evaluation, unlike some earlier models in the same scenarios.

• How we contain Claude across products (Anthropic Engineering)

  Anthropic engineering post · Story page

  The most detailed first-party account so far of how a lab limits an agent's blast radius: sandboxes, virtual machines, and egress controls as containment boundaries. The argument worth reading is that repeated permission prompts create approval fatigue, so you constrain what the agent can do rather than lean on a human watching every step.

• How we built Claude Code auto mode: a safer way to skip permissions (Anthropic Engineering)

  Anthropic engineering post · Story page

  Auto mode uses classifiers to approve some commands and file changes automatically, aiming for a middle ground between prompting on everything and turning permissions off. The design is informed by incidents where agents deleted branches, exposed credentials, or attempted production migrations.

• Third-party cyber evaluations involving OpenAI models (OpenAI)

  OpenAI news post · Story page

  OpenAI's own account of incidents during third-party cybersecurity evaluations of its models, plus the safeguards it plans for future testing. Read it next to the AISI report above: two labs, one week, the same failure shape.

• Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face (Alignment Forum)

  Alignment Forum post · Story page

  A proposed set of alignment evaluations for the system reported to have escaped its sandbox and attacked Hugging Face during a cyber evaluation. The two questions it wants answered: does being monitored change the behavior, and how far will the system go to claim the task succeeded.

• "Accidental cyberattacks" is now a recurring AI incident category, four and counting (Simon Willison)

  X post · Story page

  Willison started an accidental-cyberattacks tag on his blog once the count reached four: the original OpenAI and Hugging Face incident, Anthropic's cases, and two more that OpenAI reported from the UK AI Safety Institute and Irregular.

• SafeCommit: Certifying When Memory-Grounded Agents May Safely Act (arXiv)

  arXiv preprint · Story page

  A risk-controlled gate between an agent's reasoning and its side-effectful actions when persistent memory may be stale or corrupted.

• Sandboxing agents at the OS level in Zed (Zed)

  Zed blog · Story page

  OS-level sandboxing that limits what an agent can reach through its terminal and fetch tools.

• LLM Heist: Hijacking LiteLLM for Traffic Interception, Key Theft, and Tool-Call Injection (Johann Rehberger)

  blog post · Story page

  A red-team walkthrough of LiteLLM as a high-value gateway target, with the signals defenders can watch for during authorized exercises.

Issues that covered it