Infra & sandboxes

Stories

• Homa: The End of TCP for AI Clusters (John Ousterhout (Stanford), AI Engineer)

  talk · Story page

  Why small coordination messages queue behind big transfers

• Qwen3.8 Flash Next now at 1.2k t/s prefill on Strix Halo (r/LocalLLaMA)

  Reddit thread · Story page

  One author's llama.cpp optimizations for Qwen3.8 Flash Next on Strix Halo

• Moli Builds a Rust Browser That Uses 10x Less Memory Than Chrome (AlphaSignal)

  newsletter item · Story page

  Rust headless browser that skips rendering, per AlphaSignal

• RASER: Resilient Agent Scheduling and Execution Runtime for HPC Clusters (Semantic Scholar)

  paper · Story page

  RASER runs agent workflows on Slurm with work stealing through shared-filesystem queues, application-level checkpointing paired with Slurm requeue, and Apptainer isolation without image changes. It reports nearly 39% lower makespan than static partitioning at near-full CPU utilization, and it tests recovery after simulated preemption.

• vLLM x AgentX: Optimizing for Real-World Agentic Serving (vLLM Blog)

  Story page

  vLLM walks through KV cache management, parallelism, scheduling and prefill/decode disaggregation for agent traffic, and reports up to 130K tokens per GPU-second on SemiAnalysis AgentX plus a 14.6x to 106x serving-cost advantage over Opus 5. Those are the project's own numbers.

• Optimizing delta weight syncs for managed rollouts (Baseten)

  Story page

  Baseten syncs RL weight deltas across clusters in under 40 seconds

• Latency-Aware Orchestration for Multi-Agent LLM Workflows on Heterogeneous GPUs (Semantic Scholar)

  paper · Story page

  Prediction-guided scheduling for multi-agent workflows on mixed GPU pools

• Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving (arXiv)

  paper · Story page

  An eighty-episode agentic tool-use workload measures how prefix caching changes trajectories under fixed seeds, across two engines and four weight formats.

• Cursor Cloud Agents can now run in Vercel Sandbox (Vercel)

  blog post · Story page

  Cursor keeps the agent harness and inference loop, and Vercel Sandbox supplies an isolated Firecracker microVM per agent request, with Vercel Functions and Workflow as the control plane that claims queued requests, provisions workers, monitors sessions, and cleans up. You get a scale-to-zero worker pool, durable retries, and short-lived user-scoped credentials, and Self-Hosted Machines requires a Cursor Enterprise plan.

• Introducing Sandbox Computer Use for Mastra Agents (Mastra)

  Story page

  Mastra agents can now drive a sandboxed Linux desktop through E2B Desktop or Daytona, with 11 computer tools for browsing, forms, downloads, screenshots, and terminal programs. A SandboxComputer interface exposes screen size, cursor position, and a stream URL so you can watch the agent work.

• Set per-user budgets on AI Gateway (Vercel)

  blog post · Story page

  per-user spending limits on Vercel AI Gateway

• The Half Life of Agent Infrastructure (Ben Kus (Box))

  talk · Story page

  Agent infrastructure's half life, measured in months

• vLLM v0.28.0 (Hacker News)

  HN thread · 106 points on HN · Story page

  The vLLM project ships version 0.28.0

• FinOps for AI Agents: Who Spent All the Tokens? (Tisha Chawla & Susheem Koul (Microsoft))

  talk · Story page

  Microsoft's agent cost control plane, steering runs instead of killing them

• I hosted Kimi K3 (2.8T parameters) using 8 B300s. 92 tok/s, $190 per million tokens (r/LocalLLaMA)

  Reddit thread · Story page

  Real serving numbers for a 2.8T-parameter model on eight B300s

• From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems (arXiv)

  paper · Story page

  AgentSysBench: where latency and memory land across ten agentic apps

• Docker Sandboxes – Disposable, isolated sandboxes for AI agents (Hacker News)

  HN thread · 375 points on HN · Story page

  Disposable, isolated sandboxes for agents; 375 points on Hacker News.

• Smart Routing in Unity AI Gateway: Match frontier quality with 30%+ lower cost per task (Databricks)

  blog post · Story page

  Databricks says smart routing in Unity AI Gateway spreads coding tasks across a pool of models and harnesses, matching frontier quality at more than 30% lower cost per task.

• Monitor on-premises and multi-cloud AI agents with AgentCore Observability (AWS ML Blog)

  blog post · Story page

  AgentCore observability for agents running outside AWS

• Databricks Acquires Electric to Give Every AI Agent its Own Postgres (AlphaSignal)

  newsletter item · Story page

  Databricks is acquiring ElectricSQL to embed a full Postgres inside every AI-agent sandbox, syncing local state back to its Lakebase platform.

• Always-on agents run production without the on-call tax (Justin Smith (Resolve AI))

  talk · Story page

  Resolve AI's background agents notice a release tag dropped in Slack, read what changed, and write a monitoring plan for that release alone, deciding on their own when to check back. The target is everything that routes around CI/CD: feature flags and infrastructure changes that ship with no monitoring at all.

Issues that covered it