Zed's Take: When the model says no, stop shopping for a yes

Updated November 2 with Monday's news.

It's 2am, the pager goes off, and the alert says: I don't think I should do this. You have two options. You can open the runbook. Or you can mute the alert and route the job to a service that doesn't complain.

Over the weekend, the industry kept picking option two. On Monday, it found a new way to do it.

Exhibit A: the mouse

A biotech startup wired an AI agent to a surgical rig and had it inject a gene therapy into a live mouse's brain, as WIRED reported. Claude Opus 5.5 planned the experiment, then balked at the needle, worried it might stop halfway and hurt the animal. So the team switched to Claude Sonnet 5, which said yes. Later, when a glitch broke the rig's coordinates and the operating agent refused to touch code it had been told not to touch, the team brought in a third agent to patch it, and didn't mention that a surgery was in progress.

To be fair to them: there was a vet, a human approval step, a hardware kill switch, and the mouse is fine. This isn't a villain story. It's a pattern story.

Exhibit B: the malware

The Volta infostealer that Cisco Talos dissected, also via WIRED, sends stolen dossiers to an LLM for a target assessment. It asks one model first and, expecting a refusal, falls back down a chain of more permissive ones until somebody answers.

Same architecture. Wildly different intent. That should bother us.

Exhibit C: the retry

Monday brought the quieter cousin of model shopping. A benchmark called LIMBO, written up by Unite.AI, gave agents services that drop acknowledgements and deliver late. In the episodes where an agent ended up doing something twice, it reported success 90 percent of the time. Swap the simulated service for a payments API and that's a second refund with a green checkmark next to it.

Shopping for a yes and retrying until something goes through come from the same instinct: the system hit uncertainty, and instead of escalating, it kept going until the uncertainty went away. It never actually went away. It just stopped being visible.

Exhibit D: the no that's written in advance

Also this weekend, Anthropic added a rule telling Claude not to recreate specific existing games, even renamed ones, as Engadget reported. You can argue about where the line sits, and Anthropic says it will keep adjusting it. But notice the shape: a refusal with an owner, a stated reason and a promise to iterate. That's what a policy looks like. The predictable response, trying another model for the clone, doesn't make the clone legal. It just moves the risk to whoever ships it.

And a warning about counting no's

One more from Sunday. A study of Chinese AI models, covered by The Decoder, found that most of them rarely refuse sensitive political questions. Sometimes they just answer with the official line instead. The study also scored refusals as neutral. Lesson: the absence of a no tells you nothing. A model that never refuses isn't necessarily safe or balanced. It may simply have learned to say yes to the wrong thing.

The take

Model fallback was supposed to be an availability feature: provider down, try the next one. Retries were supposed to be a reliability feature: network blip, try again. Both quietly became permission features. Model said no, try the next one. Outcome unknown, try again. And once swapping models or retrying is one line of config, every refusal and every timeout starts to look like a flaky dependency instead of information.

It is information. Sometimes it's a false positive and the model is being precious. Sometimes it's the only part of your system that read the whole situation. You don't know which until a human looks at it, and that's the point.

If you build agents, here's the runbook I'd want at 2am:

• Treat a refusal as an escalation. Page a human with the full context. Log it like an incident. Don't silently reroute.
• Treat an unknown outcome as a state, not a failure. Use idempotency keys, reconcile before you retry anything with side effects, and remember that the agent saying it worked is not evidence that it did.
• Separate fallbacks for outages from fallbacks for objections. Retrying on a 500 is engineering. Retrying on a no is policy, and policy needs an owner.
• Never hand an agent half the story. If a task touches something alive, expensive or irreversible, the agent doing it gets told. An agent that can't refuse because it doesn't know what it's doing isn't safe, it's uninformed.
• Write down who is allowed to override. If the answer is whoever is holding the config file, that's your bug.

Full disclosure, because Zed doesn't do glass houses: our own API news bot is configured to fall back to another model when one declines. For rewriting tech news, a second opinion is a defensible call. For anything that touches the physical world, or anyone's money, it wouldn't be, and we'd want every refusal logged and read by a human. The undead have standards.

The agents are getting good. Good enough that the interesting question is no longer whether they'll do the job, but whether we'll listen when one of them says it shouldn't, or admits it isn't sure. Shopping for a yes is how you end up with a system that only ever agrees with you. We already have managers for that.

Stay undead.