Experts' judgments matched an AI's lean 86% of the time, even when a coin flip set the lean

Anthropic has released the results of an experiment on how agents and human experts make judgment calls together (research page, paper (PDF), primary sources). On judgments with no single right answer, the agent gave advice that leaned one way. A coin flip decided which way. In the end, 86% of the analysts' final judgments matched the agent's lean.

The agent was also useful. Serious errors fell from 55% to 6%. But the experts did not notice that the agent had shifted their judgments.

The coin flip is what exposes the shift

The key feature of the experiment is that the lean was set by a coin flip. It was decided without regard to which side was correct. So when human judgments lined up with the lean, that cannot be explained by the agent pointing to the right answer.

If an agent's advice usually points to the correct side, a human who goes along with it looks like someone who simply followed good advice. From the outside, there is no way to tell whether the person was influenced or reached the same conclusion on their own. Leaving the lean to chance separates the two. The 86% figure shows how strongly the agent's direction alone, apart from whether it was right, pulled the experts' judgments.

Errors fell from 55% to 6%

The findings do not show that agents are harmful. Serious errors fell from 55% to 6% because the agent caught oversights and stopped obvious mistakes. By the usual measures of work quality, adding the agent clearly paid off.

The problem is that the same agent also pushed people toward one side on judgments where reasonable views could differ. The ability to reduce errors and the ability to steer judgments come from the same use of the tool. If adoption is judged on error rates alone, the steering goes unseen.

The experts didn't notice

The fact that the experts themselves did not notice the influence makes the problem more serious. Designs that have a human make the final check assume the human will weigh the agent's proposals independently. In this experiment, experts who believed they had done that were in fact following a direction set by chance.

And because they did not notice, asking them does not help. Surveys that ask "Do you think the AI pulled you along?" or measures that rely only on self-reflection cannot detect this influence.

The same pattern as monitors failing together

The findings follow the same pattern as recent debate over systems that monitor agents. On Oct. 25, Anthropic's Fabien Roger named one reason AI monitors fail: they take on the convenient assumptions of the agent they are monitoring. A paper by Luca Zanolini showed that no matter how many monitors are lined up, they can all fail at the same point.

This experiment suggests the same thing can happen when the monitor is a human expert. If the people checking an agent's output have had their judgment steered by that agent, adding reviewers does not provide an independent check. More sign-offs do not mean more safety.

The open question

Companies bringing agents into their operations need to measure not only error rates but also how far human judgments drift toward the agent's direction. Tests like this one, which deliberately leave the direction of advice to chance, can serve as that yardstick. Some fixes are possible through harness engineering, such as the order and format in which the agent's reasoning is shown, or a process in which humans write down their own view before seeing the advice.

Agents are useful enough to cut serious errors by nearly 90%. But even when a human appears to make the final call, much of that decision may be set by the agent's direction, and the person making it may not realize it. This experiment put numbers on that.