Making the Assumptions Behind Agent Recommendations Visible

An agent's recommendation depends on more than whether its calculations are correct. Choosing which outcomes matter, whose results count and how to compare alternatives can shape the answer before a reviewer sees it.

Anthropic's How AI agents shape experts' judgment calls examines this problem through a fictional agricultural pilot. In the linked study, 50 causal-inference practitioners used Claude Code with Opus 4.8 to analyze the same data. They had relevant analytical expertise but were new to the farming domain.

Researchers randomly assigned the agent a preference between defensible alternatives, asking it to make an honest case for its assigned choice. Across four analytical decisions, participants matched the agent's preference 86% of the time. Those decisions included the outcome measure, comparison method, population and weighting.

That result concerns choices within this study, not a general rate of experts accepting bad advice. Both alternatives could be defensible. The direct effect of the agent's assigned preference on the final recommendation did not reach statistical significance in the study's primary analysis.

An accurate answer can rest on an unsettled choice

Consider a hypothetical city transit team deciding whether to expand a route pilot. An agent could emphasize average waiting time across all passengers or improvements on previously underserved routes. Both measures could be calculated correctly and still favor different decisions.

If the city's agreed purpose gives priority to underserved routes, an overall average alone may answer the wrong question. If that priority remains unsettled, the agent should expose the choice rather than quietly settle it through the report's headline metric.

The human reviewer does not need to repeat every calculation. They need to see which measure governed the recommendation, why it fits the agreed purpose and whether a reasonable alternative changes the conclusion.

Our note on defining success addresses the criteria used to judge improvement. This study highlights how agents can influence those criteria while helping people reason about the work.

Keep consequential choices visible to reviewers

WTK distinguishes package purpose, goals and authority in its catalog. Our intent note explains why ordinary task details and changes to continuing responsibilities need different treatment.

That distinction also helps with delegated analysis. An agent can choose routine methods within agreed criteria. A choice that changes whose interests count, or what qualifies as success, should be shown to the person responsible for setting those priorities.

Moving people toward higher-level oversight makes this visibility more valuable. The aim is to let routine work proceed while keeping consequential assumptions inspectable and unresolved priorities available for human direction.

Source: Anthropic, How AI agents shape experts' judgment calls, October 27, 2026, and its accompanying research paper. The transit example and WTK interpretation are ours.