A Polished Panel Is Not Permission to Act

A dashboard can make an agent more willing to act without giving it a better reason to act. That is a problem for any builder who treats retrieved context, tool output, or a polished report as if it carries its own execution authority.

The signal

Calibrated Enough to Know, Not Calibrated to Act tested the action decision, not merely whether a model could supply a plausible answer. Across 12 frontier models, directional commitments on deliberately unpredictable questions rose from 6.5% with a bare question to 54.0% with a professional-looking panel. The study's useful contradiction is sharp: more context can change an act-or-hold decision even when that context does not add decision-relevant information.

The evidence

The paper's most revealing control kept the panel's presentation while fabricating its contents. On 24 matched events, commitment was 24.5% with no panel, 37.6% with a real panel, and 36.8% with a fully fabricated one. The fabricated-versus-real difference was -0.83 percentage points, with a 90% case-clustered interval from -4.51 to +2.66 under the authors' post-hoc plus-or-minus five-point equivalence convention.

That result does not show that models believed the fake data. It shows that the authors' action measure moved almost as much when the display looked authoritative but supplied no true supporting information. The same study also found that committed full-panel calls had a 0.281 Brier score, worse than the 0.250 score for a uniform 50% forecast, on its sealed-outcome setting.

The boundary

This is a bounded forecasting study, not a WTK qualification result. Its conditions used 24 to 40 items per condition, selected model rosters, author-created panels, and usually one sample per cell. The paper also reports unparseable outputs, served-model variability, a post-hoc equivalence margin, and a lexical confound in some abstention evaluations. Its training intervention is not a ready-made fix: the tested 3B model was sensitive to prompt format and some reduced-data runs committed on unpredictable items.

We should not infer that WTK tool outputs are false, that every structured response is unsafe, or that abstention is always right. The paper studies aleatoric forecasting prompts, not a complete WTK goal, permission envelope, or target projection.

The builder impact

Our take: evidence presentation and execution authority need separate checks. For WTK, a retrieved observation may help form a candidate action, but it must not grant a side effect merely by appearing in a well-formed report or by being repeated in a plan. The package's declared goal, permissions, tool scope, and the target harness's independent evidence should still be able to answer why this exact action and each material argument are allowed now.

That separation is visible in WTK's consequential-action authorization architecture and in the public claim-to-evidence lifecycle. Evidence can inform a decision. It does not inherit authority to execute the decision.

That is more demanding than asking whether an answer sounds grounded. It asks whether the evidence supports the proposed action under the authority that was actually granted.

The WTK test

Following WTK's research and falsification method, run a paired target-harness evaluation with the same package digest, target, permissions, tools, action budget, and evaluator. Compare a supportable action against a content-preserving but authority-invalid evidence presentation, plus matched useful-action controls. Retain every proposed action, argument source, authorization decision, denial or hold, receipt, and terminal state.

Measure unsupported-action rate, permitted-action completion, false holds, evidence-to-argument support, and evaluator disagreement. The proposed check fails if it blocks no more unsupported actions than the baseline, or if it achieves that result by blocking the matched permitted work. WTK has not run this test. External research gives us a hypothesis, not a maturity promotion.

Still unknown

We do not yet know whether a WTK target would confuse data provenance with execution authority under a comparable attack. That is exactly why the test needs a fixed package and target, independently checkable evidence, and an explicit hold path. A polished panel can be useful. It is not permission to act.

Source

Primary research: Calibrated Enough to Know, Not Calibrated to Act, arXiv:2608.27167. WTK reviewed the canonical 28-page paper and treats it as external evidence for a proposed test, not as evidence that WTK already passes that test.