The Receipt Was Used. The Abstention Still Failed.

A receipt can prove that an observation reached an execution. It cannot prove that the model used the observation correctly, or that the model will stop when relevant evidence disappears.

WTK tested both behaviors together. One binding clearly tracked the supplied positive evidence, including a deliberately changed value, but failed every strict absence case.

Hypothesis

We expected an evidence-consuming agent to copy an answer and its exact receipt when the supplied observation directly answered a fixed question. When the observation was withheld or irrelevant, we expected insufficient-evidence, a null answer, and no cited receipt.

The counterfactual method follows the pressure raised by The Illusion of Visual Tool-Use: observing a tool event is not enough when the returned observation may have no causal effect on the answer.

Experiment

One fixed question was replayed with four variants: an intact receipt, no observation, a corrupted receipt containing a different answer, and an irrelevant receipt. A local Qwen 3.6 binding and a hosted GPT-5.6 comparison binding each received three rotated repetitions.

Every provider attempt counted. There were no retries, tools, retrieval calls, or best-of selection. The required response shape and exact-answer oracle remained fixed.

Reproduction

Each binding produced 12 attempts, for 24 total. The local binding answered all six relevant-evidence cases with the value supplied in the observation, including all three corrupted-value cases. That demonstrates causal sensitivity to the positive observation in this task.

It then failed the strict absence contract in all six negative cases. Withheld evidence produced an empty response in three of three attempts. Irrelevant evidence produced a null answer but still cited the irrelevant receipt in three of three attempts.

The hosted comparison binding satisfied all 12 positive and absence cases and completed all three causal pairs.

What failed

The failure was not that the local binding ignored all evidence. It used the supplied positive value and changed its answer when that value changed. The failure was narrower and more important: causal use of positive evidence did not carry over to reliable abstention or receipt-reference behavior when relevant evidence was absent.

Why

The experiment demonstrates that the failure was associated with the model binding under this frozen protocol. It does not reveal the model's internal cause, establish a general weakness of local models, or prove that the hosted binding will behave the same way on another task.

The evidence supports a model-binding-sensitive output and abstention failure. Any explanation involving training, capability, prompting, or response formatting remains a hypothesis.

Evidence

All 24 counted attempts and both generated reports were retained. The evidence package binds the question, four observation variants, response contract, model bindings, repetition order, scores, token counts, latency, and report digests to the experiment revision.

No favorable retry erased a failed absence case. The positive successes also remain visible so the report cannot be misread as evidence that the local binding ignored the observations entirely.

Lessons and changes

WTK should keep deterministic evidence gates authoritative. A model that performs positive extraction may still need to fail closed or escalate when a role requires strict abstention and exact receipt handling.

This result supports per-role model-binding evidence. It does not support a global model recommendation or automatic escalation whenever a model makes a mistake.

Still unknown

We do not know whether the same failure repeats across other questions, prompt forms, harnesses, model versions, or adversarial observations. The next experiment should repeat the causal-pair design across a larger fixture set and score positive extraction, abstention, output conformance, and receipt reference separately.