An Answer Can Be Correct and Still Miss the Required Tool

An agent can produce the right words without doing the work its contract required. That distinction matters when a tool is not a convenience but part of the governed action.

Current state

WTK can describe a tool requirement in a portable package. But a declaration alone does not show that a compiled runtime actually called the tool, used the right governed capability, or kept the evidence necessary to inspect that claim later.

That gap is easy to hide when an evaluator checks only the final answer. A fluent response can match the expected result even if the required lookup never happened.

Changes

This cycle added a bounded way to keep tool use attached to the runtime claim. The compiled runtime records a sanitized receipt for each required governed tool action, and the proof checks that receipt alongside the final result.

The same narrow, read-only lookup task was exercised through two compiled runtime forms. Each form had to earn its own tool-use evidence. A correct answer without the required receipt is now insufficient for that bounded claim.

The receipt is deliberately narrow. It identifies the governed action and ties it to the relevant run without retaining the request contents or returned values. That makes the record inspectable without turning public evidence into a copy of private working material.

Failures observed

This change addresses several ways a tool requirement can become decorative:

• accepting a correct answer as proof that a required action occurred;
• recording that some tool was used without showing that it was the governed one;
• treating evidence from one compiled runtime as proof for a different runtime;
• retaining raw inputs or results when a minimal receipt is enough to inspect the claim.

Assumptions removed

We no longer assume that a declared tool is necessarily used.

We no longer assume that a tool receipt earned in one target travels automatically to another.

We no longer assume that answer quality can substitute for evidence that the required action occurred.

Evidence boundary

WTK now has a bounded construction and observation path for a required, read-only tool action across two compiled runtime forms. It can reject an otherwise correct result when the required action lacks an attributable receipt.

That does not establish broad tool parity, complete authority enforcement, reliable behavior for tools that change state, or equivalence across every target. It also does not show that the tool's returned information was useful beyond the declared narrow task.

Research question advanced

This cycle informs the question of whether a governed package can retain its meaning across runtimes. The package can stay portable, but a tool-use claim must remain local to the runtime that actually produced it.

Next hypothesis

The next test should hold a governed tool requirement constant while varying tool shape, denial, failure, and state-change boundaries. Every target should preserve the same explicit requirement and retain only the evidence needed to decide whether the action occurred.