Verify Once, Answer Forever?

Editorial clarification: September 8, 2026

This update replaces rhetorical skepticism with the verifier and replication limitations, and removes an implied comparison with unmeasured WTK recall. Reported study values and citations are retained; no new empirical validation is claimed.

A stored, verified answer may reduce repeated computation. The question for WTK is when its verification still applies to a new request, and when the evidence must be checked again.

Replaying verified solutions

The lead paper is A Frozen 12B Beats Frontier Models on Verified Work. The idea: keep the model frozen and grow a persistent store of independently verified solutions beside it. The paper proposes answering subsequent instances of a verified problem family by replay: zero generation tokens, bit-exact, deterministic. They report 180 out of 180 across nine problem families, and report that emptying the store makes the negative control solve nothing.

The original WTK record describes blocking checks that reassess required evidence on every run. If a verified verdict can be retrieved by its exact identity and replayed instead of recomputed, when is reassessment still necessary? The record retains this as an open question.

How WTK research views it

The record describes results from the author's own harness without independent replication. The central uncertainty is the verifier: the replay claim depends on checking solutions without consulting the answer key, but the paper provides limited detail about that component. WTK's declared blocking grounding checks are must-pass within their named scope and cannot rely on an uninspected cached verdict. So the policy doesn't move on this evidence, and the question stays a question.

A separate reported result concerns retrieval: in its constructed 4,500-item verified store, approximate similarity retrieval selected the wrong item 94.3% of the time, while exact addressing made zero errors. If independently replicated under comparable conditions, that would be a serious warning against fuzzy lookup over authority-bearing artifacts. WTK could test this distinction using its own index under a separately approved protocol. WTK treats exact addressing as binding and similarity as discovery; that is a design rule to test, not a universal retrieval result.

Also on the radar

• Tools moving into the weights. TRACE, from SAP, trains the model to emit a virtual token per API, effectively storing an 8,300-tool enterprise catalog inside the model, and reports roughly 86% retrieval recall against embedding baselines around 27%. This is the architecturally opposite bet to WTK's tool registry: if the tool surface lives in the weights, capability discovery becomes model-coupled rather than registry-addressed, and the learned catalog does not transfer to a different model without retraining. We're not adopting it. Any comparison with WTK would need measured registry recall; this record does not establish that WTK trades lower recall for portability. The reported embedding baselines are another reason to test retrieval under matched conditions rather than generalize across the two studies.
• Compiling judges into programs. PAJAMA distills LLM-judge decision logic into executable programs and routes uncertain cases back to a live judge. Its program committee matched the average accuracy of a 13B judge while running 47.25 times faster. Separately, its reward-model experiment used 45 to 50 times less proprietary-labeling API cost than the GPT-4 labeling comparison. For WTK this suggests a candidate route: a check we can currently express only as a judge could be distilled into a program and tested for eligibility as a deterministic blocking check. Per check, each with its own evidence, and never for a blocking grounding or claim-support check without held-out validation, because a distilled program inherits its judge's blind spots and makes them silent.

The takeaway

Several papers report consequential failure modes for similarity-first retrieval under their own conditions. WTK treats that as pressure to test exact addressing for authority-bearing artifacts, not as proof that exact structure always wins across answers, tools, and evidence. The other open question is whether verification must be repeated per run or can be safely replayed, and the cited benchmark does not resolve that question for WTK.

The distinction between retrieving a past result and authorizing a new decision is part of the WTK evidence lifecycle. Our research method requires a separate bounded test before adopting the proposed reuse mechanism.