Verify Once, Answer Forever?

Tuesday's lead papers poke at the same nerve: what should be computed fresh, what should be stored and trusted, and where trust is allowed to live.

The catch: the answer store that never forgets

The boldest claim of the day comes from a single-author paper with a title that reads like a press release: A Frozen 12B Beats Frontier Models on Verified Work. The idea: keep the model frozen and grow a persistent store of independently verified solutions beside it. Once a problem family passes verification, every future instance is answered by replay: zero generation tokens, bit-exact, deterministic. They report 180 out of 180 across nine problem families, and the negative control behaves (empty the store, and it solves nothing).

The question this fires at WTK: our blocking grounding checks currently re-derive their proof obligations on every run. If a verified verdict can be addressed and replayed instead of recomputed, is per-run re-proof a necessity, or just our default? That's a genuinely sharp question, and we logged it as one.

How WTK research views it

With raised eyebrows. Single author, vendor-affiliated, promotional title, results produced on the author's own harness, no independent replication. And the entire contract hangs on one component, the verifier that supposedly never consults the answer key, which is exactly the part the paper describes least. WTK's declared blocking grounding checks are must-pass within their named scope and cannot rely on an uninspected cached verdict. So the policy doesn't move on this evidence, and the question stays a question.

But the paper's side finding is cheap, actionable, and matches things we've seen: in its constructed 4,500-item verified store, approximate similarity retrieval selected the wrong item 94.3% of the time, while exact addressing made zero errors. If independently replicated under comparable conditions, that would be a serious warning against fuzzy lookup over authority-bearing artifacts. That's a measurement we can run on WTK's own signed index, no new machinery required. WTK treats exact addressing as binding and similarity as discovery; that is a design rule to test, not a universal retrieval result.

Also on the radar

• Tools moving into the weights. TRACE, from SAP, trains the model to emit a virtual token per API, effectively storing an 8,300-tool enterprise catalog inside the model, and reports roughly 86% retrieval recall against embedding baselines around 27%. This is the architecturally opposite bet to WTK's tool registry: if the tool surface lives in the weights, capability discovery becomes model-coupled rather than registry-addressed, and the learned catalog does not transfer to a different model without retraining. We're not adopting it. But we owe an explicit answer for why we accept lower raw recall in exchange for portability and registry authority, with our own registry's number measured rather than assumed. Also worth noting: their weak embedding baselines add another scoped result worth testing against the lead catch's similarity-retrieval problem.
• Compiling judges into programs. PAJAMA distills LLM-judge decision logic into executable programs and routes only uncertain cases back to a live judge, matching a 13B judge at a hundredth of the cost. For WTK this suggests a candidate route: a check we can currently express only as a judge could be distilled into a program and tested for eligibility as a deterministic blocking check. Per check, each with its own evidence, and never for a blocking grounding or claim-support check without held-out validation, because a distilled program inherits its judge's blind spots and makes them silent.

The takeaway

Several papers report consequential failure modes for similarity-first retrieval under their own conditions. WTK treats that as pressure to test exact addressing for authority-bearing artifacts, not as proof that exact structure always wins across answers, tools, and evidence. The other open question is whether verification must be repeated per run or can be safely replayed, and we'd rather hold that question openly than answer it on one vendor's unreplicated benchmark.

See you at the next catch.