Does the Running Agent Enforce Its Configuration?

An agent can carry a rule in its configuration without enforcing it during a task. We need to distinguish three observations: the running system received the rule, a component read it, and the rule constrained an action. Evidence of one does not establish the others.

Picture a support agent with a rule to ask before sending an external message. The rule might appear in the package, contract, and target configuration. If the execution path never checks it, the declaration is documentation, not a working control.

The signal

PaperCompiler studies a related problem in paper-to-code generation. It labels source information as paper-supported, externally delegated, inferred, or unresolved, then carries requirements into owners, producer-consumer handoffs, and file-level contracts before code is generated.

In the authors' 90-paper controlled reruns, their reference-based fidelity score was 4.152 versus 3.647 for PaperCoder, a stated 13.8% relative improvement. Their analysis also reports high-severity evaluator critiques for 162 of 2,649 PaperCompiler items and 337 of 2,561 PaperCoder items. The extra structure had a cost: 1.71 million average tokens per generated repository versus 0.98 million.

The evidence

The proposed mechanism is explicit. A requirement has a source, a status, an owner, a producer, a consumer, and a fail-fast condition if its expected consumer is missing. That is stronger than a free-form plan, but it is still a generation-time aid, not proof that a repository works.

The paper makes that limit visible. Its SEABO example preserves much of the intended topology yet leaves an external protocol incomplete, uses a dummy dataset, and raises NotImplementedError for a required state-only path. A well-labeled declaration did not turn an unavailable dependency into a working runtime.

For WTK, this separates what a package declares from what its deployment does. Target compilation should carry a package declaration into a particular environment. Qualification evidence has to distinguish delivery, reading, and enforcement under the stated conditions. A trace showing that a component read a rule does not prove that the rule stopped an unauthorized action.

The boundary

This September 2026 preprint evaluates selected paper-to-code tasks with its own prompts, o3-mini generation, and model-based or reference-based fidelity measures. It does not show that generated repositories reproduce their papers, work across all models or targets, or provide semantic-correctness guarantees.

It also does not show that any WTK package has a fidelity defect. A manifest field, a compiler output, or an evaluator critique is not by itself runtime consumption evidence, target qualification, or deployment approval. WTK has not run this comparison, and a changed package, contract, target, binding, or tool surface would require a fresh test.

The builder impact

Our take: each required rule or setting should name the component expected to use it. If the deployment environment cannot support a required control, block activation and explain the gap. Do not substitute a placeholder.

That gives reviewers specific questions: which component read this rule, what action did it constrain, and what evidence shows the effect? It also helps separate static fidelity from a much stronger claim that the agent ran correctly.

The WTK test

We propose one bounded comparison on a frozen WTK package that declares tools, permissions, hooks, guardrails, memory, and handoffs. Compare current WTK's normal deployment preparation with the same process plus a tracking record. For each setting, that record identifies its source, the component expected to use it, and any unresolved dependency. Keep existing safeguards in both arms.

Keep package and contract digests, target, model binding, goals, evaluator, permissions, and budget fixed. Measure delivery, reading, and behavioral effect separately, alongside unsupported-setting refusals, unnecessary blocks, cost, and task outcome. Test both permitted and prohibited actions. Record all attempts and freeze acceptance thresholds before execution.

The candidate fails if it allows an unsupported required control to pass, weakens enforcement, or exceeds the agreed tolerance for unnecessary blocks. If current WTK already handles every case correctly, the comparison may show no measurable benefit or remain inconclusive. That is not automatically a failure. This is a proposed WTK experiment, not a demonstrated defect or permission to run it.

Source

Primary research: PaperCompiler: Faithful Paper-to-Code Generation via Repository-Level Specification Compilation, Yunhao Liu, Hong Phuc Pham, and Jaehong Yoon, September 2026. WTK reviewed the complete 25-page primary PDF. This Research Note translates external evidence into a testable WTK question; it reports no WTK experimental result.