A Valid Target Needs More Than Valid Syntax

A target artifact can be tidy, complete, and entirely wrong about the system it is meant to represent. That is the useful contradiction in a new executable-world study: a generated specification may look valid until execution checks the relationship between its services.

The signal

AgentMercury separates an executable business world from a task performed inside it. A world has services, tools, persistent state, transition rules, and cross-service invariants. A task adds a seeded state and task-specific rubric. Execution and grading remain separate, so a final-state check can test whether the world actually preserved its declared constraints.

The authors tested world authoring on 30 held-out business briefs. A generated world counted only if it passed all 12 structural validators. One telling failure appeared when an invariant whose trigger and target belonged in different services was placed inside one service instead. GPT-5.4 made that mistake on 10 of 30 briefs. The resulting specification could be syntactically complete and still miss the interaction it claimed to model.

That is the part we want to keep. A representation is not a proof of the behavior its parts imply.

The evidence

The study uses an executable construction oracle, not a similarity judgment against reference prose. It evaluates whether the generated world can run and satisfy structural checks, including cross-service constraints. The authors report that fine-tuning Qwen3.5-35B-A3B on construction traces raised full-oracle success from 3.3 percent to 83.3 percent on the same held-out set, with a reported Fisher exact test of p = 1.2 × 10^-10. That result is about the paper's selected model, training traces, briefs, and validators.

The sharper result for builders is not the score. Adding a recipe did not systematically improve the API models, and it reduced the fine-tuned model from 83.3 percent to 10.0 percent. Twenty-seven of those 30 recipe-conditioned generations failed the cross-service check. More instruction did not guarantee a more faithful artifact.

The boundary

This is a 2026 preprint about synthetic business worlds and author-defined validators. It is not a WTK package study, target-adapter evaluation, or proof that any WTK projection executes correctly. The 30 briefs and 12 checks do not establish production realism, complete safety coverage, or portability across targets. External research is not a WTK Finding and does not promote the maturity of a WTK mechanism.

The durable contribution is a mechanism signal: surface validity can miss a dependency that only an executable, target-aware check exposes. The reported pass rates are dated observations, not a claim about current models or WTK behavior.

The builder impact

For WTK, a package remains source truth, and a target projection remains a derived artifact. Neither should be trusted merely because it parses or has all expected fields. A target-specific proof needs to exercise the compiled runtime and test the declared relationship that spans its components, including the negative path where that relationship is deliberately broken.

Our take: keep structure, execution, and qualification distinct. A contract may say which tools and permissions should meet. A projection shows how a target receives them. A proof must show whether that target can preserve the relevant semantics without quietly substituting a plausible local arrangement.

The WTK test

WTK could freeze one package, contract, target adapter, evaluator, tool declarations, and initial state. Compare an executable projection with a surface-valid variant that deliberately breaks one declared cross-component invariant. Predeclare state assertions, denied and failed cases, package and target digests, and the attempt count. Retain every outcome.

The proposed test fails if the proof accepts the invalid variant, cannot distinguish it from the executable projection, or relies only on manifest inspection. Passing would still not establish general target safety. It would show that this one declared semantic boundary survived a test designed to break it.