A Valid Error Is Not a Passing Answer

Structured output is valuable because it makes a response checkable. It is not a permission slip to count every valid object as an accomplishment. An error response can have the right shape and still be the wrong answer to a success obligation.

Current state

WTK can turn feedback from a local run into a candidate fixture for later review. That path needs to preserve the captured input faithfully and check the fixture against the package contract before the feedback can influence an improvement loop.

Changes

One completed local engineering cycle tightened that seam. Structured captured input keeps its fields through replay, and plain text is mapped only when the contract identifies one unambiguous required field. A proposed fixture is checked against the contract before it can enter the next stage.

The same cycle made the review boundary explicit. A structured input error may be correct when a fixture asks for rejection. It cannot satisfy an ordinary success fixture merely because the response is schema-valid. The evaluation path now keeps those two obligations separate.

Failures observed

Without this distinction, malformed replay evidence can enter a review path under the wrong contract. A valid error object can also look like a successful answer to a mechanical check, leaving a semantic reviewer to repair an error that should have been blocked earlier.

Assumptions removed

We cannot assume that captured feedback has the same meaning after it is replayed. We also cannot assume that a schema-valid response satisfies the fixture's purpose. Success and expected rejection are distinct evidence obligations.

Evidence

This is one completed local engineering observation with contract-faithful replay checks, a blocked invalid fixture path, and a deterministic distinction between an ordinary success fixture and an explicit rejection fixture.

Limitations

This is not evidence that every contract is complete, that every semantic obligation is mechanically expressible, or that a tribunal will reach the right judgment in ambiguous cases. It does not establish cross-target behavior, task success, or general evaluation reliability.

Next hypothesis

Use paired fixtures that share the same output shape while differing only in whether success or rejection is required. Replay them across package shapes and target harnesses, retaining each contract check, deterministic verdict, semantic review, and disagreement.