The Repair Loop Needs an Exit Ramp

Repeatedly rewriting instructions is not a repair strategy. Sometimes the failure belongs to the output contract, a missing capability, the evidence policy, or the goal itself.

This factory cycle taught the improvement path to recognize when another instruction candidate is the wrong move.

Current state

WTK already kept operator feedback separate from deterministic verdicts and required candidates to earn qualification before promotion. The remaining weakness was inside the repair path.

After a failed review, an instruction-level candidate could change the output without satisfying the failed obligation. A new candidate might also pass fewer checks than its parent. Without explicit lineage comparison, the system could mistake activity for progress and ask for another rewrite at the same layer.

Changes

WTK now derives a bounded repair plan from the exact reviewed session and its failed goal obligations. Each proposed repair names the observed gap, the responsible contract area, and the acceptance check that would show improvement.

Candidate review compares the new result with its lineage parent. A newly failing check or a lower pass count is a regression. An exact repeat is no progress. A changed but still failing manifestation remains unresolved.

When an instruction candidate cannot resolve the obligation, WTK can stop the instruction loop and identify the next repair layer: output contract or model behavior, capability binding, evidence policy, or goal contract. The prior evidence remains intact, and the operator receives an explicit next decision instead of an automatic retry.

Retesting reuses the captured input when available, so the candidate is checked against the same problem rather than a friendlier replacement. Every additional repair still requires a fresh operator action.

Failures observed

The cycle preserved several forms of non-progress that optimistic repair loops tend to hide:

• a candidate changes its wording but still fails the same obligation;
• a repair improves one check while creating a new regression;
• another instruction rewrite is attempted after the evidence points to a different repair layer;
• a retest uses a different input and makes the candidate look better;
• successful checks disappear from the repair record while attention stays on failures;
• an automatic loop continues without a new operator decision.

Assumptions removed

We no longer assume that changed output means improved output.

We no longer assume that every evaluation failure can be repaired with instructions.

We no longer assume that the newest candidate is the best available fallback.

Evidence

WTK can now bind repair plans to exact reviewed evidence, preserve passing and failing obligations, compare candidate manifestations across lineage, retain the parent as fallback, and stop at a typed repair-layer boundary.

A retained development case progressed through instruction repair and then stopped honestly with one unresolved obligation assigned to a different layer. That is a bounded observation, not a success-rate measurement. The work has not yet been tested across a counted set of goals, models, teams, or runtimes.

Research question advanced

This cycle advances safe improvement loops by making non-progress and regression visible while keeping operator authority outside the candidate. It also informs review reliability because the repair decision remains tied to the exact obligations and evidence that produced it.

It does not establish safe autonomous promotion or long-term evaluator stability.

Next hypothesis

A governed repair loop should spend less effort and create fewer regressions when it can escalate to the correct repair layer instead of repeatedly rewriting instructions. The next experiment will use a declared set of failures across all supported repair layers, retain every attempt, and compare typed escalation with an instruction-only baseline on resolution, regression, abstention, cost, latency, and operator intervention.