Keeping Review Diagnoses Intact in Repair Guidance

An agent review can identify a problem correctly and still give the operator misleading instructions for fixing it. WTK caught this at the handoff from the recorded review failure to the repair plan: the review distinguished an invalid returned answer, but the plan still described it as a reviewer execution failure.

Current state

Our earlier change separated reviewer execution errors from invalid results. A reviewer that disconnects before returning an answer is different from one that completes its call but returns evidence that fails validation. Both leave the review incomplete.

That distinction was present in the review record. It was not consistently preserved in the explanation attached to the next repair step. The factory had a more precise diagnosis than its repair guidance showed.

Changes

The merged correction carries the invalid-result diagnosis into newly generated repair plans and gives it a matching explanation. Execution failures retain their existing description. Earlier records are preserved rather than rewritten to fit the new wording.

Consider a reviewer that returns a passing answer associated with a different evaluation. WTK rejects that answer. The repair plan now explains that the call completed but its result was invalid, instead of saying the evaluator did not complete successfully.

The required response stays the same: inspect the evaluation path, retain the failed attempt and keep the agent package unchanged. A misleading diagnosis could prompt an unnecessary package change; this correction improves the explanation without granting permission to retry or rewrite the package.

Failures observed

The product checks reproduced the lost distinction in both the failure classification and the repair proposal. An independent review also exercised invalid returned answers alongside exceptions, missing adapters, older error records and legitimate semantic results.

That review exposed errors in its own test fixtures before the comparison was usable. The original fixtures and later corrections were retained. Those corrections limit the independence claim; this was narrow regression coverage, not a complete independent evaluation of the factory.

Assumptions removed

We cannot assume that a correct diagnostic record guarantees a correct explanation downstream. The explanation needs to preserve what failed, which evaluation it belongs to and the evidence attached to it.

This continues the practical question in A Gate Failure Has to Name Its Own Layer. A later summary should not erase a distinction that the preceding stage already established.

Evidence

Focused product checks and the offline repository suite passed against the exact merged revision. A separate shared-handler check confirmed that the returned repair plan matched the persisted plan and that the agent package remained unchanged. Existing readiness and operator-decision checks remained in place.

These records support the implemented diagnostic correction. They do not establish live interface behavior, reviewer accuracy, successful recovery, fewer package edits or lower cost. The broader work on generation, execution and scoring attribution remains open. WTK's research method keeps that distinction visible.

Next hypothesis

The practical question is whether preserving the diagnosis in repair guidance helps an operator choose the responsible correction without rerunning unrelated checks. That benefit has not been measured, and this engineering cycle does not specify or authorize a new experiment.