Retaining Agent Skill Edits After a Rejected Revision

Rejecting an agent update answers whether that version should replace the current one. It does not tell a builder which individual edits might be worth trying again.

The October preprint EVISKILL studies that distinction for skills, reusable procedural instructions outside the model's weights. Its improvement loop can retain selected edits from a rejected revision for further development while leaving the validated skill unchanged.

Keep the proposal separate from the rejected combination

Consider a hypothetical enterprise access-request agent. It collects required details and routes requests to the designated system owner; it cannot grant access itself.

A proposed skill revision adds two instructions: request a missing expiry date and simplify owner selection. Evaluation finds that the combined revision routes some requests to the wrong owner, so it is rejected.

The expiry-date instruction might still help. To investigate it, start from the accepted skill and apply that instruction alone, rather than replaying the rejected combination. Check the relevant request context and retain the result with the exact edit. A later candidate still needs evaluation of the complete workflow, including correct routing and the prohibition on granting access.

That differs from reversing the rejection. The original candidate failed; the isolated edit is a proposal for another candidate.

How EVISKILL carries edits forward

EVISKILL links proposed changes to evidence cards that identify the source task, relevant execution steps and intended correction. Replay reconstructs the earlier context and lets the agent act again under the edited skill.

After a complete revision is rejected, selected edits undergo another replay against the validated skill without the other provisional changes. Supported edits can enter a separate working version used during subsequent training interactions. Only a later improvement on the validation set updates the validated skill. Unpromoted edits are excluded from final test-time use.

Across 18 dataset-and-model configurations over four rounds, the researchers tracked 155 edits from rejected revisions. Of these, 128 received post-rejection replay, 96 were accepted locally and 70 later entered globally accepted skills. The other 26 locally accepted edits remained provisional when the budget ended.

Those are selected, pooled edit counts, not independent replications or proof that each incorporated edit caused an improvement. Extraction, editing and replay judgment used GPT-5.5 with medium reasoning. Replay covered at most three supporting ranges per edit, and its judge could tolerate failures deemed unrelated to the edit. Local support therefore remains narrower than evidence about the complete workflow.

The question for WTK

Our repair-prediction note asks whether a change produced its stated behavior. EVISKILL adds a lifecycle question: what happens to that proposal when the larger revision containing it is rejected?

For WTK's cataloged agent and team packages, could an improvement record retain an edit's source version, supporting observations and later revisions without inheriting the rejected candidate's qualification? The package-revision log already explains why changed source must earn fresh evidence.

People could define the goal, permitted changes and release requirements while automation organizes proposals and gathers test evidence. Uncertain attribution, conflicting results or proposed authority changes would remain exceptions for review, rather than requiring a person to supervise every edit.

The useful distinction is between retaining something worth investigating and accepting something ready to run. Revisit that distinction whenever the package, model, execution environment or evaluation criteria change.

Source

Yan Zhou, Yili Wang, Yiwei Dai, Qinggang Zhang and Xin Wang, Jilin University, EVISKILL: Grounding Skill Evolution in Replayable Evidence, October 4, 2026, version 1. The enterprise example and WTK interpretation are ours.