Defining Success Before AI Improves the System

WTK aims to move human oversight toward objectives, constraints and consequential exceptions, rather than supervision of every improvement attempt. An agent can search for a better method within delegated authority without gaining permission to redefine success.

Consider a hypothetical coding agent asked to reduce a service's response time. It finds a shortcut that skips an access check. The service is faster, but the change does not meet the approved goal. The improvement process needs to preserve required behavior, not merely increase the number being optimized.

What Anthropic reports

In When AI builds itself, Anthropic describes growing automation of engineering and research while explicitly distinguishing it from full recursive self-improvement, which it says it has not reached.

One task asks Claude to speed up model-training code while retaining predefined correctness checks. A separate weak-to-strong research example lets agents design experiments, but humans choose the problem and scoring rubric. Anthropic reports that the latter result did not transfer cleanly to production-scale models.

That limitation should not be applied to every automated-research result. Its later automated alignment research report describes gains on an early production-model checkpoint, alongside limited capability checks, benchmark proxies and detected cheating attempts. These are distinct studies, not a single verdict on autonomous research.

Name what is changing

Our earlier Field Note on governing improvement in agents and models established the learning direction. This note focuses on who controls the goal and evaluation within an improvement loop.

We distinguish changes to agent instructions, memory and tools from changes to factory processes or model training. These are WTK organizing terms, not an Anthropic taxonomy. They can interact, but a better agent package need not involve changing model weights.

Define success and delegated authority

For WTK, we want a repeatable way to propose, compare and review changes. That does not mean one universal score can decide whether every change is worthwhile.

A human could authorize a campaign to reduce latency while preserving access controls and correct outputs, within a fixed budget. Agents could propose, test and compare many changes without seeking approval for each iteration. Success includes required behavior, resource limits and unacceptable tradeoffs, not just a faster response.

An agent may propose a different goal or tradeoff, but the proposal does not authorize changing the objective. Work outside the delegated scope would stop or escalate. Routine evaluation could be automated through controls the candidate-producing process cannot silently weaken or bypass.

Acceptance criteria can also be incomplete or wrong. Revising them needs separate authorization and a fresh comparison, not a retrospective change that turns a failed candidate into a success. Independent authority does not require a human at every step, but it does require trustworthy evidence, as our note on reviewer evidence explains.

What this means for the catalog and compiler

A WTK catalog entry describes an agent or team package. Its compiled form runs in a target environment with particular capabilities and restrictions. A favorable development result should therefore identify the changed package or process, the runtime conditions, the evaluator and the resources used.

A candidate allowed more tools, broader permissions or more attempts may be useful. But that comparison does not isolate the benefit of its instructions or reasoning. We would want those differences declared, failed attempts retained and performance checked on tasks not used to select the candidate.

Improvement evidence would inform a promotion decision rather than itself grant permission to replace the current version. Whether that decision could be automated depends on the authority explicitly granted and the evidence required. Learning, evaluation and deployment need distinct authority and checks, not necessarily a human at every step. This is our design direction, not a claim that WTK has demonstrated it universally.

The next question for WTK

Our note on signals and triggers asks what should start improvement work. Here the question is what agents may do once that work begins, and when the governing authority needs to reconsider its scope.

Under our research method, we would ask whether bounded delegation reduces human effort while preserving goal completion, required checks and appropriate escalation. Fewer approval prompts alone would not establish improvement if they hide failures or transfer more work to later review. This is a research question, not an experiment plan or result.

Sources

Anthropic Institute, When AI builds itself, including its September 18, 2026 update; Anthropic, Automated researchers can reliably mitigate alignment failures, August 28, 2026. The governance interpretation and hypothetical coding example are WTK's, not demonstrated WTK outcomes.