Reviewing Deployment Safeguards for Distilled Models

When an agent switches to a distilled model, its evaluation needs to cover the new model and the service that runs it. Distillation means training a model on responses from another model. Useful answers do not establish that the original service's controls are still present.

Suppose a coding assistant switches to a smaller model trained on authorized examples and still handles ordinary bug fixes well. If the original service screened requests and restricted reuse of protected conversation material, similar coding answers would not show that those protections remain. The replacement service needs its own evidence for those controls.

What Anthropic reports

Anthropic's October 12 technical report describes defenses against unauthorized extraction of Claude's reasoning. Its measures combine restrictions on reasoning reuse, a two-stage classifier, and account enforcement. Anthropic estimates that extracted reasoning tokens on its own platform fell by a factor of 10 to 100 after classifier introduction. The accompanying chart normalizes daily exposure against a measurable quiet day; it does not report absolute token counts.

The report acknowledges integration changes for legitimate users, occasional benign blocks, and charges for blocked requests. It also says durability is unknown and a highly sophisticated, persistent circumvention effort has yet to occur. These are provider-reported observations, without a public raw traffic ledger or a controlled comparison isolating every defense.

The WTK question

Our governed-improvement research direction includes learned model changes alongside changes to instructions and tools. This report sharpens that direction: which protections came from the original model, which came from its service, and which remain when the replacement runs elsewhere?

WTK's catalog describes agent and team packages. Compilation prepares a package for a target environment, whose runtime supplies the capabilities and restrictions used during execution. Qualification means evaluating a specified setup. A learned replacement would therefore need evidence for the model binding and surrounding service that actually run, as our runtime-configuration note explains.

We would want to distinguish learned behavior from service enforcement. Does the replacement decline an out-of-scope request? Does an external control block an unauthorized effect even if the model requests it? Can the normal workflow still complete when those controls intervene? A favorable answer to one question would leave the others open.

Humans would set the objective, constraints, escalation conditions and promotion decision. Agents could carry out work within those boundaries. The assessment question is whether a model change preserves that arrangement, including useful completion and required escalation, across the receiving runtime.

WTK has not evaluated a distilled replacement through this approach. An assessment would need an identified candidate, authorized training-data provenance, the receiving service's observable controls and acceptance criteria. Missing visibility would limit the conclusions.

Source

Anthropic, Strengthening our defenses against distillation attacks, October 12, 2026; canonical report and figures reviewed October 13. The proposed deployment questions are WTK's interpretation. New model bindings, service controls or integration behavior would require renewed assessment.