A Portable Package Does Not Qualify Its Runtime

A governed agent package can carry the same purpose, authority, and evidence obligations into two runtimes and still behave differently. If the harness controls what actions happen and what proof counts, the runtime is part of the safety claim.

The signal

Agent Safety Should Be a Runtime Contract argues that training-time safety is structurally insufficient for agents that execute code, change files, send messages, and modify databases. The authors propose a harness-enforced contract with two faces. The preventive face blocks dangerous actions. The evidential face requires checkable proof that the requested work actually happened.

The paper grounds that position in four public audits: 52 documented agent and language-model safety incidents, 31 non-contested false-completion cases plus one disputed illustrative case, trajectory schemas from 12 public systems and harnesses, and the titles of 28,560 papers accepted at three major machine-learning conferences from 2023 through 2025. The title audit reports 8 to 12 times more training-oriented than deployment-oriented publication.

The evidence

The useful idea is not that a monitor makes an agent safe. It is that the unit being evaluated should include the trajectory and its checkable evidence. A model label cannot tell us whether a file write was contained, a permission was enforced, a test actually ran, or a completion claim was supported.

That fits an uncomfortable result already on our radar. OpenART found meaningful safety variation across runtime implementations even after accounting for the underlying model. Together, the papers make the same pressure visible from two directions: a portable artifact does not erase the behavior of the harness that executes it.

The boundary

This paper is a position and synthesis paper, not a controlled WTK experiment. Its incident survey does not estimate a production failure rate. Its schema audit does not show that two runtimes are behaviorally equivalent, and paper titles cannot measure the quality of deployed safety controls. A trajectory receipt can also prove that an event was recorded without proving that the event was correct or sufficient.

So we should not read this as evidence that WTK's current runtime is unsafe. We should read it as a precise challenge to any safety claim that stops at the package boundary.

The builder impact

Our take: builders should version and identify the harness alongside the package, then keep preventive and evidential claims separate. A sandbox denial, an approval gate, a test receipt, and a grounded completion claim answer different questions. Combining them into one green status hides which part of the contract actually held.

This does not require another governance subsystem. It requires refusing to upgrade artifact portability into runtime equivalence without measured evidence.

The WTK test

WTK already binds packages, contracts, evaluators, and runtime identities to exact evidence. The missing test is comparative. Hold the goal, package, authority, model, fixtures, and evaluator plan fixed. Run them through at least two harness implementations, then compare denials, false completions, evidence sufficiency, task outcomes, cost, and latency.

The conclusion should remain target-bound. A passing runtime earns a claim about that exact compiled target under the tested conditions. It does not lend its proof to another harness with the same package name.

Still unknown

We do not yet know how large the harness residual is for WTK packages, which controls account for it, or whether equivalent receipts imply equivalent enforcement. That is the next experiment: measure the runtime difference instead of treating portability as evidence that the difference disappeared.