A Loop That Forgets Can Be Patiently Fooled

An agent can pass every single-run safety check and still be led into a bad action one ordinary-looking step at a time. That is the uncomfortable result in a new study of autonomous loops, and it matters whenever an agent keeps a work queue, memory, retries, or authority state across tasks.

Imagine a maintenance agent that receives a harmless-looking task today, an ambiguous receipt tomorrow, and a plausible follow-up next week. If the evidence is only dangerous when those records are connected, resetting the safety view between tasks can turn a long con into a sequence of clean-looking moments.

The signal

The authors of Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents make a precise claim: a monitor restricted to one trajectory cannot distinguish an attack whose evidence is distributed across iterations from benign work with the same local view. Their proposed response retains a narrowly defined risk signal at the loop level rather than treating every new task as a blank slate.

In the paper's matched cooling-off test, a decaying retained score blocked the target after one evidence-free interval, then released it after two. The reported attack-success rate moved from 0.0% to 97.0%. A latching version retained the risk signal and stayed at 0.0% across the tested intervals. That is a strong contradiction to the comforting shortcut that some remembered risk is automatically enough.

The evidence

The study uses a 200-task cohort and paired clean and attack episodes. Its full configuration reported 0.1% overall attack success with 96.9% clean target completion in its stated outer-state matrix, while comparison configurations ranged from 88.4% to 97.6% attack success. Those are results for the paper's benchmark, models, controller, and attack contracts, not rates for WTK.

Our take is narrower. WTK's package and target qualification can describe a single compiled execution form precisely, but that alone does not establish the behavior of a long-running loop that carries history forward. A cross-iteration state boundary is part of the execution form that needs its own evidence. Target compilation and qualification evidence should make that boundary inspectable instead of assuming it from a package or a passing smoke test.

The boundary

The paper is a preprint with one frozen model-role assignment, one execution seed, and no multi-model or multi-seed replication. Its formal bound also depends on mediated irreversible calls, bounded commits between governor checks, a non-decreasing counter, and a stated arbiter detection floor. The finite red-team grid probes that floor, but does not prove it.

The authors also identify the important tradeoff: their evaluated latch has no authenticated clearance transition. A safety mechanism that can only halt can become an availability failure. The result does not show that WTK has this defect, that every retained state must never decay, or that the paper's mechanism transfers to any WTK target.

The builder impact

For a long-running agent, retain the difference between risk evidence, ordinary task importance, and reviewer uncertainty. A high-impact but permitted task is not itself proof of compromise. A past rejection is not new authority. The useful design question is whether a later action can name the earlier evidence that should still constrain it, and whether a scoped, authenticated clearance can retire that constraint.

The WTK test

We propose a bounded target-harness comparison: the actual current per-run state policy, a declared decaying retained-state policy, and a retained-state policy with an explicit authenticated, scope-specific clearance transition. Hold the package and contract digests, target, tools, permissions, action budget, fixtures, task sequence, evaluator, and stop conditions fixed.

Measure unauthorized effects, clean completion, false holds, correct clearance, evidence integrity, and every denial or escalation. The candidate fails if a patient cross-iteration fixture can reopen an unsafe path, if clearance grants more authority than its scope, or if the control loses acceptable work without a declared reason. This is a proposed WTK experiment, not evidence that WTK has the problem or permission to run a test.

Source

Primary research: Safety Does Not Compose: Non-Decaying Loop State for Autonomous LLM Agents, Chenhao Wu and colleagues, August 2026. WTK reviewed the complete 22-page primary PDF. This Research Note translates external evidence into a testable WTK question; it reports no WTK experimental result.