Checking Known Failure Modes During AI Agent Team Review

An agent team can identify a likely problem during planning and still miss it in the finished work. Listing a risk is useful, but review needs to establish whether that risk is present in the current output.

A known artifact survived two reviews

In Anthropic's October 8 The missing map of the sky, astrophysicist Brice Ménard describes directing Claude Science to combine ultraviolet sky surveys and estimate missing regions.

The team produced a map, but Ménard noticed faint circular patterns left by individual observations. Residual atmospheric glow had not been fully removed. Claude had identified this as a known issue at the start, yet two reviews by other agents had not caught it. After his feedback, the team corrected the observations.

The account also describes separate measured and predicted layers with uncertainty estimates. A complete-looking image therefore does not mean every region was directly observed.

This first-person account documents a review miss, but does not establish its cause or how often similar misses occur.

Connect a known risk to a concrete check

For WTK, the interesting detail is that the problem was already named. More planning alone would not necessarily address the gap.

Consider a hypothetical team that merges monthly reports. During planning, it identifies duplicate rows as a likely problem. The team later approves the merged file, but no review record shows which duplicate keys were checked or whether apparent duplicates were resolved.

A useful review would inspect that specific risk against the current artifact. Its evidence could support absence, presence or an unresolved result. A general statement that the file looks good cannot settle the duplicate-row question.

That connects to our requirement-aligned verification note, but adds a different point: a recognized failure mode needs an actual check, not merely a place in the plan.

Higher-level oversight can remain practical

Ménard describes giving high-level direction while agents performed extended computations. That working pattern puts human attention on direction and consequential corrections rather than every processing step.

Our question is how automated review can make that oversight more effective. Humans could set the required quality boundaries and escalation conditions. Agent teams could then supply evidence about those boundaries, with unresolved checks visible for consequential decisions.

WTK defines agents and teams as catalog packages compiled for target environments. For a team review, we would want to know which output version was inspected and what evidence answered each relevant concern. As our independent-review note explains, another reviewer is useful only when its evidence makes the decision answerable.

Measured inputs, inferred values and reviewer conclusions should also remain distinguishable. In the sky-map account, those distinctions matter because filling a gap does not turn a prediction into an observation.

What remains open

A risk checklist can itself be incomplete, and a check can miss the defect it was intended to catch. WTK has not tested a review mechanism based on this case.

The question for WTK's research is concrete: does the review show what happened to the known concern in the current output? That is more informative than counting how many agents approved the work.

Source

Brice Ménard, The missing map of the sky, Anthropic, October 8, 2026, especially “Trial and error.”