Evaluating Security Agents by Additional Verified Findings

Adding another security agent can increase the number of reports without uncovering additional vulnerabilities. Its useful contribution depends on what it finds beyond the existing tools and reviewers.

Anthropic's report on discovering and exploiting vulnerabilities with Claude describes running several scanning approaches in parallel. The team assesses a scanner by the additional vulnerabilities it contributes to the mix. Approaches include examining functions, pursuing particular security goals, reasoning from documented trust boundaries and looking for variants of confirmed findings.

The report also describes keeping a standard verification and risk-rating process while trying new scanners. That separation matters: changing how candidates are discovered need not change what counts as a verified vulnerability.

This is an operational account using an unreleased internal research model. It provides neither a controlled comparison of scanner strategies nor enough cost and workload detail to establish a generally best combination.

Count the additional problem, then the work it creates

Consider a hypothetical retailer introducing an agent that reviews its customer-account service. Existing scanners already identify several access-control weaknesses. A new agent may describe those same issues more persuasively, produce duplicates under different names or discover a separate path that exposes another customer's records.

Those outcomes have different value. Clearer explanations can help remediation, but duplicate reports should not become additional coverage. A genuinely new finding needs evidence that the path exists under the service's actual permissions and configuration.

A large report count can hide a gap in protection if the scanners repeatedly inspect the same paths. Review effort also matters; an agent that produces many unsupported reports can consume the time needed to investigate stronger leads.

Our vulnerability-report review note follows a candidate report through reproduction and repair review. This source adds a question about selecting the agents that produce those candidates.

Evaluate the contribution to the team

For WTK, a catalog entry can describe a security agent's intended role, required tools and permitted scope. Evaluating that role would also need to identify the existing scanning coverage against which its contribution is judged. A finding that is additional today may become routine coverage after another scanner improves.

Our team-contribution note asks what each agent adds to a shared outcome. Security work gives that question a tangible form: which additional, verified weaknesses become visible, at what severity and review cost?

Discovery remains separate from permission to change a production service. Keeping that distinction allows agents to broaden investigation within approved limits while repair decisions follow the applicable release process.

Source: Anthropic, Discovering and exploiting vulnerabilities in critical, real-world software with Claude, October 27, 2026. The retailer example and WTK interpretation are ours.