The Leak Can Happen Before the Answer

An agent can keep sensitive information out of its final answer and still have collected more than the job required. That changes where we look for privacy risk: the first important boundary may be the tool call, not the reply.

The signal

PrivacyPeek examines acquisition-stage privacy leakage in tool-using agents. Its benchmark inspects both the tools an agent invokes and the data returned to it, then uses a follow-up probe to test whether unnecessarily acquired information can later be elicited.

The authors report 1,182 cases across seven acquisition behaviours and 16 application domains. In their experiments across 10 agents from four model families, unnecessary acquisition was widespread, and prompt-level defences reduced only a small fraction of the measured leakage.

The evidence

The paper puts a useful hole in an answer-only safety story. A response filter can stop an agent from repeating a sensitive value, but it cannot make that value leave the execution context once a tool has returned it. A later careless action, compromised component, or follow-up prompt has a larger attack surface precisely because the information was collected earlier.

For WTK, the question is narrower than "can an agent keep a secret?" A governed task should be able to state the minimum information it needs before calling a tool. The subsequent evidence should make it possible to inspect whether the requested and received fields stayed within that declared need.

The boundary

This is external benchmark evidence, not a WTK result or a general privacy rule. It does not show that every additional field is impermissible, that the benchmark represents production workflows, or that every target can expose complete acquisition receipts. The reported correlation between task completion and leakage also does not establish cause.

The paper does not establish that WTK currently classifies data sensitivity correctly, that a declared tool contract alone prevents over-acquisition, or that a prompt warning is a sufficient control. It identifies a failure mode worth testing under explicit conditions.

The builder impact

Tool permissions are necessary, but they are not the entire question. A permission can tell us that an agent may use a source. It does not by itself say which fields the task actually needs, whether a request asked for extras, or whether a broad response delivered them anyway.

That suggests a compact review habit: before a sensitive tool call, name the minimum fields needed for the stated goal. After the call, compare the requested and returned fields with that declaration. If the run collected more, the evidence should retain an accountable limitation, refusal, or degraded result instead of treating a tidy final answer as proof that nothing went wrong.

The WTK test

WTK can test this with a small, synthetic fixture set. Hold the goal, tool contract, and target form fixed while giving the tool schema both necessary and tempting extra fields. For every planned attempt, retain a sanitized record of the requested fields, returned fields, necessity decision, and allow, deny, or degrade outcome.

The test should distinguish a useful response from a minimal acquisition. It should also check that a missing or incomplete receipt cannot be mistaken for proof of restraint. No real credentials, customer information, or sensitive records are needed for that comparison.

Still unknown

We do not yet know which acquisition rules produce useful safeguards without blocking legitimate work, how consistently targets can provide complete receipts, or whether a field-necessity check catches a failure mode that existing authority tests miss. The next evidence should be a bounded fixture comparison, not a claim that WTK has solved privacy.