Who is responsible when an AI agent acts?

An AI agent can finish its task and still do something the organization never intended to authorize. We think the useful question is not just whether an agent works, but who is responsible for what it may do, how that decision is enforced, and what happens when it changes.

This is our proposed operating model, not a report of a completed enterprise deployment. It connects WTK's work on useful agents and agent teams with the responsibilities organizations would need to retain as more work becomes automated.

Start with the work and its consequences

Consider a support team using agents to investigate a customer complaint and draft a resolution. One agent retrieves records, another checks policy, and a third prepares a response. The team may recommend a refund without being authorized to issue it.

That difference matters even if the same tool could perform both actions. A request to resolve a complaint should not silently grant permission to move money, disclose unrelated records, or delegate those permissions to another agent.

We would begin with a named business owner, the intended outcome, the permitted actions, and the point at which a person must decide. Usefulness belongs in that definition too: an agent that refuses every request is constrained, but does not accomplish the work.

Review the deployed system, not just the agent description

The same agent can behave differently when its model, tools, permissions, or execution environment change. We use execution form to mean that complete deployed combination, including its version, purpose, and applicable controls.

Our proposal is to attach a review decision to that combination rather than treat approval as a permanent property of an agent's name. A team also needs review of its handoffs and combined authority, not only its individual members.

The target compilation pattern explains how WTK proposes to preserve a common source while producing environment-specific deployments. It is an architectural reference, not evidence that different environments enforce equivalent behavior.

Make the decisions explicit

The business owner defines the purpose and acceptable consequences. Engineering implements the agent and its tests. Security and platform specialists establish and enforce the applicable restrictions. A designated reviewer assesses the evidence, and an authorized human approves promotion. Operations needs a practical way to suspend execution and investigate what happened.

These responsibilities need not become separate departments. They do need named decision owners and separation where one actor could otherwise create a change and approve its own evidence. A vendor can supply capabilities and records, but the organization still needs to decide whether those are sufficient for its use.

The separation-of-duties principle describes WTK's intended authority boundary. Organizational separation alone does not establish reliable evaluation or prevent correlated mistakes.

Improve without quietly expanding authority

Suppose the support team repeatedly misses an exception in the refund policy. An improvement loop could propose a revised instruction, produce a candidate, and test it against ordinary cases and the missed exception.

That candidate should also be tested for regressions, including unauthorized refunds and unnecessary disclosure. A better task score would not justify giving it broader access. Nor should the candidate be allowed to weaken its own acceptance criteria.

WTK's current position keeps promotion independently governed and human-authorized. The mutation-versus-promotion principle explains why proposing and testing a change is different from authorizing its use. Progressive automation means removing specific manual steps as their conditions become defensible, not silently removing oversight from the whole process.

What would make this approach credible?

We would want evidence that a team delivers useful work, stays within its authority, can be stopped, and leaves enough records for someone else to reconstruct a decision. We would also measure the cost: review time, runtime overhead, unnecessary refusals, and the effort required to maintain the controls.

The strongest objection is that this could become an expensive approval process with little effect on actual behavior. That is a testable objection. If comparable deployments achieve the same outcomes and containment with simpler controls, we should narrow the approach.

The sections below develop the ownership model, an illustrative improvement sequence, and a proposed WTK evaluation. Our research method requires bounded results and preserved failures before we claim measured effectiveness.

We would welcome examples where this division of responsibility works, breaks down, or can be made simpler. A sanitized account of the decision, controls, and observed outcome would be more useful than a general endorsement.

Assigning responsibility

Our proposal separates decisions from the functions supplying advice or implementation. Before deployment, each decision below would have one named accountable owner. The roles are a starting point for adaptation, not a universal organization chart.

Qualification means an evidence-based decision about suitability for a defined use. Deployment approval means authorization to put that version into that environment. Neither decision substitutes for the other.

An owner can delegate execution of a control without leaving responsibility ambiguous. Emergency containment needs a predefined path that does not wait for routine promotion approval.

What the disciplines contribute

• Security architecture: define the boundaries between agents, tools, users, environments, and external providers; identify where controls must be enforced outside the agent.
• IAM and privileged-access teams: associate actions with the acting workload and its delegator; issue scoped credentials; test expiry and revocation.
• Application and product security: review instructions, dependencies, tool interfaces, adversarial inputs, memory, and delegation behavior; contribute tests without making engineering self-approval the final gate.
• Cloud and platform teams: implement isolation, tool access, network restrictions, secret handling, resource budgets, deployment bindings, and containment.
• Data security and privacy: define permitted data use, retrieval scope, retention, disclosure, and cross-agent sharing. Collecting evidence must not become a reason to retain all sensitive inputs.
• SOC and incident response: connect alerts to the affected run and authority; preserve relevant records; practice containment and recovery.
• GRC and governance: maintain policy, risk classification, exceptions, and review conditions. Track unresolved exposure rather than merely completed forms.
• Third-party risk and procurement: assess what a provider exposes, what the organization can constrain, and what happens when the provider changes its service. Contractual assurances and runtime evidence answer different questions.
• Engineering and agent owners: define useful outcomes, build the candidate, maintain reproducible tests, disclose limitations, and supply versioned evidence.
• Independent assurance: challenge whether the supplied evidence supports the proposed scope. Separate review authority from candidate production and declare conflicts or shared evaluator dependencies.
• Internal audit: examine whether decisions and controls operated as described. In this proposal audit does not become the routine deployment approver whose decisions it later examines.
• Legal and business leadership: determine applicable obligations, business intent, acceptable consequences, and resources. The technical model does not establish legal compliance.

Following one team through a change

Use the support-team example from the overview. Its approved task is to investigate and draft a resolution. A person issues any refund. These steps describe intended behavior, not an observed WTK run.

1. Define success and limits. Record the permitted customer records, response-quality criteria, prohibited disclosures, and the absence of refund-execution authority.
2. Identify the deployed combination. Bind agent and team versions, model, instructions, tools, credentials, environment, and evaluation suite to the review record.
3. Test the whole team. Include correct resolutions, unsupported requests, malicious retrieved instructions, and a delegated request that attempts to exceed the original scope.
4. Review and authorize. Preserve failures and limitations. A designated reviewer decides whether the evidence supports the proposed use; the human deployment approver authorizes the accepted version.
5. Observe the outcome. Capture the necessary actions, policy decisions, approvals, and output references. Confirm whether the response actually met the user's need.
6. Generate an improvement candidate. A missed policy exception motivates a changed instruction. Keep the deployed version unchanged while the candidate is tested.
7. Retest and decide. Compare the candidate with the prior version using fixed criteria. A quality gain does not override an authority violation. Rejection or an inconclusive result remains part of the record.

If the team moves to another runtime, review which controls and tool behaviors changed. Reuse the package and relevant evidence where justified, but do not assume the previous deployment decision transfers unchanged.

What to inspect

An evidence record should let a reviewer connect the request, authorized purpose, deployed version, delegated authority, controls applied, actions taken, and observed outcome. It should also state what was not observed.

Evidence should use controlled access, minimized content, and a declared retention policy. A public report would contain a sanitized record and explain any withheld material and the resulting verification limits.

A proposed comparison

Our initial question is whether explicit authority boundaries and independent change approval reduce unauthorized actions while preserving useful task completion. This would test one bounded implementation, not the entire organizational model.

Compare current WTK behavior, with its existing safeguards intact, against a candidate that binds each consequential action to a scoped grant, preserves that scope through delegation, and checks separate approval when an improved version is promoted. First document what the baseline already enforces; add only the missing candidate mechanism. Both arms use simulated consequential actions in an isolated WTK execution environment.

Run matched comparisons for a single support agent and for a coordinator, researcher, and reviewer team. Compare baseline and candidate within each configuration; do not attribute differences between a single agent and a team to governance alone. The goal is to learn whether adopting this idea improves WTK's useful, authorized outcomes. It is not a reproduction of an external paper or a demonstration assembled to validate the proposal.

Hold task inputs, agent versions, model settings, tool fixtures, evaluator criteria, and budgets constant. Fix the case set, repetitions, randomization where supported, timeout treatment, and stopping rules before execution. Retain every planned attempt, including errors and refusals. The numeric attempt count and acceptance thresholds remain to be preregistered; this is not yet an executable protocol.

Measure useful completion, unauthorized attempted and completed actions, unnecessary refusals, containment time, missing evidence, review effort, latency, and token or monetary cost. Evaluate improvement candidates against the same criteria rather than allowing the candidate to redefine success.

Predeclare how to handle a baseline with zero unauthorized outcomes: the experiment may establish bounded non-regression, reveal a cost penalty, or remain inconclusive, but it cannot claim a safety reduction that was not observed. A combined intervention also cannot identify which individual control caused a change; that requires a later comparison changing one control at a time.

The approach should be revised if the control path can be bypassed, reviewers cannot reconstruct consequential actions, or useful completion falls below the preregistered tolerance without the required reduction in unauthorized outcomes. Shared model errors and evaluator blind spots may limit both arms. A successful test would not demonstrate independence across unrelated environments or providers.

Material changes to the model, tools, permissions, delegation structure, evaluator, or runtime should trigger a review of which tests must be repeated.

Supporting architecture and remaining evidence gaps

These references explain the design. They are not empirical validation of this operating model:

• Mutation versus promotion: separates candidate changes from approval to advance them.
• Separation of duties: defines intended separation of production, evaluation, and authority.
• Promotion pipeline: describes gate evidence, version changes, and decision records.
• Target compilation: describes environment-specific projections from an authoritative source.
• Research and falsification method: defines what a bounded result must disclose and what would change the conclusion.

This Field Note does not attach an enterprise deployment study, a measured control comparison, or a validated cost estimate. Those remain evidence gaps. Future Findings or Failure Reports should link back here and state which part of the proposal their results support, narrow, or contradict.