Testing Approval Rules in AI Configured Workflows

An agent setting up a purchasing workflow has to do more than create the right forms. If requests take the wrong approval path, the finished configuration can look complete while failing the business rule it was meant to enforce.

OpenAI evaluates the finished workflow

In Advancing computer use with Ironclad, OpenAI describes tasks selected with Ironclad employees and experienced users. The work includes configuring procurement approvals and reusable legal clauses.

One example requires Finance approval above a spending threshold. The agent must configure the rule and check that requests on both sides follow the intended paths. OpenAI describes detailed task criteria and hosted product environments for practice.

That is the useful contribution for WTK: the deliverable is a process that will handle later requests, not just a document or a successful sequence of clicks. Its behavior across relevant situations belongs in the evaluation.

Test the boundary the rule names

Imagine an approved requirement saying purchases above $10,000 need Finance approval. An agent configures the workflow and confirms that a $12,000 request reaches Finance. That check leaves open what happens at $10,000 and below it, or when an amount is missing.

The required cases in this hypothetical example depend on the business specification. The agent should not invent a new policy for ambiguous inputs merely to finish the configuration.

A useful record would connect each required case to the observed path through the configured workflow. Where the expected behavior is unresolved, that uncertainty should be explicit.

Partial progress and required controls

A rubric score shows how much of the task the agent completed. For WTK, we also want to know whether every required approval rule worked. Partial progress can be useful without making the workflow ready for deployment. A good form layout cannot compensate for a missing approval rule.

OpenAI's comparison covers a small set of research tasks, and its time estimates are simulated rather than measured customer savings. The broader question is how to evaluate the rules a business depends on, not just the amount of work completed.

Our earlier note on matching checks to the required result asks whether verification reaches the user's requested conclusion. Here the narrower question is whether a configured process preserves the rule across its branches.

Our note on repeated business workflow reliability considers a complementary question: how consistently does an agent complete the work? A rule can be incorrectly configured, and an agent can inconsistently execute a correctly defined task.

The question for WTK

WTK's catalog describes an agent or team package. Compilation prepares it for a target environment, whose runtime supplies the actual capabilities. Qualification evaluates that specified setup; it does not automatically establish every behavior of a workflow the agent creates.

Could its evidence show which approval rules were tested, which situations were covered and which required behaviors remain unresolved? If the configured workflow changes later, which checks need to run again?

This supports higher-level oversight. People define business rules, acceptable exceptions and limits on changes. Agents can configure and inspect the process within that scope, while failures or unresolved policy choices return to the appropriate authority. Human oversight need not mean approving every click.

WTK has not tested this Ironclad workflow. The question for our research is what qualification evidence would show that a configured process preserves its required rules.

Source

OpenAI, Advancing computer use with Ironclad, October 6, 2026. The threshold example and proposed WTK evidence requirements are our interpretation of its engineering account.