A production approval workflow is not an Approve button added before an agent calls a tool. It is an operating control: a defined point where automation stops, an authorised person receives decision-grade evidence, and the system waits without losing state or creating duplicate work.
The design challenge is therefore not to maximise human involvement. Too many low-value requests produce queues and rubber-stamping; too few leave consequential decisions to an agent without adequate oversight. The objective is controlled delegation: automate bounded work, require judgment where consequences are material, and prohibit actions that exceed current policy or technical safeguards.
Gate actions by consequence, not confidence
Model confidence is a weak foundation for operational authority. It may be unavailable, poorly calibrated or unrelated to the cost of a mistake. An agent can be highly confident while sending an accurate message to the wrong recipient. It can be uncertain while preparing a reversible internal draft.
Classify the proposed action instead. The policy should consider:
- Reversibility: Can the action be undone completely and quickly?
- Blast radius: Will it affect one record, one customer or an entire environment?
- Data sensitivity: Does it use personal, financial, confidential or regulated information?
- External commitment: Will it publish, send, pay, refund, promise or change a deadline?
- Privilege: Does it alter access, permissions or production configuration?
- Novelty: Has this action pattern been tested under representative conditions?
- Evidence quality: Is the proposal grounded in current, authoritative records?
- Policy exception: Does it cross a limit or require discretionary judgment?
Use those dimensions to assign one of four operating modes: auto-execute; execute with retrospective sampling; require approval before execution; or prohibit. A low-consequence, reversible update may earn narrow autonomy after testing. A payment destination change should remain gated. An action outside legal or security policy should not become permissible because one person clicks Approve.
This makes the autonomy boundary legible across operations, security, legal and engineering. An action can move between modes when incident data, evaluation results and control maturity justify it.
Make the approval contract explicit
Define each gate in plain language before selecting a framework or designing the interface. The approval contract should state:
- the exact action being paused and the policy rule that triggered review;
- which roles may decide, including value or scope limits;
- the available decisions: approve, reject, edit, request information, reassign or escalate;
- how long the decision remains valid and what happens at expiry;
- what evidence must be retained;
- which conditions must be checked again before execution.
Approval must bind to a specific, immutable proposal. “Approve refund” is not sufficient. The decision should cover the customer, amount, currency, reason, payment destination, source records and request version. If a material field changes, the original decision no longer authorises the action.
The same rule applies to communications, access changes and production operations. Approval of an email body does not authorise a different recipient list. Store the approved payload or a verifiable digest and compare it at execution.
The reviewer also needs an approval packet rather than the agent’s self-authored summary. Show the exact operation and target, a before-and-after diff, the business reason, source evidence, the triggering risk, affected people and systems, expiry, and any uncertainty or conflicting information. Include recovery options and meaningful alternatives, such as reducing the scope or returning the request for correction.
Progressive disclosure helps: lead with the decision, consequence and exact change, while keeping detailed evidence easy to inspect. Do not use interface defaults that steer people towards approval.
Persist first, pause safely, execute once
Approval can take minutes or days. Treat the wait as a durable business workflow, not a process held in memory.
Before contacting a reviewer, persist the workflow ID, action ID, immutable payload and version, policy result, evidence references, eligible approvers, status, timestamps, expiry and idempotency key. Only then should the agent stop before the side effect.
When a decision arrives, the workflow should reload that state, authenticate the reviewer, confirm their current authority, check that the request and source data are still valid, and resume from a defined checkpoint. The OpenAI Agents SDK guidance on human-in-the-loop execution illustrates serialising run state while sensitive tool calls await a decision. Temporal’s human-in-the-loop example demonstrates durable timers and external signals for long-running approval waits. The implementations differ; the requirement does not: a restart or delayed response must not erase context or repeat an action.
Idempotency protects the external side effect. If an approval event is delivered twice, execution should occur once. If the action succeeds but its acknowledgement is lost, a retry should reconcile the existing result rather than send a second message, refund or update. Use a stable idempotency key at the workflow and downstream API layers where supported, and record the external result against the approved action.
Revalidation is equally important. If a price, account status, recipient, policy or source record changes while approval is pending, invalidate the request or generate a fresh diff and obtain a new decision. Approval is not a timeless credential.
Enforce authority and design the non-happy paths
Human involvement adds accountability only when the person has the right authority and enough competence to decide. Connect the workflow to enterprise identity and access controls, and check the reviewer’s role at decision time—not only when the request enters the queue.
Route by business attributes rather than a generic management hierarchy. Support leads may approve limited service credits; finance may own larger refunds and payment changes; data owners decide access to restricted knowledge; system owners approve production configuration. Enforce separation of duties where requester, agent owner and approver should differ. Use multiple approvals only where the additional control is proportionate to the delay.
The agent should still have least privilege. Approval must not make a permanently broad credential safe. Prefer short-lived, action-scoped authority issued after approval, with the approved parameters enforced by the execution service.
Then design every outcome, not only approval. Decide what happens when:
- nobody responds before expiry;
- a reviewer requests more information or edits the proposal;
- eligible reviewers disagree;
- the underlying record changes;
- the reviewer loses authority;
- the service restarts or a notification channel fails;
- execution becomes impossible or partially succeeds;
- the organisation activates an emergency stop.
For consequential side effects, fail closed: silence is not approval. Escalation may route a request to a backup role after a service threshold, but it must not reduce the original authority requirement. If no qualified reviewer is available, cancel safely or move the work to a defined manual process.
The EU AI Act’s human-oversight provisions are a useful operational reference where applicable: people need enough information to recognise limitations and automation bias, the ability to override outputs, and the power to interrupt a system safely. Legal obligations vary, but nominal human presence is not meaningful control.
Measure control quality, not approval volume
Approval fatigue is a control failure. Repetitive prompts increase queue time and encourage reflexive decisions or unofficial bypasses. Reserve synchronous review for situations where human judgment can materially change the risk. Narrow and automate low-consequence actions that have earned trust; sample appropriate actions after execution instead.
An audit trail should reconstruct what the system knew, proposed, displayed, decided and executed. Record the workflow, action, policy and agent versions; source evidence; approval packet; reviewer identity and authority result; decision, edits and reason; expiry and escalation events; final payload; execution result, retries and recovery. Flag any difference between approved and executed content.
Protect this evidence according to its sensitivity. Approval logs can contain customer data, confidential policy or security-relevant details. Observability should not create a new data leak.
Measure whether the gate improves outcomes. Useful indicators include decision time, expiry and escalation rates, edits and rejections by policy, duplicate prevention, bypass attempts, downstream incidents and reviewer workload. Sample approved outcomes for quality and check whether reviewers opened supporting evidence.
Do not optimise for a high approval rate. It may indicate accurate proposals, or it may indicate rubber-stamping. Repeated edits to the same field, recurring requests for missing evidence or near-universal rapid approvals are signals to inspect the policy, source data, interface and workload. The NIST AI RMF Govern playbook similarly treats oversight as an organisational system of roles, responsibilities, policy and accountability—not merely a technical interrupt.
Audit readiness before granting autonomy
Before a consequential agent workflow enters production, verify that:
- [ ] Every action type has an owner, consequence tier and allowed operating mode.
- [ ] Gates follow consequence and policy, not model confidence alone.
- [ ] Reviewers see the exact action, evidence, risk, alternatives and expiry.
- [ ] Each decision is bound to an immutable payload or version.
- [ ] Identity, authority and separation-of-duties rules are enforced at decision time.
- [ ] State survives restarts and long waits; resume and execution are idempotent.
- [ ] Timeout, rejection, edit, conflict, reassignment and cancellation paths are tested.
- [ ] Material source changes invalidate or refresh a pending request.
- [ ] The agent cannot bypass the gate through another tool or credential.
- [ ] Reviewers can interrupt the workflow and trigger a safe stop.
- [ ] Logs reconstruct proposal through execution without unnecessary data exposure.
- [ ] Metrics reveal fatigue, stale requests, weak policy, bypass attempts and incidents.
- [ ] A named team reviews exceptions and updates policy on a defined cadence.
A practical audit should trace several real action types from trigger to side effect. For each one, identify the governing policy, decision owner, evidence shown, persisted state, expiry behaviour, credentials used, recovery path and audit event. Then run tabletop tests for a duplicate decision, stale data, unavailable approver, service restart and partial downstream failure.
The result should be a prioritised control map: what can be automated now, what needs a stronger approval contract, what requires engineering work, and what must remain prohibited. If your team needs an independent view of that boundary, discuss a Workflow Audit with OpenCraft. We can map decision rights, systems, failure paths and measurable control points before implementation.