Approve refunds over $500: the four things that decide whether a threshold holds
Every guide says auto-approve under $100 and escalate above it. That is the easy part. Four properties decide whether an approval threshold survives contact with production.
Search for how to add human approval to an AI agent that spends money and every answer gives you the same shape. Auto-approve refunds under $100. Escalate anything above. Adjust the number to your margins.
That advice is correct and it is the easy tenth of the problem. Picking a number takes an afternoon. What takes longer is the discovery that your threshold has four ways to not hold, and that each one looks fine in testing.
Key takeaways
- A rule that reads the action's parameters, not just its name, is what makes a threshold expressible at all.
- Rule order must not decide the outcome. Most restrictive wins, or two people writing rules on one afternoon produce a policy nobody can predict.
- A call that omits the guarded field has to escalate. Skipping the rule you cannot evaluate turns omission into a bypass.
- A verdict says an action may happen. It does not say it happened once, and a refund issued twice cannot be taken back.
Naming an action is not reading it
Most permission systems for agents decide on the name of a thing. git push is gated, git status is not. That works while the agent runs commands on your own machine, because the command name really does carry the risk.
It stops working the moment the action has a number attached. "Issue a refund" is not risky or safe as a category. A $12 refund and a $12,000 refund are the same action name and completely different decisions, and no pattern matching on refund.create tells them apart.
So the rule has to read the arguments:
{
"toolPattern": "refund.create",
"effect": "require_approval",
"conditions": [
{ "parameter": "amount", "operator": "gte", "value": 500 }
]
}Now $480 resolves on its own and $900 goes to a person. That is the part every guide covers. Here is the part they do not.
1. Order must not decide the outcome
You write the threshold rule. A colleague writes one that allows refund.create for the support team. Both match the same call. Which wins?
If the answer is "whichever was created first", or "whichever has the higher priority number", then your policy's behaviour depends on facts nobody looks at while writing a rule. Two people working on the same afternoon can produce a policy that neither of them predicted, and the failure is silent: the refund goes through and nothing logs that a rule you thought was governing was quietly outranked.
The property you want is that the most restrictive matching rule wins. deny beats require_approval beats allow, always, with no priority field to get wrong. That makes rule-writing composable: adding a rule can tighten the policy and can never loosen it by accident.
2. A missing field has to escalate
A rule decides on amount. A call arrives without one.
The tempting implementation skips a rule it cannot evaluate and moves on to the next. It reads as reasonable, and it is a bypass: omitting the field becomes the way past the threshold. Anything that constructs the call, including the model, can find that path without ever intending to attack anything.
The safe behaviour is that a rule which cannot be evaluated sends the call to a person. Not skipped, not allowed. It costs you an interruption in exactly the case where your policy does not have enough information to be sure, which is when you wanted the interruption anyway.
3. A numeric string is not a number
The caller sends "600". The rule compares amount against 500.
A permissive comparison reads that string as 600 and behaves the way you hoped. It is still wrong, because the same coercion reads "" as 0 and "1e3" as 1000, and now the question of whether a refund needs approval depends on JavaScript's type rules rather than on your policy. A threshold that silently reinterprets somebody's money is not a threshold.
"600" should escalate, the same as a missing field. Nothing about the string is trustworthy enough to compare.
4. Authorized once is not executed once
This is the one that is missing from every version of this advice, and it is the one that costs real money.
A verdict says an action may happen. It does not say it happened once. Between the decision and the side effect sit every framework retry, every resumed run, and every concurrent worker. A refund that was authorized once and issued twice cannot be taken back, and the approval log will look perfect: one request, one human, one yes.
Two things have to hold. A replayed call has to resolve to the same decision rather than asking a second time, which means the decision is keyed on the call and the run rather than on wall-clock arrival. And the authorization has to be spendable exactly once, so the second execution of an approved action is refused even though the approval was genuine.
That second part is a uniqueness guarantee, not a lock. Anything that depends on a lock the caller holds, a transaction it keeps open, or a read it does first has a window where a crash duplicates a real-world side effect.
What that looks like in one call
protect() in @pushary/server wraps the four properties around the side effect itself, so the action cannot run except through them:
const outcome = await protect({
action: "refund.create",
target: "order_4471",
externalId: "customer_123",
facts: { amount: 480, currency: "USD" },
callId,
runId,
question: "Approve a $480 refund on order #4471?",
run: () => issueRefund("order_4471", 480),
})facts are what a rule may decide on, passed by hand rather than derived, because the person writing a protected action knows which of its arguments matter and the raw input does not say. Whatever unit you choose, the rule and the call have to agree on it: a rule written against dollars and a call passing cents compares 500 to 48000 and escalates every refund you have. callId with runId is what makes a replay resolve to the same decision instead of asking twice. run is called only after the action is authorized, and never twice.
A rule that names the action settles it. Nothing else does: an all-agents wildcard, which is a statement about your own coding agents, never authorizes a customer's refund. Anything your rules do not name goes to a person, so a missing rule and an empty policy fail the same safe way.
The number was never the hard part
Auto-approve under $100 is a good instinct and a fine place to start. It is also the part of this you can get right in an afternoon and then be wrong about for a year, because none of the four failures above announce themselves. The rule that got outranked, the call that omitted a field, the string that compared as a number, and the retry that refunded twice all look like the system working until somebody reconciles the ledger.
If you are evaluating a policy layer for agents, these are four questions worth asking it, including of ours. Human-in-the-loop for AI agents covers the API and the framework adapters, the Partner plan is what you embed when the person deciding is your own customer, and the four levels of AI agent oversight puts thresholds in context with notification, approval and audit. For what happens when nobody answers at all, fail-closed by default covers the timeout case.
Frequently asked questions
How do you require human approval only above a certain amount?
With a rule that reads the action's parameters rather than only its name. A rule on refund.create with the condition amount gte 500 sends a $900 refund to a person and lets $480 resolve on its own. Picking the number is the easy part; the rule has to survive ordering, a missing field, a numeric string, and a retry.
What happens if the agent omits the amount?
It has to escalate. A rule that decides on a parameter the call does not carry cannot be evaluated, and the tempting implementation skips the rule it cannot evaluate. That quietly turns omitting the field into a way past the policy, so the safe behaviour is to send the call to a person instead.
Is an approval enough to guarantee the action runs once?
No. A verdict says an action may happen; it does not say it happened once. Framework retries, resumed runs and concurrent workers can all replay an authorized call, and a refund authorized once and issued twice cannot be taken back. Exactly-once needs a permit that is spent on execution, not just a decision that was approved.
Should rule order decide the outcome?
No. When several rules match one call the most restrictive should win, so deny beats require_approval beats allow. If a priority field or creation order decides instead, two people adding rules on the same day can produce a policy whose behaviour depends on who saved first.