Human-in-the-loop API for AI agents
Decide what your AI agent is allowed to do.
Your agent asks before it acts. A $42 refund just runs. A $4,200 refund asks the right person first, on their phone.
@pushary/server · @pushary/ai-sdk · @pushary/eve · pip install pushary
What is a human-in-the-loop API?
Sometimes an AI agent needs a person to say yes before it acts. A human-in-the-loop API asks that person and sends the answer back. Pushary is that API, and your rules can also allow or deny on their own. Your agent calls authorize() before a consequential action, and your rules answer three ways: allow it, deny it, or send it to a named person and wait for their tap. Refunds under $100 run on their own. Refunds at or over $1,000 go to the person whose money it is. Anything your rules do not name goes to a person, so an empty policy fails safe. If nobody answers inside your window, your code gets approved: false and the action does not run. Every answer is saved, with the rule or the person that gave it.
You write the rules, not the if-statements
A rule can read the amount, so you get "refunds over $1,000 need approval" instead of "all refunds need approval".
refund.create amount < 100 allow
refund.create amount >= 1000 require_approval
refund.create amount >= 10000 deny$42 runs on its own. $4,200 asks your user first. $40,000 never happens. Anything you forgot to write a rule for asks a person, so a gap is never a yes.
- allow
- A rule allows this action. The agent carries on, nobody is asked, and the record shows which rule allowed it.
- deny
- A rule blocks it. The agent stops and gets a plain reason, so it does not just try the same thing again.
- requires_human
- A rule asks for a person, or no rule covers the action. Either way, the person your code named gets it on their phone. A missing rule asks. It never allows.
The strictest rule wins, so the order you write them in never changes the answer. The rule decides whether to ask. You decide who gets asked.
The call
Connect once. Then one call before the action.
One call sits in front of the refund, the deploy, the delete. Your rules answer it. A phone only gets involved when they cannot.
import { createPusharyServer } from "@pushary/server"
// Rules need a server key from Settings > API keys.
// The key from onboarding can ask a person, but it cannot read your rules.
const px = createPusharyServer({ apiKey: process.env.PUSHARY_API_KEY! })
// Once per person: show them this link to connect. No account needed.
const { universalLink } = await px.enroll(user.id)
// Rules answer first. This one is over your $1,000 threshold.
const decision = await px.authorize({
toolName: "refund.create",
toolTarget: "order_4471",
externalId: user.id,
parameters: { amount: 4200 },
question: "Approve a $4,200 refund for order #4471?",
})
if (!decision.approved) throw new Error(decision.reason)
await issueRefund()Every call returns the same shape
Three outcomes, one shape. Check approved and carry on. Keep the decision id if you need to pick the run back up later.
// $42. A rule allowed it. Nobody was asked.
{ approved: true, resolvedBy: "policy", reason: "Allowed by policy rule refund.create." }
// $4,200. Your >= 1000 rule sent it to a person. They tapped Approve.
{ approved: true, resolvedBy: "human", reason: "A person approved it." }
// $4,200, and nobody answered inside the window.
{ approved: false, resolvedBy: "human", reason: "Nobody answered, so this was not approved." }If nobody answers in time, the answer is no. Your agent gets a plain no with a reason it can log, and the refund does not happen. An action your rules never named comes back the same way, settled by a person instead of a rule.
After the pause
Your framework stops the agent. It does not decide anything.
LangGraph interrupts. The Vercel AI SDK marks a tool call as needing approval. The OpenAI Agents SDK records an approval item. Every one of them hands you a paused run and leaves the rest to you.
- Deciding
- Something has to read the action, like the refund amount, and pick the right rule every time. When it cannot read the amount, it should ask, not let the action through.
- Changing the answer
- Moving a limit from $500 to $750 should not need a code change. In a framework, the rule lives in your code. Here you change one rule, with no deploy.
How it works
From call to answer
Install it, connect the person who answers, write the rules, and put one call in front of the action. The deciding, the waiting, the delivery and the record are handled for you.
- 01
Install the SDK
npm i @pushary/server in TypeScript, or pip install pushary in Python. Published adapters cover the Vercel AI SDK, LangGraph, Mastra, the OpenAI Agents SDK, CrewAI and Eve.
- 02
Connect the person who answers
Call enroll() with your own user id. They tap the link once, it opens the Pushary app, and Approve and Deny land on their lock screen. No account and no key on their side.
- 03
Write the rules
A rule is an action, a condition on its parameters, and an effect. Under $100 allow. At or over $1,000 require approval. At or over $10,000 deny. Most restrictive wins, so the answer never depends on the order you wrote them in.
- 04
Call authorize() before the action
Your rules answer first, and the question reaches a person only when no rule can settle it. If nobody answers in your window it comes back denied, so a timeout never becomes an unapproved action.
A question can be a yes or no, a choice from a list, or a typed answer. Every answer is saved.
Questions, answered
How do I add human-in-the-loop to an AI agent I already built?
Who decides which person gets asked?
Does this mean more interruptions?
What happens if nobody answers?
Do my end-users need an account or app?
Which frameworks does Pushary support?
How much does it cost?
Decide it before it happens.
Write the rules once. Your agent asks before it acts and gets a yes, a no, or a real person's answer. Every answer is saved, with the rule or the person that gave it.
Start freeHuman-in-the-loop by framework
By product type
The four levels of oversight
Oversight of an AI agent stacks in four layers. The first reports an action once it has already happened. The second holds an action open until a person answers. The third resolves the routine cases from rules, so a person is asked only when it matters. The fourth keeps the record of what ran and who cleared it. Each layer is worth less without the one beneath it, which is why a notification on its own is something you watch rather than something you control.
| Level | Question it answers | When it acts |
|---|---|---|
| 1. Notify | What is the agent doing? | After the fact, no decision |
| 2. Approve | Should this run? | In the moment, blocks |
| 3. Policy | Which cases need me at all? | Before the moment, automatically |
| 4. Audit | What happened, and who decided? | After, durable record |
- Notify
- Notify is awareness with no decision: the agent acts, then tells you what happened. It is parity now that Claude Code, Codex and Cursor all ship native notifications, so the only thing left to compete on is whether the message actually reaches you. Notifications across every agent
- Approve
- Approve is a gate in the moment: the agent stops before the action and does not move until a human answers yes or no. A gate only counts if it really holds the agent, which takes a hook sitting in front of the call itself. Anything that merely announces an approval is wanted, without the power to stop what comes next, is notification wearing a different label.
- Policy
- Policy is rules that settle the routine cases without you, turning a call into an allow, an ask or a deny before any person sees it. The version worth having reads the parameters a call carries rather than only its name, so a harmless status check can clear itself while a push to a shared remote still stops for you. Permission policies
- Audit
- Audit is the durable record after the fact: every decision, whatever made it, and what changed as a result. It has to include the calls nobody was ever shown, since anything a rule cleared on its own still belongs in the record together with the rule that cleared it. Keep only the moments somebody was interrupted and the record omits nearly everything the agent was free to do. The audit trail
The full argument for why the levels stack, and why notify stopped being the product, is in The four levels of AI agent oversight: notify, approve, policy, audit.
Controlling agents you run yourself
The same two calls work on any framework, listed here or not, anywhere @pushary/server or pip install pushary runs.
Using AI agents yourself? Get their questions on your phone or your Mac, and answer from either one. See the agent plans.