Human-in-the-loop for infrastructure agents
The on-call engineer is not at the terminal.
Your agent gates the risky change correctly. Pushary is what gets the answer back before the incident gets worse.
How does an AI DevOps agent get approval from an on-call engineer?
Pushary reaches the on-call engineer on their phone and blocks the agent until they answer. Put decisions.ask() in front of the migration, the restart, or the scale-down, and the agent holds on a durable wait rather than a chat message nobody has opened. Timeouts resolve denied, and every decision is written to an exportable ledger for the review afterwards.
Where it stops
The calls that should not be automatic
- runMigration(name)
- A migration that is not cleanly reversible is the classic example of an action worth waking someone up for, and the classic example of one an agent will happily run at 02:00.
- scaleDown(service, replicas)
- Reducing capacity during an incident you have diagnosed wrong turns a degradation into an outage, and the blast radius is everyone.
- rotateSecret(name)
- Rotating credentials fixes one problem and can break every consumer that cached the old value, which is a judgement call rather than a threshold.
- deleteResource(arn)
- Cleanup agents are useful right up to the moment one of them reclaims something that looked idle and was load bearing.
- promoteToProduction(sha)
- Shipping is the step where the cost of being wrong is paid by users rather than by a test suite.
The integration
Two calls inside the step you already have
Wrap the action, not the whole agent. Everything the agent does that is safe keeps running without asking anyone.
from pushary import Pushary
px = Pushary(api_key=os.environ["PUSHARY_API_KEY"])
def remediate(incident, plan):
# Read-only diagnosis and reversible fixes run unattended.
if plan.reversible and plan.blast_radius == "single-pod":
return apply(plan)
decision = px.decisions.ask(
external_id=incident.on_call_id,
type="confirm",
question=f"{plan.summary}\nBlast radius: {plan.blast_radius}. Reversible: {plan.reversible}.",
)
if not decision.approved:
return page_secondary(incident, reason="human declined automated remediation")
return apply(plan)Who answers
The approver is usually not an engineer
The approver is whoever is on call, and the whole point of being on call is that you are not sitting at a terminal. Most remediation agents post the approval request into a chat channel, which works when the engineer happens to be in that channel with notifications on, and fails silently the rest of the time. The failure is quiet and expensive: the change is correctly blocked and the incident keeps burning. Pushary addresses the on-call engineer by the id your rota already knows and reaches their phone, so the gate you designed behaves like a gate rather than a pause of unknown length.
We already post approvals to Slack. What is different?
Slack is a good place for a request to be visible and a bad place for a run to depend on. The message arrives whether or not the right person is looking, there is no return value your code can wait on, and nothing about it fails closed. So you end up writing the missing half yourself: a poller, a timeout, a state machine that survives the deploy that lands mid-incident, and somewhere to record who clicked what for the review. Pushary is that half as one call that returns a decision. If your team lives in Slack it still routes there, after the phone, and the record is written either way rather than being scattered across a channel history.
Worth reading: Google's SRE book on release engineering and change management.
Embed it in your product. Your users never see a Pushary login or bill.
FAQ
Questions, answered
Put a person on the step that needs one.
Connect a user with one tap, ask a human with one call, and act on a fail-closed decision. Every decision is saved to a durable ledger you can audit.