Human-in-the-Loop

How it works

Human-in-the-loop lets your AI agent pause, ask you a question, and wait for your answer — all via push notification. You tap Yes or No from your phone's lock screen without switching back to your computer.

Agent hits a decision → sends push question → you tap Yes/No → agent continues

This uses three MCP tools:

1. ask_user_yes_no — sends the question as a push notification with Yes/No buttons
2. wait_for_answer — long-polls until you respond or the timeout is reached
3. cancel_question — cancels a pending question if it becomes irrelevant

The flow

1. Agent asks a question

The agent calls ask_user_yes_no with a question. This sends a push notification to your phone.

{
  "tool": "ask_user_yes_no",
  "arguments": {
    "question": "Delete the 3 unused migration files?"
  }
}

Returns a correlationId — a unique ID for this question:

{
  "correlationId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}

2. Agent waits for your answer

The agent immediately calls wait_for_answer with the correlationId. This blocks (long-polls) until you respond or the timeout is hit.

{
  "tool": "wait_for_answer",
  "arguments": {
    "correlationId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "timeoutMs": 55000
  }
}

If you answer in time:

{
  "answered": true,
  "value": "yes"
}

If the timeout is reached:

{
  "answered": false
}

3. Retry if no answer yet

If wait_for_answer times out, the agent retries. Your answer is stored in Redis for 10 minutes — even if the first poll times out, the answer will be there when you eventually tap.

Recommended retry pattern: call wait_for_answer up to 3 times (3 × 55 seconds = nearly 3 minutes of total wait time).

4. Agent acts on the answer

Once the agent receives "yes" or "no", it proceeds accordingly.

Full example

Here's the complete flow as an agent would execute it:

1. ask_user_yes_no({ question: "Delete the 3 unused migration files?" })
   → { correlationId: "abc-123" }

2. wait_for_answer({ correlationId: "abc-123", timeoutMs: 55000 })
   → { answered: false }                 // you haven't responded yet

3. wait_for_answer({ correlationId: "abc-123", timeoutMs: 55000 })
   → { answered: true, value: "yes" }    // you tapped Yes on your phone

4. Agent deletes the migration files

What the user sees

When ask_user_yes_no is called:

1. Your phone shows a push notification with the question text
2. Tapping the notification opens a decision page with large Yes and No buttons
3. You tap your answer
4. The page confirms your choice was recorded
5. The agent's wait_for_answer call returns immediately with your answer

The entire interaction takes 2-5 seconds from your side.

When to use it

Good use cases:

• "Should I delete these deprecated API routes or keep backward compatibility?"
• "This migration will drop the legacy_users table. Proceed?"
• "Found 2 approaches to fix this bug. Option A changes 3 files, Option B changes 1 file but is less readable. Go with Option A?"
• "Deploy to staging now?"

When NOT to use it:

• Questions the agent can answer from context (don't ask "should I use TypeScript?" if the project is already TypeScript)
• Every single micro-decision — use it for meaningful choices, not every line of code
• When the user already gave instructions that cover this case

Cancelling a question

If the agent figures out the answer on its own or the user responds in chat, cancel the pending question:

{
  "tool": "cancel_question",
  "arguments": {
    "correlationId": "abc-123"
  }
}

Returns { "cancelled": true } if the question was found, or { "cancelled": false } if it was already answered or expired.

Setting up your agent skill

If you're using the Pushary agent skill, this flow is already documented in the skill's instructions. Your agent will know when and how to use it.

npx skills add pushary/pushary-skill

The skill teaches agents to:

• Use ask_user_yes_no for binary decisions
• Always call wait_for_answer immediately after asking
• Retry up to 3 times on timeout
• Cancel questions that become irrelevant

Parameter reference

ask_user_yes_no

wait_for_answer

cancel_question

Tips

• Keep questions under 60 characters so they display fully on phone lock screens
• Be specific. "Delete the 3 unused migration files?" is better than "Clean up?"
• Set timeoutMs to 55000 (the maximum) since users might need a moment to reach their phone
• Have a safe default. If the user never answers after 3 retries, pick the non-destructive option
• One question at a time. Don't fire multiple ask_user_yes_no calls simultaneously — it overwhelms the user