scheduled-task-resilience

Scheduled Task Resilience

Use these rules whenever a scheduled or regular task polls or waits on a long-running operation. They prevent heartbeat collisions, lost work, duplicate delivery, and sessions that fail after holding an external job open too long.

Rule 1 — Never use ScheduleWakeup with delaySeconds >= 300

The runtime heartbeat staleness threshold may sit near the same window as the model prompt-cache TTL. Sleeping for five minutes or longer can therefore look like a dead worker and cause the heartbeat sweep to fail the task mid-poll.

Instead:

• For waits under four minutes, run one bounded sleep of at most 240 seconds followed by a single status check, as one shell call, then return control to the agent turn.
• Never batch sleeps into a single shell invocation. A loop of five 240-second sleeps blocks the session for twenty minutes with no opportunity to record progress — the same staleness failure ScheduleWakeup causes, just spelled differently.
• Between cycles, record progress as described in Rule 3, then start the next cycle in the next turn.
• For genuinely long waits such as CI builds, releases, or deploys, prefer a durable workflow or a follow-up task over ScheduleWakeup.

Aggregate every check run before you decide the poll is over. Reading a single row such as .[0].state reports one check and can look terminal while another required check is still pending or failing.

# ONE cycle per agent turn. Resolve PR_NUMBER from the current task or repository context.
sleep 240
if out=$(gh pr checks "$PR_NUMBER" --json name,state,bucket \
      --jq 'if   any(.[]; .bucket == "pending") then "PENDING"
            elif any(.[]; .bucket == "fail" or .bucket == "cancel") then "FAILING"
            else "PASSED" end' 2>&1); then
  echo "$out"
else
  case "$out" in
    *"no checks reported"*) echo "PENDING" ;;  # no check contexts yet — not a failure
    *) echo "$out" >&2; exit 1 ;;              # real gh error: auth, network, bad PR
  esac
fi