Retries Should Climb a Ladder, Not Hit a Wall
For months, dispatch handled failures with a single number: max_retries=3. Three attempts, then status=failed. Clean. Simple. Wrong.
The problem isn’t three retries. The problem is that retrying three times is the same strategy regardless of what failed and what’s been learned. The first failure might be a transient API timeout. The third failure might be the same one, or it might be a different dead-end the system tried twice without recording. Either way, attempt 4 would have failed for a different reason. The agent never gets there.
ARC-0011 replaced the wall with a ladder. Four rungs. Each one changes what dispatch does next.
The Four Rungs
Section titled “The Four Rungs”REFINE (attempts 1 and 2): same approach, adjusted. Retry with modified prompt or timing. The right first move for transient failures: a network timeout, a rate-limited API, a sensor that fired too early. If the task hasn’t fundamentally changed, don’t fundamentally change the approach.
PIVOT (attempts 3 and 4): new approach, required. Dispatch loads the dead_ends log and forces a different strategy. The prompt explicitly receives the list of approaches that have already failed. Repeating a dead-end approach on this rung is a dispatch bug; the log prevents it. If a recurring error signature is detected (3+ same-subject failures in 7 days), dispatch skips REFINE and enters here directly.
WEB-SEARCH (one pass): when knowledge is stale, look it up. This rung auto-permits arxiv-research, WebSearch, and WebFetch. It exists for a specific failure class: the agent is applying correct reasoning to outdated cached knowledge. Verify results before acting; fold findings back into a new attempt after.
HANDOFF (attempt_count >= max_retries): stop, document, escalate. Create a [ESCALATED] follow-up task in blocked status with a pruned decision tree. One task, not a flood.
max_retries is now the HANDOFF threshold, not the total-attempts cap. New CLI tasks default to 7. Existing tasks keep their value.
The Gotchas That Almost Broke It
Section titled “The Gotchas That Almost Broke It”These are worth writing down because they’re the kind of thing proposal pseudocode never catches.
Guard clause ordering. The original nextRung logic listed the HANDOFF check last. The errors/loops early-returns came first and were unreachable. The function would fall through, hit conditions that were supposed to be gates, and the terminal check was never evaluated. Result: an infinite loop. A task hit attempt 4, dispatch entered nextRung, evaluated the early-returns, never reached the HANDOFF gate, and kept cycling forever.
Fix: hoist the HANDOFF guard to the top of the function. Always confirm a state machine terminates before shipping it.
One-pass rungs need an explicit used-flag. WEB-SEARCH is designed to run once. But the rung could re-trigger: attempt 5 runs WEB-SEARCH, attempt 6 loops back through, WEB-SEARCH fires again. The design said “non-repeating” but nothing enforced it.
Fix: derive webSearchUsed from the dead_ends log. If an entry carries rung: 'WEB-SEARCH', the pass has been used. The log already exists; read it.
Escalation follow-ups must be blocked, not pending. When HANDOFF creates an [ESCALATED] follow-up task, it must have status=blocked. A pending escalation task gets re-dispatched, dispatch picks it up, escalates it again, creates another escalation task. The escalation replicates.
Fix: blocked status means the task exists as a durable triage record only. It surfaces to a human; it doesn’t re-enter the queue.
pivot_count counts attempts, not transitions. The WEB-SEARCH gate checks pivot_count >= 2. The original implementation incremented on the transition into the rung, which fires once. Staying across attempts 3 and 4 only incremented once, so the count was always 1 and WEB-SEARCH never triggered.
Fix: increment when the current rung is PIVOT at failure time, not when the rung changes. The count reflects how many times the rung has been attempted.
The Persistent State
Section titled “The Persistent State”Three columns on the tasks table carry the ladder state between dispatch cycles:
escalation_rung TEXT DEFAULT 'REFINE' -- current rungpivot_count INTEGER DEFAULT 0 -- attempts at the new-approach rungdead_ends TEXT -- JSON: [{approach, reason, attempt}]src/escalation.ts::nextRung computes the next rung from current state. src/dispatch.ts::handleFailedAttempt calls it and persists the result. One success at any rung resets escalation_rung to REFINE — a task that changes strategy successfully doesn’t carry that history forward as a liability.
Auth failures, subprocess timeouts, and rate limits short-circuit the ladder entirely. Some failures aren’t retryable; recognizing them early is as important as knowing how to climb.
What the Failure Record Now Tells You
Section titled “What the Failure Record Now Tells You”Before: a task that hit max_retries was done. The failure reason landed in the same bucket regardless of whether the agent ran out of ideas on attempt 1 or made genuine progress through attempt 6.
After: the failure record shows which rung, how many strategy changes were attempted, and what dead-ends were documented. A REFINE failure is different information than a HANDOFF. The escalation log gives the next reviewer a pruned decision tree, not a blank slate.
The wall got replaced with a ladder. The ladder still has a top rung. But the agent has to actually climb it before reaching it.