An agent recovery runbook should let an operator establish the run's last durable state, determine whether external effects occurred, and choose a tested replay, rollback, or review path without depending on the original developer's memory.
Start from observable symptoms and durable state
Write entries for the failures an operator can actually observe: an expected run is absent, progress stopped, a tool returned an incompatible result, output exists without a completion event, or a downstream action may have occurred before a timeout. Each entry should point to the exact dashboard, query, run identifier, state record, and owner needed to confirm the symptom. Avoid directions such as check the logs without the saved query and expected fields.
The first decision establishes the last durable checkpoint and known external effects. The runbook should list state transitions in order, identify which operations have idempotency keys, and show where the downstream system records the effect. If the effect remains uncertain, the procedure stops automatic replay and routes the case to the named reviewer. That stop condition is part of recovery, not a missing branch.
Give every action a precondition and verification
A recovery command needs the run state in which it is permitted, the credential role that may execute it, the change it makes, and the query that proves the result. Separate replay from restart: replay resumes from a stored checkpoint, while restart creates a new run and needs a rule for earlier effects. Include rate limits, retry ceilings, and a rollback or compensating action where the integration supports one.
Google's SRE guidance treats incident response as a practiced process and notes that untested rollback procedures can lengthen an outage. Exercise each recovery path with test data or a staging dependency, including the point where automation hands control to a person. Store the date, versions, operator, and result of the exercise so an old command does not masquerade as current evidence.
Preserve the incident record and feed the repair back
After recovery, attach the trigger record, state history, tool attempts, downstream identifiers, alerts, operator actions, and final verification to one incident. Name the causal uncertainty separately from the confirmed cause. Corrective actions should change a monitor, test, schema, retry rule, permission, or runbook step, with an owner and verification case, so the document records an actual control rather than an intention.
Reality Contact, LLC prepares and tests the runbook for Agent Runtime Reliability. The buyer controls production access, approves every recovery action, and assigns the continuing owner. The runbook describes the accepted workflows and versions; it does not provide an availability assurance or authorize Reality Contact, LLC to act during later incidents without a separate agreement.
Where the service stops
Reality Contact, LLC implements and verifies bounded runtime changes but does not certify security or availability, approve credentials, authorize consequential actions, or operate the service indefinitely. The buyer approves boundaries and scenarios, controls every production credential and release, and moves workflows only after reviewing the evidence. This is software implementation and technical verification; it does not replace the buyer's security, privacy, legal, compliance, or production-readiness review. We do not promise continuous availability, error-free execution, complete incident causation, safe behavior outside the accepted scenarios, or recovery from every possible failure.
Sources: Google SRE guidance for emergency response; Google SRE guidance for postmortem practice.