Onboarding On-Call Engineers to Work Alongside Agents

On-call has to trust the agent. The 30-day onboarding curriculum, the shadow-mode period, and the first agent decisions a human should be expected to override.

Week 1: shadow only

The first week is observation. The agent runs read-only and the on-call sees the agent’s hypothesis alongside their own; no expectation that the on-call defers (they form their own opinion, the agent is a peer not an authority); daily debrief surfaces where the agent agreed, where it disagreed, where each was right.

• Read-only operation. Agent runs alongside; doesn’t act.
• Peer not authority. On-call forms own opinion; agent is parallel input.
• Daily debrief. Where did agreement happen; where did disagreement; who was right.
• Per-day learning capture. Each debrief produces signal; the team builds intuition.

Weeks 2-3: agent-first triage

The next two weeks shift the order. The agent triages first, the on-call reads the triage and decides whether to follow or override; most triage cases follow the agent; override is logged with a reason and reasons feed back into agent improvement; on-call still does the action because the agent does not act in this phase.

• Agent triages first. On-call reads; decides follow or override.
• Most cases follow agent. Override is the exception; the default is trust.
• Override with reason. Logged; reasons feed agent improvement.
• On-call still acts. Agent does not act in this phase; the human is the final layer.

Week 4: agent action with monitoring

Week 4 is the first action phase. The agent takes specific low-risk actions while the on-call monitors and intervenes if needed; action allowlist starts narrow (tag the alert, post a Slack update, create a ticket) and adds up over weeks; confidence builds gradually because trust is earned per action class.

• Specific low-risk actions. Agent acts; on-call monitors.
• Narrow allowlist. Tag, post update, create ticket; the safe primitives.
• Adds up over weeks. Allowlist grows as trust builds.
• Per-action-class trust. Trust earned per action; the discipline is granular.

First overrides matter

The first override is a trust test. The first time the agent is wrong about something the on-call had to override, trust is tested; make the override visible (“agent said X, on-call did Y, on-call was right because Z”) because transparency builds trust; track override patterns because repeated overrides on the same cause means the prompt needs work.

• First wrong override tests trust. The team learns whether the agent can be challenged.
• Make override visible. “Agent said X, on-call did Y, on-call was right because Z”.
• Transparency builds trust. The override visible; the team sees the system works.
• Track override patterns. Repeated same-cause overrides mean prompt needs work.

When the on-call has "graduated"

Three signs mark graduation. Comfortable defaulting to the agent on standard cases with confidence to override when needed; adds new cases to the eval suite from their experience so the on-call becomes an active participant in agent improvement; the agent becomes part of the team’s tooling, not a curiosity (that is the success state).

• Default to agent on standard cases. With confidence to override; the comfortable equilibrium.
• Adds new eval cases. Becomes an active participant in agent improvement.
• Tooling not curiosity. Agent is part of the team’s default workflow.
• Per-team graduation rate. The pattern repeats per onboard; supports continued investment.