An escalation protocol is the written rule set that says when an incident moves from the person holding it to the next level, who that next level is, how they get contacted and how long they have to respond. It sits underneath the escalation policy (the why) and above the contact matrix (the who), and it is the document the on-call engineer actually reads at 3 AM. This post gives you the pieces to write one in an afternoon: escalation criteria, a levels table, an escalation path template and two worked examples.

If you want the long version with training plans, compliance and metrics, the escalation policies guide covers all of it at 9,000 words. This page is the short operational version.

Quick summary

  • An escalation protocol answers four questions per level: who is contacted, through which channel, how long they have to acknowledge and what happens if they do not.
  • Four criteria trigger a step up: severity at detection, time without acknowledgement (I recommend 5 minutes for SEV1 and 15 for SEV2), impact growth (more than 10% of users, or a paying customer's core workflow) and lack of expertise at the current level.
  • Three technical levels plus one management level cover most teams: L1 on-call responder, L2 service owner, L3 domain specialist, then an engineering manager who owns customer and executive communication.
  • The most common failure is a protocol with no time limits, so incidents sit with the first responder until someone notices. Every level needs an acknowledgement deadline, including the last one.
  • In Hyperping, the protocol becomes an escalation policy with delayed steps, on-call schedules as recipients and acknowledgement that stops the chain.

What is an escalation protocol, and how is it different from a policy?

People searching for "escalation protocol", "escalation policy", "escalation procedure" and "escalation matrix" mostly mean the same thing. Inside a team the words are worth separating, because each document has a different reader.

Document Question it answers Typical length Who reads it during an incident
Escalation policy Why and when do we escalate at all? 1-2 pages Nobody, it is read when the process is designed
Escalation protocol At which thresholds, to which level, through which channel, with what deadline? 1 page, mostly tables The responder deciding whether to step up
Escalation procedure What exact steps do I take to escalate? 2-5 pages The responder executing the step up
Escalation matrix Which name and number for this service and severity? One table Anyone looking up a contact

The protocol is the piece most teams skip. They have a policy that says "escalate promptly" and a matrix with phone numbers, and nothing that says at which minute or severity the phone gets picked up. Our escalation procedure template has the surrounding sections (contact directory, communication rules, checklists); this post fills in the criteria and path parts.

Why listen to us

I run Hyperping, an uptime monitoring and on-call tool, so I sell software that executes escalation protocols. That is a conflict of interest, and you should read the Hyperping section at the end with it in mind. The thresholds below come from the escalation policies I see customers configure and from our own incidents, so treat them as a starting point rather than a standard.

What escalation criteria trigger a step up?

Four types of criteria cover almost every case. Write concrete thresholds for each one, because "significant impact" and "reasonable time" are the two phrases that stall escalations while people argue about whether the bar is met.

Criterion Question Concrete threshold to start from
Severity How bad did it look at detection? SEV1 (full outage, data loss, security breach) starts at L2 with L1 paged in parallel. SEV2 (core feature degraded) starts at L1. SEV3 and SEV4 do not page outside business hours.
Time How long has the current level held it without progress? No acknowledgement: 5 min for SEV1, 15 min for SEV2, 60 min for SEV3. Acknowledged but no mitigation: 30 min for SEV1, 2 hours for SEV2.
Impact Is it growing or touching something that changes the stakes? More than 10% of requests failing, more than one region down, a top-20 account's core workflow broken, or an SLA breach becoming likely. Any of these moves the incident up one severity, which re-runs the severity rule.
Expertise Does the current level have the access or knowledge to fix it? If the responder cannot name the next diagnostic step within 10 minutes, or the fix needs production database access, a vendor, or a security decision they cannot make, escalate at once without waiting for the timer.

Time and expertise criteria move the incident one level up. Impact criteria raise the severity, which means shorter deadlines and a higher starting level, so the same incident can jump two levels in one move. Both are correct; write them down so nobody feels they need permission.

Escalation levels: who is at each tier?

Levels describe roles rather than people, and the on-call schedule maps role to person on the day. What the protocol fixes is what each level is expected to do, how fast, and through which channel.

Level Who Acknowledge within Contact channel Owns
L1 Primary on-call engineer for the service 5 min (SEV1 and SEV2), 15 min (SEV3) Push or SMS, then phone call Triage, first mitigation (rollback, restart, failover), first status page update
L2 Service owner or senior engineer on the secondary rotation 10 min Phone call and Slack DM Diagnosis needing code or infrastructure knowledge, rollback or hotfix decision
L3 Domain specialist (database, network, security, payments) or platform lead 15 min Phone call, plus the vendor's support line if a third party is involved Fixes needing privileged access or deep system knowledge
Management Engineering manager, then head of engineering or CTO 15 min Phone call Incident commander role, customer and executive communication, resourcing, opening a SEV1 bridge

Management does not debug. Their entry turns an engineering incident into a company incident, so write that trigger down separately: any SEV1 that reaches 30 minutes, any customer-facing data issue, or any contract customer asking for an update. On small teams L2 and L3 collapse into one person, which works as long as the founder's or CTO's phone number appears at the end of the chain rather than being implied.

Escalation path template

Copy this table into your runbook, one copy per service or service group, and replace every placeholder in square brackets. Keep it to one screen; a path that needs scrolling is not read under pressure.

Step Trigger Level Contact Channel Acknowledge within If not acknowledged
0 Monitor [name] fails [N] checks from [N] regions, or ticket tagged [SEV1/SEV2] Detection Alerting tool Slack #[incidents] n/a Step 1 fires at once
1 Immediately for SEV1 and SEV2 L1 [on-call schedule] SMS + push, phone call at +2 min [5] min Step 2
2 Step 1 not acknowledged in [5] min, or L1 escalates on expertise L2 [secondary schedule or service owner role] Phone call + Slack DM [10] min Step 3
3 Step 2 not acknowledged in [10] min, or impact threshold [X] met L3 [specialist or platform lead] Phone call [15] min Step 4
4 Step 3 not acknowledged in [15] min, or SEV1 older than [30] min Management [engineering manager], then [CTO] Phone call [15] min Repeat every [15] min until acknowledged

Add three lines under the table so the path stands on its own:

  • Acknowledging stops the timer for the current step, but the next step still fires if the impact or time-since-mitigation criteria are hit. Say this explicitly: the alerting tool stops its automatic chain on acknowledgement, so those two criteria depend on the responder escalating by hand.
  • Every level can call any higher level directly when the expertise criterion applies. Skipping levels is allowed, as long as the jump is posted in the incidents channel.
  • L1 owns the status page and customer updates until management joins.

The downloadable procedure template has the sections that go around this table: contact directory, business-hours coverage and the post-incident checklist.

Two worked examples

SaaS API outage on a Saturday night

Detection at 23:04: the API monitor fails from three regions and the error rate on the /v1/orders endpoint is 100%. Severity at detection is SEV1 (full outage of a core endpoint), so step 1 pages the on-call engineer by SMS and push at 23:04 and calls at 23:06.

The engineer acknowledges at 23:07 and starts a rollback of the 22:50 deploy. At 23:20 the endpoint is still down and she cannot tell whether the failure is in the app or in the database connection pool. Expertise criterion: she calls L2 (the backend service owner) directly at 23:21 rather than waiting for a timer. L2 finds a connection pool exhausted by a migration and the fix is deployed by 23:38.

The SEV1 30-minute rule fired at 23:34, so the engineering manager was already on the call, wrote the second status page update and emailed the two enterprise accounts that have an SLA. Total time down: 34 minutes. Levels touched: L1, L2 and management. L3 was never needed, which is what a well-set path looks like most of the time.

Suspected credential leak

Detection at 14:12 on a weekday: a developer notices an API key for the production object store in a public GitHub repository. There is no outage, so nothing pages automatically. Severity is SEV1 by rule (any suspected exposure of production credentials), and the security path for this team starts at L2 with the security lead as L3 paged in parallel.

The security lead revokes the key at 14:19 and rotates the dependent services by 14:40. The management trigger "any customer-facing data issue" applies, so at 14:20 the CTO is on the thread checking whether access logs show any read of customer objects. They do not, so the incident closes at 15:30 with a post-mortem scheduled and no customer notification required. Had the logs shown a read, the path would have continued to legal and the CEO, which is why security paths need those two rows even if they rarely fire. Our post on security incident types covers which categories deserve their own row.

Common escalation protocol mistakes

  • No deadline on the first level. L1 gets paged, nothing states when the alert moves on, and a missed page becomes a 40-minute outage. Every level gets an acknowledgement window, including the last one.
  • Named people instead of roles. The protocol says "call Sam", Sam left in March, and nobody updated the document. Reference on-call schedules and roles, and let the schedule resolve the name.
  • Same channel at every level. If L1 missed a Slack message, L2 will miss it too. Channels should get more intrusive as the level rises: chat, then SMS, then phone call.
  • Escalation treated as failure. If responders feel that paging L2 is an admission they could not cope, they wait too long. Write the expertise criterion so calling up early is the expected behaviour, and say so in retros when someone does it well.
  • Protocol never tested outside a real incident. Phone numbers and schedule membership drift within a quarter. A 20-minute drill each quarter that fires a test alert through every level catches most of it.

How to run an escalation protocol in Hyperping

The document above translates directly into Hyperping's escalation policies: each row of the path template becomes a step with a delay, a recipient and channels. On-call schedules and escalation policies are included from the Essentials plan at $29 per month ($24 billed yearly).

1. Create the escalation policy with one step per level

Go to Escalation Policies, create a policy named after the service and severity, for example "Orders API, SEV1", and add a step per row of your path. Step 1 has a delay of 0 and fires when the monitor goes down; each following step's delay is counted from the previous step, so set it to the previous level's acknowledgement window. With the template above the delays are 0, 5, 10 and 15 minutes, which means the steps fire at minute 0, 5, 15 and 30 of the incident. If a step is acknowledged or the incident is resolved, the remaining steps are cancelled.

2. Build on-call schedules and use them as recipients

Create an on-call schedule per rotation, with its own timezone and handoff time, and pick the schedule as the SMS or phone call recipient in a step instead of a named person. This turns "L1 primary on-call" from a role on paper into the person on duty at alert time, and overrides for holidays or swapped shifts are resolved when the alert fires. If you are still choosing a tool for the scheduling side, my round-up of on-call scheduling tools goes through the options.

3. Match the channel to the level

Within a step you can add several channels that fire together: Slack or Microsoft Teams to the incidents channel, email, SMS and phone calls to individuals or schedules. Follow the "more intrusive as it rises" rule: Slack plus SMS at step 1, phone call and Slack DM at step 2, phone calls only after that. Teammates need their phone number set in their account settings before SMS or calls reach them.

4. Assign the policy to monitors and test the chain

Open each monitor's notification settings and pick the policy from the Escalation Policy dropdown; without one, a monitor pages every channel at once. Then run the drill: point a test monitor at a URL you control, take it down, and watch the steps fire at minute 0, 5, 15 and 30. Acknowledge at step 2 to confirm the chain stops there. Turn on grouped alerts as well, so five monitors failing together produce one page rather than five parallel escalations.

What to write first

If you have nothing today, write the severity definitions and the levels table, in that order, and give each level an acknowledgement deadline. That one page removes the "should I call someone" hesitation that costs the most minutes. The deep escalation policies guide covers training, metrics and compliance once the one page is in use.