Post-mortem template
Run a short, blameless review after every significant incident. The point is not to find who broke production, it is to find what made breaking production possible.
Blameless is a practical choice, not a soft one. Engineers who fear blame leave out details, and missing details become repeat incidents. Name systems and gaps, never people.
The template
Copy this into your wiki or repository and fill it in within 48 hours, while memories are fresh. Replace the example values as you go.
# Post-mortem: [incident title]
- Date: YYYY-MM-DD
- Authors: [names]
- Status: Draft / In review / Final
- Severity: Critical / Major / Minor
## Summary
Two or three sentences a new hire could follow.
What broke, who felt it, how long it lasted, and what fixed it.
## Impact
- Duration: 42 minutes, from 14:03 to 14:45 UTC
- Users affected: all API consumers, roughly 18% of active users
- Requests lost or degraded: about 12,000 failed requests
- SLA or revenue impact: [if any]
## Timeline (UTC)
- 14:03 Hyperping alert fires: api.example.com returns 503 from 3 regions
- 14:05 [name] acknowledges the alert
- 14:11 Cause identified: connection pool exhausted after the 13:58 deploy
- 14:19 Mitigation: deploy rolled back, pool size doubled
- 14:45 Monitors report recovery, incident marked resolved
## Root cause
Explain the underlying condition, not the trigger.
Keep asking why until you reach a process or design gap.
## What went well
- Alert fired within one minute of the first failed check
- Rollback took four minutes end to end
## What went poorly
- The runbook linked from the alert channel was outdated
- Only one person knew how to roll back the migration
## Action items
| Action | Owner | Deadline | Status |
| --------------------------------------- | ------ | ---------- | ------ |
| Alert on connection pool saturation | [name] | YYYY-MM-DD | Open |
| Update the rollback runbook | [name] | YYYY-MM-DD | Open |
| Document the migration rollback process | [name] | YYYY-MM-DD | Open |
How to run the review
- Cap it at 45 minutes. Invite the responders, a facilitator, and whoever will own action items. If it does not fit, the scope is too big; split the incident into two reviews.
- Keep it blameless. Replace "who deployed this" with "what allowed a bad deploy to reach production". People stay honest when the answer cannot be their own name.
- Give every action item an owner and a deadline. An unowned action item is a wish. Reject vague entries like "be more careful during deploys".
- Revisit at day 30. Book the follow-up before leaving the room and check what shipped. A post-mortem whose action items never land is theater.
If the incident was customer-visible, publish a public summary once it is resolved. Your status page incident already holds the updates you posted during the outage, so most of the writing is done.
Next steps
Monitor logs
Per-check results to anchor your timeline.
Reports and outage history
Uptime, outage durations, MTTA and MTTR.
Status page incidents
Publish a public summary once resolved.
Alert triage that scales
Fix the routing gaps your review uncovered.
