Working with incidents

The Incidents page is where the on-call lives during a problem. Every incident is either a fired alert that's been routed to a channel, or a manually-created incident.

  • Title: Incidents · Track and resolve service incidents.
  • Refresh button top-right.
  • + New Incident button top-right — admins and managers can manually open an incident (useful when the alert system didn't catch something but you want to track the ongoing investigation).

Filter tabs

Five lifecycle filters across the top: All, Open (default), Investigating, Mitigated, Resolved. For what each state means and how incidents move between them (acknowledge, reopen, ack-timeout), see Alerts → Incidents — the canonical lifecycle reference.

Free-text match against the incident title.

Results table

Per-row columns:

ColumnMeans
(checkbox)Bulk-select for batch actions.
TITLEIncident summary, click to open the detail page.
SEVERITYCritical, High, Medium or Low when set by hand — or Warning / Info, copied unchanged from the alert rule that raised the incident. Six values in total; see the two severity fields.
STATUSLifecycle state pill (OPEN / INVESTIGATING / MITIGATED / RESOLVED).
GROUP LABELSLabel=value pills showing what tied alerts together (e.g., service=payments). when none.
ACKNOWLEDGEDWho acked, with a clock icon if their ack expired. when none.
ALERTSNumber of underlying alert events grouped into this incident.
CREATEDWhen the first alert in the group fired.
(last column)External-link icon to open the detail in a new tab.

Detail page

Click an incident:

  • Title + severity + status + action buttons (Acknowledge, Mitigate, Resolve). Acknowledging moves an Open incident to Investigating — see Alerts → Incidents.
  • Group labels.
  • Routing policy that matched, with its channels.
  • Ack history (who acked when, comments, ack expiries).
  • Alert events timeline (firing, resolved, reopened).
  • Notification log (channel, status, timestamp).
  • "Open in Logs" deep-link scoped to the alert's source and time range.
  • Post-mortem editor (see below).

Post-mortems

The right-hand column of the detail page has a Post-Mortem panel: a free-text box and a Save Post-Mortem button. What you write is stored on the incident and rendered on the page from then on; until something is written it reads "No post-mortem written yet."

Writing one requires admin or manager. Viewers see the text but cannot edit it.

It is plain text — Markdown is not rendered — so keep the structure simple. A workable shape is: what broke, how you noticed, what you did, what would have caught it sooner.

Nothing prompts you to write it. Resolving an incident does not ask, and no state depends on the field being filled in, so this is a habit rather than a gate. The reason to keep it here rather than in a separate document is that it sits beside the timeline, the ack history and the notification log — the evidence you would otherwise have to reassemble from memory later.

Bulk actions

Checkbox the rows you want, then apply a state change to all at once. Useful when a deploy creates a swarm of related incidents you want to clear together.

See also