Authoring rules

A rule is the question ObserveKit asks: *is this thing wrong right now?* Click + New Alert on the Rules tab to author one.

Where notifications get attached

Channels are not picked on the rule. Once a rule fires, the routing policies decide which channels see it (see Routing policies). So this form is purely about detection and severity.

The form, section by section

The Create Alert modal is a single-scroll layout with five numbered sections plus a Preview pane at the bottom.

Name + Description (top)

• Name — required. Use a phrase the on-call person will read at 3 AM. "High CPU on production payments" beats "rule-3".
• Description — optional one-line note about what the alert means.

1. Detect

Pick a template or switch to expression mode (link top-right).

Template mode (default):

• Dropdown of 15 built-in templates with plain-English names (High CPU usage, HTTP 5xx error rate, Pod stuck pending, Anomaly detected, etc.).
• Sustained for — duration the condition must hold before firing. Examples: 30s, 5m, 1h. Use 0 for log-pattern alerts that should fire instantly.

Expression mode:

• Free-form PromQL/LogQL-style query. Reference a metric or log stream and compare against a value.
• Same Sustained for field.

2. Scope — Label Matchers

Restrict the rule to a subset of series. + Add matcher to add a row of Label key, Operator (= != =~ !~), Value.

Empty matchers = match every series. The form shows: "No matchers — rule applies to all series. Click '+ Add matcher' to narrow scope."

3. Labels

• Severity dropdown: Critical, Warning (default), Info. Routing policies match on severity, so this drives where the alert is sent.
• Output labels (key=value rows) — extra labels stamped on every event this rule emits. Use them as routing-policy match targets (e.g., add team=platform here, then match team=platform in a routing policy).

4. Message (Annotations)

• Summary — short message included in every notification.
• Description — longer details. Markdown allowed.
• Runbook URL — optional but strongly recommended. Where the on-call should look first.

5. No-Data Behavior

What should happen if the metric stops reporting? Three radio choices:

• Ignore (default) — do nothing. Best when the metric is sparse.
• Notify — fire a nodata alert. Best when missing data itself is a problem.
• Auto-resolve — treat as resolved. Best for hosts that may go offline gracefully.

Preview with current data

The Preview button replays your rule against the last 24 hours of data and reports what would have fired. Use it before saving — it's the cheapest way to catch a too-loose threshold.

Save

When you save, the rule starts evaluating immediately. The Active tab shows any matches. Routing decides where notifications go — see Routing policies next.

Editing or deleting

The Rules tab has pencil and trash icons on every row. Edits take effect on the next evaluation tick (within seconds). Deletions are immediate; existing incidents linked to the deleted rule stay in history.