Authoring rules

A rule is the question ObserveKit asks: *is this thing wrong right now?* Click + New Alert on the Rules tab to author one.

Where notifications get attached

Channels are not picked on the rule. Once a rule fires, the routing policies decide which channels see it (see Routing policies). So this form is purely about detection and severity.

The form, section by section

The Create Alert modal is a single-scroll layout with five numbered sections plus a Preview pane at the bottom.

Name + Description (top)

• Name — required. Use a phrase the on-call person will read at 3 AM. "High CPU on production payments" beats "rule-3".
• Description — optional one-line note about what the alert means.

1. Detect

Pick a template or switch to expression mode (link top-right).

Template mode (default):

• Dropdown of 18 built-in templates with plain-English names (High CPU usage, HTTP 5xx error rate, Pod stuck pending, Anomaly detected, etc.). The full list is in the template catalog.
• Sustained for — duration the condition must hold before firing. Examples: 30s, 5m, 1h. Use 0 for log-pattern alerts that should fire instantly.

Expression mode:

• Free-form PromQL/LogQL-style query. Reference a metric or log stream and compare against a value.
• Same Sustained for field.

2. Scope — Label Matchers

Restrict the rule to a subset of series. + Add matcher to add a row of Label key, Operator (= != =~ !~), Value.

Empty matchers = match every series. The form shows: "No matchers — rule applies to all series. Click '+ Add matcher' to narrow scope."

3. Labels

• Severity dropdown: Critical, Warning (default), Info. Routing policies match on severity, so this drives where the alert is sent.
• Output labels (key=value rows) — extra labels stamped on every event this rule emits. Use them as routing-policy match targets (e.g., add team=platform here, then match team=platform in a routing policy).

4. Message (Annotations)

• Summary — short message included in every notification.
• Description — longer details. Markdown allowed.
• Runbook URL — optional but strongly recommended. Where the on-call should look first.

5. No-Data Behavior

What should happen if the metric stops reporting? Three radio choices:

• Ignore (default) — do nothing. Best when the metric is sparse.
• Notify — fire a nodata alert. Best when missing data itself is a problem.
• Auto-resolve — treat as resolved. Best for hosts that may go offline gracefully.

Preview with current data — admin only

The Preview button replays your rule against the last 24 hours of data and reports what would have fired. Use it before saving — it's the cheapest way to catch a too-loose threshold.

Preview requires the Admin role. The endpoint behind it is admin-only and enforced on the server, so the button is hidden for Managers and Viewers rather than returning a 403 with no explanation.

If you are a Manager, you can still validate a rule — it just takes one more step:

1. Save the rule with the toggle disabled, so it cannot notify anyone.
2. Reproduce the condition you are targeting in Metrics → Explorer or the Log Explorer over the same window, and check the threshold against what you see.
3. Enable the rule and watch the Active tab.

Or ask an Admin to run Preview on it. If your team writes rules regularly and this is friction every time, that is an argument for granting Admin rather than for working around it.

Save

When you save, the rule starts evaluating immediately. The Active tab shows any matches. Routing decides where notifications go — see Routing policies next.

Editing or deleting

The Rules tab has pencil and trash icons on every row. Edits take effect on the next evaluation tick (within seconds). Deletions are immediate; existing incidents linked to the deleted rule stay in history.