Building a pipeline

A log pipeline transforms, masks, or drops log entries on the way in — before they're stored. Use it to redact PII, drop noisy log lines, add labels, or extract structured fields out of unstructured messages. Pipelines run server-side on the ingestion path and apply to logs only (including OTLP logs) — never metrics or traces.

Header

• Title: Log Pipelines · Transform, mask, and filter logs before storage.
• + New Pipeline button top-right.

Empty state

When no pipelines exist, the page shows a funnel icon and:

> "No pipelines configured. Create a pipeline to transform logs before they are stored."

> [+ Create First Pipeline]

Create Pipeline modal

Click + New Pipeline (or + Create First Pipeline in the empty state) to open the modal:

Below the basic fields is a tab toggle:

• Stage Builder — add and order the pipeline's stages.
• Test Panel — paste a sample log line and see what each stage does to it before saving.

Stage types

In the Stage Builder, click any of the five stage buttons to add a stage of that type:

• + Extract Fields — pull structured fields out of a log field using a regex with named groups ((?P<status>\d+)); captures land in the log's parsed fields.
• + Mask / Redact — replace the value of named fields (or the message body) with a redaction marker (default ***REDACTED***).
• + Drop Logs — discard log lines matching a condition (by level, message substring, or label value) — saves storage and ingest cost.
• + Add Field — stamp a static key/value label onto log lines.
• + Rename Field — move a label or parsed field to a new key.

Stages run top to bottom on each incoming log line; a Drop stops processing that line immediately. Only Drop takes a match condition, and Extract applies where its regex matches — the other stages apply to every line the pipeline sees. Exact per-stage config and the Drop condition syntax are in the Pipeline stage reference.

Test Panel

Switch to the Test Panel tab to paste a sample log line and see the result of running each stage on it. Used to validate a regex or check that the right lines are being dropped before saving.

Saving

Create Pipeline persists the configuration. The server picks it up within ~30 seconds and starts applying it to incoming log lines. Pipelines are fully editable and deletable after creation.

Performance

Pipelines run server-side on the ingestion path (not on the agent). Complex regex over high log volume costs server CPU. Keep stages simple — most installations need only one or two Drop / Mask stages. To cut cost *before* data leaves the cluster, filter at the agent instead.

What this is not

• Not for outbound delivery. Pipelines transform on ingest. To transform on the way out (e.g., to a webhook), customise notification templates instead.
• Not for routing alerts. Use Alerts → Routing policies.
• Not for metrics or traces. Pipelines only operate on log lines.