Pipeline stage reference
The exact configuration for each of the five stage types, the Drop condition syntax, and the rules. Pipelines are logs-only and run server-side — see Building a pipeline.
Stages
Extract Fields (extract)
Runs a regex with named groups over a field; each named capture becomes a parsed field on the log.
| Config | Meaning |
|---|---|
pattern | The regex (required). Named groups: (?P<name>…). |
field | Source field to match against — defaults to message; can be a label key. |
Example — pull status and path out of an access log message:
pattern: (?P<method>\w+)\s(?P<path>/\S*)\s(?P<status>\d+) field: message
Mask / Redact (mask)
Replaces the value of named fields with a marker.
| Config | Meaning |
|---|---|
fields | Comma-separated field names to mask (required). The literal message masks the body. |
replacement | The marker (default ***REDACTED***). |
Drop Logs (drop)
Discards entries matching a condition (this is the only stage with a match filter). The condition is a small fixed DSL — not regex:
| Form | Example |
|---|---|
level == 'X' / level != 'X' | level == 'debug' (case-insensitive) |
message contains 'X' | message contains 'health check' (substring) |
label.<key> == '<value>' | label.app == 'linkerd-proxy' |
Add Field (add_field)
| Config | Meaning |
|---|---|
field | Label key to add (required). |
value | Static value. |
Rename Field (rename)
| Config | Meaning |
|---|---|
from | Existing key (required). |
to | New key (required). Applies to labels and parsed fields. |
Regex flavor
Extract patterns use Go RE2 (the standard-library regexp engine) — named groups (?P<name>…), but no backreferences and no lookaround. Invalid patterns are rejected when you save. Max pattern length 512 bytes.
Ordering
- Within a pipeline, stages apply in order; a matching Drop removes the line and stops the rest.
- If multiple pipelines match one source, they apply in sequence; the first Drop across any of them removes the line.
- Disabled pipelines are skipped. A pipeline with empty source scope is global (applies to every source).
Permissions & limits
- List: any signed-in user (scoped to their sources). Create / edit / delete: admin or manager.
- Only admins can create a global pipeline (empty source scope), and only admins can create a global pipeline containing a Drop stage (it would silently discard everyone's logs).
- Max 20 stages per pipeline. Test-panel sample capped at 32 KB.
- Config changes take effect within ~30s (server-side cache).