Skip to content

Commit e3fd929

Browse files
authored
docs: add more transform examples (#2684)
1 parent 04894a6 commit e3fd929

File tree

3 files changed

+59
-2
lines changed

3 files changed

+59
-2
lines changed

docs/user/filter-and-process/ottl-transform-and-filter/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ This sequence means that your OTTL rules only operate on data that has already p
3030
## Limitations
3131

3232
### Always Use the Full Context Path
33+
3334
You must specify the full path for every field. Short-hand references are not supported.
3435

3536
- Correct: `resource.attributes["k8s.namespace.name"] == "default"`, `log.attributes["level"]`, `datapoint.value_int`, `span.name`

docs/user/filter-and-process/ottl-transform-and-filter/ottl-filter.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,4 +63,4 @@ spec:
6363
filter:
6464
- conditions:
6565
- 'IsMatch(metric.name, "^envoy_") == true and IsMatch(metric.name, ".*outlier_detection.*") == false'
66-
```
66+
```

docs/user/filter-and-process/ottl-transform-and-filter/ottl-transform.md

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Use transformations to modify telemetry data as it flows through a pipeline. You
77
You define these rules in the `transform` section of your Telemetry pipeline's `spec`.
88

99
Each rule in the `transform` list contains:
10+
1011
- `statements`: One or more OTTL functions that modify the data. These are the actions that you want to perform.
1112
- `conditions` (optional): One or more OTTL conditions that must be met. If you provide conditions, the statements only run on data that matches at least one of the conditions.
1213

@@ -51,4 +52,59 @@ spec:
5152
- 'IsMatch(resource.attributes["k8s.namespace.name"], ".*-system")'
5253
statements:
5354
- 'set(span.attributes["system"], "true")'
54-
```
55+
```
56+
57+
## Example: Parse Custom Log Formats
58+
59+
If your application writes logs to stdout in a custom format, you can use a series of OTTL statements to parse these logs into structured data, so you can easily query and analyze them in your observability backend. The following example parses the payload and enriches core attributes:
60+
61+
```yaml
62+
# In your LogPipeline spec
63+
spec:
64+
input:
65+
application:
66+
enabled: true
67+
output:
68+
otlp:
69+
endpoint:
70+
value: http://traces.example.com:4317
71+
transform:
72+
# Try to parse the body as custom parser (default spring boot logback)
73+
# 2025-11-12T14:40:36.828Z INFO 1 --- [demo] [ main] c.e.restservice.RestServiceApplication : Started Application
74+
- statements:
75+
- merge_maps(log.attributes, ExtractPatterns(log.body,"^(?P<timestamp>\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d+Z)\\s+(?P<level>[A-Z]+)\\s+(?P<pid>\\d+)\\s+---\\s+\\[(?P<mdc>[^\\]]+)\\]\\s+\\[\\s*(?P<thread>[^\\]]+)\\s*\\]\\s+(?P<logger>[^\\s:]+)\\s*:\\s*(?P<msg>.*)$"), "upsert")
76+
77+
# Try to enrich core attributes if custom parsing was successful
78+
- conditions:
79+
- log.attributes["msg"] != nil
80+
statements:
81+
- set(log.body, log.attributes["msg"])
82+
- delete_key(log.attributes, "msg")
83+
- conditions:
84+
- log.attributes["timestamp"] != nil
85+
statements:
86+
- set(log.time, Time(log.attributes["timestamp"], "2006-01-02T15:04:05.000Z"))
87+
- delete_key(log.attributes, "timestamp")
88+
- conditions:
89+
- log.attributes["level"] != nil
90+
statements:
91+
- set(log.severity_text, ToUpperCase(log.attributes["level"]))
92+
- delete_key(log.attributes, "level")
93+
```
94+
95+
## Example: Redact Sensitive Data in Spans
96+
97+
To improve security and meet compliance requirements, you can redact sensitive information from your telemetry data before it leaves the cluster.
98+
99+
```yaml
100+
# In your TracePipeline spec
101+
spec:
102+
output:
103+
otlp:
104+
endpoint:
105+
value: http://traces.example.com:4317
106+
transform:
107+
- statements:
108+
- replace_pattern(span.attributes["http.url"], "client_id=[^&]+", "client_id=[REDACTED]")
109+
- replace_pattern(span.attributes["http.url"], "client_secret=[^&]+", "client_secret=[REDACTED]")
110+
```

0 commit comments

Comments
 (0)