Skip to content

[processor/redaction] Add Support for Redacting log.body in the Redaction Processor #37239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
karakayasemi opened this issue Jan 15, 2025 · 6 comments · Fixed by #37369
Closed

[processor/redaction] Add Support for Redacting log.body in the Redaction Processor #37239

karakayasemi opened this issue Jan 15, 2025 · 6 comments · Fixed by #37369
Labels
enhancement New feature or request help wanted Extra attention is needed processor/redaction Redaction processor Stale

Comments

@karakayasemi
Copy link
Contributor

Component(s)

processor/redaction

Is your feature request related to a problem? Please describe.

The redaction processor currently supports redacting attributes. However, sensitive data often resides in log.body, where user-generated messages or critical information is logged.

Describe the solution you'd like

Add support for redacting log.body in the redaction processor.

Describe alternatives you've considered

No response

Additional context

If this proposal makes sense to the community, I am willing to contribute to the implementation as a feature PR.
@karakayasemi karakayasemi added enhancement New feature or request needs triage New item requiring triage labels Jan 15, 2025
@github-actions github-actions bot added the processor/redaction Redaction processor label Jan 15, 2025
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@TylerHelmuth TylerHelmuth removed the needs triage New item requiring triage label Jan 15, 2025
@TylerHelmuth
Copy link
Member

I agree that the body is a logical field for this component to comprehend

@TylerHelmuth TylerHelmuth added the help wanted Extra attention is needed label Jan 15, 2025
@akstron
Copy link
Contributor

akstron commented Jan 18, 2025

Hi @karakayasemi , I would like to work on this if possible. As I can see, we want to add something like a blocked_values (similar to the current implementation which masks content in the value of the attributes) for content in log.body. Let me know if this sounds good or do you have something else in mind?

@karakayasemi
Copy link
Contributor Author

karakayasemi commented Jan 20, 2025 •
edited
Loading

Yes, you can work on this if you are available. We can extend the existing blocked_values logic to log.body for consistency. The current implementation adds redaction.masked.keys and redaction.masked.count as resource attributes and attributes for summary. Similarly, we could introduce redaction.body.masked.keys and redaction.body.masked.count and hold it on attributes to track redactions in log.body. How does that sound?

@github-actions github-actions bot mentioned this issue Jan 21, 2025
Closed
karakayasemi pushed a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Jan 21, 2025
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
787420d
karakayasemi pushed a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Jan 21, 2025
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
5a8656c
@karakayasemi karakayasemi mentioned this issue Jan 21, 2025
Merged
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Jan 21, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
8c9b0ca
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Jan 21, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
3db3e73
@karakayasemi
Copy link
Contributor Author

@akstron I have already submitted a PR, feel free to add any comment.

karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Feb 14, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
035a5e2
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Feb 14, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
a883245
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Feb 14, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
3afc5d2
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Feb 14, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
41f94cb
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 7, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
8ca8458
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 7, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
531b24a
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 7, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
8eb1482
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 7, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
f9bf49a
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 10, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
701fe2b
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 13, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
f7d214c
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 13, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
ecd6c77
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 19, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
11733db
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Mar 19, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
552720b
@LucaLanziani LucaLanziani mentioned this issue Mar 23, 2025
Closed
@LucaLanziani LucaLanziani mentioned this issue Mar 23, 2025
Closed
@github-actions GitHub Actions
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Mar 24, 2025
@github-actions github-actions bot mentioned this issue Mar 25, 2025
Closed
This was referenced Apr 1, 2025
Closed
Closed
This was referenced Apr 15, 2025
Closed
Closed
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Apr 22, 2025
@karakayasemi
[processor/redaction] apply redaction to log.body (open-telemetry#37239)
0e184c8
karakayasemi pushed a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Apr 22, 2025
[processor/redaction] use AsString() instead of Str() to get string v…
a7a1791 
…alue (open-telemetry#37239)
karakayasemi added a commit to karakayasemi/opentelemetry-collector-contrib that referenced this issue Apr 22, 2025
@karakayasemi
[processor/redaction] use AsString() instead of Str() to get string v…
4ebc42c 
…alue (open-telemetry#37239)
@github-actions github-actions bot mentioned this issue Apr 29, 2025
Closed
@github-actions github-actions bot mentioned this issue May 6, 2025
Closed
dmitryax pushed a commit that referenced this issue May 7, 2025
@karakayasemi @atoulme
[processor/redaction] apply redaction to log.body (#37239) (#37369)
9f2f46b 
## Description  
This update applies redaction rules to the `log.body` field by
supporting multiple data types, including string, slices, maps, and
nested maps. It applies key-based configurations consistently across all
map structures.

- **Key-Based Configurations**: Applies redaction rules at all nesting
levels.
- **New Debug Attributes**:  
  - `redaction.body.redacted.keys` / `redaction.body.redacted.count`  
  - `redaction.body.masked.keys` / `redaction.body.masked.count`  
  - `redaction.body.allowed.keys` / `redaction.body.allowed.count`  
  - `redaction.body.ignored.count`  

#### Link to tracking issue
Fixes #37239

## Testing & Refactoring  
- **Extended Tests**: Log body redaction added to all existing cases +
new tests for detailed body structure validation.
- **Code Cleanup**: Introduced helper methods for redaction, masking,
allowing, and ignoring logic.
- **Improved Readability**: Renamed some const variables for clarity and
consistency.

---------

Co-authored-by: Antoine Toulme <[email protected]>
dragonlord93 pushed a commit to dragonlord93/opentelemetry-collector-contrib that referenced this issue May 23, 2025
@karakayasemi @atoulme @dragonlord93
[processor/redaction] apply redaction to log.body (open-telemetry#37239…
eb1a938 
…) (open-telemetry#37369)

## Description  
This update applies redaction rules to the `log.body` field by
supporting multiple data types, including string, slices, maps, and
nested maps. It applies key-based configurations consistently across all
map structures.

- **Key-Based Configurations**: Applies redaction rules at all nesting
levels.
- **New Debug Attributes**:  
  - `redaction.body.redacted.keys` / `redaction.body.redacted.count`  
  - `redaction.body.masked.keys` / `redaction.body.masked.count`  
  - `redaction.body.allowed.keys` / `redaction.body.allowed.count`  
  - `redaction.body.ignored.count`  

#### Link to tracking issue
Fixes open-telemetry#37239

## Testing & Refactoring  
- **Extended Tests**: Log body redaction added to all existing cases +
new tests for detailed body structure validation.
- **Code Cleanup**: Introduced helper methods for redaction, masking,
allowing, and ignoring logic.
- **Improved Readability**: Renamed some const variables for clarity and
consistency.

---------

Co-authored-by: Antoine Toulme <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request help wanted Extra attention is needed processor/redaction Redaction processor Stale
Projects
None yet
Milestone
No milestone
3 participants
@TylerHelmuth @karakayasemi @akstron