Skip to content

Define splunk.trace.snapshot.profiling Baggage Key and Propagation Details #342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Define splunk.trace.snapshot.profiling Baggage Key and Propagation Details #342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b318890
Define how traces should be selected for snapshotting.
tduncan May 30, 2025
1c96d07
Add changelog entry.
tduncan May 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,20 @@

## [Unreleased]

### Semantic Conventions

#### Enhancements

- Define the `splunk.trace.snapshot.volume` OpenTelemetry baggage entry with valid vales
of `highest`, `off`, and `unspecified`.
[#342](https://github.com/signalfx/gdi-specification/pull/342)

#### Behaviors

- Define how traces should be selected for snapshotting and how the `splunk.trace.snapshot.volume` baggage entry should
be propagated.
[#342](https://github.com/signalfx/gdi-specification/pull/342)

## [1.7.0] - 2025-01-07

### Configuration
Expand Down
29 changes: 29 additions & 0 deletions specification/behaviors.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,32 @@ metrics).

See [integration_context.md](integration_context.md) for specifics about
exchanging additional context between AppD and splunk-otel based agents.

## Trace Snapshot Volume

**Status**: [Experimental](../README.md#versioning-and-status-of-the-specification)

### Context Propagation

The trace snapshot volume MUST be propagated using the OpenTelemetry [`baggage`](https://opentelemetry.io/docs/concepts/signals/baggage/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/signalfx/gdi-specification/blob/main/specification/integration_context.md uses headers to propagate info. Why should we use baggage here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unfamiliar with the constraints of that effort but for trace snapshot profiling we utilize the Baggage so the signal will propagate beyond services instrumented with agents that do not have trace snapshotting capabilities.

Is the ask to include the WHY in the document?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the ask to include the WHY in the document?

no, I was just curious why we are using different solutions for a similar problem. Note that with java api users can easily clear existing baggage when using Baggage.builder().put("mybaggage", "foo").build().makeCurrent(); instead of Baggage.current().toBuilder().put("mybaggage", "foo").build().makeCurrent();

cc @breedx-splk

The baggage key MUST be `splunk.trace.snapshot.volume` with a value as listed in [semantic_conventions.md](semantic_conventions.md).

### Trace Selection
Agents SHOULD make a trace selection decision when a trace root is detected. Trace selection MUST be randomized with the
following constraints:
* Default selection rate of 0.01
* Maximum selection rate of 0.10

Agents SHOULD make trace selection decisions based on trace ID when `splunk.trace.snapshot.volume` has not been set.
Trace ID-based selection MUST follow the same approach as described in (https://github.com/open-telemetry/opentelemetry-specification/blob/9eee5293f95b9fd74f6f1c280b97f87aaec872d7/specification/trace/sdk.md#traceidratiobased-sampler-algorithm)

When a trace is selected for snapshotting the `splunk.trace.snapshot.volume` value MUST be set to `highest`.
When a trace is not selected for snapshotting the `splunk.trace.snapshot.volume` value MUST be set to `off`.

When baggage entry is set:
* Agents MUST use previously set `splunk.trace.snapshot.volume` value internally.
* Agents MUST propagate the same `splunk.trace.snapshot.volume` value to downstream agents
* Agents MUST NOT set the `splunk.trace.snapshot.volume` baggage entry to any other value

When baggage entry is not set:
* Agents SHOULD use a value of `unspecified` internally.
20 changes: 20 additions & 0 deletions specification/semantic_conventions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,3 +272,23 @@ For each `cpu` sample:
in milliseconds if this sample represents a periodic event
- label `thread.state` of type `string` OPTIONALLY can be set to describe
the state of the thread

## Trace Snapshot Volume

**Status**: [Experimental](../README.md#versioning-and-status-of-the-specification)

The trace snapshot volume MUST be propagated using the OpenTelemetry [`baggage`](https://opentelemetry.io/docs/concepts/signals/baggage/)

Trace snapshot volume is specified by the `splunk.trace.snapshot.volume` baggage key. Valid values are as follows:
* `highest`
* `off`
* `unspecified`

Note: a value of `highest` rather than `on` was chosen intentionally to allow for potential volume ranges in the future.

The `splunk.trace.snapshot.volume` baggage entry MUST be set to either `highest` or `off` when a snapshotting selection
decision is made by an agent. When the `splunk.trace.snapshot.volume` the value assumed is assumed to be `unspecified`.

When a `splunk.trace.snapshot.volume` other than `unspecified` is set an agent MUST use that value internally.

Agents MUST NOT set the `splunk.trace.snapshot.volume` baggage entry to any other value when already set by an upstream agent.