Skip to content

[metricstarttimeprocessor] Add subtract_initial_point strategy #38379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
dashpole opened this issue Mar 5, 2025 · 2 comments · Fixed by #38594
Closed

[metricstarttimeprocessor] Add subtract_initial_point strategy #38379

dashpole opened this issue Mar 5, 2025 · 2 comments · Fixed by #38594
Assignees
@ridwanmsharif
Labels
enhancement New feature or request processor/metricstarttime Stale

Comments

@dashpole
Copy link
Contributor

dashpole commented Mar 5, 2025

Component(s)

processor/metricstarttime

Is your feature request related to a problem? Please describe.

This is part of implementing the metricstarttime processor. This strategy is useful because is preserves semantics for cumulative metrics, meaning that the cumulative value between the start and end time is from observations that occured in that window.

Describe the solution you'd like

  • For each resource, and for each timeseries
    • Skip if the point is not a cumulative sum, histogram, exp histogram, or summary, or if it already has a start timestamp.
    • Lookup the timeseries in the cache. If it is not present, drop this point from the batch, and store the timestamp and value (including bucket counts, etc) in the cache.
    • Subtract cumulative values (e.g. sum, bucket counts) of the cached timeseries from the current timeseries, and set the start timestamp to the cached point's timestamp.
    • Drop any points where the start timestamp is after the timestamp.

Describe alternatives you've considered

No response

Additional context

This is similar to what is done in the google exporters: https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/internal/normalization/standard_normalizer.go
@dashpole dashpole added enhancement New feature or request needs triage New item requiring triage labels Mar 5, 2025
@dashpole dashpole added processor/metricstarttime and removed needs triage New item requiring triage labels Mar 5, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 5, 2025

Pinging code owners for processor/metricstarttime: @dashpole @ridwanmsharif. See Adding Labels via Comments if you do not have permissions to add labels yourself. For example, comment '/label priority:p2 -needs-triaged' to set the priority and remove the needs-triaged label.

@dashpole dashpole mentioned this issue Mar 5, 2025
Merged
atoulme pushed a commit that referenced this issue Mar 7, 2025
@dashpole
Add subtract_initial_point strategy skeleton (#38380)
1aed17d 
#### Description

Add a skeleton for the `subtract_initial_point` strategy. The
implementation will follow in subsequent PRs.

#### Link to tracking issue

Part of
#37186,
and #38379

#### Testing

Added config_test.go

#### Documentation

Updated the README.
@github-actions github-actions bot mentioned this issue Mar 11, 2025
Closed
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue Mar 12, 2025
@ridwanmsharif
metricstarttimeprocessor: Add subtractinitial strategy implementation
f6df1f1 
This change implements the strategy defined in open-telemetry#38379.
@ridwanmsharif ridwanmsharif mentioned this issue Mar 12, 2025
Closed
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue Mar 13, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
0460076 
This change addresses open-telemetry#38379.
@ridwanmsharif ridwanmsharif mentioned this issue Mar 13, 2025
Merged
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue Mar 19, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
a176135 
This change addresses open-telemetry#38379.
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue Mar 19, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
c5ae763 
This change addresses open-telemetry#38379.
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue Apr 14, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
e67ac9a 
This change addresses open-telemetry#38379.
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue May 2, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
cd881f1 
This change addresses open-telemetry#38379.
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue May 2, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
7ee15b5 
This change addresses open-telemetry#38379.
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 5, 2025

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label May 5, 2025
ridwanmsharif added a commit to ridwanmsharif/opentelemetry-collector-contrib that referenced this issue May 5, 2025
@ridwanmsharif
metricstarttimeprocessor: add subtractinitial strategy implementation
d558917 
This change addresses open-telemetry#38379.
atoulme pushed a commit that referenced this issue May 5, 2025
@ridwanmsharif
metricstarttimeprocessor: Implementation of the subtractinitial strat…
d23e351 
…egy (#38594)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Addresses #38379.

Some of this logic already existed in the [GCM
exporter](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/internal/normalization/standard_normalizer.go).

For each resource, and for each timeseries
- Skip if the point is not a cumulative sum, histogram, exp histogram,
or summary.
- Lookup the timeseries in the cache. If it is not present, drop this
point from the batch, and store the timestamp and value (including
bucket counts, etc) in the cache.
- Subtract cumulative values (e.g. sum, bucket counts) of the cached
timeseries from the current timeseries, and set the start timestamp to
the cached point's timestamp.

<!-- Issue number (e.g. #1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes #38379.

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added unit and integration tests.

<!--Describe the documentation added.-->
#### Documentation
Readme already covers this strategy

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Ridwan Sharif <[email protected]>
vincentfree pushed a commit to ing-bank/opentelemetry-collector-contrib that referenced this issue May 6, 2025
@ridwanmsharif @vincentfree
metricstarttimeprocessor: Implementation of the subtractinitial strat…
5fbab05 
…egy (open-telemetry#38594)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Addresses open-telemetry#38379.

Some of this logic already existed in the [GCM
exporter](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/internal/normalization/standard_normalizer.go).

For each resource, and for each timeseries
- Skip if the point is not a cumulative sum, histogram, exp histogram,
or summary.
- Lookup the timeseries in the cache. If it is not present, drop this
point from the batch, and store the timestamp and value (including
bucket counts, etc) in the cache.
- Subtract cumulative values (e.g. sum, bucket counts) of the cached
timeseries from the current timeseries, and set the start timestamp to
the cached point's timestamp.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#38379.

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added unit and integration tests.

<!--Describe the documentation added.-->
#### Documentation
Readme already covers this strategy

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Ridwan Sharif <[email protected]>
vincentfree pushed a commit to ing-bank/opentelemetry-collector-contrib that referenced this issue May 20, 2025
@ridwanmsharif @vincentfree
metricstarttimeprocessor: Implementation of the subtractinitial strat…
4542f60 
…egy (open-telemetry#38594)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Addresses open-telemetry#38379.

Some of this logic already existed in the [GCM
exporter](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/internal/normalization/standard_normalizer.go).

For each resource, and for each timeseries
- Skip if the point is not a cumulative sum, histogram, exp histogram,
or summary.
- Lookup the timeseries in the cache. If it is not present, drop this
point from the batch, and store the timestamp and value (including
bucket counts, etc) in the cache.
- Subtract cumulative values (e.g. sum, bucket counts) of the cached
timeseries from the current timeseries, and set the start timestamp to
the cached point's timestamp.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#38379.

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added unit and integration tests.

<!--Describe the documentation added.-->
#### Documentation
Readme already covers this strategy

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Ridwan Sharif <[email protected]>
dragonlord93 pushed a commit to dragonlord93/opentelemetry-collector-contrib that referenced this issue May 23, 2025
@ridwanmsharif @dragonlord93
metricstarttimeprocessor: Implementation of the subtractinitial strat…
e3528fe 
…egy (open-telemetry#38594)

<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
#### Description
Addresses open-telemetry#38379.

Some of this logic already existed in the [GCM
exporter](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/internal/normalization/standard_normalizer.go).

For each resource, and for each timeseries
- Skip if the point is not a cumulative sum, histogram, exp histogram,
or summary.
- Lookup the timeseries in the cache. If it is not present, drop this
point from the batch, and store the timestamp and value (including
bucket counts, etc) in the cache.
- Subtract cumulative values (e.g. sum, bucket counts) of the cached
timeseries from the current timeseries, and set the start timestamp to
the cached point's timestamp.

<!-- Issue number (e.g. open-telemetry#1234) or full URL to issue, if applicable. -->
#### Link to tracking issue
Fixes open-telemetry#38379.

<!--Describe what testing was performed and which tests were added.-->
#### Testing
Added unit and integration tests.

<!--Describe the documentation added.-->
#### Documentation
Readme already covers this strategy

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Ridwan Sharif <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ridwanmsharif ridwanmsharif

Labels
enhancement New feature or request processor/metricstarttime Stale
Projects
None yet
Milestone
No milestone
2 participants
@dashpole @ridwanmsharif