-
Notifications
You must be signed in to change notification settings - Fork 3k
[pkg/ottl] Add support for scaling values #33246
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
TylerHelmuth
merged 43 commits into
open-telemetry:main
from
bacherfl:feat/16214/scale-metric-values
Jul 25, 2024
Merged
Changes from 41 commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
b782416
[processor/transform] Add support for scaling values
bacherfl adcda48
revert change to e2e test
bacherfl 4ebb38a
rename helper function
bacherfl 3ba07ac
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl d610bf0
add documentation for Scale function
bacherfl 9d741ac
Merge remote-tracking branch 'bacherfl/feat/16214/scale-metric-values…
bacherfl d098a77
adapt to possible types returned by metric context path
bacherfl f019ec0
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl 15a2006
additional test for exponential histogram
bacherfl 213fcce
Merge remote-tracking branch 'bacherfl/feat/16214/scale-metric-values…
bacherfl 285a37d
use type switch for handling different types as Scale argument
bacherfl fc1dc7f
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl 4084b53
incorporate feedback from PR review
bacherfl 35064e1
Merge remote-tracking branch 'bacherfl/feat/16214/scale-metric-values…
bacherfl 343ce4e
add license comment
bacherfl 549bdc3
address PR review comments, add changelog entry
bacherfl fb04e21
fix linting
bacherfl 75e24b1
Merge branch 'main' into feat/16214/scale-metric-values
evan-bradley 49321a2
Update pkg/ottl/ottlfuncs/func_scale.go
bacherfl 47d6834
fix e2e tests
bacherfl 313cd7e
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl d76e6cc
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl 44997a1
Merge branch 'main' into feat/16214/scale-metric-values
evan-bradley 0f07a49
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl 92d33ed
adapt scale function to be an editor function
bacherfl f660ca6
remove unused testing struct
bacherfl 37f16bb
remove unused testing struct; adapt test
bacherfl 62a539b
adapt test
bacherfl 61e2a09
fix linting
bacherfl 75a6d67
adapt the function to only work with metrics
bacherfl 14bbdd8
remove obsolete test
bacherfl 0090fcc
apply suggestions from PR review
bacherfl d398d64
move scale function to transformprocessor
bacherfl e495e03
fix failing checks
bacherfl c83e5ba
fix linting
bacherfl 66c5bd5
appease linter
bacherfl bf746e7
make unit an optional value
bacherfl ffed4f1
Merge branch 'main' into feat/16214/scale-metric-values
bacherfl 719daca
adapt unit tests to recent changes
bacherfl 721c537
apply suggestion from code review
bacherfl a6c39a1
adapt to pr review
bacherfl 876afc1
Apply suggestions from code review
bacherfl 42d3140
incorporate review comments
bacherfl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# Use this changelog template to create an entry for release notes. | ||
|
||
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' | ||
change_type: enhancement | ||
|
||
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver) | ||
component: processor/transform | ||
|
||
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). | ||
note: Add `scale_metric` function that scales all data points in a metric. | ||
|
||
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. | ||
issues: [16214] | ||
|
||
# (Optional) One or more lines of additional information to render under the primary note. | ||
# These lines will be padded with 2 spaces and then inserted directly into the document. | ||
# Use pipe (|) for multiline entries. | ||
subtext: | ||
|
||
# If your change doesn't affect end users or the exported elements of any package, | ||
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label. | ||
# Optional: The change log or logs in which this entry should be included. | ||
# e.g. '[user]' or '[user, api]' | ||
# Include 'user' if the change is relevant to end users. | ||
# Include 'api' if there is a change to a library API. | ||
# Default: '[user]' | ||
change_logs: [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
130 changes: 130 additions & 0 deletions
130
processor/transformprocessor/internal/metrics/func_scale.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package metrics // import "github.com/open-telemetry/opentelemetry-collector-contrib/processor/transformprocessor/internal/metrics" | ||
|
||
import ( | ||
"context" | ||
"errors" | ||
"fmt" | ||
|
||
"go.opentelemetry.io/collector/pdata/pmetric" | ||
|
||
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl" | ||
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottlmetric" | ||
) | ||
|
||
type ScaleArguments struct { | ||
Multiplier float64 | ||
Unit ottl.Optional[ottl.StringGetter[ottlmetric.TransformContext]] | ||
} | ||
|
||
func newScaleMetricFactory() ottl.Factory[ottlmetric.TransformContext] { | ||
return ottl.NewFactory("scale_metric", &ScaleArguments{}, createScaleFunction) | ||
} | ||
|
||
func createScaleFunction(_ ottl.FunctionContext, oArgs ottl.Arguments) (ottl.ExprFunc[ottlmetric.TransformContext], error) { | ||
args, ok := oArgs.(*ScaleArguments) | ||
|
||
if !ok { | ||
return nil, fmt.Errorf("ScaleFactory args must be of type *ScaleArguments[K]") | ||
} | ||
|
||
return Scale(*args) | ||
} | ||
|
||
func Scale(args ScaleArguments) (ottl.ExprFunc[ottlmetric.TransformContext], error) { | ||
return func(ctx context.Context, tCtx ottlmetric.TransformContext) (any, error) { | ||
metric := tCtx.GetMetric() | ||
|
||
var unit string | ||
evan-bradley marked this conversation as resolved.
Show resolved
Hide resolved
|
||
var err error | ||
if !args.Unit.IsEmpty() { | ||
unit, err = args.Unit.Get().Get(ctx, tCtx) | ||
if err != nil { | ||
return nil, fmt.Errorf("could not get unit from ScaleArguments: %w", err) | ||
} | ||
} | ||
|
||
switch metric.Type() { | ||
case pmetric.MetricTypeGauge: | ||
scaleMetric(metric.Gauge().DataPoints(), args.Multiplier) | ||
case pmetric.MetricTypeHistogram: | ||
scaleHistogram(metric.Histogram().DataPoints(), args.Multiplier) | ||
case pmetric.MetricTypeSummary: | ||
scaleSummarySlice(metric.Summary().DataPoints(), args.Multiplier) | ||
case pmetric.MetricTypeSum: | ||
scaleMetric(metric.Sum().DataPoints(), args.Multiplier) | ||
case pmetric.MetricTypeExponentialHistogram: | ||
return nil, errors.New("exponential histograms are not supported by the 'scale_metric' function") | ||
default: | ||
return nil, fmt.Errorf("unsupported metric type: '%v'", metric.Type()) | ||
} | ||
if !args.Unit.IsEmpty() { | ||
metric.SetUnit(unit) | ||
} | ||
|
||
return nil, nil | ||
}, nil | ||
} | ||
|
||
func scaleExemplar(ex *pmetric.Exemplar, multiplier float64) { | ||
switch ex.ValueType() { | ||
case pmetric.ExemplarValueTypeInt: | ||
ex.SetIntValue(int64(float64(ex.IntValue()) * multiplier)) | ||
case pmetric.ExemplarValueTypeDouble: | ||
ex.SetDoubleValue(ex.DoubleValue() * multiplier) | ||
} | ||
} | ||
|
||
func scaleSummarySlice(values pmetric.SummaryDataPointSlice, multiplier float64) { | ||
for i := 0; i < values.Len(); i++ { | ||
dp := values.At(i) | ||
|
||
dp.SetSum(dp.Sum() * multiplier) | ||
|
||
for i := 0; i < dp.QuantileValues().Len(); i++ { | ||
qv := dp.QuantileValues().At(i) | ||
qv.SetValue(qv.Value() * multiplier) | ||
} | ||
} | ||
} | ||
|
||
func scaleHistogram(datapoints pmetric.HistogramDataPointSlice, multiplier float64) { | ||
for i := 0; i < datapoints.Len(); i++ { | ||
dp := datapoints.At(i) | ||
|
||
if dp.HasSum() { | ||
dp.SetSum(dp.Sum() * multiplier) | ||
} | ||
if dp.HasMin() { | ||
dp.SetMin(dp.Min() * multiplier) | ||
} | ||
if dp.HasMax() { | ||
dp.SetMax(dp.Max() * multiplier) | ||
} | ||
|
||
for bounds, bi := dp.ExplicitBounds(), 0; bi < bounds.Len(); bi++ { | ||
bounds.SetAt(bi, bounds.At(bi)*multiplier) | ||
} | ||
|
||
for exemplars, ei := dp.Exemplars(), 0; ei < exemplars.Len(); ei++ { | ||
exemplar := exemplars.At(ei) | ||
scaleExemplar(&exemplar, multiplier) | ||
} | ||
} | ||
} | ||
|
||
func scaleMetric(points pmetric.NumberDataPointSlice, multiplier float64) { | ||
for i := 0; i < points.Len(); i++ { | ||
dp := points.At(i) | ||
switch dp.ValueType() { | ||
case pmetric.NumberDataPointValueTypeInt: | ||
dp.SetIntValue(int64(float64(dp.IntValue()) * multiplier)) | ||
|
||
case pmetric.NumberDataPointValueTypeDouble: | ||
dp.SetDoubleValue(dp.DoubleValue() * multiplier) | ||
default: | ||
} | ||
} | ||
} |
204 changes: 204 additions & 0 deletions
204
processor/transformprocessor/internal/metrics/func_scale_test.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,204 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package metrics | ||
|
||
import ( | ||
"context" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/assert" | ||
"go.opentelemetry.io/collector/pdata/pcommon" | ||
"go.opentelemetry.io/collector/pdata/pmetric" | ||
|
||
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl" | ||
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl/contexts/ottlmetric" | ||
) | ||
|
||
func TestScale(t *testing.T) { | ||
TylerHelmuth marked this conversation as resolved.
Show resolved
Hide resolved
|
||
type testCase struct { | ||
name string | ||
args ScaleArguments | ||
valueFunc func() pmetric.Metric | ||
wantFunc func() pmetric.Metric | ||
wantErr bool | ||
} | ||
tests := []testCase{ | ||
{ | ||
name: "scale gauge float metric", | ||
valueFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptyGauge() | ||
metric.Gauge().DataPoints().AppendEmpty().SetDoubleValue(10.0) | ||
|
||
return metric | ||
}, | ||
args: ScaleArguments{ | ||
Multiplier: 10.0, | ||
Unit: ottl.NewTestingOptional[ottl.StringGetter[ottlmetric.TransformContext]](ottl.StandardStringGetter[ottlmetric.TransformContext]{ | ||
Getter: func(_ context.Context, _ ottlmetric.TransformContext) (any, error) { | ||
return "kWh", nil | ||
}, | ||
}), | ||
}, | ||
wantFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptyGauge() | ||
metric.SetUnit("kWh") | ||
metric.Gauge().DataPoints().AppendEmpty().SetDoubleValue(100.0) | ||
|
||
return metric | ||
}, | ||
wantErr: false, | ||
}, | ||
{ | ||
name: "scale gauge int metric", | ||
valueFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptyGauge() | ||
metric.Gauge().DataPoints().AppendEmpty().SetIntValue(10) | ||
|
||
return metric | ||
}, | ||
args: ScaleArguments{ | ||
Multiplier: 10.0, | ||
}, | ||
wantFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptyGauge() | ||
metric.Gauge().DataPoints().AppendEmpty().SetIntValue(100.0) | ||
|
||
return metric | ||
}, | ||
wantErr: false, | ||
}, | ||
{ | ||
name: "scale sum metric", | ||
valueFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptySum() | ||
metric.Sum().DataPoints().AppendEmpty().SetDoubleValue(10.0) | ||
|
||
return metric | ||
}, | ||
args: ScaleArguments{ | ||
Multiplier: 10.0, | ||
}, | ||
wantFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptySum() | ||
metric.Sum().DataPoints().AppendEmpty().SetDoubleValue(100.0) | ||
|
||
return metric | ||
}, | ||
wantErr: false, | ||
}, | ||
{ | ||
name: "scale histogram metric", | ||
valueFunc: func() pmetric.Metric { | ||
metric := getTestScalingHistogramMetric(1, 4, 1, 3, []float64{1, 10}, []uint64{1, 2}, []float64{1.0}, 1, 1) | ||
return metric | ||
}, | ||
args: ScaleArguments{ | ||
Multiplier: 10.0, | ||
}, | ||
wantFunc: func() pmetric.Metric { | ||
metric := getTestScalingHistogramMetric(1, 40, 10, 30, []float64{10, 100}, []uint64{1, 2}, []float64{10.0}, 1, 1) | ||
return metric | ||
}, | ||
wantErr: false, | ||
}, | ||
{ | ||
name: "scale summary metric", | ||
valueFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
dp := metric.SetEmptySummary().DataPoints().AppendEmpty() | ||
dp.SetSum(10.0) | ||
qv := dp.QuantileValues().AppendEmpty() | ||
qv.SetValue(10.0) | ||
|
||
return metric | ||
}, | ||
args: ScaleArguments{ | ||
Multiplier: 10.0, | ||
}, | ||
wantFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
dp := metric.SetEmptySummary().DataPoints().AppendEmpty() | ||
dp.SetSum(100.0) | ||
qv := dp.QuantileValues().AppendEmpty() | ||
qv.SetValue(100.0) | ||
|
||
return metric | ||
}, | ||
wantErr: false, | ||
}, | ||
{ | ||
name: "unsupported: exponential histogram metric", | ||
valueFunc: func() pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetEmptyExponentialHistogram() | ||
return metric | ||
}, | ||
args: ScaleArguments{ | ||
Multiplier: 10.0, | ||
}, | ||
wantFunc: func() pmetric.Metric { | ||
// value should not be modified | ||
metric := pmetric.NewMetric() | ||
metric.SetEmptyExponentialHistogram() | ||
return metric | ||
}, | ||
wantErr: true, | ||
}, | ||
} | ||
for _, tt := range tests { | ||
t.Run(tt.name, func(t *testing.T) { | ||
target := ottlmetric.NewTransformContext( | ||
tt.valueFunc(), | ||
pmetric.NewMetricSlice(), | ||
pcommon.NewInstrumentationScope(), | ||
pcommon.NewResource(), | ||
pmetric.NewScopeMetrics(), | ||
pmetric.NewResourceMetrics(), | ||
) | ||
|
||
expressionFunc, _ := Scale(tt.args) | ||
_, err := expressionFunc(context.Background(), target) | ||
|
||
if tt.wantErr { | ||
assert.Error(t, err) | ||
} else { | ||
assert.NoError(t, err) | ||
} | ||
assert.EqualValues(t, tt.wantFunc(), target.GetMetric()) | ||
}) | ||
} | ||
} | ||
|
||
func getTestScalingHistogramMetric(count uint64, sum, min, max float64, bounds []float64, bucketCounts []uint64, exemplars []float64, start, timestamp pcommon.Timestamp) pmetric.Metric { | ||
metric := pmetric.NewMetric() | ||
metric.SetName("test-metric") | ||
metric.SetEmptyHistogram() | ||
histogramDatapoint := metric.Histogram().DataPoints().AppendEmpty() | ||
histogramDatapoint.SetCount(count) | ||
histogramDatapoint.SetSum(sum) | ||
histogramDatapoint.SetMin(min) | ||
histogramDatapoint.SetMax(max) | ||
histogramDatapoint.ExplicitBounds().FromRaw(bounds) | ||
histogramDatapoint.BucketCounts().FromRaw(bucketCounts) | ||
for i := 0; i < len(exemplars); i++ { | ||
exemplar := histogramDatapoint.Exemplars().AppendEmpty() | ||
exemplar.SetTimestamp(1) | ||
exemplar.SetDoubleValue(exemplars[i]) | ||
} | ||
histogramDatapoint.SetStartTimestamp(start) | ||
histogramDatapoint.SetTimestamp(timestamp) | ||
return metric | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.