[exporter/signalfx] Remove deprecated translation_rules configuration option

.chloggen/signalfx_exporter_remove_translation_rules.yaml

@@ -0,0 +1,29 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: breaking
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: signalfxexporter
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Remove the deprecated configuration option `translation_rules`
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [35332]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: |
+  Please use processors to handle desired metric transformations instead. Find migration guidance in the
+  [translation rules migration guide](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/signalfxexporter/docs/translation_rules_migration_guide.md).
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: []

exporter/signalfxexporter/README.md

@@ -81,7 +81,7 @@ The following configuration options can also be configured:
   updates.
 - `disable_default_translation_rules` (default = `false`): Disable default translation
   of the OTel metrics to a SignalFx compatible format. The default translation rules are
-  defined in `translation/constants.go`.
+  defined in [`internal/translation/constants.go`](./internal/translation/constants.go).
 - `timeout` (default = 10s): Amount of time to wait for a send operation to
   complete.
 - `http2_read_idle_timeout` (default = 10s): Send a ping frame for a health check if the connection has been idle for the configured value.
@@ -179,7 +179,7 @@ One of `realm` and `api_url` are required.
 ## Default Metric Filters
 [List of metrics excluded by default](./internal/translation/default_metrics.go)
 
-Some OpenTelemetry receivers may send metrics that SignalFx considers to be categorized as custom metrics. In order to prevent unwanted overage usage due to custom metrics from these receivers, the SignalFx exporter has a [set of metrics excluded by default](./internal/translation/default_metrics.go). Some exclusion rules use regex to exclude multiple metric names. Some metrics are only excluded if specific resource labels (dimensions) are present. If `translation_rules` are configured and new metrics match a default exclusion, the new metric will still be excluded. Users may configure the SignalFx exporter's `include_metrics` config option to override the any of the default exclusions, as `include_metrics` will always take precedence over any exclusions. An example of `include_metrics` is shown below.
+Some OpenTelemetry receivers may send metrics that SignalFx considers to be categorized as custom metrics. In order to prevent unwanted overage usage due to custom metrics from these receivers, the SignalFx exporter has a [set of metrics excluded by default](./internal/translation/default_metrics.go). Some exclusion rules use regex to exclude multiple metric names. Some metrics are only excluded if specific resource labels (dimensions) are present. If the default translation rules are enabled and match an exclusion rule, the exclusion takes precedence. Users may configure the SignalFx exporter's `include_metrics` config option to override the any of the default exclusions, as `include_metrics` will always take precedence over any exclusions. An example of `include_metrics` is shown below.
 
 ```
 exporters:
@@ -205,25 +205,9 @@ exporters:
 
 ## Translation Rules and Metric Transformations
 
-The `translation_rules` metrics configuration field accepts a list of metric-transforming actions to
-help ensure compatibility with custom charts and dashboards when using the OpenTelemetry Collector. It also provides the ability to produce custom metrics by copying, calculating new, or aggregating other metric values without requiring an additional processor.
-The rule language is expressed in yaml mappings and is [documented here](./internal/translation/translator.go).  Translation rules currently allow the following actions:
-
-* `aggregate_metric` - Aggregates a metric through removal of specified dimensions
-* `calculate_new_metric` - Creates a new metric via operating on two constituent ones
-* `convert_values` - Convert float values to int or int to float for specified metric names
-* `copy_metrics` - Creates a new metric as a copy of another
-* `delta_metric` - Creates a new delta metric for a specified non-delta one
-* `divide_int` - Scales a metric's integer value by a given factor
-* `drop_dimensions` - Drops dimensions for specified metrics, or globally
-* `drop_metrics` - Drops all metrics with a given name
-* `multiply_float` - Scales a metric's float value by a given float factor
-* `multiply_int` - Scales a metric's int value by a given int factor
-* `rename_dimension_keys` - Renames dimensions for specified metrics, or globally
-* `rename_metrics` - Replaces a given metric name with specified one
-* `split_metric` - Splits a given metric into multiple new ones for a specified dimension
-
-The translation rules defined in [`translation/constants.go`](./internal/translation/constants.go) are used by default for this value.  The default rules will create the following aggregated metrics from the [`hostmetrics` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md):
+The default translation rules defined in [`translation/constants.go`](./internal/translation/constants.go) are used by the SignalFx exporter
+to help ensure compatibility with custom charts and dashboards when using the OpenTelemetry Collector.
+The default rules will create the following aggregated metrics from the [`hostmetrics` receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/README.md):
 
 * cpu.idle
 * cpu.interrupt
@@ -264,7 +248,7 @@ The CPU number is assigned to the dimension `cpu`
 * cpu.user
 * cpu.wait
 
-These metrics are intended to be reported directly to Splunk IM by the SignalFx exporter.  Any desired changes to their attributes or values should be made via additional translation rules or from their constituent host metrics.
+These metrics are intended to be reported directly to Splunk IM by the SignalFx exporter.  Any desired changes to their attributes or values should be made via their constituent host metrics.
 
 ## Example Config
 

exporter/signalfxexporter/config.go

@@ -15,7 +15,6 @@ import (
 	"go.opentelemetry.io/collector/config/configtls"
 	"go.opentelemetry.io/collector/confmap"
 	"go.opentelemetry.io/collector/exporter/exporterhelper"
-	"go.uber.org/zap"
 	"gopkg.in/yaml.v3"
 
 	"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/correlation"
@@ -24,17 +23,27 @@ import (
 	"github.com/open-telemetry/opentelemetry-collector-contrib/internal/splunk"
 )
 
-const (
-	translationRulesConfigKey = "translation_rules"
-)
+// This mimics the original translation_rules config option (now deleted), but is not reachable
+// by user configuration. It's only used by defaultTranslationRules to parse the
+// YAML into a []translation.Rule object.
+type translationRulesConfig struct {
+	TranslationRules []translation.Rule `mapstructure:"translation_rules"`
+}
 
 var defaultTranslationRules = func() []translation.Rule {
-	cfg, err := loadConfig([]byte(translation.DefaultTranslationRulesYaml))
+	var data map[string]any
+	var defaultRules translationRulesConfig
+
 	// It is safe to panic since this is deterministic, and will not fail anywhere else if it doesn't fail all the time.
-	if err != nil {
+	if err := yaml.Unmarshal([]byte(translation.DefaultTranslationRulesYaml), &data); err != nil {
 		panic(err)
 	}
-	return cfg.TranslationRules
+
+	if err := confmap.NewFromStringMap(data).Unmarshal(&defaultRules); err != nil {
+		panic(fmt.Errorf("failed to load default translation rules: %w", err))
+	}
+
+	return defaultRules.TranslationRules
 }()
 
 var defaultExcludeMetrics = func() []dpfilters.MetricFilter {
@@ -89,11 +98,6 @@ type Config struct {
 
 	splunk.AccessTokenPassthroughConfig `mapstructure:",squash"`
 
-	// TranslationRules defines a set of rules how to translate metrics to a SignalFx compatible format
-	// Rules defined in translation/constants.go are used by default.
-	// Deprecated: Use metricstransform processor to do metrics transformations.
-	TranslationRules []translation.Rule `mapstructure:"translation_rules"`
-
 	DisableDefaultTranslationRules bool `mapstructure:"disable_default_translation_rules"`
 
 	// DeltaTranslationTTL specifies in seconds the max duration to keep the most recent datapoint for any
@@ -150,25 +154,17 @@ type DimensionClientConfig struct {
 	Timeout             time.Duration `mapstructure:"timeout"`
 }
 
-func (cfg *Config) getMetricTranslator(logger *zap.Logger, done chan struct{}) (*translation.MetricTranslator, error) {
-	rules := defaultTranslationRules
-	if cfg.TranslationRules != nil {
-		// Previous way to disable default translation rules.
-		if len(cfg.TranslationRules) == 0 {
-			logger.Warn("You are using the deprecated `translation_rules` option that will be removed soon; Use `disable_default_translation_rules` to disable the default rules in a gateway mode.")
-			rules = []translation.Rule{}
-		} else {
-			logger.Warn("You are using the deprecated `translation_rules` option that will be removed soon; Use metricstransform processor instead.")
-			rules = cfg.TranslationRules
-		}
-	}
+func (cfg *Config) getMetricTranslator(done chan struct{}) (*translation.MetricTranslator, error) {
+	var rules []translation.Rule
 	// The new way to disable default translation rules. This override any setting of the default TranslationRules.
 	if cfg.DisableDefaultTranslationRules {
 		rules = []translation.Rule{}
+	} else {
+		rules = defaultTranslationRules
 	}
 	metricTranslator, err := translation.NewMetricTranslator(rules, cfg.DeltaTranslationTTL, done)
 	if err != nil {
-		return nil, fmt.Errorf("invalid \"%s\": %w", translationRulesConfigKey, err)
+		return nil, fmt.Errorf("invalid default translation rules: %w", err)
 	}
 
 	return metricTranslator, nil

exporter/signalfxexporter/config_test.go

@@ -21,7 +21,6 @@ import (
 	"go.opentelemetry.io/collector/confmap/confmaptest"
 	"go.opentelemetry.io/collector/confmap/xconfmap"
 	"go.opentelemetry.io/collector/exporter/exporterhelper"
-	"go.uber.org/zap"
 
 	apmcorrelation "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/apm/correlations"
 	"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/signalfxexporter/internal/correlation"
@@ -86,7 +85,6 @@ func TestLoadConfig(t *testing.T) {
 					IdleConnTimeout:     30 * time.Second,
 					Timeout:             10 * time.Second,
 				},
-				TranslationRules:    nil,
 				ExcludeMetrics:      nil,
 				IncludeMetrics:      nil,
 				DeltaTranslationTTL: 3600,
@@ -162,40 +160,6 @@ func TestLoadConfig(t *testing.T) {
 					IdleConnTimeout:     2 * time.Hour,
 					Timeout:             20 * time.Second,
 				},
-				TranslationRules: []translation.Rule{
-					{
-						Action: translation.ActionRenameDimensionKeys,
-						Mapping: map[string]string{
-							"k8s.cluster.name": "kubernetes_cluster",
-						},
-					},
-					{
-						Action: translation.ActionDropDimensions,
-						DimensionPairs: map[string]map[string]bool{
-							"foo":  nil,
-							"foo1": {"bar": true},
-						},
-					},
-					{
-						Action:     translation.ActionDropDimensions,
-						MetricName: "metric",
-						DimensionPairs: map[string]map[string]bool{
-							"foo":  nil,
-							"foo1": {"bar": true},
-						},
-					},
-					{
-						Action: translation.ActionDropDimensions,
-						MetricNames: map[string]bool{
-							"metric1": true,
-							"metric2": true,
-						},
-						DimensionPairs: map[string]map[string]bool{
-							"foo":  nil,
-							"foo1": {"bar": true},
-						},
-					},
-				},
 				ExcludeMetrics: []dpfilters.MetricFilter{
 					{
 						MetricName: "metric1",
@@ -313,7 +277,7 @@ func TestConfigGetMetricTranslator(t *testing.T) {
 		wantErr bool
 	}{
 		{
-			name: "Test empty config",
+			name: "Test default translation rules",
 			cfg: &Config{
 				DeltaTranslationTTL: 3600,
 			},
@@ -323,18 +287,6 @@ func TestConfigGetMetricTranslator(t *testing.T) {
 				return translator
 			}(),
 		},
-		{
-			name: "Test empty rules",
-			cfg: &Config{
-				TranslationRules:    []translation.Rule{},
-				DeltaTranslationTTL: 3600,
-			},
-			want: func() *translation.MetricTranslator {
-				translator, err := translation.NewMetricTranslator([]translation.Rule{}, 3600, done)
-				require.NoError(t, err)
-				return translator
-			}(),
-		},
 		{
 			name: "Test disable rules",
 			cfg: &Config{
@@ -347,37 +299,10 @@ func TestConfigGetMetricTranslator(t *testing.T) {
 				return translator
 			}(),
 		},
-		{
-			name: "Test disable rules overrides rules",
-			cfg: &Config{
-				TranslationRules:               []translation.Rule{{Action: translation.ActionDropDimensions}},
-				DisableDefaultTranslationRules: true,
-				DeltaTranslationTTL:            3600,
-			},
-			want: func() *translation.MetricTranslator {
-				translator, err := translation.NewMetricTranslator([]translation.Rule{}, 3600, done)
-				require.NoError(t, err)
-				return translator
-			}(),
-		},
-		{
-			name: "Test invalid translation rules",
-			cfg: &Config{
-				Realm:       "us0",
-				AccessToken: "access_token",
-				TranslationRules: []translation.Rule{
-					{
-						Action: translation.ActionRenameDimensionKeys,
-					},
-				},
-				DeltaTranslationTTL: 3600,
-			},
-			wantErr: true,
-		},
 	}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
-			got, err := tt.cfg.getMetricTranslator(zap.NewNop(), done)
+			got, err := tt.cfg.getMetricTranslator(done)
 			if tt.wantErr {
 				assert.Error(t, err)
 				return

exporter/signalfxexporter/docs/translation_rules_migration_guide.md

@@ -0,0 +1,28 @@
+# Translation Rules Migration Guide
+
+## Context
+
+The `translation_rules` configuration option of the SignalFx exporter
+[has been deprecated](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/18218)
+and [removed](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/35332) in an effort to
+reduce the exporter's complexity and leverage existing functionality in existing
+OpenTelemetry Collector processors.
+
+## Migration Guide
+
+The following table can be referenced to map existing translation rule actions to processors that
+can be used to accomplish the same functionality.
+
+| Deleted Translation rule | Replacement option | Replacement example |
+  | -----------------|--------------------|----------------------|
+| aggregate_metric | `transform` processor's `aggregate_on_attributes` function with the `metric` context | [aggregate example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/transformprocessor#aggregate_on_attributes) |
+| calculate_new_metric | `metricsgeneration` processor's `calculate` functionality | [calculate example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricsgenerationprocessor#example-configurations) |
+| convert_values | `transform` processor's `Double` or `Int` converter on a `datapoint` context | [`Double` example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#double), [`Int` example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#int) |
+| copy_metrics | `metricstransform` processor's `insert` functionality | [copy all datapoints](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#create-a-new-metric-from-an-existing-metric), [conditionally copy datapoints](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#create-a-new-metric-from-an-existing-metric-with-matching-label-values) |
+| delta_metric | `cumulativetodelta` processor. To preserve original metrics, first copy the original metric, then use the copied metric in the `cumulativetodelta` processor | [specify which metrics to convert example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/cumulativetodeltaprocessor#examples)
+| drop_dimensions | `transform` processor's `delete_keys` function with the `datapoint` context | [simple example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl/ottlfuncs#delete_key), use a `where` clause with the given example to filter based upon the metric name or dimension value |
+| drop_metrics | `filter` processor | [drop by name and value example](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/filterprocessor#dropping-specific-metric-and-value) |
+| multiply_int, divide_int, multiply_float | `metricstransform` processor's `scale` value functionality | [one metric](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#scale-value) |
+| rename_dimension_keys | `metricstransform` processor's update label function | [one metric](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#rename-labels), [multiple metrics](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#rename-labels-for-multiple-metrics) |
+| rename_metrics | `metricstransform` processor's rename metric functionality | [one metric](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#rename-metric), [multiple metrics](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#rename-multiple-metrics-using-substitution) |
+| split_metric | `metricstransform` processor's `insert` functionality and `filter` processor | Refer to the replacement guidance for the `copy_metrics` and `drop_metrics` translation rules |

exporter/signalfxexporter/exporter.go

@@ -69,7 +69,7 @@ func newSignalFxExporter(
 		return nil, errors.New("nil config")
 	}
 
-	metricTranslator, err := config.getMetricTranslator(createSettings.TelemetrySettings.Logger, make(chan struct{}))
+	metricTranslator, err := config.getMetricTranslator(make(chan struct{}))
 	if err != nil {
 		return nil, err
 	}

exporter/signalfxexporter/testdata/config.yaml

@@ -20,29 +20,6 @@ signalfx/allsettings:
     added-entry: "added value"
     dot.test: test
   access_token_passthrough: false
-  translation_rules:
-  - action: rename_dimension_keys
-    mapping:
-      k8s.cluster.name: kubernetes_cluster
-  - action: drop_dimensions
-    dimension_pairs:
-      foo:
-      foo1:
-        bar: true
-  - action: drop_dimensions
-    metric_name: metric
-    dimension_pairs:
-      foo:
-      foo1:
-        bar: true
-  - action: drop_dimensions
-    metric_names:
-      metric1: true
-      metric2: true
-    dimension_pairs:
-      foo:
-      foo1:
-        bar: true
   exclude_metrics:
     - metric_name: metric1
     - metric_names: [metric2, metric3]