[receiver/azuremonitorreceiver] feat: Allow to not split result by dimension (#37168)

Recreated from a diffferent fork. Context:
#36240

> <!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
> Ex. Adding a feature - Explain what this achieves.-->
> #### Description
> 
> Currently there is a mechanism allowing to split the result by
dimension, thanks to a filter param hack.
> This is great that all the dimensions are collected as labels in the
metrics, but for some resources types it could be unwanted. In cause
some concern about cardinality, or continuity of the queries between one
version to another (e.g with prometheus exporter, if one does not do a
"sum by ..." and the otelcol version is updated, the query can display
different results)
> 
> To mitigate that I propose to put an optout for this collection so if
we want for some resource types and not for others, we can create two
separate receivers for example. Or we completely opt out if we don't
want additional labels.
> 
> ### Edit: 
> After a first review, we agreed on the fact that it could do a bit
more like allowing us to specify a list of dimensions for a particular
metric.
> e.g from added documentation
> ```yaml
> receivers:
>   azuremonitor:
>     dimensions:
>       enabled: true
>       overrides:
>         "Microsoft.Network/azureFirewalls":
>           # Real example of an Azure limitation here:
>           # Dimensions exposed are Reason, Status, Protocol,
> # but when selecting Protocol in the filters, it returns nothing.
> # Note here that the metric display name is ``Network rules hit
count`` but it's programmatic value is ``NetworkRuleHit``
> # Ref:
https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-network-azurefirewalls-metrics
>           "NetworkRuleHit": [Reason, Status]
> ```
> Without this config you won't have the
``azure_networkrulehit_average_Count`` metric at all in your results
> But with the config we have it with reason and status labels:
> 
> ```# HELP azure_networkrulehit_average_Count``` 
> ```# TYPE azure_networkrulehit_average_Count gauge```
>
```azure_networkrulehit_average_Count{azuremonitor_resource_id="/subscriptions/<redacted>/resourceGroups/<redacted>/providers/Microsoft.Network/azureFirewalls/<redacted>",location="<redacted>",metadata_reason="<redacted>",metadata_status="Allow",name="<redacted>",resource_group="<redacted>",type="Microsoft.Network/azureFirewalls"}
21.875```
>
```azure_networkrulehit_average_Count{azuremonitor_resource_id="/subscriptions/<redacted>/resourceGroups/<redacted>/providers/Microsoft.Network/azureFirewalls/<redacted>",location="<redacted>",metadata_reason="<redacted>",metadata_status="Deny",name="<redacted>",resource_group="<redacted>",type="Microsoft.Network/azureFirewalls"}
10```
>  
> <!-- Issue number (e.g. #1234) or full URL to issue, if applicable.
-->
> #### Link to tracking issue
> Fixes #36611 
> 
> <!--Describe what testing was performed and which tests were added.-->
> #### Testing
> 
> <!--Describe the documentation added.-->
> #### Documentation
> 
> <!--Please delete paragraphs that you did not use before
submitting.-->

Signed-off-by: Célian Garcia <[email protected]>

Author: celian-garcia

.chloggen/split-dimensions-optout.yaml

@@ -0,0 +1,27 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: receiver/azuremonitorreceiver
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: "Add dimensions.enabled and dimensions.overrides which allows to opt out from automatically split by all the dimensions of the resource type"
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [36611]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext:
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: []

receiver/azuremonitorreceiver/README.md

@@ -31,6 +31,8 @@ The following settings are optional:
 - `maximum_number_of_records_per_resource` (default = 10): Maximum number of records to fetch per resource.
 - `initial_delay` (default = `1s`): defines how long this receiver waits before starting.
 - `cloud` (default = `AzureCloud`): defines which Azure cloud to use. Valid values: `AzureCloud`, `AzureUSGovernment`, `AzureChinaCloud`.
+- `dimensions.enabled` (default = `true`): allows to opt out from automatically split by all the dimensions of the resource type.
+- `dimensions.overrides` (default = `{}`): if dimensions are enabled, it allows you to specify a set of dimensions for a particular metric. This is a two levels map with first key being the resource type and second key being the metric name. Programmatic value should be used for metric name https://learn.microsoft.com/en-us/azure/azure-monitor/reference/metrics-index
 
 Authenticating using service principal requires following additional settings:
 
@@ -101,6 +103,22 @@ receivers:
     auth: "default_credentials"
 ```
 
+Overriding dimensions for a particular metric:
+```yaml
+receivers:
+  azuremonitor:
+    dimensions:
+      enabled: true
+      overrides:
+        "Microsoft.Network/azureFirewalls":
+          # Real example of an Azure limitation here:
+          # Dimensions exposed are Reason, Status, Protocol,
+          # but when selecting Protocol in the filters, it returns nothing.
+          # Note here that the metric display name is ``Network rules hit count`` but it's programmatic value is ``NetworkRuleHit``
+          # Ref: https://learn.microsoft.com/en-us/azure/azure-monitor/reference/supported-metrics/microsoft-network-azurefirewalls-metrics
+          "NetworkRuleHit": [Reason, Status]
+```
+
 
 ## Metrics
 

receiver/azuremonitorreceiver/config.go

@@ -228,6 +228,11 @@ var (
 	}
 )
 
+type DimensionsConfig struct {
+	Enabled   *bool                          `mapstructure:"enabled"`
+	Overrides map[string]map[string][]string `mapstructure:"overrides"`
+}
+
 // Config defines the configuration for the various elements of the receiver agent.
 type Config struct {
 	scraperhelper.ControllerConfig    `mapstructure:",squash"`
@@ -246,6 +251,7 @@ type Config struct {
 	MaximumNumberOfMetricsInACall     int                           `mapstructure:"maximum_number_of_metrics_in_a_call"`
 	MaximumNumberOfRecordsPerResource int32                         `mapstructure:"maximum_number_of_records_per_resource"`
 	AppendTagsAsAttributes            bool                          `mapstructure:"append_tags_as_attributes"`
+	Dimensions                        DimensionsConfig              `mapstructure:"dimensions"`
 }
 
 const (

receiver/azuremonitorreceiver/dimension_test.go

@@ -0,0 +1,171 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package azuremonitorreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azuremonitorreceiver"
+
+import (
+	"testing"
+
+	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
+	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/monitor/armmonitor"
+	"github.com/stretchr/testify/require"
+)
+
+func newDimension(value string) *armmonitor.LocalizableString {
+	return to.Ptr(armmonitor.LocalizableString{Value: to.Ptr(value)})
+}
+
+func TestFilterDimensions(t *testing.T) {
+	type args struct {
+		dimensions   []*armmonitor.LocalizableString
+		cfg          DimensionsConfig
+		resourceType string
+		metricName   string
+	}
+
+	tests := []struct {
+		name     string
+		args     args
+		expected []string
+	}{
+		{
+			name: "always empty if dimensions disabled",
+			args: args{
+				dimensions: []*armmonitor.LocalizableString{
+					newDimension("foo"),
+					newDimension("bar"),
+				},
+				cfg: DimensionsConfig{
+					Enabled: to.Ptr(false),
+				},
+				resourceType: "rt1",
+				metricName:   "m1",
+			},
+			expected: nil,
+		},
+		{
+			name: "split by dimensions should be enabled by default",
+			args: args{
+				dimensions: []*armmonitor.LocalizableString{
+					newDimension("foo"),
+					newDimension("bar"),
+				},
+				cfg:          DimensionsConfig{}, // enabled by default
+				resourceType: "rt1",
+				metricName:   "m1",
+			},
+			expected: []string{"foo", "bar"},
+		},
+		{
+			name: "overrides takes precedence over input",
+			args: args{
+				dimensions: []*armmonitor.LocalizableString{
+					newDimension("foo"),
+					newDimension("bar"),
+				},
+				cfg: DimensionsConfig{
+					Enabled: to.Ptr(true),
+					Overrides: map[string]map[string][]string{
+						"rt1": {
+							"m1": {
+								"foo",
+							},
+						},
+					},
+				},
+				resourceType: "rt1",
+				metricName:   "m1",
+			},
+			expected: []string{"foo"},
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			actual := filterDimensions(tt.args.dimensions, tt.args.cfg, tt.args.resourceType, tt.args.metricName)
+			require.Equal(t, tt.expected, actual)
+		})
+	}
+}
+
+func TestBuildDimensionsFilter(t *testing.T) {
+	type args struct {
+		dimensionsStr string
+	}
+
+	tests := []struct {
+		name     string
+		args     args
+		expected *string
+	}{
+		{
+			name: "empty given dimensions string",
+			args: args{
+				dimensionsStr: "",
+			},
+			expected: nil,
+		},
+		{
+			name: "build dimensions filter",
+			args: args{
+				dimensionsStr: "bar,foo",
+			},
+			expected: to.Ptr("bar eq '*' and foo eq '*'"),
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			actual := buildDimensionsFilter(tt.args.dimensionsStr)
+			require.EqualValues(t, tt.expected, actual)
+		})
+	}
+}
+
+func TestSerializeDimensions(t *testing.T) {
+	type args struct {
+		dimensions []string
+	}
+
+	tests := []struct {
+		name     string
+		args     args
+		expected string
+	}{
+		{
+			name: "empty given dimensions",
+			args: args{
+				dimensions: []string{},
+			},
+			expected: "",
+		},
+		{
+			name: "nil given dimensions",
+			args: args{
+				dimensions: []string{},
+			},
+			expected: "",
+		},
+		{
+			name: "reorder dimensions",
+			args: args{
+				dimensions: []string{"foo", "bar"},
+			},
+			expected: "bar,foo",
+		},
+		{
+			name: "trim spaces dimensions",
+			args: args{
+				dimensions: []string{"  bar", "foo "},
+			},
+			expected: "bar,foo",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			actual := serializeDimensions(tt.args.dimensions)
+			require.EqualValues(t, tt.expected, actual)
+		})
+	}
+}

receiver/azuremonitorreceiver/dimensions.go

@@ -0,0 +1,66 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package azuremonitorreceiver // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azuremonitorreceiver"
+
+import (
+	"bytes"
+	"sort"
+	"strings"
+
+	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/monitor/armmonitor"
+)
+
+// filterDimensions transforms a list of azure dimensions into a list of string, taking in account the DimensionConfig
+// given by the user.
+func filterDimensions(dimensions []*armmonitor.LocalizableString, cfg DimensionsConfig, resourceType, metricName string) []string {
+	// Only skip if explicitly disabled. Enabled by default.
+	if cfg.Enabled != nil && !*cfg.Enabled {
+		return nil
+	}
+
+	// If dimensions are overridden for that resource type and metric name, we take it
+	if _, resourceTypeFound := cfg.Overrides[resourceType]; resourceTypeFound {
+		if newDimensions, metricNameFound := cfg.Overrides[resourceType][metricName]; metricNameFound {
+			return newDimensions
+		}
+	}
+	// Otherwise we get all dimensions
+	var result []string
+	for _, dimension := range dimensions {
+		result = append(result, *dimension.Value)
+	}
+	return result
+}
+
+// serializeDimensions build a comma separated string from trimmed, sorted dimensions list.
+// It is designed to be used as a key in scraper maps.
+func serializeDimensions(dimensions []string) string {
+	var dimensionsSlice []string
+	for _, dimension := range dimensions {
+		if trimmedDimension := strings.TrimSpace(dimension); len(trimmedDimension) > 0 {
+			dimensionsSlice = append(dimensionsSlice, trimmedDimension)
+		}
+	}
+	sort.Strings(dimensionsSlice)
+	return strings.Join(dimensionsSlice, ",")
+}
+
+// buildDimensionsFilter takes a serialized dimensions input to build an Azure Request filter that will allow us to
+// receive metrics values split by these dimensions.
+func buildDimensionsFilter(dimensionsStr string) *string {
+	if len(dimensionsStr) == 0 {
+		return nil
+	}
+	var dimensionsFilter bytes.Buffer
+	dimensions := strings.Split(dimensionsStr, ",")
+	for i, dimension := range dimensions {
+		dimensionsFilter.WriteString(dimension)
+		dimensionsFilter.WriteString(" eq '*'")
+		if i < len(dimensions)-1 {
+			dimensionsFilter.WriteString(" and ")
+		}
+	}
+	result := dimensionsFilter.String()
+	return &result
+}