[exporter/azuremonitor] Add Connection String Support to Azure Monitor Exporter (#28854)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This pull request introduces the ability to configure the Azure Monitor
Exporter using a connection string, aligning the exporter configuration
with Azure Monitor's recommended practices. The current implementation
requires users to set the instrumentation key directly, which will soon
be deprecated in favor of using the connection string.

**Changes Made:**

1. Configuration Update: Modified the `Config` struct and related
configuration parsing logic to support a `ConnectionString` field.
2. Parsing Logic: Implemented functionality to parse the connection
string and extract necessary details, such as `InstrumentationKey` and
`IngestionEndpoint`.
3. Updated Tests: Revised existing tests and added new ones to ensure
coverage of the new configuration option.

**Benefits:**

* Streamlines the configuration process for end-users.
* Aligns with Azure Monitor's best practices and recommended
configuration approach.
* Paves the way for the upcoming deprecation of direct instrumentation
key configuration.

**Backwards Compatibility:**
This update maintains full backwards compatibility. Users currently
utilizing the instrumentation key for configuration can continue to do
so but are advised to transition to using the connection string.

**To-Do** 

* Documentation Update in a follow up PR
* Deprecation Notice: A future update will introduce a deprecation
warning for users still configuring the exporter with the
instrumentation key, encouraging them to switch to using a connection
string.
* Add support for `EndpointSuffix` in connection string -
https://learn.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=dotnet5#connection-string-with-an-endpoint-suffix

**Link to tracking Issue:** <Issue number if applicable> 
#28853

**Testing:** <Describe what testing was performed and which tests were
added.>

Conducted comprehensive testing, including unit tests, to validate that
the new configuration option works as expected and does not introduce
regressions. All tests are currently passing.

```
[Wed Nov  1 12:53:42 PDT 2023] --------- Transmitting 27 items ---------
[Wed Nov  1 12:53:43 PDT 2023] Telemetry transmitted in 331.926261ms
[Wed Nov  1 12:53:43 PDT 2023] Response: 200
[Wed Nov  1 12:53:43 PDT 2023] Items accepted/received: 27/27
[Wed Nov  1 12:53:53 PDT 2023] --------- Transmitting 30 items ---------
[Wed Nov  1 12:53:53 PDT 2023] Telemetry transmitted in 73.171392ms
[Wed Nov  1 12:53:53 PDT 2023] Response: 200
[Wed Nov  1 12:53:53 PDT 2023] Items accepted/received: 30/30
[Wed Nov  1 12:54:04 PDT 2023] --------- Transmitting 27 items ---------
[Wed Nov  1 12:54:04 PDT 2023] Telemetry transmitted in 68.037724ms
[Wed Nov  1 12:54:04 PDT 2023] Response: 200
[Wed Nov  1 12:54:04 PDT 2023] Items accepted/received: 27/27
```

**Documentation:** <Describe the documentation added.>

TODO, in a follow up PR.

Author: rajkumar-rangaraj

.chloggen/add-connection-string-azure-monitor-exporter.yaml

@@ -0,0 +1,33 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: azuremonitorexporter
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Added connection string support to the Azure Monitor Exporter 
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [28853]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: |
+  This enhancement simplifies the configuration process and aligns the exporter with Azure Monitor's recommended practices.
+  The Connection String method allows the inclusion of various fields such as the InstrumentationKey and IngestionEndpoint
+  within a single string, facilitating an easier and more integrated setup.
+  While the traditional InstrumentationKey method remains supported for backward compatibility, it will be phased out.
+  Users are encouraged to adopt the Connection String approach to ensure future compatibility and to leverage the broader
+  configuration options it enables.
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: [user]

cmd/otelcontribcol/exporters_test.go

@@ -17,6 +17,7 @@ import (
 	"github.com/stretchr/testify/require"
 	"go.opentelemetry.io/collector/component"
 	"go.opentelemetry.io/collector/config/configgrpc"
+	"go.opentelemetry.io/collector/config/configopaque"
 	"go.opentelemetry.io/collector/exporter"
 	"go.opentelemetry.io/collector/exporter/exportertest"
 	"go.opentelemetry.io/collector/exporter/otlpexporter"
@@ -298,6 +299,7 @@ func TestDefaultExporters(t *testing.T) {
 			getConfigFn: func() component.Config {
 				cfg := expFactories["azuremonitor"].CreateDefaultConfig().(*azuremonitorexporter.Config)
 				cfg.Endpoint = "http://" + endpoint
+				cfg.ConnectionString = configopaque.String("InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=" + cfg.Endpoint)
 
 				return cfg
 			},

cmd/otelcontribcol/go.mod

@@ -186,6 +186,7 @@ require (
 	github.com/stretchr/testify v1.8.4
 	go.opentelemetry.io/collector/component v0.88.1-0.20231026220224-6405e152a2d9
 	go.opentelemetry.io/collector/config/configgrpc v0.88.1-0.20231026220224-6405e152a2d9
+	go.opentelemetry.io/collector/config/configopaque v0.88.1-0.20231026220224-6405e152a2d9
 	go.opentelemetry.io/collector/confmap v0.88.1-0.20231026220224-6405e152a2d9
 	go.opentelemetry.io/collector/connector v0.88.1-0.20231026220224-6405e152a2d9
 	go.opentelemetry.io/collector/connector/forwardconnector v0.88.1-0.20231026220224-6405e152a2d9
@@ -637,7 +638,6 @@ require (
 	go.opentelemetry.io/collector/config/configcompression v0.88.1-0.20231026220224-6405e152a2d9 // indirect
 	go.opentelemetry.io/collector/config/confighttp v0.88.1-0.20231026220224-6405e152a2d9 // indirect
 	go.opentelemetry.io/collector/config/confignet v0.88.1-0.20231026220224-6405e152a2d9 // indirect
-	go.opentelemetry.io/collector/config/configopaque v0.88.1-0.20231026220224-6405e152a2d9 // indirect
 	go.opentelemetry.io/collector/config/configtelemetry v0.88.1-0.20231026220224-6405e152a2d9 // indirect
 	go.opentelemetry.io/collector/config/configtls v0.88.1-0.20231026220224-6405e152a2d9 // indirect
 	go.opentelemetry.io/collector/config/internal v0.88.1-0.20231026220224-6405e152a2d9 // indirect

exporter/azuremonitorexporter/README.md

@@ -17,20 +17,37 @@ This exporter sends logs, traces and metrics to [Azure Monitor](https://docs.mic
 
 ## Configuration
 
-The following settings are required:
+To configure the Azure Monitor Exporter, you must specify one of the following settings:
 
-- `instrumentation_key` (no default): Application Insights instrumentation key, which can be found in the Application Insights resource in the Azure Portal.
+- `connection_string` (recommended): The Azure Application Insights Connection String is required to send telemetry data to the monitoring service. It is the recommended method for configuring the exporter, aligning with Azure Monitor's best practices. If you need guidance on creating Azure resources, please refer to the step-by-step guides to [Create an Application Insights resource](https://docs.microsoft.com/azure/azure-monitor/app/create-new-resource) and [find your connection string](https://docs.microsoft.com/azure/azure-monitor/app/sdk-connection-string?tabs=net#find-your-connection-string).
+- `instrumentation_key`: Application Insights instrumentation key, which can be found in the Application Insights resource in the Azure Portal. While it is currently supported, its use is discouraged and it is slated for deprecation. It is highly encouraged to use the `connection_string` setting for new configurations and migrate existing configurations to use the `connection_string` as soon as possible.
+
+**Important**: Only one of `connection_string` or `instrumentation_key` should be specified in your configuration. If both are provided, `connection_string` will be used as the priority setting.
 
 The following settings can be optionally configured:
 
-- `endpoint` (default = `https://dc.services.visualstudio.com/v2/track`): The endpoint URL where data will be submitted.
+- `endpoint` (default = `https://dc.services.visualstudio.com/v2/track`): The endpoint URL where data will be submitted. While this option remains available, it is important to note that the use of the `connection_string` is recommended, as it encompasses the endpoint information. The direct configuration of the `endpoint` is considered to be on a deprecation path.
 - `maxbatchsize` (default = 1024): The maximum number of telemetry items that can be submitted in each request. If this many items are buffered, the buffer will be flushed before `maxbatchinterval` expires.
 - `maxbatchinterval` (default = 10s): The maximum time to wait before sending a batch of telemetry.
 - `spaneventsenabled` (default = false): Enables export of span events.
 
 Example:
 
 ```yaml
+# Recommended Configuration:
+# It is highly recommended to use the connection string which includes the InstrumentationKey and IngestionEndpoint
+# This is the preferred method over using 'instrumentation_key' alone.
+exporters:
+  azuremonitor:
+    connection_string: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/"
+```
+
+(or)
+
+```yaml
+# Legacy Configuration:
+# The use of 'instrumentation_key' alone is not recommended and will be deprecated in the future. It is advised to use the connection_string instead.
+# This example is provided primarily for existing configurations that have not yet transitioned to the connection string.
 exporters:
   azuremonitor:
     instrumentation_key: b1cd0778-85fc-4677-a3fa-79d3c23e0efd

exporter/azuremonitorexporter/config.go

@@ -12,6 +12,7 @@ import (
 // Config defines configuration for Azure Monitor
 type Config struct {
 	Endpoint           string              `mapstructure:"endpoint"`
+	ConnectionString   configopaque.String `mapstructure:"connection_string"`
 	InstrumentationKey configopaque.String `mapstructure:"instrumentation_key"`
 	MaxBatchSize       int                 `mapstructure:"maxbatchsize"`
 	MaxBatchInterval   time.Duration       `mapstructure:"maxbatchinterval"`

exporter/azuremonitorexporter/config_test.go

@@ -35,7 +35,8 @@ func TestLoadConfig(t *testing.T) {
 			id: component.NewIDWithName(metadata.Type, "2"),
 			expected: &Config{
 				Endpoint:           defaultEndpoint,
-				InstrumentationKey: "abcdefg",
+				ConnectionString:   "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/",
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
 				MaxBatchSize:       100,
 				MaxBatchInterval:   10 * time.Second,
 				SpanEventsEnabled:  false,

exporter/azuremonitorexporter/connection_string_parser.go

@@ -0,0 +1,80 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package azuremonitorexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuremonitorexporter"
+
+import (
+	"fmt"
+	"net/url"
+	"path"
+	"strings"
+)
+
+type ConnectionVars struct {
+	InstrumentationKey string
+	IngestionURL       string
+}
+
+const (
+	DefaultIngestionEndpoint  = "https://dc.services.visualstudio.com/"
+	IngestionEndpointKey      = "IngestionEndpoint"
+	InstrumentationKey        = "InstrumentationKey"
+	ConnectionStringMaxLength = 4096
+)
+
+func parseConnectionString(exporterConfig *Config) (*ConnectionVars, error) {
+	connectionString := string(exporterConfig.ConnectionString)
+	instrumentationKey := string(exporterConfig.InstrumentationKey)
+	connectionVars := &ConnectionVars{}
+
+	if connectionString == "" && instrumentationKey == "" {
+		return nil, fmt.Errorf("ConnectionString and InstrumentationKey cannot be empty")
+	}
+	if len(connectionString) > ConnectionStringMaxLength {
+		return nil, fmt.Errorf("ConnectionString exceeds maximum length of %d characters", ConnectionStringMaxLength)
+	}
+	if connectionString == "" {
+		connectionVars.InstrumentationKey = instrumentationKey
+		connectionVars.IngestionURL = getIngestionURL(DefaultIngestionEndpoint)
+		return connectionVars, nil
+	}
+
+	pairs := strings.Split(connectionString, ";")
+	values := make(map[string]string)
+	for _, pair := range pairs {
+		kv := strings.SplitN(strings.TrimSpace(pair), "=", 2)
+		if len(kv) != 2 {
+			return nil, fmt.Errorf("invalid format for connection string: %s", pair)
+		}
+
+		key, value := strings.TrimSpace(kv[0]), strings.TrimSpace(kv[1])
+		if key == "" {
+			return nil, fmt.Errorf("key cannot be empty")
+		}
+		values[key] = value
+	}
+
+	var ok bool
+	if connectionVars.InstrumentationKey, ok = values[InstrumentationKey]; !ok || connectionVars.InstrumentationKey == "" {
+		return nil, fmt.Errorf("%s is required", InstrumentationKey)
+	}
+
+	var ingestionEndpoint string
+	if ingestionEndpoint, ok = values[IngestionEndpointKey]; !ok || ingestionEndpoint == "" {
+		ingestionEndpoint = DefaultIngestionEndpoint
+	}
+
+	connectionVars.IngestionURL = getIngestionURL(ingestionEndpoint)
+
+	return connectionVars, nil
+}
+
+func getIngestionURL(ingestionEndpoint string) string {
+	ingestionURL, err := url.Parse(ingestionEndpoint)
+	if err != nil {
+		ingestionURL, _ = url.Parse(DefaultIngestionEndpoint)
+	}
+
+	ingestionURL.Path = path.Join(ingestionURL.Path, "/v2/track")
+	return ingestionURL.String()
+}

exporter/azuremonitorexporter/connection_string_parser_test.go

@@ -0,0 +1,134 @@
+// Copyright The OpenTelemetry Authors
+// SPDX-License-Identifier: Apache-2.0
+
+package azuremonitorexporter
+
+import (
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"go.opentelemetry.io/collector/config/configopaque"
+)
+
+func TestParseConnectionString(t *testing.T) {
+	tests := []struct {
+		name      string
+		config    *Config
+		want      *ConnectionVars
+		wantError bool
+	}{
+		{
+			name: "Valid connection string and instrumentation key",
+			config: &Config{
+				ConnectionString:   "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/",
+				InstrumentationKey: "00000000-0000-0000-0000-00000000IKEY",
+			},
+			want: &ConnectionVars{
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
+				IngestionURL:       "https://ingestion.azuremonitor.com/v2/track",
+			},
+			wantError: false,
+		},
+		{
+			name: "Empty connection string with valid instrumentation key",
+			config: &Config{
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
+			},
+			want: &ConnectionVars{
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
+				IngestionURL:       DefaultIngestionEndpoint + "v2/track",
+			},
+			wantError: false,
+		},
+		{
+			name: "Valid connection string with empty instrumentation key",
+			config: &Config{
+				ConnectionString: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/",
+			},
+			want: &ConnectionVars{
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
+				IngestionURL:       "https://ingestion.azuremonitor.com/v2/track",
+			},
+			wantError: false,
+		},
+		{
+			name: "Empty connection string and instrumentation key",
+			config: &Config{
+				ConnectionString:   "",
+				InstrumentationKey: "",
+			},
+			want:      nil,
+			wantError: true,
+		},
+		{
+			name: "Invalid connection string format",
+			config: &Config{
+				ConnectionString: "InvalidConnectionString",
+			},
+			want:      nil,
+			wantError: true,
+		},
+		{
+			name: "Missing InstrumentationKey in connection string",
+			config: &Config{
+				ConnectionString: "IngestionEndpoint=https://ingestion.azuremonitor.com/",
+			},
+			want:      nil,
+			wantError: true,
+		},
+		{
+			name: "Empty InstrumentationKey in connection string",
+			config: &Config{
+				ConnectionString: "InstrumentationKey=;IngestionEndpoint=https://ingestion.azuremonitor.com/",
+			},
+			want:      nil,
+			wantError: true,
+		},
+		{
+			name: "Extra parameters in connection string",
+			config: &Config{
+				ConnectionString: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/;ExtraParam=extra",
+			},
+			want: &ConnectionVars{
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
+				IngestionURL:       "https://ingestion.azuremonitor.com/v2/track",
+			},
+			wantError: false,
+		},
+		{
+			name: "Spaces around equals in connection string",
+			config: &Config{
+				ConnectionString: "InstrumentationKey = 00000000-0000-0000-0000-000000000000 ; IngestionEndpoint = https://ingestion.azuremonitor.com/",
+			},
+			want: &ConnectionVars{
+				InstrumentationKey: "00000000-0000-0000-0000-000000000000",
+				IngestionURL:       "https://ingestion.azuremonitor.com/v2/track",
+			},
+			wantError: false,
+		},
+		{
+			name: "Connection string too long",
+			config: &Config{
+				ConnectionString: configopaque.String(strings.Repeat("a", ConnectionStringMaxLength+1)),
+			},
+			want:      nil,
+			wantError: true,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			got, err := parseConnectionString(tt.config)
+			if tt.wantError {
+				require.Error(t, err, "Expected an error but got none")
+			} else {
+				require.NoError(t, err, "Unexpected error: %v", err)
+				require.NotNil(t, got, "Expected a non-nil result")
+				assert.Equal(t, tt.want.InstrumentationKey, got.InstrumentationKey, "InstrumentationKey does not match")
+				assert.Equal(t, tt.want.IngestionURL, got.IngestionURL, "IngestionEndpoint does not match")
+			}
+		})
+	}
+}