Skip to content

Commit 6d876f2

Browse files
ChrsMarkChrsMark
committed
[receiver/receiver_creator] Add support for metrics' hints
Signed-off-by: ChrsMark <[email protected]>
1 parent 6ce4397 commit 6d876f2

File tree

6 files changed

+565
-71
lines changed

6 files changed

+565
-71
lines changed

‎.chloggen/hints.yaml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
7+
component: receivercreator
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Add support for generating metrics receivers based on provided annotations' hints
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [34427]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# If your change doesn't affect end users or the exported elements of any package,
21+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
22+
# Optional: The change log or logs in which this entry should be included.
23+
# e.g. '[user]' or '[user, api]'
24+
# Include 'user' if the change is relevant to end users.
25+
# Include 'api' if there is a change to a library API.
26+
# Default: '[user]'
27+
change_logs: []

‎receiver/receivercreator/README.md

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -437,3 +437,143 @@ service:
437437
438438
The full list of settings exposed for this receiver are documented [here](./config.go)
439439
with detailed sample configurations [here](./testdata/config.yaml).
440+
441+
442+
## Generate receiver configurations from provided Hints
443+
444+
Currently this feature is only supported for K8s environments and the `k8sobserver`.
445+
446+
The feature for K8s is enabled with the following setting:
447+
448+
```yaml
449+
receiver_creator/metrics:
450+
watch_observers: [ k8s_observer ]
451+
hints:
452+
k8s:
453+
metrics:
454+
enabled: true
455+
```
456+
457+
Users can use the following annotations to automatically enable receivers to start collecting metrics from the target Pods/containers.
458+
459+
### Supported metrics annotations
460+
1. `io.opentelemetry.collector.receiver-creator.metrics/receiver` (example: `nginx`)
461+
2. `io.opentelemetry.collector.receiver-creator.metrics/endpoint` (example: ```"http://`endpoint`/nginx_status"```, if not provided it defaults to `endpoint` which is of form `pod_ip:container_port`.)
462+
3. `io.opentelemetry.collector.receiver-creator.metrics/collection_interval` (example: `20s`)
463+
4. `io.opentelemetry.collector.receiver-creator.metrics/timeout` (example: `1m`)
464+
5. `io.opentelemetry.collector.receiver-creator.metrics/username` (example: `admin`)
465+
6. `io.opentelemetry.collector.receiver-creator.metrics/password` (example: `passpass`)
466+
467+
468+
### Support multiple target containers
469+
470+
Users can target the annotation to a specific container by suffixing it with the name of the port that container exposes,
471+
for example ```io.opentelemetry.collector.receiver-creator.metrics/endpoint.webserver: "http://`endpoint`/nginx_status"```
472+
where `webserver` is the name of the port the target container exposes.
473+
474+
If a Pod is annotated with both container level hints and pod level hints the container level hints have priority and
475+
the Pod level hints are used as a fallback (see detailed example bellow).
476+
477+
The current implementation relies on the implementation of `k8sobserver` extension and specifically
478+
the [pod_endpoint](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.111.0/extension/observer/k8sobserver/pod_endpoint.go).
479+
The hints are evaluated per container by extracting the annotations from each `Port` endpoint that is emitted.
480+
481+
482+
### Examples
483+
484+
#### Metrics example
485+
486+
Collector's configuration:
487+
```yaml
488+
receivers:
489+
receiver_creator/metrics:
490+
watch_observers: [ k8s_observer ]
491+
hints:
492+
k8s:
493+
metrics:
494+
enabled: true
495+
receivers:
496+
497+
service:
498+
extensions: [ k8s_observer]
499+
pipelines:
500+
metrics:
501+
receivers: [ receiver_creator/metrics ]
502+
processors: []
503+
exporters: [ debug ]
504+
```
505+
506+
Target Pod annotated with hints:
507+
508+
```yaml
509+
apiVersion: v1
510+
kind: ConfigMap
511+
metadata:
512+
name: nginx-conf
513+
data:
514+
nginx.conf: |
515+
user nginx;
516+
worker_processes 1;
517+
error_log /dev/stderr warn;
518+
pid /var/run/nginx.pid;
519+
events {
520+
worker_connections 1024;
521+
}
522+
http {
523+
include /etc/nginx/mime.types;
524+
default_type application/octet-stream;
525+
526+
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
527+
'$status $body_bytes_sent "$http_referer" '
528+
'"$http_user_agent" "$http_x_forwarded_for"';
529+
access_log /dev/stdout main;
530+
server {
531+
listen 80;
532+
server_name localhost;
533+
534+
location /nginx_status {
535+
stub_status on;
536+
}
537+
}
538+
include /etc/nginx/conf.d/*;
539+
}
540+
---
541+
apiVersion: v1
542+
kind: Pod
543+
metadata:
544+
name: redis
545+
annotations:
546+
io.opentelemetry.collector.receiver-creator.metrics/receiver: redis
547+
io.opentelemetry.collector.receiver-creator.metrics/collection_interval: '20s'
548+
io.opentelemetry.collector.receiver-creator.metrics/receiver.webserver: nginx
549+
io.opentelemetry.collector.receiver-creator.metrics/endpoint.webserver: "http://`endpoint`/nginx_status"
550+
labels:
551+
k8s-app: redis
552+
app: redis
553+
spec:
554+
volumes:
555+
- name: nginx-conf
556+
configMap:
557+
name: nginx-conf
558+
items:
559+
- key: nginx.conf
560+
path: nginx.conf
561+
containers:
562+
- name: webserver
563+
image: nginx:latest
564+
ports:
565+
- containerPort: 80
566+
name: webserver
567+
volumeMounts:
568+
- mountPath: /etc/nginx/nginx.conf
569+
readOnly: true
570+
subPath: nginx.conf
571+
name: nginx-conf
572+
- image: redis
573+
imagePullPolicy: IfNotPresent
574+
name: redis
575+
ports:
576+
- name: redis
577+
containerPort: 6379
578+
protocol: TCP
579+
```

‎receiver/receivercreator/config.go

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,19 @@ type Config struct {
7878
// ResourceAttributes is a map of default resource attributes to add to each resource
7979
// object received by this receiver from dynamically created receivers.
8080
ResourceAttributes resourceAttributes `mapstructure:"resource_attributes"`
81+
Hints HintsConfig `mapstructure:"hints"`
82+
}
83+
84+
type HintsConfig struct {
85+
K8s K8sHintsConfig `mapstructure:"k8s"`
86+
}
87+
88+
type K8sHintsConfig struct {
89+
Metrics MetricsHints `mapstructure:"metrics"`
90+
}
91+
92+
type MetricsHints struct {
93+
Enabled bool `mapstructure:"enabled"`
8194
}
8295

8396
func (cfg *Config) Unmarshal(componentParser *confmap.Conf) error {

‎receiver/receivercreator/hints.go

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
// Copyright The OpenTelemetry Authors
2+
// SPDX-License-Identifier: Apache-2.0
3+
4+
package receivercreator // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/receivercreator"
5+
6+
import (
7+
"fmt"
8+
9+
"go.uber.org/zap"
10+
11+
"github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer"
12+
)
13+
14+
const (
15+
hintsMetricsReceiver = "io.opentelemetry.collector.receiver-creator.metrics/receiver"
16+
hintsMetricsEndpoint = "io.opentelemetry.collector.receiver-creator.metrics/endpoint"
17+
hintsMetricsCollectionInterval = "io.opentelemetry.collector.receiver-creator.metrics/collection_interval"
18+
hintsMetricsTimeout = "io.opentelemetry.collector.receiver-creator.metrics/timeout"
19+
hintsMetricsUsername = "io.opentelemetry.collector.receiver-creator.metrics/username"
20+
hintsMetricsPassword = "io.opentelemetry.collector.receiver-creator.metrics/password"
21+
)
22+
23+
// HintsTemplatesBuilder creates configuration templates from provided hints.
24+
type HintsTemplatesBuilder interface {
25+
createReceiverTemplatesFromHints() ([]receiverTemplate, error)
26+
}
27+
28+
// K8sHintsBuilder creates configurations from hints provided as Pod's annotations.
29+
type K8sHintsBuilder struct {
30+
logger *zap.Logger
31+
}
32+
33+
func (builder *K8sHintsBuilder) createReceiverTemplatesFromHints(env observer.EndpointEnv) ([]receiverTemplate, error) {
34+
var endpointType string
35+
var podUID string
36+
var port uint16
37+
var annotations map[string]string
38+
var receiverTemplates []receiverTemplate
39+
40+
builder.logger.Debug("handling hints for added endpoint", zap.Any("env", env))
41+
42+
if pod, ok := env["pod"]; ok {
43+
endpointPod, ok := pod.(observer.EndpointEnv)
44+
if !ok {
45+
return receiverTemplates, fmt.Errorf("could not extract endpoint's pod: %v", zap.Any("endpointPod", pod))
46+
}
47+
ann := endpointPod["annotations"]
48+
if ann != nil {
49+
annotations, ok = ann.(map[string]string)
50+
if !ok {
51+
return receiverTemplates, fmt.Errorf("could not extract annotations: %v", zap.Any("annotations", ann))
52+
}
53+
}
54+
podUID = endpointPod["uid"].(string)
55+
} else {
56+
return receiverTemplates, nil
57+
}
58+
59+
if valType, ok := env["type"]; ok {
60+
endpointType, ok = valType.(string)
61+
if !ok {
62+
return receiverTemplates, fmt.Errorf("could not extract endpointType: %v", zap.Any("endpointType", valType))
63+
}
64+
} else {
65+
return receiverTemplates, fmt.Errorf("could not get endpoint type: %v", zap.Any("env", env))
66+
}
67+
68+
if len(annotations) > 0 {
69+
if endpointType == string(observer.PortType) {
70+
// Only handle Endpoints of type port for metrics
71+
portName := env["name"].(string)
72+
metricsReceiverEnabled := getHintAnnotation(annotations, hintsMetricsReceiver, portName)
73+
if metricsReceiverEnabled != "" {
74+
subreceiverKey := metricsReceiverEnabled
75+
if subreceiverKey == "" {
76+
return receiverTemplates, nil
77+
}
78+
builder.logger.Debug("handling added hinted receiver", zap.Any("subreceiverKey", subreceiverKey))
79+
80+
userConfMap := createMetricsConfig(annotations, env, portName)
81+
82+
if p, ok := env["port"]; ok {
83+
port = p.(uint16)
84+
if port == 0 {
85+
return receiverTemplates, fmt.Errorf("could not extract port: %v", zap.Any("env", env))
86+
}
87+
} else {
88+
return receiverTemplates, fmt.Errorf("could not extract port: %v", zap.Any("env", env))
89+
}
90+
subreceiver, err := newReceiverTemplate(fmt.Sprintf("%v/%v_%v", subreceiverKey, podUID, port), userConfMap)
91+
if err != nil {
92+
builder.logger.Error("error adding subreceiver", zap.Any("err", err))
93+
return receiverTemplates, err
94+
}
95+
96+
subreceiver.Rule = fmt.Sprintf("type == \"port\" && port ==%v", port) //
97+
subreceiver.rule, err = newRule(subreceiver.Rule)
98+
if err != nil {
99+
builder.logger.Error("error adding subreceiver rule", zap.Any("err", err))
100+
return receiverTemplates, err
101+
}
102+
builder.logger.Debug("adding hinted receiver", zap.Any("subreceiver", subreceiver))
103+
receiverTemplates = append(receiverTemplates, subreceiver)
104+
}
105+
}
106+
}
107+
return receiverTemplates, nil
108+
}
109+
110+
func createMetricsConfig(annotations map[string]string, env observer.EndpointEnv, portName string) userConfigMap {
111+
confMap := map[string]any{}
112+
113+
defaultEndpoint := env["endpoint"]
114+
// get endpoint directly from the Port endpoint
115+
if defaultEndpoint != "" {
116+
confMap["endpoint"] = defaultEndpoint
117+
}
118+
119+
subreceiverEndpoint := getHintAnnotation(annotations, hintsMetricsEndpoint, portName)
120+
if subreceiverEndpoint != "" {
121+
confMap["endpoint"] = subreceiverEndpoint
122+
}
123+
subreceiverColInterval := getHintAnnotation(annotations, hintsMetricsCollectionInterval, portName)
124+
if subreceiverColInterval != "" {
125+
confMap["collection_interval"] = subreceiverColInterval
126+
}
127+
subreceiverTimeout := getHintAnnotation(annotations, hintsMetricsTimeout, portName)
128+
if subreceiverTimeout != "" {
129+
confMap["timeout"] = subreceiverTimeout
130+
}
131+
subreceiverUsername := getHintAnnotation(annotations, hintsMetricsUsername, portName)
132+
if subreceiverUsername != "" {
133+
confMap["username"] = subreceiverUsername
134+
}
135+
subreceiverPassword := getHintAnnotation(annotations, hintsMetricsPassword, portName)
136+
if subreceiverPassword != "" {
137+
confMap["password"] = subreceiverPassword
138+
}
139+
return confMap
140+
}
141+
142+
func getHintAnnotation(annotations map[string]string, hintKey string, portName string) string {
143+
containerLevelHint := annotations[fmt.Sprintf("%s.%s", hintKey, portName)]
144+
if containerLevelHint != "" {
145+
return containerLevelHint
146+
}
147+
148+
// if there is no container level hint defined try to scope the hint more on container level by suffixing with .<port_name>
149+
podLevelHint := annotations[hintKey]
150+
if podLevelHint != "" {
151+
return podLevelHint
152+
}
153+
return ""
154+
}

0 commit comments

Comments
 (0)