Skip to content

Commit 7c65ab2

Browse files
committed
[receiver/receiver_creator] Add support for metrics' hints
Signed-off-by: ChrsMark <[email protected]>
1 parent 47118f2 commit 7c65ab2

File tree

6 files changed

+599
-71
lines changed

6 files changed

+599
-71
lines changed

.chloggen/hints.yaml

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Use this changelog template to create an entry for release notes.
2+
3+
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
4+
change_type: enhancement
5+
6+
# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
7+
component: receivercreator
8+
9+
# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
10+
note: Add support for generating metrics receivers based on provided annotations' hints
11+
12+
# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
13+
issues: [34427]
14+
15+
# (Optional) One or more lines of additional information to render under the primary note.
16+
# These lines will be padded with 2 spaces and then inserted directly into the document.
17+
# Use pipe (|) for multiline entries.
18+
subtext:
19+
20+
# If your change doesn't affect end users or the exported elements of any package,
21+
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
22+
# Optional: The change log or logs in which this entry should be included.
23+
# e.g. '[user]' or '[user, api]'
24+
# Include 'user' if the change is relevant to end users.
25+
# Include 'api' if there is a change to a library API.
26+
# Default: '[user]'
27+
change_logs: []

receiver/receivercreator/README.md

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -437,3 +437,144 @@ service:
437437
438438
The full list of settings exposed for this receiver are documented [here](./config.go)
439439
with detailed sample configurations [here](./testdata/config.yaml).
440+
441+
442+
## Generate receiver configurations from provided Hints
443+
444+
Currently this feature is only supported for K8s environments and the `k8sobserver`.
445+
446+
The feature for K8s is enabled with the following setting:
447+
448+
```yaml
449+
receiver_creator/metrics:
450+
watch_observers: [ k8s_observer ]
451+
hints:
452+
k8s:
453+
metrics:
454+
enabled: true
455+
```
456+
457+
Users can use the following annotations to automatically enable receivers to start collecting metrics from the target Pods/containers.
458+
459+
### Supported metrics annotations
460+
1. `io.opentelemetry.collector.receiver-creator.metrics/receiver` (example: `nginx`)
461+
2. `io.opentelemetry.collector.receiver-creator.metrics/endpoint` (example: ```"http://`endpoint`/nginx_status"```, if not provided it defaults to `endpoint` which is of form `pod_ip:container_port`.)
462+
3. `io.opentelemetry.collector.receiver-creator.metrics/collection_interval` (example: `20s`)
463+
4. `io.opentelemetry.collector.receiver-creator.metrics/timeout` (example: `1m`)
464+
5. `io.opentelemetry.collector.receiver-creator.metrics/username` (example: `admin`)
465+
6. `io.opentelemetry.collector.receiver-creator.metrics/password` (example: `passpass`)
466+
467+
468+
### Support multiple target containers
469+
470+
Users can target the annotation to a specific container by suffixing it with the name of the port that container exposes:
471+
`io.opentelemetry.collector.receiver-creator.metrics.<container_port_name>/endpoint`.
472+
For example ```io.opentelemetry.collector.receiver-creator.metrics.webserver/endpoint: "http://`endpoint`/nginx_status"```
473+
where `webserver` is the name of the port the target container exposes.
474+
475+
If a Pod is annotated with both container level hints and pod level hints the container level hints have priority and
476+
the Pod level hints are used as a fallback (see detailed example bellow).
477+
478+
The current implementation relies on the implementation of `k8sobserver` extension and specifically
479+
the [pod_endpoint](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/v0.111.0/extension/observer/k8sobserver/pod_endpoint.go).
480+
The hints are evaluated per container by extracting the annotations from each `Port` endpoint that is emitted.
481+
482+
483+
### Examples
484+
485+
#### Metrics example
486+
487+
Collector's configuration:
488+
```yaml
489+
receivers:
490+
receiver_creator/metrics:
491+
watch_observers: [ k8s_observer ]
492+
hints:
493+
k8s:
494+
metrics:
495+
enabled: true
496+
receivers:
497+
498+
service:
499+
extensions: [ k8s_observer]
500+
pipelines:
501+
metrics:
502+
receivers: [ receiver_creator/metrics ]
503+
processors: []
504+
exporters: [ debug ]
505+
```
506+
507+
Target Pod annotated with hints:
508+
509+
```yaml
510+
apiVersion: v1
511+
kind: ConfigMap
512+
metadata:
513+
name: nginx-conf
514+
data:
515+
nginx.conf: |
516+
user nginx;
517+
worker_processes 1;
518+
error_log /dev/stderr warn;
519+
pid /var/run/nginx.pid;
520+
events {
521+
worker_connections 1024;
522+
}
523+
http {
524+
include /etc/nginx/mime.types;
525+
default_type application/octet-stream;
526+
527+
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
528+
'$status $body_bytes_sent "$http_referer" '
529+
'"$http_user_agent" "$http_x_forwarded_for"';
530+
access_log /dev/stdout main;
531+
server {
532+
listen 80;
533+
server_name localhost;
534+
535+
location /nginx_status {
536+
stub_status on;
537+
}
538+
}
539+
include /etc/nginx/conf.d/*;
540+
}
541+
---
542+
apiVersion: v1
543+
kind: Pod
544+
metadata:
545+
name: redis
546+
annotations:
547+
io.opentelemetry.collector.receiver-creator.metrics/receiver: redis
548+
io.opentelemetry.collector.receiver-creator.metrics/collection_interval: '20s'
549+
io.opentelemetry.collector.receiver-creator.metrics.webserver/receiver: nginx
550+
io.opentelemetry.collector.receiver-creator.metrics.webserver/endpoint: "http://`endpoint`/nginx_status"
551+
labels:
552+
k8s-app: redis
553+
app: redis
554+
spec:
555+
volumes:
556+
- name: nginx-conf
557+
configMap:
558+
name: nginx-conf
559+
items:
560+
- key: nginx.conf
561+
path: nginx.conf
562+
containers:
563+
- name: webserver
564+
image: nginx:latest
565+
ports:
566+
- containerPort: 80
567+
name: webserver
568+
volumeMounts:
569+
- mountPath: /etc/nginx/nginx.conf
570+
readOnly: true
571+
subPath: nginx.conf
572+
name: nginx-conf
573+
- image: redis
574+
imagePullPolicy: IfNotPresent
575+
name: redis
576+
ports:
577+
- name: redis
578+
containerPort: 6379
579+
protocol: TCP
580+
```

receiver/receivercreator/config.go

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,19 @@ type Config struct {
7878
// ResourceAttributes is a map of default resource attributes to add to each resource
7979
// object received by this receiver from dynamically created receivers.
8080
ResourceAttributes resourceAttributes `mapstructure:"resource_attributes"`
81+
Hints HintsConfig `mapstructure:"hints"`
82+
}
83+
84+
type HintsConfig struct {
85+
K8s K8sHintsConfig `mapstructure:"k8s"`
86+
}
87+
88+
type K8sHintsConfig struct {
89+
Metrics MetricsHints `mapstructure:"metrics"`
90+
}
91+
92+
type MetricsHints struct {
93+
Enabled bool `mapstructure:"enabled"`
8194
}
8295

8396
func (cfg *Config) Unmarshal(componentParser *confmap.Conf) error {

receiver/receivercreator/hints.go

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
// Copyright The OpenTelemetry Authors
2+
// SPDX-License-Identifier: Apache-2.0
3+
4+
package receivercreator // import "github.com/open-telemetry/opentelemetry-collector-contrib/receiver/receivercreator"
5+
6+
import (
7+
"fmt"
8+
9+
"go.uber.org/zap"
10+
11+
"github.com/open-telemetry/opentelemetry-collector-contrib/extension/observer"
12+
)
13+
14+
const (
15+
otelHints = "io.opentelemetry.collector.receiver-creator"
16+
metricsHint = "metrics"
17+
hintsMetricsReceiver = "receiver"
18+
hintsMetricsEndpoint = "endpoint"
19+
hintsMetricsCollectionInterval = "collection_interval"
20+
hintsMetricsTimeout = "timeout"
21+
hintsMetricsUsername = "username"
22+
hintsMetricsPassword = "password"
23+
)
24+
25+
// HintsTemplatesBuilder creates configuration templates from provided hints.
26+
type HintsTemplatesBuilder interface {
27+
createReceiverTemplatesFromHints() ([]receiverTemplate, error)
28+
}
29+
30+
// K8sHintsBuilder creates configurations from hints provided as Pod's annotations.
31+
type K8sHintsBuilder struct {
32+
logger *zap.Logger
33+
config K8sHintsConfig
34+
}
35+
36+
// createReceiverTemplateFromHints creates a receiver configuration based on the provided hints.
37+
// Hints are extracted from Pod's annotations.
38+
// Metrics configurations are only created for Port Endpoints.
39+
// TODO: Logs configurations are only created for Pod Container Endpoints.
40+
func (builder *K8sHintsBuilder) createReceiverTemplateFromHints(env observer.EndpointEnv) (*receiverTemplate, error) {
41+
var endpointType string
42+
var podUID string
43+
var annotations map[string]string
44+
45+
builder.logger.Debug("handling hints for added endpoint", zap.Any("env", env))
46+
47+
if pod, ok := env["pod"]; ok {
48+
endpointPod, ok := pod.(observer.EndpointEnv)
49+
if !ok {
50+
return nil, fmt.Errorf("could not extract endpoint's pod: %v", zap.Any("endpointPod", pod))
51+
}
52+
ann := endpointPod["annotations"]
53+
if ann != nil {
54+
annotations, ok = ann.(map[string]string)
55+
if !ok {
56+
return nil, fmt.Errorf("could not extract annotations: %v", zap.Any("annotations", ann))
57+
}
58+
}
59+
podUID = endpointPod["uid"].(string)
60+
} else {
61+
return nil, nil
62+
}
63+
64+
if valType, ok := env["type"]; ok {
65+
endpointType, ok = valType.(string)
66+
if !ok {
67+
return nil, fmt.Errorf("could not extract endpointType: %v", zap.Any("endpointType", valType))
68+
}
69+
} else {
70+
return nil, fmt.Errorf("could not get endpoint type: %v", zap.Any("env", env))
71+
}
72+
73+
if len(annotations) > 0 {
74+
if endpointType == string(observer.PortType) && builder.config.Metrics.Enabled {
75+
// Only handle Endpoints of type port for metrics
76+
return builder.createMetricsReceiver(annotations, env, podUID)
77+
}
78+
}
79+
return nil, nil
80+
}
81+
82+
func (builder *K8sHintsBuilder) createMetricsReceiver(
83+
annotations map[string]string,
84+
env observer.EndpointEnv,
85+
podUID string) (*receiverTemplate, error) {
86+
87+
var port uint16
88+
89+
portName := env["name"].(string)
90+
subreceiverKey := getHintAnnotation(annotations, metricsHint, hintsMetricsReceiver, portName)
91+
92+
if subreceiverKey == "" {
93+
// no metrics hints detected
94+
return nil, nil
95+
}
96+
builder.logger.Debug("handling added hinted receiver", zap.Any("subreceiverKey", subreceiverKey))
97+
98+
userConfMap := createMetricsConfig(annotations, env, portName)
99+
100+
if p, ok := env["port"]; ok {
101+
port = p.(uint16)
102+
if port == 0 {
103+
return nil, fmt.Errorf("could not extract port: %v", zap.Any("env", env))
104+
}
105+
} else {
106+
return nil, fmt.Errorf("could not extract port: %v", zap.Any("env", env))
107+
}
108+
subreceiver, err := newReceiverTemplate(fmt.Sprintf("%v/%v_%v", subreceiverKey, podUID, port), userConfMap)
109+
if err != nil {
110+
builder.logger.Error("error adding subreceiver", zap.Any("err", err))
111+
return nil, err
112+
}
113+
114+
builder.logger.Debug("adding hinted receiver", zap.Any("subreceiver", subreceiver))
115+
return &subreceiver, nil
116+
117+
}
118+
119+
func createMetricsConfig(annotations map[string]string, env observer.EndpointEnv, portName string) userConfigMap {
120+
confMap := map[string]any{}
121+
122+
defaultEndpoint := env["endpoint"]
123+
// get endpoint directly from the Port endpoint
124+
if defaultEndpoint != "" {
125+
confMap["endpoint"] = defaultEndpoint
126+
}
127+
128+
subreceiverEndpoint := getHintAnnotation(annotations, metricsHint, hintsMetricsEndpoint, portName)
129+
if subreceiverEndpoint != "" {
130+
confMap["endpoint"] = subreceiverEndpoint
131+
}
132+
subreceiverColInterval := getHintAnnotation(annotations, metricsHint, hintsMetricsCollectionInterval, portName)
133+
if subreceiverColInterval != "" {
134+
confMap["collection_interval"] = subreceiverColInterval
135+
}
136+
subreceiverTimeout := getHintAnnotation(annotations, metricsHint, hintsMetricsTimeout, portName)
137+
if subreceiverTimeout != "" {
138+
confMap["timeout"] = subreceiverTimeout
139+
}
140+
subreceiverUsername := getHintAnnotation(annotations, metricsHint, hintsMetricsUsername, portName)
141+
if subreceiverUsername != "" {
142+
confMap["username"] = subreceiverUsername
143+
}
144+
subreceiverPassword := getHintAnnotation(annotations, metricsHint, hintsMetricsPassword, portName)
145+
if subreceiverPassword != "" {
146+
confMap["password"] = subreceiverPassword
147+
}
148+
return confMap
149+
}
150+
151+
func getHintAnnotation(annotations map[string]string, hintType string, hintKey string, suffix string) string {
152+
// try to scope the hint more on container level by suffixing with .<port_name>
153+
containerLevelHint := annotations[fmt.Sprintf("%s.%s.%s/%s", otelHints, hintType, suffix, hintKey)]
154+
if containerLevelHint != "" {
155+
return containerLevelHint
156+
}
157+
158+
// if there is no container level hint defined try to use the Pod level hint
159+
podHintKey := fmt.Sprintf("%s.%s/%s", otelHints, hintType, hintKey)
160+
podLevelHint := annotations[podHintKey]
161+
if podLevelHint != "" {
162+
return podLevelHint
163+
}
164+
return ""
165+
}

0 commit comments

Comments
 (0)