Skip to content

[datadogexporter] Improve support of semantic conventions for K8s, Azure and ECS #2623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

mx-psi
Copy link
Member

@mx-psi mx-psi commented Mar 9, 2021

Description:

TL;DR:

  • Add support for the new Azure resource detector
  • Improve support for Kubernetes semantic conventions
  • Improve support for Kubernetes labels
  • Improve support for ECS semantic conventions

Azure

  • The host.id (Azure VM ID) is sent as a host alias now, to replicate the Datadog Agent behavior.
  • The cluster name is taken from the resource group ID.

ECS

  • Task family, cluster ARN and task revision are mapped to Datadog conventions. container.id priority to be taken as a hostname is lowered since we only want this as a last resort.

Kubernetes

  • OpenTelemetry Kubernetes semantic conventions are mapped to Datadog conventions.
  • Kubernetes labels conventions (both Datadog-specific and general recommendations) are mapped to Datadog conventions.
  • We now take the Kubernetes cluster name on Azure (see above) and AWS.

Link to tracking Issue: n/a

Testing: Added unit tests

@codecov
Copy link

codecov bot commented Mar 9, 2021

Codecov Report

Merging #2623 (376a6b4) into main (838be57) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2623      +/-   ##
==========================================
+ Coverage   91.30%   91.32%   +0.02%     
==========================================
  Files         429      430       +1     
  Lines       21457    21492      +35     
==========================================
+ Hits        19591    19628      +37     
+ Misses       1396     1395       -1     
+ Partials      470      469       -1     
Flag Coverage Δ
integration 69.18% <ø> (-0.07%) ⬇️
unit 90.23% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
exporter/datadogexporter/attributes/attributes.go 100.00% <100.00%> (ø)
exporter/datadogexporter/metadata/azure/azure.go 100.00% <100.00%> (ø)
exporter/datadogexporter/metadata/ec2/ec2.go 53.33% <100.00%> (+5.83%) ⬆️
exporter/datadogexporter/metadata/host.go 85.71% <100.00%> (+4.46%) ⬆️
exporter/datadogexporter/metadata/metadata.go 90.14% <100.00%> (+0.43%) ⬆️
receiver/k8sclusterreceiver/watcher.go 97.64% <0.00%> (+2.35%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 838be57...376a6b4. Read the comment docs.

kisieland pushed a commit to kisieland/opentelemetry-collector-contrib that referenced this pull request Mar 16, 2021
It would conflict with the ECS hostname
Comment on lines +76 to +80
if ok && cloudProvider.StringVal() == conventions.AttributeCloudProviderAzure {
return azure.ClusterNameFromAttributes(attrs)
} else if ok && cloudProvider.StringVal() == conventions.AttributeCloudProviderAWS {
return ec2.ClusterNameFromAttributes(attrs)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might not have followed everything here, is there a reason why we're not also doing this for GCP? Is it because there are no ways to get a cluster name from GCP-specific attributes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I should have mentioned it. There are none currently, there will be once #570 is resolved

Copy link
Contributor

@KSerrania KSerrania left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a comment to fix the codecov warning

Copy link
Contributor

@julien-lebot julien-lebot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mx-psi mx-psi marked this pull request as ready for review March 26, 2021 11:38
@mx-psi mx-psi requested a review from a team March 26, 2021 11:38
@github-actions
Copy link
Contributor

github-actions bot commented Apr 3, 2021

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Apr 3, 2021
@tigrannajaryan tigrannajaryan merged commit a280b63 into open-telemetry:main Apr 9, 2021
@mx-psi mx-psi deleted the mx-psi/semantic-conventions branch April 9, 2021 19:52
pmatyjasek-sumo referenced this pull request in pmatyjasek-sumo/opentelemetry-collector-contrib Apr 28, 2021
…ure and ECS (#2623)

TL;DR:
- Add support for the new Azure resource detector
- Improve support for Kubernetes semantic conventions
- Improve support for Kubernetes labels
- Improve support for ECS semantic conventions

### Azure

- The `host.id` (Azure VM ID) is sent as a host alias now, to replicate the Datadog Agent behavior.
- The cluster name is taken from the resource group ID.

### ECS

- Task family, cluster ARN and task revision are mapped to Datadog conventions. `container.id` priority to be taken as a hostname is lowered since we only want this as a last resort.

### Kubernetes

- OpenTelemetry Kubernetes semantic conventions are mapped to Datadog conventions.
- Kubernetes labels conventions (both Datadog-specific and general recommendations) are mapped to Datadog conventions.
- We now take the Kubernetes cluster name on Azure (see above) and AWS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants