Support Oracle Cloud with Resource Detection Processor

[tonychoe]

Component(s)

processor/resourcedetection

Is your feature request related to a problem? Please describe.

The Resource Detection Processor do not support Oracle's Cloud ("Oracle Cloud Infrastructure" or "OCI") yet.

Describe the solution you'd like

We propose to develop this processor to support detecting OCI compute node and (Oracle Kubernetes Engine) node resources.

OCI offers the Instance Metadata Service which works like the one from AWS.

We're from Oracle and happy to contribute to implement and maintain this feature.

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/resourcedetection: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dashpole]

This works for me. This isn't a "new component" exactly, since it is part of an existing one, but it would be helpful to still roughly follow the new component process: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#adding-new-components. I'll act as your sponsor, and will review PRs.

Before we get started, it would be good to know:

• what will the name of the detector be?
• Who will the codeowners be (2 codeowners would be ideal)
• Which resource attributes will it detect?

[tonychoe]

Thanks @dashpole

• We'd like to call it "OCI Metadata"
• codeowners are @tonychoe @raphaelfan (both are from Oracle)
• The resource attributes we'd like to collect are:

I wrote this to explain that

OCI Metadata
uses OCI SDK for Go to read resource information from the OCI instance metadata API to retrieve the resource attributes of OCI Compute instances and OKE worker nodes.

OCI Metadata

• cloud.provider ("oci")
• cloud.platform ("oci_compute")
• cloud.region (e.g. us-phoenix-1)
• cloud.availability_zone (e.g. aOft:PHX-AD-1)
• host.id (instance id)
• host.name (instance display name)
• host.type (instance shape)

OKE Metadata

• cloud.provider ("oci")
• cloud.platform ("oci_kubernetes engine")
• cloud.region (e.g. us-phoenix-1)
• cloud.availability_zone (e.g. aOft:PHX-AD-1)
• k8s.cluster.name (cluster display name)
• k8s.cluster.id (cluster id)
• host.id (instance id)
• host.name (instance display name)
• host.type (instance shape)

[dashpole]

We'd like to call it "OCI Metadata"

It would likely be "oci", "oracle", or "oracle_cloud_infrastructure", based on naming of other detectors: https://github.com/search?q=repo%3Aopen-telemetry%2Fopentelemetry-collector-contrib+%22type%3A+resourcedetectionprocessor%22&type=code

We might want to stay away from OCI, as that is currently used for the open container initiative in the conventions: https://github.com/open-telemetry/semantic-conventions/blob/156daec14fe8ca12d3fd8c3fad9602e4f80493b2/model/registry/oci.yaml

I would also encourage you to update the list of providers, platforms, etc in the conventions so we get constants we can use from the semconv package: https://github.com/open-telemetry/semantic-conventions/blob/156daec14fe8ca12d3fd8c3fad9602e4f80493b2/model/registry/cloud.yaml#L101. It doesn't need to block the initial release, but we should have those eventually.

[tonychoe]

Thanks for the pointers. "oracle_cloud_infrastructure" makes sense.

[dashpole]

Cool, you will have a folder in https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor/internal. You should be able to follow https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#vendor-specific-components, and open the "First PR". You won't need to do every part of that. Just look at other detectors if you aren't sure.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/resourcedetection: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[geekdave]

@dashpole Hello! I recently joined Oracle and I've been assigned this issue. I see it was marked stale. I plan to begin work on this within the next couple weeks. Thanks for your support and guidance!

[dashpole]

@geekdave sounds good. Let me know if you have any questions

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/resourcedetection: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[geekdave]

I'm still working on this. I'm new to a lot of this so it's been a bit slower than anticipated. Just got the upstream semantic-conventions change merged last week. Thanks! :)

[geekdave]

@dashpole Looks like Oracle Cloud is now recognized in the latest release of semantic-conventions v1.30.0! I'm not super clear about how that upstream project gets consumed by the collector-contrib project. Could you please point me to any references to how that project is integrated, so that I can test out using the newly-available oracle cloud conventions as I work on the resource detection processor?

[dashpole]

Someone will need to implement a resource detector here: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor/internal

[dashpole]

These packages also are generated based on the semantic conventions: https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv. You will need to wait for v1.30.0 to be generated there

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/resourcedetectionprocessor: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[tonychoe]

We're still working on this, waiting for semconv v1.30.0 is released.

[geekdave]

@dashpole Regarding your guidance:

These packages also are generated based on the semantic conventions: https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv. You will need to wait for v1.30.0 to be generated there

Would it be OK for me to generate the updated semantic convention packages there myself and submit a PR?

I see that there have been 3 semantic conventions releases over the past few months that include the needed Oracle Cloud conventions that were first introduced in v1.30.0. The latest is:
https://github.com/open-telemetry/semantic-conventions/releases/tag/v1.32.0

...yet the collector's last semantic conventions update was v1.27.0 about 6 months ago.

I'm happy to take a shot at following the docs, updating to v1.32.0 and submitting a PR to the collector repo. Just didn't want to step on any toes in case there is already a process or schedule for this. Thanks again for your help and guidance!

[dashpole]

I believe the tracking issue for updating semconv is here: open-telemetry/opentelemetry-collector#11828. Looks more complex than just running a regeneration script.

Given the amount of time that has elapsed, I think we should move forward with adding resource detection, and just use local constants that match the semconv, rather than blocking on semconv generation. Once that PR is open, we should track migrating to the new semconv once it is available in an issue, and link it from the code.

[dashpole]

Actually, it looks like the collector is moving to use https://github.com/open-telemetry/opentelemetry-go/tree/main/semconv/v1.30.0 instead of the collector's generated semconv libraries.

[geekdave]

@dashpole I chatted with Adriel Perkins recently in the #otel-collector-dev channel in the CNCF Slack org.

I asked:

Is it ok for outside contributors to submit a PR to bump the semantic conventions version used in the collector? Or is there some official process? A few of us are waiting for a more recent version here: https://github.com/open-telemetry/opentelemetry-collector/tree/main/semconv

Adriel responded:

that's the old generator and was replaced by weaver. 1.30.0 is available from this package https://pkg.go.dev/go.opentelemetry.io/otel/[email protected] which was what replaced the one in your link. it all comes from the go sdk now instead of being unique to the collector.

Each component will usually define which version of semconv it's using in the metadata.yaml file. Of course, it's not guaranteed that all attributes emitted by a component are in the semantic conventions. Just doing a search shows that there are components using 1.9.0 which was quite a while ago.
semconv releases occur about once a month, and a lot stopped at 1.27 because of waiting for release of the new package I mentioned above.
TL;DR, it's a bit all over the place and decided at a component level.

So it sounds like instead of doing a whole repo-level migration, I can just reference the right version directly from my own resource detection processor component.

I agree it's not worth it to block on this, so if I am unable to make it work in a reasonable time I'll proceed with hardcoded values for now.

[dashpole]

sounds great