[receiver/datadog] service.name not set at trace resource resulting in OTLPResourceNoServiceName

[mvadu]

Component(s)

receiver/datadog

What happened?

Description

Currently in addResourceData service.name is not set. Its set in the span attributes. Because of this traces exported to backends like Grafana tempo are appearing as OTLPResourceNoServiceName.

Steps to Reproduce

• Run an app instrumented with datadog tracer (https://docs.datadoghq.com/tracing/trace_collection/dd_libraries/java/?tab=containers)
• Forward the traces to Grafana tempo using otel-collector-contrib with

receivers:
 otlp:
   protocols:
     grpc:
     http:
       cors:
         allowed_origins:
           - "http://*"
           - "https://*"
 datadog:
   endpoint: 0.0.0.0:8126
   read_timeout: 60s

exporters:
 otlp:
   endpoint: tempo:4317
   tls:
     insecure: true 
 logging:
 prometheus:
   endpoint: "otelcol:9464"
   resource_to_telemetry_conversion:
     enabled: true
   enable_open_metrics: true

processors:
 batch:

service:
 pipelines:
   traces:
     receivers: [datadog]
     processors: [batch]
     exporters: [otlp]

• Validate the traces received in Tempo

Expected Result

The service Name should be populated in Service Name filter which uses the Resources section
ref:

Actual Result

The Resource section does not contains a service.name tag, thus resulting in OTLPResourceNoServiceName

Collector version

v0.75.0

Environment information

Environment

OS: Ubuntu 20

OpenTelemetry Collector configuration

receivers:
 otlp:
   protocols:
     grpc:
     http:
       cors:
         allowed_origins:
           - "http://*"
           - "https://*"
 datadog:
   endpoint: 0.0.0.0:8126
   read_timeout: 60s

exporters:
 otlp:
   endpoint: tempo:4317
   tls:
     insecure: true 
 logging:
 prometheus:
   endpoint: "otelcol:9464"
   resource_to_telemetry_conversion:
     enabled: true
   enable_open_metrics: true

processors:
 batch:

service:
 pipelines:
   traces:
     receivers: [datadog]
     processors: [batch]
     exporters: [otlp]

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/datadog: @boostchicken @gouthamve @jpkrohling @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[boostchicken]

taking a look

[boostchicken]

Upon further review I yanked the PR.
I ran a test locally and it looks good, also our unit tests show copying service name just fine. here is what we are doing

opentelemetry-collector-contrib/receiver/datadogreceiver/translator.go

Lines 81 to 83 in 84d9f48

	attrs := newSpan.Attributes()
	attrs.EnsureCapacity(len(span.GetMeta()) + 1)
	attrs.PutStr(semconv.AttributeServiceName, span.Service)

span being th Datadog Span. Since DD is a snowflake, a name to them is not driven off a tag, it has a special field. That is where we pull service.name from.

So everything looks fine to me.

Is your request to also copy service.name from a tag?

[mvadu]

@boostchicken sorry for delay in responding John. Please find below the steps to recreate the issue. Core of it is with Tempo expecting the service name to be available in Resource section of the trace, where as the translator is only adding it in Attributes section of the trace.

Steps to reproduce:

1. clone and run the sample app in https://github.com/mvadu/dd-otel-tempo with docker compose up

2. navigate to http://localhost:3000/explore? you will see a Tempo based search UI. Select Search, under span names select one of the GET requests and select Run queries

3. You will see bunch of traces generated from java app sent to tempo via dd agent-> dd receiver -> otlp exporter

4. expand any of them by clicking the trace id

5. observe the right hand side.
   

6. Now in the same search UI select the Service Name dropdown and select loadgenerator, and click run queries. These are generated by Open Telemetry Python SDK and sent directly to tempo.

7. If you expand any of them now you will see the service name properly.
   

@gouthamve Could you please have someone from Tempo team review this and clarify why Tempo is expecting the service.name under Resource section?

[MovieStoreGuy]

Will be fixed in the next released.

[mvadu]

Thank you @MovieStoreGuy for fixing this.. I did hear from Tempo team on their expectations, adding it here incase someone comes across this at a future date.

Based on this section of Otel Spec (https://opentelemetry.io/docs/specs/otel/resource/sdk/#sdk-provided-resource-attributes) Tempo expects service.name and telemetry.sdk group be present in the Resource section of the trace.