[exporter/datadog] Error while validaing API key: i/o timeout in serverless

[RangelReale]

Component(s)

exporter/datadog

What happened?

Description

Error while validaing API key: i/o timeout in serverless

Steps to Reproduce

Collector initialization in lambda/cloudrun frequently have timeouts on initialization trying to validate the api key, this may take a lot of time and times out the function initialization.

Expected Result

Have an option for not doing this check on serverless environments. We can't depend on a external service responding on each function startup.

Actual Result

Collector version

0.64.0

Environment information

Environment

OS: Docker Distroless
Compiler(if manually compiled): go 1.17

OpenTelemetry Collector configuration

No response

Log output

023-02-01T01:27:36.162Z	warn	Error while validating API key	{"kind": "exporter", "data_type": "metrics", "name": "datadog", "error": "Get \"https://api.datadoghq.com/api/v1/validate\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/datadog: @mx-psi @gbbr @dineshg13 @liustanley @songy23

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[RangelReale]

I managed to "avoid" it for now descreasing the timeout from the default of 15s to 1s, but 1s in lambda is still a lot. Cloud run limits are higher.

[songy23]

I worked on implementing #13732 (comment) before, will send a PR.