[connector/routing] Dynamic routing config from discovery

[Amoolaa]

Component(s)

connector/routing

Is your feature request related to a problem? Please describe.

I have a multi-tenanted kubernetes platform where I wish to send telemetry to different backend tenants/destinations based on the namespace, annotations used in deployments on the cluster. I can't think of a way to do this dynamically on the collector, such that a new pod comes up and it's dynamically included as part of the routing table.

Describe the solution you'd like

Ability to discover routing configs and apply them dynamically. This would still require being able to dynamically create exporters, but this is the critical part for my use case

Describe alternatives you've considered

Writing a basic controller that watches namespaces/whatever resource I'm interested in that patches the collector configMap with new routing configs/exporters and hot-reloads config from sidecar.

Additional context

No response

[github-actions]

Pinging code owners:

• connector/routing: @mwear

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[Frapschen]

I have a few questions regarding your scenario. Is your backend static, or does it also involve dynamic elements? Could you please provide more detailed examples to better demonstrate your case?

[Amoolaa]

The backend is Loki/Mimir/Tempo (so relies on X-Scope-OrgId header for tenancy) or another collector with a URL that is dynamic to the namespace it's deployed to.

It's one of a few other use cases I've been thinking about - such as the ability to ingest cloudwatch logs via kinesis and fan out to different tenancies based on the AWS account.

[Frapschen]

if the backend is also dynamic, I believe it a very complex case.

[Amoolaa]

I'm happy to consider other solutions outside the collector (such as patch on event + hot-reload I've mentioned above) - just need to know if there is appetite to have something native to the collector or if there is a better way to do it