[receiver/k8sobjects] allow options to exclude namespace

[KyriosGN0]

Component(s)

receiver/k8sobjects

Is your feature request related to a problem? Please describe.

yes, in our cluster we have seen that in certain namespace that is has a heavy spike pattern and generates around 50k events, this causes our otel daemonset to get killed my oom

Describe the solution you'd like

we would like the optiont to exclude namespace when using the k8sobjects receiver

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/k8sobjects: @dmitryax @hvaghani221 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[atoulme]

You can already set namespaces to scrape here: https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/k8sobjectsreceiver/README.md?plain=1#L54

Do you want to enhance that option?

[KyriosGN0]

@atoulme as far as i understand its a whitelist, i want the option to blacklist namespace

[atoulme]

OK, please explain how those 2 options would coexist. Please offer a configuration example.

[KyriosGN0]

@atoulme
namespaceWhitelist: [<Regex of namespace in include>]
namespaceBlacklist: [<Regex of namespace to exclude>]
or we could use labels to select which namespaces are exclude/included
as for coexist, we would need to build a list of namespace to watch based on those regex/labels and then pass it to if statement here

i do think its more useful to exclude namespace since in my experience i want to stop watching namespaces with high amount of events

opentelemetry-collector-contrib/receiver/k8sobjectsreceiver/receiver.go

Line 113 in 2612314

case WatchMode:

[atoulme]

whitelist and blacklist are no longer in use ; typically we use allowlist and denylist.
We also try really hard to keep configuration backwards compatible ; there is already a namespaces option taking a list of namespaces.
What is the smallest change that will get you what you need?

[KyriosGN0]

@atoulme a list of namespace to exclude

[KyriosGN0]

hey @atoulme i would like to contribute this feature myself, can we agree on how (if you agree with my suggestion)?

[atoulme]

I am not a codeowner and cannot accept this enhancement. @dmitryax @hvaghani221 @TylerHelmuth please review

[ChrsMark]

@KyriosGN0 sorry for the delay on this. Would you like to get assigned to this issue?

There is a related issue #39677 to add namespace setting for k8s_observer.

I think it makes sense to have a coordination here for all the k8s components that access the K8s APIServer, and check if we could have 2 options like:

namespaceAllowList: []
namespaceDenyList: []

/cc @dmitryax @hvaghani221 @TylerHelmuth

[KyriosGN0]

@ChrsMark yes please assign it to me, I will check the other k8s components as well
Thanks in advance

[ChrsMark]

@KyriosGN0 as discussed at #39777 (comment), I wonder if the namespaces setting and/or field_selector would be enough for your use-case.

Having a namespacesDenyList option does not look really feasible as noted at #39777 (comment).

[KyriosGN0]

@ChrsMark if the field_selector supports regex they yeah it should be good i hope, my problem was that some namespace generated a lot of events and i would like to not watch them at all

[ChrsMark]

I see. I don't think field selectors in k8s support regexps.

In that case, it would make sense to provide a namespacesDenyList. The way it could be implemented would be to first list all namespaces on startup and only start watchers for the namespaces not in the namespacesDenyList. That's just a first idea though, so if there is something more efficient happy to hear it.

[KyriosGN0]

I see. I don't think field selectors in k8s support regexps.

In that case, it would make sense to provide a namespacesDenyList. The way it could be implemented would be to first list all namespaces on startup and only start watchers for the namespaces not in the namespacesDenyList. That's just a first idea though, so if there is something more efficient happy to hear it.

My one issue is namespace created after collector has started running, i wouldn't want to restart the collector when namespace is created