Variable sampling frequency for interval processor

[KalmanMeth]

Component(s)

processor/interval

Is your feature request related to a problem? Please describe.

I have a use case where I would like to set different sampling rates for different metrics collected by the Collector.
For example, I may want most of the metrics to be updated once per minute, except for some specific metrics that I want to be updated every 10 seconds.

Describe the solution you'd like

A sample syntax of what I have in mind is:

intervalprocessor:
  [ interval: <duration> | default = 60s ]
  pass_through:
    [ gauge: <bool> | default = false ]
    [ summary: <boo>l | default = false ]
  [ rules:
    - name: <metric_name1>
      interval: <duration>
    - name: <metric_name2>
      interval: <duration>
    ...
  ]

The values of the <duration> specified in the rules would take precedence over the value specified in the default interval parameter.

Describe alternatives you've considered

I looked through the various processors, but I didn't find one that allows me such fine-grained control over the sampling frequencies for different metrics. I have implemented an approximate proof-of-concept and placed it at https://github.com/observ-vol-mgt/research-otel-collector/tree/main/processor/intervalprocessor, with the main code in https://github.com/observ-vol-mgt/research-otel-collector/blob/main/processor/intervalprocessor/processor.go.

Additional context

No response

[github-actions]

Pinging code owners:

• processor/interval: @RichieSams @sh0rez

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[RichieSams]

While we could implement this behavior, I feel like it's duplicating work done by other processors. For example, you could have multiple pipelines, with different interval processors. And you could use the filter processor to select which metrics go where.

[KalmanMeth]

While we could implement this behavior, I feel like it's duplicating work done by other processors. For example, you could have multiple pipelines, with different interval processors. And you could use the filter processor to select which metrics go where.

@RichieSams It is not immediately clear to me how to achieve this in a straightforward manner. The filter processor removes matching items from the pipeline. I would want to pass through only the specified items, exactly opposite of what the filter processor does. If I have multiple metrics with different sampling rates, I can filter out each one in a separate filter, but then all the other metrics would be passed through all of the pipelines and would be processed multiple times. It seems to me much more efficient to have a single processor that can achieve this in a single pass of the data. I need the opposite of the filter processor, to include only those metrics that are included in some condition.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/intervalprocessor: @RichieSams @sh0rez

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[pjanotti]

Pinging @RichieSams, @sh0rez as code owners.

[RichieSams]

It is not immediately clear to me how to achieve this in a straightforward manner. The filter processor removes matching items from the pipeline. I would want to pass through only the specified items, exactly opposite of what the filter processor does. If I have multiple metrics with different sampling rates, I can filter out each one in a separate filter, but then all the other metrics would be passed through all of the pipelines and would be processed multiple times. It seems to me much more efficient to have a single processor that can achieve this in a single pass of the data. I need the opposite of the filter processor, to include only those metrics that are included in some condition.

Sorry for the long delay. I agree the filter processor would be awkward. The routing processor looks better suited: https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/routingprocessor#tech-preview-opentelemetry-transformation-language-statements-as-routing-conditions

Ultimately, in the implementation I would basically have to do the same thing: Instantiating multiple inner interval processors, based on the input config.