Missing events count when switching to "use_otlp_decoding: true"

[stepbeta]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

I'm running some tests following #22696
Basically I'm sending logs to my vector (I'm using docker image 0.46.1-alpine) and while sending logs I deploy the new 0.51.1-alpine with the flag use_otlp_decoding: true

The result of this is that the events count disappears.

The above is the result of this query:

sum by (component_id) (rate(vector_component_received_events_total{component_kind="source", component_type="opentelemetry"}[1m]))

Has the metric changed between 0.46 and 0.51, by any chance? I cannot find a similar one

Configuration

api:
  address: 0.0.0.0:8686
  enabled: true
  playground: false
data_dir: /vector-data-dir
sinks:
  loki_opentelemetry:
    buffer:
      max_size: 3221225472
      type: disk
      when_full: drop_newest
    inputs:
    - enriched_logs_raw
    protocol:
      acknowledgements:
        enabled: false
      batch:
        max_events: 1
      encoding:
        codec: json
      framing:
        method: newline_delimited
      method: post
      request:
        headers:
          content-type: application/json
        retry_attempts: 20
      type: http
      uri: http://loki-gateway:80/otlp/v1/logs
    type: opentelemetry
sources:
  open_telemetry_raw:
    grpc:
      address: 0.0.0.0:4320
    http:
      address: 0.0.0.0:4319
    type: opentelemetry
    use_otlp_decoding: true
transforms:
  enriched_logs_raw:
    file: /vector-data-dir/vrl-scripts/enrichment-otlp/enrichment-otlp.vrl
    inputs:
    - open_telemetry_raw.logs
    type: remap

The enrichment-otlp.vrl only does the following

resLogs = []
for_each(array!(.resourceLogs)) -> |_index, rl|{
    ogResAttrs = get!(rl.resource.attributes, [])
    if is_null(ogResAttrs) {
        ogResAttrs = []
    }
    rl.resource.attributes = push(array!(ogResAttrs), {
        "key": "mylabel",
        "value": {
            "stringValue": "myvalue"
        }
    })

    resLogs = push(array(resLogs), rl)
}
.resourceLogs = resLogs

Version

0.51.1-alpine

Debug Output

Example Data

No response

Additional Context

Vector is running on kubernetes 1.33.5

References

• Provide OOTB mapping (with OTLP) for incoming Opentelemetry logs to outgoing Opentelemetry logs #22696

[thomasqueirozb]

Hi @stepbeta, can you confirm the metric shows 0 or is it way lower than expected? Due to the change I'd assume that events are now batched together and the metric should report lower values

[stepbeta]

Hi @thomasqueirozb, it's actually 0 as soon as I switch to the new version