Incorrect/misleading Documentation for receiver/kafkametrics TLS Configuration

[enrico-agile]

Component(s)

receiver/kafkametrics

Describe the issue you're reporting

Today I was trying to configure the kafkametrics receiver to monitor a cluster that is using TLS w/ plain_text authentication.
In the end I've managed to make it work but I've admittedly lost a lot of time trying to configure it.

The documentation currently states:

• auth.tls.ca_file: path to the CA cert. For a client this verifies the server certificate. Should only be used if insecure is set to true
• auth.tls.insecure: (default = false) Disable verifying the server's certificate chain and host name (InsecureSkipVerify in the tls config)

However, after reviewing the code (here and here), I found discrepancies:

• insecure: Does not disable certificate verification, it actually disables TLS entirely
• ca_file: The statement Should only be used if insecure is set to true is misleading - In reality, configuring a ca_file implicitly neutralizes an insecure=true, enabling TLS even if insecure is set to false
• insecure_skip_verify: This is the actual parameter that disables certificate verification, but it is undocumented

Additionally, the way the configuration is structured is a little confusing. There are valid Kafka configurations with plain_text authentication (like my case) or even Kerberos authentication that require TLS encryption. The only way to enable TLS in these scenarios is to pass a non-nil auth.tls configuration, as shown in my example configuration below, even if I'm not actually using mTLS authn.

The following configuration successfully enabled monitoring in my setup:

receivers:
  kafkametrics:
    brokers:
      - broker0:9092
      - broker1:9092
      - broker3:9093
    protocol_version: 2.0.0
    scrapers:
      - brokers
      - topics
      - consumers
    auth:
      plain_text:
        username: username
        password: password
      tls:
        insecure: false

[github-actions]

Pinging code owners:

• receiver/kafkametrics: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

@enrico-agile thanks for opening this. I've recently been doing some refactoring across the Kafka components, including receiver/kafkametrics, and I also found the auth bits in general and the TLS part specifically to be confusing.

I'm thinking of making a couple of changes:

• move tls out of auth, to the top level of the receiver config
• deprecate and remove auth.plain_text, since you can do the same thing with sasl by setting the mechanism to PLAIN

So your example config above would look like:

receivers:
  kafkametrics:
    brokers:
      - broker0:9092
      - broker1:9092
      - broker3:9093
    protocol_version: 2.0.0
    scrapers:
      - brokers
      - topics
      - consumers
    tls: {} # enable TLS with defaults, i.e. verify server cert & DNS
    auth:
      sasl:
        mechanism: PLAIN
        username: username
        password: password

As far as documentation goes, I would be inclined to mostly remove it and refer readers to the TLS config docs, like the OTLP receiver does -- see https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver#advanced-configuration

How does this sound to you?

[enrico-agile]

Thanks @axw for addressing this!
The proposed solution looks perfect 👍

[axw]

#39115 should fix this