exporter/kafka: Expose `ProducerLinger` configuration option

[marclop]

Component(s)

exporter/kafka

Is your feature request related to a problem? Please describe.

Starting on version v.1.20.0, franz-go now defaults to using a 10ms ProducerLinger (changelog). This is expected to increase throughput by allowing a bit more data to be buffered rather than eagerly creating produce requests when records are received.

While this is better than before, it would be useful to allow the value of ProducerLinger to be configured by users since different use cases and deployments may require different linger values.

Describe the solution you'd like

Add a new producer_linger configuration option with a 10ms default value.

Describe alternatives you've considered

Do nothing and rely on the default ProducerLinger=10ms.

Additional context

See twmb/franz-go#1072 for more details on the analysis for the ProducerLinger value.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[marclop]

I'm happy to work on this right away.

[github-actions]

Pinging code owners:

• exporter/kafka: @pavolloffay @MovieStoreGuy @axw @paulojmdias

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[axw]

SGTM! Thanks @marclop, assigned to you