feat: prom rw exporter add support for rw2 #38235

jmichalek132 · 2025-02-26T19:37:30Z

Description

PR for adding rw2 support in the prometheus remote write exporter.

Restart of a PR #35888

Link to tracking issue #33661

Fixes

Signed-off-by: Juraj Michalek <[email protected]>

jmichalek132 · 2025-03-04T21:46:56Z

Tests failing due to go leak on go.opencensus.io asked about it in this issue #30438 (comment)

ArthurSens

Great start! I know we previously talked about how it was OK to duplicate a lot of code between v1 and v2 because we would quickly clean it up with follow-up PRs. Seeing how difficult it was for us all to find time for this, it may be wise not to duplicate so much. What do you think?

Also, for the goleak test failures, you can add this to the metadata file and run the Makefile shenanigans :)

opentelemetry-collector-contrib/receiver/prometheusremotewritereceiver/metadata.yaml

Lines 10 to 16 in a9d8412

    
           tests: 
        
             config: 
        
             goleak: 
        
               ignore: 
        
                 top: 
        
                   # See https://github.com/census-instrumentation/opencensus-go/issues/1191 for more information. 
        
                   - "go.opencensus.io/stats/view.(*worker).start"

ArthurSens · 2025-03-10T17:47:59Z

exporter/prometheusremotewriteexporter/config.go

@@ -57,6 +58,9 @@ type Config struct {

 	// SendMetadata controls whether prometheus metadata will be generated and sent


Should we update this comment to mention that this boolean flag only applies to v1?

ArthurSens · 2025-03-10T17:54:44Z

exporter/prometheusremotewriteexporter/README.md

@@ -50,7 +50,7 @@ The following settings can be optionally configured:
  - *Note the following headers cannot be changed: `Content-Encoding`, `Content-Type`, `X-Prometheus-Remote-Write-Version`, and `User-Agent`.*
 - `namespace`: prefix attached to each exported metric name.
 - `add_metric_suffixes`: If set to false, type and unit suffixes will not be added to metrics. Default: true.
- `send_metadata`: If set to true, prometheus metadata will be generated and sent. Default: false.
+- `send_metadata`: If set to true, prometheus metadata will be generated and sent. Default: false. This option is ignored when using PRW 2.0, which already includes metadata.


Suggested change

- `send_metadata`: If set to true, prometheus metadata will be generated and sent. Default: false. This option is ignored when using PRW 2.0, which already includes metadata.

- `send_metadata`: If set to true, prometheus metadata will be generated and sent. Default: false. This option is ignored when using PRW 2.0, which always includes metadata.

ArthurSens · 2025-03-10T17:58:29Z

exporter/prometheusremotewriteexporter/exporter.go

+	if err := config.RemoteWriteProtoMsg.Validate(cfg.RemoteWriteProtoMsg); err != nil {
+		return nil, err
+	}


I'm confused about why we need this here. Don't we validate the Proto Message when in config.Validate() already? I'm not sure when the config validation happens though 🤔

ArthurSens · 2025-03-10T18:10:36Z

exporter/prometheusremotewriteexporter/exporter_test.go

+					if tt.enableSendingRW2 {
+						cfg.RemoteWriteProtoMsg = config.RemoteWriteProtoMsgV2
+						oldValue := enableSendingRW2FeatureGate.IsEnabled()
+						testutil.SetFeatureGateForTest(t, enableSendingRW2FeatureGate, tt.enableSendingRW2)
+						defer testutil.SetFeatureGateForTest(t, enableSendingRW2FeatureGate, oldValue)
+					} else {
+						cfg.RemoteWriteProtoMsg = config.RemoteWriteProtoMsgV1
+					}


We can get rid of the else here, easier to read :)

cfg.RemoteWriteProtoMsg = config.RemoteWriteProtoMsgV1 if tt.enableSendingRW2 { cfg.RemoteWriteProtoMsg = config.RemoteWriteProtoMsgV2 . . }

ArthurSens · 2025-03-10T18:13:34Z

exporter/prometheusremotewriteexporter/exporter_v2.go

+// exportV2 sends a Snappy-compressed writev2.Request containing writev2.TimeSeries to a remote write endpoint.
+func (prwe *prwExporter) exportV2(ctx context.Context, requests []*writev2.Request) error {
+	input := make(chan *writev2.Request, len(requests))


Hm, I'm still not sure how to do it, but it's not looking very nice that we're duplicating a lot of code between v1 and v2 now 😬

The exportv2 function is quite lengthy; it might be hard to maintain both of them in the long run. Have we explored other ideas?

They are literally the same, actually. The only difference is the signature 🤔, could we pass a byte array and a boolean that signals which RW version it represents?

I'm also open for other ideas in case anyone has any

atoulme · 2025-03-17T22:35:14Z

Please resolve conflicts and address feedback and mark the PR as ready for review. Thanks!

github-actions · 2025-04-01T05:21:56Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

github-actions · 2025-04-16T05:21:27Z

Closed as inactive. Feel free to reopen if this PR is still being worked on.

jmichalek132 added 2 commits February 26, 2025 20:35

feat: prom rw exporter add support for rw2

5d00da8

Signed-off-by: Juraj Michalek <[email protected]>

chore: add changelog entry

74864b3

Signed-off-by: Juraj Michalek <[email protected]>

github-actions bot added the exporter/prometheusremotewrite label Feb 26, 2025

github-actions bot requested review from Aneurysm9, ArthurSens, dashpole and rapphil February 26, 2025 19:37

Merge branch 'main' into jm-prom-rw-exporter-add-support-for-rw2-2

c7c73f8

jmichalek132 mentioned this pull request Feb 26, 2025

[exporter/prometheusremotewrite] Support Prometheus Remote-Write v2 #33661

Open

23 tasks

chore: ran go mod tidy

821bd79

Signed-off-by: Juraj Michalek <[email protected]>

github-actions bot added receiver/prometheus Prometheus receiver receiver/purefb labels Feb 26, 2025

github-actions bot requested review from chrroberts-pure, dgoscn and jpkrohling February 26, 2025 19:51

chore: make gotidy and make generate

1d6daed

Signed-off-by: Juraj Michalek <[email protected]>

github-actions bot added receiver/purefa receiver/simpleprometheus labels Feb 26, 2025

github-actions bot requested a review from fatsheep9146 February 26, 2025 20:11

jmichalek132 added 4 commits March 4, 2025 17:36

Merge branch 'main' into jm-prom-rw-exporter-add-support-for-rw2-2

51035f7

chore: make gotidy and make generate

3629b12

Signed-off-by: Juraj Michalek <[email protected]>

Merge branch 'main' into jm-prom-rw-exporter-add-support-for-rw2-2

4f28f70

chore: make gotidy and make generate

c94868d

Signed-off-by: Juraj Michalek <[email protected]>

github-actions bot added the receiver/tcpcheck label Mar 4, 2025

github-actions bot requested review from atoulme, chengchuanpeng and michael-burt March 4, 2025 17:28

jmichalek132 added 2 commits March 4, 2025 22:01

Merge branch 'main' into jm-prom-rw-exporter-add-support-for-rw2-2

c59a5a4

chore: fix failing tests

e9e2a1d

Signed-off-by: Juraj Michalek <[email protected]>

jmichalek132 marked this pull request as ready for review March 4, 2025 21:28

jmichalek132 requested a review from a team as a code owner March 4, 2025 21:28

github-actions bot assigned bogdandrutu Mar 4, 2025

Merge branch 'main' into jm-prom-rw-exporter-add-support-for-rw2-2

09ceb1f

ArthurSens reviewed Mar 10, 2025

View reviewed changes

atoulme marked this pull request as draft March 17, 2025 22:34

github-actions bot added the Stale label Apr 1, 2025

github-actions bot closed this Apr 16, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: prom rw exporter add support for rw2 #38235

feat: prom rw exporter add support for rw2 #38235

Uh oh!

jmichalek132 commented Feb 26, 2025 •

edited

Loading

Uh oh!

jmichalek132 commented Mar 4, 2025

Uh oh!

ArthurSens left a comment

Uh oh!

ArthurSens Mar 10, 2025

Uh oh!

ArthurSens Mar 10, 2025

Uh oh!

ArthurSens Mar 10, 2025

Uh oh!

ArthurSens Mar 10, 2025

Uh oh!

ArthurSens Mar 10, 2025

Uh oh!

ArthurSens Mar 10, 2025

Uh oh!

atoulme commented Mar 17, 2025

Uh oh!

github-actions bot commented Apr 1, 2025

Uh oh!

github-actions bot commented Apr 16, 2025

Uh oh!

Uh oh!

	tests:
	config:
	goleak:
	ignore:
	top:
	# See https://github.com/census-instrumentation/opencensus-go/issues/1191 for more information.
	- "go.opencensus.io/stats/view.(*worker).start"

		@@ -57,6 +58,9 @@ type Config struct {

		// SendMetadata controls whether prometheus metadata will be generated and sent

	- `send_metadata`: If set to true, prometheus metadata will be generated and sent. Default: false. This option is ignored when using PRW 2.0, which already includes metadata.
	- `send_metadata`: If set to true, prometheus metadata will be generated and sent. Default: false. This option is ignored when using PRW 2.0, which always includes metadata.

feat: prom rw exporter add support for rw2 #38235

feat: prom rw exporter add support for rw2 #38235

Uh oh!

Conversation

jmichalek132 commented Feb 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Link to tracking issue #33661

Uh oh!

jmichalek132 commented Mar 4, 2025

Uh oh!

ArthurSens left a comment

Choose a reason for hiding this comment

Uh oh!

ArthurSens Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

ArthurSens Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

ArthurSens Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

ArthurSens Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

ArthurSens Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

ArthurSens Mar 10, 2025

Choose a reason for hiding this comment

Uh oh!

atoulme commented Mar 17, 2025

Uh oh!

github-actions bot commented Apr 1, 2025

Uh oh!

github-actions bot commented Apr 16, 2025

Uh oh!

Uh oh!

jmichalek132 commented Feb 26, 2025 •

edited

Loading