Prometheus upgrade to support native histogram aggregations in promxy

[rishabhkumar92]

Starting this thread to discuss upgrading prometheus for promxy to support native histograms.
Promxy is currently based on Prometheus 2.37 (last commit on main branch) which is 2 year old and a lot of improvement and new features has been introduced since then and mainly support around native histogram.

• Is there active development planned ongoing to address this? If not, I would like to use this as discussion forum and upgrade Prometheus dependency.
• Are there any known issues/risk I should be aware of before upgrading library as there are some custom implementations on top of main branch changes.

cc: @jacksontj

[jacksontj]

Is there active development planned ongoing to address this?

There isn't as of now; but it is probably about time to upgrade the prometheus fork -- as you mentioned it has been a while.

Are there any known issues/risk I should be aware of before upgrading library as there are some custom implementations on top of main branch changes.

This has historically been a bit of a pain to do the upgrade for a couple main reasons:

1. the web UI is a bunch of codegen/etc. that isn't built to be embeddable; so this requires making those changes in the fork such that the UI can be embedded into promxy (different project)

2. NodeSelector: the NodeSelector feature is entirely unique to this fork which is the "secret sauce" that makes promxy work. I have attempted to get this upstreamed before but there was negative interest. This historically is somewhat time consuming as you have to take the time to review changes to the promql package and see how they would impact this feature.

3. Unknowns: the biggest annoyance so far is that each time there is an update there will undoubtedly be some change that breaks compatibility that we'll have to sort out.

None of these are show-stoppers; they all just require time. I'd be happy to review a branch/PR to prometheus if you'd like to take a stab at it! Otherwise when I get a free day or two I can take a crack at it (not sure when that'll be, this is in my free time) :)

[jacksontj]

One other thought on this; I generally don't pick the "latest" branch of prometheus as there historically have been some odd bugs that are hard to deal with. So I usually pick a release where its been "stable" for a while. I see that prometheus now has a concept of LTS releases -- so maybe that would be a good choice to base off of (not that we'd get more support of the fork; but that if there were an issue it might be easier to get a fix upstream).

[rishabhkumar92]

I was able to get prometheus to be a compilable and address any test failures with your custom changes here. Can you take a quick look into that change if you see anything off. I am trying to make this updated version work with promxy now.

[rishabhkumar92]

@jacksontj I have started to work on upgrading prometheus and adding native histogram support. Here is how I proceeded with the changes:

• Created fork from prometheus main branch here and cherry-picked NodeReplacer and other related changes Prometheus upgrade + custom Changes. Can you help reviewing this.
• Upgraded Promxy to include custom prometheus changes above and added native histogram translation logic Promxy upgrade and native histogram. I will be adding test coverage on this path next month or early January.

Let me know your thoughts on this, once Promxy PR is ready for review, I can use your help in reviewing the change

[rishabhkumar92]

cc: @jacksontj to review the above proposed approach.

[rishabhkumar92]

bumping it up @jacksontj