feat(http/prom): introduce NewRecordStatusCode middleware
#4298
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cratelyn
changed the title
NewRecordStatusCode<N> middleware
cratelyn
commented
Nov 14, 2025
Comment on lines
+261
to
+268
| let ugh = RequestCancelled.into(); // XXX(kate) | ||
|
|
||
| stream_label.end_response(match eos { | ||
| EosRef::None => Ok(None), | ||
| EosRef::Trailers(trls) => Ok(Some(trls)), | ||
| EosRef::Error(error) => Err(error), | ||
| EosRef::Cancelled => Err(&ugh), | ||
| }); |
Member
Author
There was a problem hiding this comment.
note: this is one wrinkle that i would like to sort out before this is merged.
Member
Author
There was a problem hiding this comment.
see #4306, which explores altering StreamLabel::end_response() such that it accepts the end-of-stream object directly.
cratelyn
force-pushed
the
kate/decouple-record-response.pt-1.status-code-middleware
branch
from
34ce5ea to
ea7c47b
Compare
cratelyn
changed the base branch from
main
to
kate/app-outbound.refactor-backend-metrics-nits
Member
Author
|
this is based on #4299. |
cratelyn
commented
Nov 14, 2025
| /// | ||
| /// This generates [`LabelGrpcStatus`] labelers. | ||
| #[derive(Debug)] | ||
| #[derive(Clone, Debug)] |
Member
Author
There was a problem hiding this comment.
this ended up not being required in more recent iterations of #4300, but seemed like a reasonable and harmless addition.
i'm happy to (a) keep this, (b) back out of this change altogether, or (c) put it in a different pull request. what do you think?
cratelyn
changed the title
NewRecordStatusCode<N> middlewareNewRecordStatusCode middleware
olix0r
approved these changes
Nov 17, 2025
Base automatically changed from
kate/app-outbound.refactor-backend-metrics-nits
to
main
November 17, 2025 18:18
these types do not contain any internal state, and could thus be clonable. this commit makes `MkLabelGrpcStatus` and `MkLabelHttpStatus` clonable. Signed-off-by: katelyn martin <[email protected]>
this commit removes the uninhabited `()` state in `RequestCancelled`. this will allow our forthcoming status middleware to construct this type when a request is cancelled. Signed-off-by: katelyn martin <[email protected]>
this commit introduces a new submodule to `linkerd-http-prom`. this submodule includes a `NewRecordStatusCode<N>` middleware that can be used with the status code labeling implementations in `stream_label` to record Prometheus metrics counting response status codes for HTTP and gRPC traffic. a test suite demonstrates that this works properly for HTTP and gRPC traffic, including edge cases like errors or cancelled bodies. Signed-off-by: katelyn martin <[email protected]>
cratelyn
force-pushed
the
kate/decouple-record-response.pt-1.status-code-middleware
branch
from
ea7c47b to
b1b7fe2
Compare
Member
Author
|
rebased on |
cratelyn
deleted the
kate/decouple-record-response.pt-1.status-code-middleware
branch
cratelyn
added a commit
that referenced
this pull request
Nov 17, 2025
…e` (#4300) **nb:** this branch is based upon #4299, and #4298. in #4299 we made some prepatory adjustments to the outbound proxy's route-backend metrics layer, and in #4298 we introduced a new `linker-http-prom::status` metrics layer that can be used to count response status codes, in a manner that is agnostic with respect to the particular protocol that instrumented traffic is using. this branch performs a sequences of changes oriented towards two concrete goals: **1:** integrate the `NewRecordStatusCode` middleware into the outbound proxy's route and backend metrics layers, and **2:** remove status code measurement from the `NewRecordResponse` middleware. it's worth stating explicitly that this maintains the existing behavior of the outbound proxy, and that (1) and (2) must be performed at the same time to maintain parity. <img width="498" height="262" alt="image" src="https://github.com/user-attachments/assets/5030e7cd-33df-4cc3-b5cc-5f4db8247bd5" /> to demonstrate that parity is maintained, let's compare metrics from this branch against a snapshot of metrics exported in `main`. **🟰 comparing the metrics** metrics were scraped from the proxy, using a small traffic generation deployment that runs `curl` against a container running `http-echo`. a snapshot of the route and backend status/duration counters on main: ``` # HELP outbound_http_route_request_duration_seconds The time between request initialization and response completion. # TYPE outbound_http_route_request_duration_seconds histogram # UNIT outbound_http_route_request_duration_seconds seconds outbound_http_route_request_duration_seconds_sum{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 0.5594713199999999 outbound_http_route_request_duration_seconds_count{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 284 outbound_http_route_request_duration_seconds_bucket{le="0.05",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 284 outbound_http_route_request_duration_seconds_bucket{le="0.5",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 284 outbound_http_route_request_duration_seconds_bucket{le="1.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 284 outbound_http_route_request_duration_seconds_bucket{le="10.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 284 outbound_http_route_request_duration_seconds_bucket{le="+Inf",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 284 # HELP outbound_http_route_request_statuses Completed request-response streams. # TYPE outbound_http_route_request_statuses counter outbound_http_route_request_statuses_total{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname="",http_status="200",error=""} 284 # HELP outbound_http_route_backend_response_duration_seconds The time between request completion and response completion. # TYPE outbound_http_route_backend_response_duration_seconds histogram # UNIT outbound_http_route_backend_response_duration_seconds seconds outbound_http_route_backend_response_duration_seconds_sum{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 0.4626854930000003 outbound_http_route_backend_response_duration_seconds_count{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="0.025",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="0.05",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="0.1",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="0.25",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="0.5",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="1.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="10.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 outbound_http_route_backend_response_duration_seconds_bucket{le="+Inf",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 284 # HELP outbound_http_route_backend_response_statuses Completed responses. # TYPE outbound_http_route_backend_response_statuses counter outbound_http_route_backend_response_statuses_total{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name="",http_status="200",error=""} 284 ``` this was repeated after applying `config.linkerd.io/proxy-version` to use a proxy with these patches applied. a snapshot of the route and backend status/duration counters with these changes: ``` # HELP outbound_http_route_request_duration_seconds The time between request initialization and response completion. # TYPE outbound_http_route_request_duration_seconds histogram # UNIT outbound_http_route_request_duration_seconds seconds outbound_http_route_request_duration_seconds_sum{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 0.2557158190000001 outbound_http_route_request_duration_seconds_count{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 128 outbound_http_route_request_duration_seconds_bucket{le="0.05",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 128 outbound_http_route_request_duration_seconds_bucket{le="0.5",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 128 outbound_http_route_request_duration_seconds_bucket{le="1.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 128 outbound_http_route_request_duration_seconds_bucket{le="10.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 128 outbound_http_route_request_duration_seconds_bucket{le="+Inf",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 128 # HELP outbound_http_route_request_statuses Completed request-response streams. # TYPE outbound_http_route_request_statuses counter outbound_http_route_request_statuses_total{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname="",http_status="200",error=""} 128 # HELP outbound_http_route_backend_response_duration_seconds The time between request completion and response completion. # TYPE outbound_http_route_backend_response_duration_seconds histogram # UNIT outbound_http_route_backend_response_duration_seconds seconds outbound_http_route_backend_response_duration_seconds_sum{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 0.209469089 outbound_http_route_backend_response_duration_seconds_count{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="0.025",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="0.05",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="0.1",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="0.25",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="0.5",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="1.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="10.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 outbound_http_route_backend_response_duration_seconds_bucket{le="+Inf",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 128 # HELP outbound_http_route_backend_response_statuses Completed responses. # TYPE outbound_http_route_backend_response_statuses counter outbound_http_route_backend_response_statuses_total{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name="",http_status="200",error=""} 128 ``` because there are a large number of metrics and labels, a character-level diff of the two looked like so. as we'd hope, the only differences observed were values of particular time series. in other words, the same traffic yielded the same number of time series with the same labels. ``` diff --git a/recorded.txt b/recorded.txt index 41d4c53..5bf5c13 100644 --- a/recorded.txt +++ b/recorded.txt @@ -1,29 +1,29 @@ # HELP outbound_http_route_request_duration_seconds The time between request initialization and response completion. # TYPE outbound_http_route_request_duration_seconds histogram # UNIT outbound_http_route_request_duration_seconds seconds outbound_http_route_request_duration_seconds_sum{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} 0.{+2+}55[-94-]71[-3-]{+58+}19[-9999999-]{+0000001+} outbound_http_route_request_duration_seconds_count{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} {+1+}28[-4-] outbound_http_route_request_duration_seconds_bucket{le="0.05",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} {+1+}28[-4-] outbound_http_route_request_duration_seconds_bucket{le="0.5",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} {+1+}28[-4-] outbound_http_route_request_duration_seconds_bucket{le="1.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} {+1+}28[-4-] outbound_http_route_request_duration_seconds_bucket{le="10.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} {+1+}28[-4-] outbound_http_route_request_duration_seconds_bucket{le="+Inf",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname=""} {+1+}28[-4-] # HELP outbound_http_route_request_statuses Completed request-response streams. # TYPE outbound_http_route_request_statuses counter outbound_http_route_request_statuses_total{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",hostname="",http_status="200",error=""} {+1+}28[-4-] # HELP outbound_http_route_backend_response_duration_seconds The time between request completion and response completion. # TYPE outbound_http_route_backend_response_duration_seconds histogram # UNIT outbound_http_route_backend_response_duration_seconds seconds outbound_http_route_backend_response_duration_seconds_sum{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} 0.[-46-]2[-685-]{+09+}4{+6+}9[-30-]0[-00003-]{+89+} outbound_http_route_backend_response_duration_seconds_count{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="0.025",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="0.05",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="0.1",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="0.25",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="0.5",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="1.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="10.0",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] outbound_http_route_backend_response_duration_seconds_bucket{le="+Inf",parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name=""} {+1+}28[-4-] # HELP outbound_http_route_backend_response_statuses Completed responses. # TYPE outbound_http_route_backend_response_statuses counter outbound_http_route_backend_response_statuses_total{parent_group="core",parent_kind="Service",parent_namespace="simple-app",parent_name="simple-app-v1",parent_port="80",parent_section_name="",route_group="",route_kind="default",route_namespace="",route_name="http",backend_group="core",backend_kind="Service",backend_namespace="simple-app",backend_name="simple-app-v1",backend_port="80",backend_section_name="",http_status="200",error=""} {+1+}28[-4-] ``` --- * feat(app/outbound): add status counters to route metrics Signed-off-by: katelyn martin <[email protected]> * feat(app/outbound): add status counters to backend metrics Signed-off-by: katelyn martin <[email protected]> * feat(app/outbound): add route status counting middleware Signed-off-by: katelyn martin <[email protected]> * feat(app/outbound): add backend status counting middleware Signed-off-by: katelyn martin <[email protected]> * refactor(app/outbound): remove status counter from duration middleware Signed-off-by: katelyn martin <[email protected]> * nit(http/prom): nitpick a todo comment Signed-off-by: katelyn martin <[email protected]> --------- Signed-off-by: katelyn martin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
refactor(http/prom): status code labelers are
Clonethese types do not contain any internal state, and could thus be
clonable. this commit makes
MkLabelGrpcStatusandMkLabelHttpStatusclonable.Signed-off-by: katelyn martin [email protected]
refactor(http/prom): constructable
RequestCancelledthis commit removes the uninhabited
()state inRequestCancelled.this will allow our forthcoming status middleware to construct this
type when a request is cancelled.
Signed-off-by: katelyn martin [email protected]
feat(http/prom): add status counting middleware
this commit introduces a new submodule to
linkerd-http-prom.this submodule includes a
NewRecordStatusCode<N>middleware that canbe used with the status code labeling implementations in
stream_labelto record Prometheus metrics counting response status codes for HTTP and
gRPC traffic.
a test suite demonstrates that this works properly for HTTP and gRPC
traffic, including edge cases like errors or cancelled bodies.
Signed-off-by: katelyn martin [email protected]