Issue ReactiveX#354: Adapt prometheus metrics to the new format (ReactiveX#375)

* Introduce new prometheus metrics collector for bulkheads

* Introduce new prometheus metrics collector for circuit breakers

* Introduce new prometheus metrics collector for rate limiters

* Avoid label list creation on every collect

* Adapt/extend prometheus documentation

* Improve metrics descriptions

Author: voievodin

resilience4j-documentation/src/docs/asciidoc/addon_guides/prometheus.adoc

@@ -4,30 +4,43 @@
 
 Integration with https://github.com/prometheus/client_java[Prometheus simple client]
 
-Module provides exporters for `CircuitBreaker` and `RateLimiter` metrics.
+Module provides exporters for `CircuitBreaker`, `RateLimiter` and `Bulkhead` metrics.
+Every metric has `name` label indicating the relation to certain object, object category
+such as `bulkhead` is inlined to the metric name, so that the combination of metric name + `name` label
+uniquely identifies metric per backend. For example:
 
-For the circuit breaker library exports 2 metrics:
+```
+resilience4j_bulkhead_available_concurrent_calls{name="Backend1"}
+resilience4j_circuitbreaker_buffered_calls{name="Backend1"}
+resilience4j_ratelimiter_waiting_threads{name=Backend1"}
+```
 
-1. By state with default metric name `circuit_breaker_states` and label `state`:
+For every `CircuitBreaker` the library exports the following metrics:
 
-    - `closed`
-    - `open`
-    - `half_open`
+1. `resilience4j_circuitbreaker_state` - the state of the circuit breaker, possible values:
 
-2. By call result with default metric name `circuit_breaker_calls` and label `call_result`:
+    - 0 - CLOSED
+    - 1 - OPEN
+    - 2 - HALF_OPEN
 
-    - `successful`
-    - `failed`
-    - `not_permitted`
-    - `buffered`
-    - `buffered_max`
+2. `resilience4j_circuitbreaker_calls` - the number of recorded calls,
+the additional `kind` label indicates type of calls being recorded, the possible
+`kind` values are `successful`, `failed` or `not_permitted`.
 
-For the rate limiter following metric with default name `rate_limiter` and label `param` exported:
+3. `resilience4j_circuitbreaker_buffered_calls` - the number of buffered calls
 
-- `available_permissions`
-- `waiting_threads`
+4. `resilience4j_circuitbreaker_max_buffered_calls` - the maximum number of buffered calls
+
+For every `RateLimiter` the library exports the following metrics:
+
+1. `resilience4j_ratelimiter_available_permissions` - the number of available permissions
+2. `resilience4j_ratelimiter_waiting_threads` - the number of waiting threads
+
+For every `Bulkhead` the library exports the following metrics:
+
+1. `resilience4j_bulkhead_available_concurrent_calls` - the number of available concurrent calls
+2. `resilience4j_bulkhead_max_allowed_concurrent_calls` - the maximum number of allowed concurrent calls
 
-The names of the rate limiters and circuit breakers are exposed using label `name`.
 
 This module also provides `CallMeter` -- a composite metric to measure single call/request metrics such as:
     - execution time distribution,
@@ -56,7 +69,7 @@ final CircuitBreaker foo = circuitBreakerRegistry.circuitBreaker("foo");
 final CircuitBreaker boo = circuitBreakerRegistry.circuitBreaker("boo");
 
 // Registering metrics in prometeus CollectorRegistry
-collectorRegistry.register(CircuitBreakerExports.ofCircuitBreakerRegistry(circuitBreakerRegistry));
+collectorRegistry.register(CircuitBreakerMetricsCollector.ofCircuitBreakerRegistry(circuitBreakerRegistry));
 --
 
 ===== RateLimiter
@@ -71,10 +84,26 @@ final RateLimiter foo = rateLimiterRegistry.rateLimiter("foo");
 final RateLimiter boo = rateLimiterRegistry.rateLimiter("boo");
 
 // Registering metrics in prometeus CollectorRegistry
-collectorRegistry.register(RateLimiterExports.ofRateLimiterRegistry(rateLimiterRegistry));
+collectorRegistry.register(RateLimiterMetricsCollector.ofRateLimiterRegistry(rateLimiterRegistry));
+--
+
+===== Bulkhead
+
+[source,java]
+--
+final CollectorRegistry collectorRegistry = CollectorRegistry.defaultRegistry;
+
+final BulkheadRegistry bulkheadRegistry = BulkheadRegistry.ofDefaults();
+
+final Bulkhead foo = bulkheadRegistry.bulkhead("foo");
+final Bulkhead boo = bulkheadRegistry.bulkhead("boo");
+
+// Registering metrics in prometeus CollectorRegistry
+collectorRegistry.register(BulkheadMetricsCollector.ofBulkheadRegistry(rateLimiterRegistry));
 --
 
-For both it is possible to use just a collection of breakers and limiters instead of registry.
+Every collector exposes more methods for binding corresponding objects, for example,
+you can use suppliers, collections or even collect metrics for a single object.
 
 ===== Call Meter
 

resilience4j-prometheus/src/main/java/io/github/resilience4j/prometheus/BulkheadExports.java

@@ -34,7 +34,10 @@
 /**
  * An adapter from builtin {@link Bulkhead.Metrics} to prometheus
  * {@link io.prometheus.client.CollectorRegistry}.
+ *
+ * @deprecated use {@link io.github.resilience4j.prometheus.collectors.BulkheadMetricsCollector} instead
  */
+@Deprecated
 public class BulkheadExports extends Collector {
     private static final String DEFAULT_NAME = "resilience4j_bulkhead";
 

resilience4j-prometheus/src/main/java/io/github/resilience4j/prometheus/CircuitBreakerExports.java

@@ -37,7 +37,10 @@
  * {@link io.prometheus.client.CollectorRegistry}.
  *
  * Also exports {@link CircuitBreaker} state as a labeled metric
+ *
+ * @deprecated use {@link io.github.resilience4j.prometheus.collectors.CircuitBreakerMetricsCollector} instead.
  */
+@Deprecated
 public class CircuitBreakerExports extends Collector {
 
     private static final String DEFAULT_NAME = "resilience4j_circuitbreaker";

resilience4j-prometheus/src/main/java/io/github/resilience4j/prometheus/RateLimiterExports.java

@@ -34,7 +34,10 @@
 /**
  * An adapter from builtin {@link RateLimiter.Metrics} to prometheus
  * {@link io.prometheus.client.CollectorRegistry}.
+ *
+ * @deprecated use {@link io.github.resilience4j.prometheus.collectors.RateLimiterMetricsCollector} instead.
  */
+@Deprecated
 public class RateLimiterExports extends Collector {
     private static final String DEFAULT_NAME = "resilience4j_ratelimiter";
 

resilience4j-prometheus/src/main/java/io/github/resilience4j/prometheus/collectors/BulkheadMetricsCollector.java

@@ -0,0 +1,175 @@
+/*
+ * Copyright 2019 Yevhenii Voievodin
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *        http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package io.github.resilience4j.prometheus.collectors;
+
+import io.github.resilience4j.bulkhead.Bulkhead;
+import io.github.resilience4j.bulkhead.Bulkhead.Metrics;
+import io.github.resilience4j.bulkhead.BulkheadRegistry;
+import io.prometheus.client.Collector;
+import io.prometheus.client.GaugeMetricFamily;
+
+import java.util.List;
+import java.util.Objects;
+import java.util.function.Supplier;
+
+import static java.util.Arrays.asList;
+import static java.util.Collections.singletonList;
+import static java.util.Objects.requireNonNull;
+
+/** Collects bulkhead exposed {@link Metrics}. */
+public class BulkheadMetricsCollector extends Collector {
+
+    /**
+     * Creates a new collector with custom metric names and
+     * using given {@code supplier} as source of bulkheads.
+     *
+     * @param names    the custom metric names
+     * @param supplier the supplier of bulkheads, note that supplier will be called one every {@link #collect()}
+     */
+    public static BulkheadMetricsCollector ofSupplier(MetricNames names, Supplier<? extends Iterable<? extends Bulkhead>> supplier) {
+        return new BulkheadMetricsCollector(names, supplier);
+    }
+
+    /**
+     * Creates a new collector using given {@code supplier} as source of bulkheads.
+     *
+     * @param supplier the supplier of bulkheads, note that supplier will be called one very {@link #collect()}
+     */
+    public static BulkheadMetricsCollector ofSupplier(Supplier<? extends Iterable<? extends Bulkhead>> supplier) {
+        return new BulkheadMetricsCollector(MetricNames.ofDefaults(), supplier);
+    }
+
+    /**
+     * Creates a new collector using given {@code registry} as source of bulkheads.
+     *
+     * @param registry the source of bulkheads
+     */
+    public static BulkheadMetricsCollector ofBulkheadRegistry(BulkheadRegistry registry) {
+        return new BulkheadMetricsCollector(MetricNames.ofDefaults(), registry::getAllBulkheads);
+    }
+
+    /**
+     * Creates a new collector using given {@code bulkheads} iterable as source of bulkheads.
+     *
+     * @param bulkheads the source of bulkheads
+     */
+    public static BulkheadMetricsCollector ofIterable(Iterable<? extends Bulkhead> bulkheads) {
+        return new BulkheadMetricsCollector(MetricNames.ofDefaults(), () -> bulkheads);
+    }
+
+    /**
+     * Creates a new collector for a given {@code bulkhead}.
+     *
+     * @param bulkhead the bulkhead to collect metrics for
+     */
+    public static BulkheadMetricsCollector ofBulkhead(Bulkhead bulkhead) {
+        return ofIterable(singletonList(bulkhead));
+    }
+
+    private final MetricNames names;
+    private final Supplier<? extends Iterable<? extends Bulkhead>> bulkheadsSupplier;
+
+    private BulkheadMetricsCollector(MetricNames names, Supplier<? extends Iterable<? extends Bulkhead>> bulkheadsSupplier) {
+        this.names = Objects.requireNonNull(names);
+        this.bulkheadsSupplier = Objects.requireNonNull(bulkheadsSupplier);
+    }
+
+    @Override
+    public List<MetricFamilySamples> collect() {
+        GaugeMetricFamily availableCallsFamily = new GaugeMetricFamily(
+            names.getAvailableConcurrentCallsMetricName(),
+            "The number of available concurrent calls",
+            LabelNames.NAME
+        );
+        GaugeMetricFamily maxAllowedCallsFamily = new GaugeMetricFamily(
+            names.getMaxAllowedConcurrentCallsMetricName(),
+            "The maximum number of allowed concurrent calls",
+            LabelNames.NAME
+        );
+
+        for (Bulkhead bulkhead: bulkheadsSupplier.get()) {
+            List<String> labelValues = singletonList(bulkhead.getName());
+            availableCallsFamily.addMetric(labelValues, bulkhead.getMetrics().getAvailableConcurrentCalls());
+            maxAllowedCallsFamily.addMetric(labelValues, bulkhead.getMetrics().getMaxAllowedConcurrentCalls());
+        }
+
+        return asList(availableCallsFamily, maxAllowedCallsFamily);
+    }
+
+    /** Defines possible configuration for metric names. */
+    public static class MetricNames {
+
+        public static final String DEFAULT_BULKHEAD_AVAILABLE_CONCURRENT_CALLS_METRIC_NAME = "resilience4j_bulkhead_available_concurrent_calls";
+        public static final String DEFAULT_BULKHEAD_MAX_ALLOWED_CONCURRENT_CALLS_METRIC_NAME = "resilience4j_bulkhead_max_allowed_concurrent_calls";
+
+        /**
+         * Returns a builder for creating custom metric names.
+         * Note that names have default values, so only desired metrics can be renamed.
+         */
+        public static Builder custom() {
+            return new Builder();
+        }
+
+        /** Returns default metric names. */
+        public static MetricNames ofDefaults() {
+            return new MetricNames();
+        }
+
+        private String availableConcurrentCallsMetricName = DEFAULT_BULKHEAD_AVAILABLE_CONCURRENT_CALLS_METRIC_NAME;
+        private String maxAllowedConcurrentCallsMetricName = DEFAULT_BULKHEAD_MAX_ALLOWED_CONCURRENT_CALLS_METRIC_NAME;
+
+        private MetricNames() {}
+
+        /**
+         * Returns the metric name for bulkhead concurrent calls,
+         * defaults to {@value DEFAULT_BULKHEAD_AVAILABLE_CONCURRENT_CALLS_METRIC_NAME}.
+         */
+        public String getAvailableConcurrentCallsMetricName() {
+            return availableConcurrentCallsMetricName;
+        }
+
+        /**
+         * Returns the metric name for bulkhead max available concurrent calls,
+         * defaults to {@value DEFAULT_BULKHEAD_MAX_ALLOWED_CONCURRENT_CALLS_METRIC_NAME}.
+         */
+        public String getMaxAllowedConcurrentCallsMetricName() {
+            return maxAllowedConcurrentCallsMetricName;
+        }
+
+        /** Helps building custom instance of {@link MetricNames}. */
+        public static class Builder {
+
+            private final MetricNames metricNames = new MetricNames();
+
+            /** Overrides the default metric name {@value MetricNames#DEFAULT_BULKHEAD_AVAILABLE_CONCURRENT_CALLS_METRIC_NAME} with a given one. */
+            public Builder availableConcurrentCallsMetricName(String availableConcurrentCallsMetricNames) {
+                metricNames.availableConcurrentCallsMetricName = requireNonNull(availableConcurrentCallsMetricNames);
+                return this;
+            }
+
+            /** Overrides the default metric name {@value MetricNames#DEFAULT_BULKHEAD_MAX_ALLOWED_CONCURRENT_CALLS_METRIC_NAME} with a given one. */
+            public Builder maxAllowedConcurrentCallsMetricName(String maxAllowedConcurrentCallsMetricName) {
+                metricNames.maxAllowedConcurrentCallsMetricName = requireNonNull(maxAllowedConcurrentCallsMetricName);
+                return this;
+            }
+
+            /** Builds {@link MetricNames} instance. */
+            public MetricNames build() {
+                return metricNames;
+            }
+        }
+    }
+}