[receiver/kafkametrics] Fix metrics in kafkametricsreceiver (#4327) (open-telemetry#23632)

Utilize mdatagen to modify
certain metrics from gauge to non-monotonic sum, employing delta
aggregation. Adjust the test files to align with the sum test type and
update the expected.yaml file for integration testing purposes.

Co-authored-by: zeno-splunk <[email protected]>

Author: ZenoCC-Peng

.chloggen/fix-all-kafkametric-receiver-types.yaml

@@ -0,0 +1,20 @@
+# Use this changelog template to create an entry for release notes.
+# If your change doesn't affect end users, such as a test fix or a tooling change,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: bug_fix
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: receiver/kafkametricsreceiver
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Updates certain metrics in kafkametricsreceiver to function as non-monotonic sums.
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [4327]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: Update the metrics type in KafkaMetricsReceiver from "gauge" to "nonmonotonic sum". Changes metrics are, kafka.brokers, kafka.topic.partitions, kafka.partition.replicas, kafka.partition.replicas_in_sync, kafka.consumer_group.members.

receiver/kafkametricsreceiver/documentation.md

@@ -18,7 +18,7 @@ Number of brokers in the cluster.
 
 | Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
 | ---- | ----------- | ---------- | ----------------------- | --------- |
-| {brokers} | Sum | Int | Delta | false |
+| {brokers} | Sum | Int | Cumulative | false |
 
 ### kafka.consumer_group.lag
 
@@ -55,9 +55,9 @@ Current approximate sum of consumer group lag across all partitions of topic
 
 Count of members in the consumer group
 
-| Unit | Metric Type | Value Type |
-| ---- | ----------- | ---------- |
-| {members} | Gauge | Int |
+| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
+| ---- | ----------- | ---------- | ----------------------- | --------- |
+| {members} | Sum | Int | Cumulative | false |
 
 #### Attributes
 
@@ -130,9 +130,9 @@ Oldest offset of partition of topic
 
 Number of replicas for partition of topic
 
-| Unit | Metric Type | Value Type |
-| ---- | ----------- | ---------- |
-| {replicas} | Gauge | Int |
+| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
+| ---- | ----------- | ---------- | ----------------------- | --------- |
+| {replicas} | Sum | Int | Cumulative | false |
 
 #### Attributes
 
@@ -145,9 +145,9 @@ Number of replicas for partition of topic
 
 Number of synchronized replicas of partition
 
-| Unit | Metric Type | Value Type |
-| ---- | ----------- | ---------- |
-| {replicas} | Gauge | Int |
+| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
+| ---- | ----------- | ---------- | ----------------------- | --------- |
+| {replicas} | Sum | Int | Cumulative | false |
 
 #### Attributes
 
@@ -160,9 +160,9 @@ Number of synchronized replicas of partition
 
 Number of partitions in topic.
 
-| Unit | Metric Type | Value Type |
-| ---- | ----------- | ---------- |
-| {partitions} | Gauge | Int |
+| Unit | Metric Type | Value Type | Aggregation Temporality | Monotonic |
+| ---- | ----------- | ---------- | ----------------------- | --------- |
+| {partitions} | Sum | Int | Cumulative | false |
 
 #### Attributes
 

receiver/kafkametricsreceiver/internal/metadata/generated_metrics.go

@@ -26,14 +26,16 @@ metrics:
     sum:
       monotonic: false
       value_type: int
-      aggregation: delta
+      aggregation: cumulative
   #  topics scraper
   kafka.topic.partitions:
     enabled: true
     description: Number of partitions in topic.
     unit: "{partitions}"
-    gauge:
+    sum:
+      monotonic: false
       value_type: int
+      aggregation: cumulative
     attributes: [topic]
   kafka.partition.current_offset:
     enabled: true
@@ -53,23 +55,29 @@ metrics:
     enabled: true
     description: Number of replicas for partition of topic
     unit: "{replicas}"
-    gauge:
+    sum:
+      monotonic: false
       value_type: int
+      aggregation: cumulative
     attributes: [topic, partition]
   kafka.partition.replicas_in_sync:
     enabled: true
     description: Number of synchronized replicas of partition
     unit: "{replicas}"
-    gauge:
+    sum:
+      monotonic: false
       value_type: int
+      aggregation: cumulative
     attributes: [topic, partition]
   #  consumers scraper
   kafka.consumer_group.members:
     enabled: true
     description: Count of members in the consumer group
     unit: "{members}"
-    gauge:
+    sum:
+      monotonic: false
       value_type: int
+      aggregation: cumulative
     attributes: [group]
   kafka.consumer_group.offset:
     enabled: true

receiver/kafkametricsreceiver/internal/metadata/generated_metrics_test.go

@@ -4,7 +4,7 @@ resourceMetrics:
       - metrics:
           - description: Number of brokers in the cluster.
             sum:
-              aggregationTemporality: 1
+              aggregationTemporality: 2
               dataPoints:
                 - asInt: "1"
                   startTimeUnixNano: "1685063120199110000"