Closed
Description
What's wrong?
i can see spanmetrics count is growing over the period, once grafana-agent restart then only i can see some drop on traces_spanmetrics_calls_total.
i've configured metrics_flush_interval = "5s" and i'm using docker.io/grafana/agent:v0.39.2
issue is similar to grafana/agent#5271
Steps to reproduce
config for otelcol.connector.spanmetrics
// The Spanmetrics Connector will generate RED metrics based on the incoming trace span data.
otelcol.connector.spanmetrics "tracemetrics" {
// The namespace explicit adds a prefix to all the generated span metrics names.
// In this case, we'll ensure they match as closely as possible those generated by Tempo.
namespace = "traces.spanmetrics"
// Each extra dimension (metrics label) to be added to the generated metrics from matching span attributes. These
// need to be defined with a name and optionally a default value (in the following cases, we do not want a default
// value if the span attribute is not present).
dimension {
name = "http.method"
}
dimension {
name = "http.status_code"
}
metrics_flush_interval = "5s"
// A histogram block must be present, either explicitly defining bucket values or via an exponential block.
// We do the latter here.
histogram {
explicit {
buckets= ["10ms", "50ms", "100ms", "200ms", "400ms", "1s"]
}
}
// The exemplar block is added to ensure we generate exemplars for traces on relevant metric values.
exemplars {
enabled = true
}
// Generated metrics data is in OTLP format. We send this data to the OpenTelemetry Prometheus exporter to ensure
// it gets transformed into Prometheus format data.
output {
metrics = [otelcol.processor.filter.default.input]
}
}
System information
No response
Software version
agent v0.39.2
Configuration
No response
Logs
No response