spanmetrics connector has memory leak

[bripkens]

Component(s)

connector/spanmetrics

What happened?

Description

The spanmetrics connector has a memory leak that occurs when the cumulative temporality is used. The connectorImp#resourceMetrics map is only ever cleaned up in delta temporality.

opentelemetry-collector-contrib/connector/spanmetricsconnector/connector.go

Lines 253 to 271 in 628a6cb

	func (p *connectorImp) resetState() {
	// If delta metrics, reset accumulated data
	if p.config.GetAggregationTemporality() == pmetric.AggregationTemporalityDelta {
	p.resourceMetrics = make(map[resourceKey]*resourceMetrics)
	p.metricKeyToDimensions.Purge()
	p.startTimestamp = pcommon.NewTimestampFromTime(time.Now())
	} else {
	p.metricKeyToDimensions.RemoveEvictedItems()
	// Exemplars are only relevant to this batch of traces, so must be cleared within the lock
	if p.config.Histogram.Disable {
	return
	}
	for _, m := range p.resourceMetrics {
	m.histograms.Reset(true)
	}
	}
	}

Steps to Reproduce

1. Enable cumulative temporality
2. Observe many changing resources

Expected Result

connectorImp#resourceMetrics is eventually cleaned up.

Actual Result

connectorImp#resourceMetrics keeps growing over time.

Collector version

628a6cb

Environment information

Not relevant

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• connector/spanmetrics: @albertteoh

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[bryan-aguilar]

Thanks for the report @bripkens, are you interested in working on a fix?

[bripkens]

Fix proposal added via #28847

[mfilipe]

@bripkens do you think the comments I made here are related to your issue?

[bripkens]

Sounds related, yes. I'd argue however that just pruning histograms is not sufficient. See my (open) PR which would fix this: #28847