Skip to content

Counter measurement attributes aren't immutable #4610

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
P4rk opened this issue Jun 4, 2025 · 2 comments
Open

Counter measurement attributes aren't immutable #4610

P4rk opened this issue Jun 4, 2025 · 2 comments
Labels
bug Something isn't working metrics sdk Affects the SDK package.

Comments

@P4rk
Copy link

P4rk commented Jun 4, 2025

Describe your environment

OS: macOS Sequoia 15.4.1 (24E263)
Python version: 3.12.8
SDK version: 1.33.1
API version: 1.33.1

What happened?

Attributes for a datapoint are mutable after the datapoint has been recorded.
I discovered this using counters, I assume it affects all instruments, though.

If a reference to the mapping of the attributes passed into counter.add is kept, the datapoints attributes created by calling add can be changed.

Steps to Reproduce

from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import (
    ConsoleMetricExporter,
    PeriodicExportingMetricReader,
)

reader = PeriodicExportingMetricReader(ConsoleMetricExporter())
provider = MeterProvider(metric_readers=[reader])
meter = provider.get_meter("my-meter")

counter = meter.create_counter("my_counter")

# create a reference to the attributes
attributes = {"att": "value"}
counter.add(1, attributes)

# adjust the attributes after the first increment
attributes["att"] = "value2"
counter.add(1, attributes)

provider.force_flush()
provider.shutdown()

Expected Result

counter.add creates an immutable copy of the attributes used in recording the metrics.
expected datapoints, the attributes are different for each datapoint:

"data_points": [
    {
        "attributes": {
            "att": "value"
        },
        ...
        "value": 1,
        ...
    },
    {
        "attributes": {
            "att": "value2"
        },
        ...
        "value": 1,
        ...
    }
]

Actual Result

counter.add references the attribute object passed in without recording its value at the time of incrementation.
resulting in these datapoints, the attribute values are the same for each datapoint:

"data_points": [
    {
        "attributes": {
            "att": "value2"
        },
        ...
        "value": 1,
        ...
    },
    {
        "attributes": {
            "att": "value2"
        },
        ...
        "value": 1,
        ...
    }
]

Additional context

Downstream clients can fix this by passing in copies of their objects as a workaround. However, it was not obvious to me what was happening. My previous experience using Prometheus libraries suggests that holding a reference to the attributes object is ok.

It's also likely that I have misunderstood or misconfigured something. Please, feel free to point me in the correct direction if that is the case.

Please let me know if you agree if this is something that needs fixing and I'll see if I can submit a pull request.

Would you like to implement a fix?

Yes

@P4rk P4rk added the bug Something isn't working label Jun 4, 2025
@aabmass
Copy link
Member

aabmass commented Jun 6, 2025

Thanks for the clear repro. It is a little surprising to me as well, I would expect this case to work. The reason you see two separate data points is the SDK correctly creates an immutable key, but it just lazily stores a reference to the attributes for encoding later.


self._attributes_aggregation[aggr_key].aggregate(
measurement, should_sample_exemplar
)

However this seems buggy since it takes a shallow copy and attributes may contain a list nvm frozenset would just fail here which is a separate bug.

IMO we should fix this

@aabmass aabmass added sdk Affects the SDK package. metrics labels Jun 6, 2025
@aabmass
Copy link
Member

aabmass commented Jun 6, 2025

Would you like to implement a fix?

Yes

Would be very appreciated! We don't assign issues generally but please drop a comment if you start working on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working metrics sdk Affects the SDK package.
Projects
None yet
Development

No branches or pull requests

2 participants