Feature Request for Azure Monitor Metrics Data Plane Batch API Support #38651

ashishbans · 2025-03-15T10:13:11Z

Component(s)

receiver/azuremonitor

Is your feature request related to a problem? Please describe.

Currently, the OpenTelemetry Collector's Azure integration uses the standard Azure Resource Manager (ARM) APIs for metrics collection, which has significant limitations:

Low query limits (12,000 API calls per hour)
Requirement to make separate API calls for each resource
Higher risk of throttling when monitoring many Azure resources
Lower performance for large-scale monitoring scenarios
These limitations create challenges for users monitoring large Azure environments with multiple resources, as they frequently encounter throttling issues and inefficient data collection.

Describe the solution you'd like

Implement support for the new Azure Monitor Metrics Data Plane API in the OpenTelemetry Collector. This API offers several significant advantages:

Higher query limits: Up to 360,000 API calls per hour (30x improvement over ARM API)
Batch capabilities: Collect metrics for up to 50 resource IDs in a single API call
Improved efficiency: Reduced number of API calls for the same amount of metric data
Regional endpoints: Using region-specific endpoints for better performance
The implementation should allow users to:
Configure the collector to use the new Metrics Data Plane API instead of ARM APIs
Specify Azure resource IDs to be monitored in batches
Take advantage of the higher throughput and reduced throttling risk

Describe alternatives you've considered

No response

Additional context

The Azure Monitor Metrics Data Plane API is Generally Available and represents a significant improvement in Azure monitoring capabilities. Key technical details:

API endpoint format: {endpoint}/subscriptions/{subscriptionId}/metrics:getBatch?metricnamespace={metricnamespace}&metricnames={metricnames}&api-version=2023-10-01
Uses regional endpoints (e.g., https://eastus.metrics.monitor.azure.com)
Authenticated via Azure Active Directory with the scope https://metrics.monitor.azure.com/.default
Supports comprehensive filtering and aggregation capabilities similar to existing ARM APIs
This feature would significantly improve the experience for users monitoring large Azure environments and ensure the OpenTelemetry Collector remains a competitive option for Azure monitoring.
How to migrate from the metrics API to the getBatch API
Azure Monitor: Announcing GA of Azure Metrics Data Plane API
Azure Monitor Metrics Batch API Documentation

github-actions · 2025-03-15T10:13:26Z

Pinging code owners:

receiver/azuremonitor: @nslaughter @celian-garcia

See Adding Labels via Comments if you do not have permissions to add labels yourself.

celian-garcia · 2025-03-15T11:34:39Z

Thank you for this very detailed feature request. I'm actually planning to propose a PR from my fork that uses the getBatch API. This is the fork that we are using in my company for months already with good success!
I didn't know about this new API though. This is interesting, do you have go libraries associated?

ashishbans · 2025-03-17T06:15:53Z

Hey @celian-garcia , glad to know that your company already using getBatch API

It is same. Here is the go libraries for Batch API
https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/monitor/azquery

celian-garcia · 2025-03-17T07:08:34Z

Ok we're actually using github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azmetrics
For the batch metrics API.
For ref: https://github.com/celian-garcia/opentelemetry-collector-contrib/blob/50ac164dc41074b64727deb16b7a207461b54508/receiver/azuremonitorreceiver/scraper_batch.go

We followed this doc https://github.com/Azure/azure-sdk-for-go/blob/main/sdk%2Fmonitor%2Fquery%2FMIGRATION.md

I was more asking for the dataplane API but I'm realising, reading again the doc that if we migrate to github.com/Azure/azure-sdk-for-go/sdk/monitor/query/azmetrics for everything, we should be good. Am I right?

ashishbans · 2025-03-18T07:16:09Z

Hey @celian-garcia , yes, we are good and on right path,

FYI : also mentioned doc you shared
azmetrics contains a new data plane metrics feature that wasn't previously available in azquery. This allows users to query data from one or multiple resource IDs using a data plane endpoint. azmetrics will expand in the future when the service adds more features.

ashishbans · 2025-03-18T07:19:06Z

Also would you mind sharing the repo fork or pull request where this work is in progress. we also like to contribute to speedup the rollout

celian-garcia · 2025-03-18T12:09:19Z

Okay. Currently I want to finish this #37167 (multi subscription support) before to rebase and propose a PR from my fork's branch about getBatch

I believe that it's the same for the azmetrics usage. I'd be glad to have your support on this! Can you just wait for the multi subscription config to be merged before starting to migrate to azmetrics. As otherwise we'd have some conflcts in tests.
Also you're more than welcome to review the code that we've done on both multi subscription and getBatch.

celian-garcia · 2025-03-18T12:14:24Z

/label never-stale -needs-triage good-first-issue

ashishbans · 2025-03-18T12:16:20Z

@celian-garcia Sounds good. I'll go through it.
I found some references from another scrapping tool written in c#. feat: Provide support for batch scraping in Promitor

Using Batch Scraping with Azure Monitor API (Experimental)

celian-garcia · 2025-03-18T12:30:15Z

Yes we know it already and migrated from it to otel col. I don't remember why but I'd say that it's easier for us to code in Go and otel col is the perfect place to have it, benefiting also from a larger community. That could give us some ideas of improvements. though of course!

For now, since we're using our fork now with quite a good result, I will focus on merging it as is but we take of course any feedback!

celian-garcia · 2025-03-18T12:31:18Z

I'm not very confident about this label, I remove it.

/label -good-first-issue

celian-garcia · 2025-03-19T20:33:14Z

/label -good-first-issue

celian-garcia · 2025-03-24T09:40:55Z

@ashishbans I'm currently trying to add some tests for our fork that uses azmetrics package getBatch API.
I'm surprised to not find fake API but an internal unexposed mock that you are using in your tests internally.

Is there any plan to have a fake API like other clients?

internal mock:
https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/monitor/query/azmetrics/client_test.go#L16
fake API for the armmonitor client:
https://github.com/Azure/azure-sdk-for-go/blob/main/sdk/resourcemanager/monitor/armmonitor/fake/metrics_server.go

ashishbans · 2025-03-25T13:07:02Z

Hey @celian-garcia , Glad to see we have made good progress on batchApi feature request.
Microsoft engineer has raised PR for azmetrics fake support. it should be merged soon.

celian-garcia · 2025-03-25T14:50:09Z

@ashishbans oh great ! I thought it would take time when I saw the milestone so I started today implement my own fake server haha 😂

ashishbans · 2025-03-26T03:24:10Z

Hey @celian-garcia
FYI: fake support PR has been meeged now. Thanks

ashishbans · 2025-04-08T14:52:54Z

Hello @celian-garcia , I wanted to check if there are any updates on this feature request. Thank you."

celian-garcia · 2025-04-08T14:56:08Z

I was waiting for another PR about subscription name label. Now it's merge so I can rebase. I will try to make it before the end of the week.

…h API (#38895)  #### Description Let me introduce the Batch Metrics API https://learn.microsoft.com/en-us/rest/api/monitor/metrics-batch/batch?view=rest-monitor-2023-10-01&tabs=HTTP The advantages of using ``azmetrics`` batch metrics API over the ``armmonitor`` metrics API are described in details in the issue + in the README.md Note that this is a revival of a previous PR that we made last year, and that has not been followed up properly but we're using that forked code in the Amadeus company for months with good results.  #### Link to tracking issue Relates #38651  #### Testing ✅ Manual tests + mocked unit tests  #### Documentation ✅ Added config field documentation in README.md  Signed-off-by: Célian Garcia <[email protected]>

…h API (open-telemetry#38895)  #### Description Let me introduce the Batch Metrics API https://learn.microsoft.com/en-us/rest/api/monitor/metrics-batch/batch?view=rest-monitor-2023-10-01&tabs=HTTP The advantages of using ``azmetrics`` batch metrics API over the ``armmonitor`` metrics API are described in details in the issue + in the README.md Note that this is a revival of a previous PR that we made last year, and that has not been followed up properly but we're using that forked code in the Amadeus company for months with good results.  #### Link to tracking issue Relates open-telemetry#38651  #### Testing ✅ Manual tests + mocked unit tests  #### Documentation ✅ Added config field documentation in README.md  Signed-off-by: Célian Garcia <[email protected]>

celian-garcia · 2025-05-27T07:55:09Z

Hi @ashishbans, I believe that this PR #38895 is actually closing this issue. I will close it and let your reopen if needed.

celian-garcia · 2025-05-27T08:00:09Z

Okay I don't have rights to close 😄
at triager please close

celian-garcia · 2025-05-27T08:01:05Z

/label -never-stale needs-triage

ashishbans added enhancement New feature or request needs triage New item requiring triage labels Mar 15, 2025

github-actions bot added the receiver/azuremonitor label Mar 15, 2025

github-actions bot mentioned this issue Mar 18, 2025

Weekly Report: 2025-03-11 - 2025-03-18 #38702

Closed

ashishbans changed the title ~~Feature Request for Azure Monitor Metrics Data Plane API Support~~ Feature Request for Azure Monitor Metrics Data Plane Batch API Support Mar 18, 2025

github-actions bot added never stale Issues marked with this label will be never staled and automatically removed good first issue Good for newcomers and removed needs triage New item requiring triage labels Mar 18, 2025

github-actions bot removed the good first issue Good for newcomers label Mar 19, 2025

This was referenced Mar 24, 2025

fake support in azmetrics Azure/azure-sdk-for-go#24307

Closed

[receiver/azuremonitorreceiver] feat: option to use azmetrics getBatch API #38895

Merged

github-actions bot added needs triage New item requiring triage and removed never stale Issues marked with this label will be never staled and automatically removed labels May 27, 2025

github-actions bot mentioned this issue Jun 3, 2025

Weekly Report: 2025-05-27 - 2025-06-03 #40424

Open

Feature Request for Azure Monitor Metrics Data Plane Batch API Support #38651

Feature Request for Azure Monitor Metrics Data Plane Batch API Support #38651

Comments

ashishbans commented Mar 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Component(s)

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

github-actions bot commented Mar 15, 2025

Uh oh!

celian-garcia commented Mar 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ashishbans commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celian-garcia commented Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ashishbans commented Mar 18, 2025

Uh oh!

ashishbans commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celian-garcia commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celian-garcia commented Mar 18, 2025

Uh oh!

ashishbans commented Mar 18, 2025

Uh oh!

celian-garcia commented Mar 18, 2025

Uh oh!

celian-garcia commented Mar 18, 2025

Uh oh!

celian-garcia commented Mar 19, 2025

Uh oh!

celian-garcia commented Mar 24, 2025

Uh oh!

ashishbans commented Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celian-garcia commented Mar 25, 2025

Uh oh!

ashishbans commented Mar 26, 2025

Uh oh!

ashishbans commented Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celian-garcia commented Apr 8, 2025

Uh oh!

celian-garcia commented May 27, 2025

Uh oh!

celian-garcia commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

celian-garcia commented May 27, 2025

Uh oh!

ashishbans commented Mar 15, 2025 •

edited

Loading

celian-garcia commented Mar 15, 2025 •

edited

Loading

ashishbans commented Mar 17, 2025 •

edited

Loading

celian-garcia commented Mar 17, 2025 •

edited

Loading

ashishbans commented Mar 18, 2025 •

edited

Loading

celian-garcia commented Mar 18, 2025 •

edited

Loading

ashishbans commented Mar 25, 2025 •

edited

Loading

ashishbans commented Apr 8, 2025 •

edited

Loading

celian-garcia commented May 27, 2025 •

edited

Loading