Skip to content

Conversation

@joshcoughlan
Copy link

@joshcoughlan joshcoughlan commented Nov 17, 2025

Summary

This PR adds per-sink proxy support to the azure_blob sink, matching the behavior already available for AWS S3 and GCP sinks.

Highlights:

  • Each Azure Blob sink instance now uses its own proxy settings from SinkContext (ProxyConfig), not environment variables.
  • The Azure SDK transport is configured via azure_core::http::ClientOptions and a dedicated, aliased reqwest 0.12 client (reqwest_0_12_24), so it does not impact the project-wide reqwest usage.
  • Supports HTTP and HTTPS proxies, including credentials embedded in the proxy URL (handled by reqwest::Proxy).
  • Honors no_proxy patterns at the sink level to bypass the proxy for matching Azure endpoints.

Vector configuration

Example of per-sink proxy settings for Azure Blob:

[sinks.my_azure_blob]
type = "azure_blob"
connection_string = "DefaultEndpointsProtocol=https;AccountName=<account>;AccountKey=<key>;EndpointSuffix=core.windows.net"
container_name = "my-logs"
healthcheck.enabled = true

[sinks.my_azure_blob.proxy]
enabled = true
http = "http://user:[email protected]:3128"
https = "http://user:[email protected]:3128"
# Bypass proxy for these hosts/domains
no_proxy = "localhost,.internal.example.com"

Notes:

  • Credentials in the proxy URL are supported by reqwest.
  • If the Azure endpoint matches no_proxy or the proxy is disabled, the sink makes direct connections and disables any system proxy auto-detection for this client.

How did you test this PR?

  • Build and lint:
    • Ensured the updated call sites compile (e.g., integration helpers that construct the client).
  • Functional checks:
    • Verified that the Azure client is constructed with a reqwest 0.12 transport and that proxies are applied only for this sink.
    • Sanity-tested no_proxy handling by targeting a host in the exclusion list to ensure direct connections are used.
    • Am running a compiled binary of this PR in a real-world environment.
  • Suggested reviewer test plan (optional):
    • Run Azurite locally and point an azure_blob sink at it with:
      • No proxy (baseline).
      • HTTP proxy set.
      • HTTPS proxy set.
      • A no_proxy entry matching the Azurite host to confirm bypass.
    • Confirm healthcheck behavior with valid/invalid credentials and unknown containers.

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
    • Suggested changelog: enhancement(azure_blob sink): Add per-sink proxy support via reqwest + azure_core::http::ClientOptions (HTTP/HTTPS proxies, honors no_proxy, supports proxy credentials).
  • No. A maintainer will apply the no-changelog label to this PR.

References

Notes

  • Implementation uses an aliased reqwest 0.12 client (reqwest_0_12_24) to avoid affecting other components that depend on a different reqwest version.
  • No environment variables are used; proxy configuration is per-sink via ProxyConfig.
  • If this PR modifies dependencies (Cargo.lock), I’ll run make build-licenses to refresh the license inventory.
  • This (or the parent SDK PR) fixes a bug introduced by Azure SDK version bump introduced regression in token refresh #23036 in which the documented SAS connection_string ("BlobEndpoint=https://mylogstorage.blob.core.windows.net/;SharedAccessSignature=generatedsastoken") doesn't work and fails with an error indicating it requires AccountName:
    ERROR vector::topology::builder: Configuration error. error=Sink "testing-azure_blob": Account name missing in connection string

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Nov 17, 2025
@joshcoughlan joshcoughlan marked this pull request as ready for review November 17, 2025 18:28
@joshcoughlan joshcoughlan requested a review from a team as a code owner November 17, 2025 18:28
@thomasqueirozb
Copy link
Contributor

Hi @joshcoughlan, I think this one should go in after #24255 right?

@thomasqueirozb thomasqueirozb added provider: azure Anything `azure` service provider related sink: azure_blob Anything `azure_blob` sink related meta: awaiting author Pull requests that are awaiting their author. labels Dec 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sinks Anything related to the Vector's sinks meta: awaiting author Pull requests that are awaiting their author. provider: azure Anything `azure` service provider related sink: azure_blob Anything `azure_blob` sink related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants