subscriber: add arc-swap reload layer #3438
Open
+766
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
tracing_subscriber::reload::Layerwraps the innerLayer/Filterin anRwLock. In many common Tokio setups (typical async request handling on a runtime with a small, fixed number of worker threads), this does not usually show up as a problem: tasks are multiplexed onto a bounded number of OS threads and the extra read-side synchronization is often lost in the noise.However, the reload layer is also used in environments where there is real OS-thread parallelism at high log volume (e.g.,
spawn_blocking, rayon thread pools, or other dedicated thread pools) and where spans/events are emitted frequently from many threads at once. In those cases, the read-side overhead of anRwLockcan become a measurable and sometimes dominant cost, even when the filter is never reloaded.This PR addresses that niche-but-real performance cliff while keeping the existing behavior as the default.
Refs: #2658
Solution
reload::ArcSwapLayer/reload::ArcSwapHandlebehind a newreload-arc-swapfeature.arc_swap::ArcSwapfor a lock-free read path.Mutex(reloads are expected to be rare).tracing_core::callsite::rebuild_interest_cache()after updates (matchingreload::Layersemantics), and updateslogmax-level whentracing-logis enabled.L: Clonefor theLayerimpl (becauseon_layerneeds&mut L).This is additive:
reload::Layerremains unchanged.arc-swapis an optional dependency.ArcSwapLayersee no behavior or dependency changes.Benchmark
A small Criterion benchmark was added to compare baseline vs
reload::Layervsreload::ArcSwapLayer:cargo bench -p tracing-subscriber --bench reload --features reload-arc-swapNote: the multi-threaded benchmarks are intentionally constructed to create OS-thread parallelism (via
std::thread,tokio::spawn_blocking, and a Rayon pool) in order to exacerbate read-side synchronization contention. This is not representative of typical async request-handling on a small number of Tokio worker threads.Results (Apple M4 Pro, 14 cores, 48GB RAM; macOS 26.2;
rustc 1.92.0; Criterion point estimates; short run settings):reload::Layer(RwLock)reload::ArcSwapLayer(ArcSwap)single_threadedmultithreaded_16x1000(std::thread)tokio_spawn_blocking_16x1000rayon_16x1000(
*_16x1000means 16 workers/tasks; each emits 3 events (info/debug/trace) 1000x with anINFOfilter.)Testing
cargo test -p tracing-subscriber --all-featurescargo clippy -p tracing-subscriber --all-features -- -D warnings