-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[service] Share log sampler core allocations with reflect magic #13107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
[service] Share log sampler core allocations with reflect magic #13107
Conversation
Codecov ReportAttention: Patch coverage is
❌ Your patch check has failed because the patch coverage (80.00%) is below the target coverage (95.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #13107 +/- ##
==========================================
- Coverage 91.48% 91.24% -0.24%
==========================================
Files 506 508 +2
Lines 28555 28712 +157
==========================================
+ Hits 26123 26198 +75
- Misses 1917 1995 +78
- Partials 515 519 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
My comment from today's 1.0 meeting: I think we want to avoid the following situations:
|
|
Context
PR #12617, which implemented the injection of component-identifying attributes into the
zap.Logger
provided to components, introduced significant additional memory use when the Collector's pipelines contain many components (#13014). This was because we would callzapcore.NewSamplerWithOptions
to wrap the specialized logger core of each Collector component, which allocates half a megabyte's worth of sampling counters.This problem was mitigated in #13015 by moving the sampling layer to a different location in the logger core hierarchy. This meant that Collector users that do not export their logs through OTLP and only use stdout-based logs no longer saw the memory increase.
Description
This PR aims to provide a better solution to this issue, by using the
reflect
library to clone zap's sampler core and set a new inner core, while reusing the counter allocation.(This may also be "more correct" from a sampling point of view, ie. we only have one global instance of the counters instead of one for console logs and one for each component's OTLP-exported logs, but I'm not sure if anyone noticed the difference anyway).
Link to tracking issue
Fixes #13014
Testing
A new test was added which checks that the log counters are shared between two sampler cores with different attributes.