-
Notifications
You must be signed in to change notification settings - Fork 475
ci(iast): fix flaky test [backport 4.0] #15753
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The urlparse function was imported inside the async function view_iast_ssrf_secure, which caused a race condition in multiprocess mode where: - The IAST validator wrapper for urlparse is set up at module startup - But importing inside a function can bypass or inconsistently apply the wrapper - This caused the secure mark to not be applied reliably, leading to false SSRF vulnerability reports Flaky tests IDs: DD_1PGYGI DD_ONKXDT DD_U2V880 DD_954GJO DD_O236KS (cherry picked from commit 381404b) Signed-off-by: Alberto Vara <[email protected]>
|
|
abcaadd to
2e18fa2
Compare
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 240 ± 2 ms. The average import time from base is: 244 ± 3 ms. The import time difference between this PR and base is: -4.6 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
Performance SLOsComparing candidate backport-15749-to-4.0 (2e18fa2) with baseline 4.0 (03b5f92) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 0.382µs (SLO: <10.000µs 📉 -96.2%) vs baseline: ~same Memory: ✅ 37.825MB (SLO: <41.500MB -8.9%) vs baseline: +4.6% ✅ add_inplace_aspectTime: ✅ 0.379µs (SLO: <10.000µs 📉 -96.2%) vs baseline: -0.6% Memory: ✅ 37.925MB (SLO: <41.500MB -8.6%) vs baseline: +5.2% ✅ add_inplace_noaspectTime: ✅ 0.285µs (SLO: <10.000µs 📉 -97.1%) vs baseline: -1.6% Memory: ✅ 37.823MB (SLO: <41.500MB -8.9%) vs baseline: +4.3% ✅ add_noaspectTime: ✅ 0.355µs (SLO: <10.000µs 📉 -96.5%) vs baseline: ~same Memory: ✅ 37.843MB (SLO: <41.500MB -8.8%) vs baseline: +4.7% ✅ bytearray_aspectTime: ✅ 1.352µs (SLO: <10.000µs 📉 -86.5%) vs baseline: +1.2% Memory: ✅ 37.960MB (SLO: <41.500MB -8.5%) vs baseline: +5.0% ✅ bytearray_extend_aspectTime: ✅ 1.513µs (SLO: <10.000µs 📉 -84.9%) vs baseline: +0.4% Memory: ✅ 37.932MB (SLO: <41.500MB -8.6%) vs baseline: +5.2% ✅ bytearray_extend_noaspectTime: ✅ 0.616µs (SLO: <10.000µs 📉 -93.8%) vs baseline: -0.2% Memory: ✅ 37.788MB (SLO: <41.500MB -8.9%) vs baseline: +4.6% ✅ bytearray_noaspectTime: ✅ 0.485µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.2% Memory: ✅ 37.919MB (SLO: <41.500MB -8.6%) vs baseline: +4.7% ✅ bytes_aspectTime: ✅ 1.431µs (SLO: <10.000µs 📉 -85.7%) vs baseline: 📈 +11.1% Memory: ✅ 37.717MB (SLO: <41.500MB -9.1%) vs baseline: +4.1% ✅ bytes_noaspectTime: ✅ 0.494µs (SLO: <10.000µs 📉 -95.1%) vs baseline: +0.2% Memory: ✅ 37.795MB (SLO: <41.500MB -8.9%) vs baseline: +4.8% ✅ bytesio_aspectTime: ✅ 1.333µs (SLO: <10.000µs 📉 -86.7%) vs baseline: -0.2% Memory: ✅ 37.967MB (SLO: <41.500MB -8.5%) vs baseline: +5.1% ✅ bytesio_noaspectTime: ✅ 0.500µs (SLO: <10.000µs 📉 -95.0%) vs baseline: -0.4% Memory: ✅ 37.840MB (SLO: <41.500MB -8.8%) vs baseline: +4.7% ✅ capitalize_aspectTime: ✅ 0.738µs (SLO: <10.000µs 📉 -92.6%) vs baseline: ~same Memory: ✅ 38.084MB (SLO: <41.500MB -8.2%) vs baseline: +5.1% ✅ capitalize_noaspectTime: ✅ 0.432µs (SLO: <10.000µs 📉 -95.7%) vs baseline: -0.6% Memory: ✅ 37.896MB (SLO: <41.500MB -8.7%) vs baseline: +5.1% ✅ casefold_aspectTime: ✅ 0.737µs (SLO: <10.000µs 📉 -92.6%) vs baseline: +0.1% Memory: ✅ 37.915MB (SLO: <41.500MB -8.6%) vs baseline: +4.7% ✅ casefold_noaspectTime: ✅ 0.372µs (SLO: <10.000µs 📉 -96.3%) vs baseline: +0.7% Memory: ✅ 37.760MB (SLO: <41.500MB -9.0%) vs baseline: +4.4% ✅ decode_aspectTime: ✅ 0.723µs (SLO: <10.000µs 📉 -92.8%) vs baseline: -1.4% Memory: ✅ 37.926MB (SLO: <41.500MB -8.6%) vs baseline: +5.3% ✅ decode_noaspectTime: ✅ 0.423µs (SLO: <10.000µs 📉 -95.8%) vs baseline: +0.2% Memory: ✅ 37.766MB (SLO: <41.500MB -9.0%) vs baseline: +4.6% ✅ encode_aspectTime: ✅ 0.713µs (SLO: <10.000µs 📉 -92.9%) vs baseline: ~same Memory: ✅ 37.850MB (SLO: <41.500MB -8.8%) vs baseline: +5.2% ✅ encode_noaspectTime: ✅ 0.403µs (SLO: <10.000µs 📉 -96.0%) vs baseline: +0.6% Memory: ✅ 37.885MB (SLO: <41.500MB -8.7%) vs baseline: +4.8% ✅ format_aspectTime: ✅ 3.430µs (SLO: <10.000µs 📉 -65.7%) vs baseline: +0.6% Memory: ✅ 37.856MB (SLO: <41.500MB -8.8%) vs baseline: +4.9% ✅ format_map_aspectTime: ✅ 3.619µs (SLO: <10.000µs 📉 -63.8%) vs baseline: -0.6% Memory: ✅ 37.891MB (SLO: <41.500MB -8.7%) vs baseline: +4.8% ✅ format_map_noaspectTime: ✅ 0.821µs (SLO: <10.000µs 📉 -91.8%) vs baseline: +0.7% Memory: ✅ 37.905MB (SLO: <41.500MB -8.7%) vs baseline: +5.1% ✅ format_noaspectTime: ✅ 0.594µs (SLO: <10.000µs 📉 -94.1%) vs baseline: -3.3% Memory: ✅ 37.828MB (SLO: <41.500MB -8.8%) vs baseline: +4.8% ✅ index_aspectTime: ✅ 0.345µs (SLO: <10.000µs 📉 -96.5%) vs baseline: +0.6% Memory: ✅ 37.718MB (SLO: <41.500MB -9.1%) vs baseline: +4.7% ✅ index_noaspectTime: ✅ 0.315µs (SLO: <10.000µs 📉 -96.8%) vs baseline: +0.4% Memory: ✅ 37.731MB (SLO: <41.500MB -9.1%) vs baseline: +4.5% ✅ join_aspectTime: ✅ 1.279µs (SLO: <10.000µs 📉 -87.2%) vs baseline: -0.7% Memory: ✅ 37.687MB (SLO: <41.500MB -9.2%) vs baseline: +3.9% ✅ join_noaspectTime: ✅ 0.537µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +1.1% Memory: ✅ 37.757MB (SLO: <41.500MB -9.0%) vs baseline: +4.1% ✅ ljust_aspectTime: ✅ 2.573µs (SLO: <20.000µs 📉 -87.1%) vs baseline: +0.3% Memory: ✅ 37.883MB (SLO: <41.500MB -8.7%) vs baseline: +5.0% ✅ ljust_noaspectTime: ✅ 0.406µs (SLO: <10.000µs 📉 -95.9%) vs baseline: -1.6% Memory: ✅ 37.779MB (SLO: <41.500MB -9.0%) vs baseline: +4.5% ✅ lower_aspectTime: ✅ 2.300µs (SLO: <10.000µs 📉 -77.0%) vs baseline: +0.7% Memory: ✅ 37.880MB (SLO: <41.500MB -8.7%) vs baseline: +5.1% ✅ lower_noaspectTime: ✅ 0.372µs (SLO: <10.000µs 📉 -96.3%) vs baseline: +0.9% Memory: ✅ 37.785MB (SLO: <41.500MB -9.0%) vs baseline: +4.5% ✅ lstrip_aspectTime: ✅ 2.504µs (SLO: <20.000µs 📉 -87.5%) vs baseline: 📈 +15.0% Memory: ✅ 37.823MB (SLO: <41.500MB -8.9%) vs baseline: +4.5% ✅ lstrip_noaspectTime: ✅ 0.388µs (SLO: <10.000µs 📉 -96.1%) vs baseline: +0.3% Memory: ✅ 37.683MB (SLO: <41.500MB -9.2%) vs baseline: +4.5% ✅ modulo_aspectTime: ✅ 0.972µs (SLO: <10.000µs 📉 -90.3%) vs baseline: -0.6% Memory: ✅ 37.921MB (SLO: <41.500MB -8.6%) vs baseline: +5.4% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 1.470µs (SLO: <10.000µs 📉 -85.3%) vs baseline: -1.2% Memory: ✅ 37.840MB (SLO: <41.500MB -8.8%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 0.949µs (SLO: <10.000µs 📉 -90.5%) vs baseline: -1.1% Memory: ✅ 38.022MB (SLO: <41.500MB -8.4%) vs baseline: +5.6% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 1.169µs (SLO: <10.000µs 📉 -88.3%) vs baseline: ~same Memory: ✅ 37.831MB (SLO: <41.500MB -8.8%) vs baseline: +5.0% ✅ modulo_noaspectTime: ✅ 0.668µs (SLO: <10.000µs 📉 -93.3%) vs baseline: -2.0% Memory: ✅ 37.804MB (SLO: <41.500MB -8.9%) vs baseline: +4.4% ✅ replace_aspectTime: ✅ 5.362µs (SLO: <10.000µs 📉 -46.4%) vs baseline: +9.1% Memory: ✅ 37.790MB (SLO: <41.500MB -8.9%) vs baseline: +4.7% ✅ replace_noaspectTime: ✅ 0.464µs (SLO: <10.000µs 📉 -95.4%) vs baseline: +0.4% Memory: ✅ 37.759MB (SLO: <41.500MB -9.0%) vs baseline: +4.6% ✅ repr_aspectTime: ✅ 0.942µs (SLO: <10.000µs 📉 -90.6%) vs baseline: ~same Memory: ✅ 37.781MB (SLO: <41.500MB -9.0%) vs baseline: +4.6% ✅ repr_noaspectTime: ✅ 0.458µs (SLO: <10.000µs 📉 -95.4%) vs baseline: +2.0% Memory: ✅ 37.821MB (SLO: <41.500MB -8.9%) vs baseline: +5.0% ✅ rstrip_aspectTime: ✅ 1.858µs (SLO: <20.000µs 📉 -90.7%) vs baseline: +0.4% Memory: ✅ 37.959MB (SLO: <41.500MB -8.5%) vs baseline: +4.7% ✅ rstrip_noaspectTime: ✅ 0.383µs (SLO: <10.000µs 📉 -96.2%) vs baseline: +0.5% Memory: ✅ 37.848MB (SLO: <41.500MB -8.8%) vs baseline: +4.5% ✅ slice_aspectTime: ✅ 0.482µs (SLO: <10.000µs 📉 -95.2%) vs baseline: -0.4% Memory: ✅ 37.829MB (SLO: <41.500MB -8.8%) vs baseline: +4.5% ✅ slice_noaspectTime: ✅ 0.450µs (SLO: <10.000µs 📉 -95.5%) vs baseline: -0.2% Memory: ✅ 37.809MB (SLO: <41.500MB -8.9%) vs baseline: +4.7% ✅ stringio_aspectTime: ✅ 1.788µs (SLO: <10.000µs 📉 -82.1%) vs baseline: +5.8% Memory: ✅ 37.757MB (SLO: <41.500MB -9.0%) vs baseline: +4.7% ✅ stringio_noaspectTime: ✅ 0.928µs (SLO: <10.000µs 📉 -90.7%) vs baseline: ~same Memory: ✅ 37.830MB (SLO: <41.500MB -8.8%) vs baseline: +5.0% ✅ strip_aspectTime: ✅ 2.163µs (SLO: <20.000µs 📉 -89.2%) vs baseline: ~same Memory: ✅ 37.960MB (SLO: <41.500MB -8.5%) vs baseline: +5.1% ✅ strip_noaspectTime: ✅ 0.388µs (SLO: <10.000µs 📉 -96.1%) vs baseline: -1.0% Memory: ✅ 37.783MB (SLO: <41.500MB -9.0%) vs baseline: +4.6% ✅ swapcase_aspectTime: ✅ 2.469µs (SLO: <10.000µs 📉 -75.3%) vs baseline: ~same Memory: ✅ 37.799MB (SLO: <41.500MB -8.9%) vs baseline: +5.0% ✅ swapcase_noaspectTime: ✅ 0.540µs (SLO: <10.000µs 📉 -94.6%) vs baseline: +0.3% Memory: ✅ 37.847MB (SLO: <41.500MB -8.8%) vs baseline: +4.6% ✅ title_aspectTime: ✅ 2.399µs (SLO: <10.000µs 📉 -76.0%) vs baseline: -1.2% Memory: ✅ 38.012MB (SLO: <41.500MB -8.4%) vs baseline: +5.0% ✅ title_noaspectTime: ✅ 0.506µs (SLO: <10.000µs 📉 -94.9%) vs baseline: +1.2% Memory: ✅ 37.848MB (SLO: <41.500MB -8.8%) vs baseline: +4.5% ✅ translate_aspectTime: ✅ 3.329µs (SLO: <10.000µs 📉 -66.7%) vs baseline: -0.1% Memory: ✅ 37.786MB (SLO: <41.500MB -8.9%) vs baseline: +4.4% ✅ translate_noaspectTime: ✅ 1.040µs (SLO: <10.000µs 📉 -89.6%) vs baseline: -0.3% Memory: ✅ 38.007MB (SLO: <41.500MB -8.4%) vs baseline: +5.0% ✅ upper_aspectTime: ✅ 2.322µs (SLO: <10.000µs 📉 -76.8%) vs baseline: ~same Memory: ✅ 37.946MB (SLO: <41.500MB -8.6%) vs baseline: +4.9% ✅ upper_noaspectTime: ✅ 0.370µs (SLO: <10.000µs 📉 -96.3%) vs baseline: -0.6% Memory: ✅ 37.805MB (SLO: <41.500MB -8.9%) vs baseline: +4.8% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 4.299µs (SLO: <10.000µs 📉 -57.0%) vs baseline: +2.5% Memory: ✅ 37.827MB (SLO: <41.000MB -7.7%) vs baseline: +4.7% ✅ ospathbasename_noaspectTime: ✅ 1.077µs (SLO: <10.000µs 📉 -89.2%) vs baseline: -0.4% Memory: ✅ 37.945MB (SLO: <41.000MB -7.5%) vs baseline: +5.0% ✅ ospathjoin_aspectTime: ✅ 6.830µs (SLO: <10.000µs 📉 -31.7%) vs baseline: 📈 +14.3% Memory: ✅ 37.965MB (SLO: <41.000MB -7.4%) vs baseline: +5.1% ✅ ospathjoin_noaspectTime: ✅ 2.283µs (SLO: <10.000µs 📉 -77.2%) vs baseline: -0.7% Memory: ✅ 37.867MB (SLO: <41.000MB -7.6%) vs baseline: +4.8% ✅ ospathnormcase_aspectTime: ✅ 3.425µs (SLO: <10.000µs 📉 -65.7%) vs baseline: +1.2% Memory: ✅ 37.906MB (SLO: <41.000MB -7.5%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 0.570µs (SLO: <10.000µs 📉 -94.3%) vs baseline: +0.3% Memory: ✅ 37.827MB (SLO: <41.000MB -7.7%) vs baseline: +4.6% ✅ ospathsplit_aspectTime: ✅ 4.751µs (SLO: <10.000µs 📉 -52.5%) vs baseline: -0.2% Memory: ✅ 37.808MB (SLO: <41.000MB -7.8%) vs baseline: +4.8% ✅ ospathsplit_noaspectTime: ✅ 1.588µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -0.1% Memory: ✅ 37.847MB (SLO: <41.000MB -7.7%) vs baseline: +4.9% ✅ ospathsplitdrive_aspectTime: ✅ 3.636µs (SLO: <10.000µs 📉 -63.6%) vs baseline: +0.5% Memory: ✅ 37.965MB (SLO: <41.000MB -7.4%) vs baseline: +5.1% ✅ ospathsplitdrive_noaspectTime: ✅ 0.698µs (SLO: <10.000µs 📉 -93.0%) vs baseline: -0.7% Memory: ✅ 37.906MB (SLO: <41.000MB -7.5%) vs baseline: +5.1% ✅ ospathsplitext_aspectTime: ✅ 4.531µs (SLO: <10.000µs 📉 -54.7%) vs baseline: -0.1% Memory: ✅ 37.886MB (SLO: <41.000MB -7.6%) vs baseline: +5.0% ✅ ospathsplitext_noaspectTime: ✅ 1.379µs (SLO: <10.000µs 📉 -86.2%) vs baseline: +0.2% Memory: ✅ 37.867MB (SLO: <41.000MB -7.6%) vs baseline: +4.6% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.251µs (SLO: <20.000µs 📉 -83.7%) vs baseline: 📈 +11.2% Memory: ✅ 34.465MB (SLO: <35.500MB -2.9%) vs baseline: +4.8% ✅ 1-count-metrics-100-timesTime: ✅ 199.442µs (SLO: <220.000µs -9.3%) vs baseline: -0.7% Memory: ✅ 34.485MB (SLO: <35.500MB -2.9%) vs baseline: +4.9% ✅ 1-distribution-metric-1-timesTime: ✅ 3.338µs (SLO: <20.000µs 📉 -83.3%) vs baseline: +1.2% Memory: ✅ 34.465MB (SLO: <35.500MB -2.9%) vs baseline: +4.9% ✅ 1-distribution-metrics-100-timesTime: ✅ 216.076µs (SLO: <230.000µs -6.1%) vs baseline: -0.8% Memory: ✅ 34.426MB (SLO: <35.500MB -3.0%) vs baseline: +4.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.149µs (SLO: <20.000µs 📉 -89.3%) vs baseline: +0.1% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +5.0% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.500µs (SLO: <150.000µs -9.0%) vs baseline: -0.3% Memory: ✅ 34.465MB (SLO: <35.500MB -2.9%) vs baseline: +4.8% ✅ 1-rate-metric-1-timesTime: ✅ 3.048µs (SLO: <20.000µs 📉 -84.8%) vs baseline: -1.4% Memory: ✅ 34.426MB (SLO: <35.500MB -3.0%) vs baseline: +4.6% ✅ 1-rate-metrics-100-timesTime: ✅ 213.064µs (SLO: <250.000µs 📉 -14.8%) vs baseline: -0.6% Memory: ✅ 34.485MB (SLO: <35.500MB -2.9%) vs baseline: +4.6% ✅ 100-count-metrics-100-timesTime: ✅ 19.937ms (SLO: <22.000ms -9.4%) vs baseline: -0.5% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +4.7% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.286ms (SLO: <2.300ms 🟡 -0.6%) vs baseline: -0.2% Memory: ✅ 34.465MB (SLO: <35.500MB -2.9%) vs baseline: +4.9% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.401ms (SLO: <1.550ms -9.6%) vs baseline: -0.1% Memory: ✅ 34.485MB (SLO: <35.500MB -2.9%) vs baseline: +4.7% ✅ 100-rate-metrics-100-timesTime: ✅ 2.184ms (SLO: <2.550ms 📉 -14.4%) vs baseline: -0.7% Memory: ✅ 34.505MB (SLO: <35.500MB -2.8%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.608µs (SLO: <20.000µs 📉 -77.0%) vs baseline: +0.4% Memory: ✅ 34.564MB (SLO: <35.500MB -2.6%) vs baseline: +5.0% ✅ flush-100-metricsTime: ✅ 175.407µs (SLO: <250.000µs 📉 -29.8%) vs baseline: ~same Memory: ✅ 34.426MB (SLO: <35.500MB -3.0%) vs baseline: +4.8% ✅ flush-1000-metricsTime: ✅ 2.116ms (SLO: <2.500ms 📉 -15.4%) vs baseline: -0.2% Memory: ✅ 35.173MB (SLO: <36.500MB -3.6%) vs baseline: +4.7%
|
Backport 381404b from #15749 to 4.0.
The urlparse function was imported inside the async function view_iast_ssrf_secure, which caused a race condition in multiprocess mode where:
Flaky tests IDs: DD_1PGYGI DD_ONKXDT DD_U2V880 DD_954GJO DD_O236KS