-
Notifications
You must be signed in to change notification settings - Fork 475
ci: migrate lib injection tests to gitlab #15760
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
|
brettlangdon
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
questions and nits, nothing really blocking
Performance SLOsComparing candidate emmett.butler/lib-injection-tests (dc6ab0f) with baseline main (7c7d690) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 18.041µs (SLO: <20.000µs -9.8%) vs baseline: 📈 +21.0% Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +5.0% ✅ add_inplace_aspectTime: ✅ 15.031µs (SLO: <20.000µs 📉 -24.8%) vs baseline: +0.4% Memory: ✅ 42.605MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +5.1% ✅ add_inplace_noaspectTime: ✅ 0.345µs (SLO: <10.000µs 📉 -96.6%) vs baseline: +0.9% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ add_noaspectTime: ✅ 0.544µs (SLO: <10.000µs 📉 -94.6%) vs baseline: -1.2% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ bytearray_aspectTime: ✅ 18.051µs (SLO: <30.000µs 📉 -39.8%) vs baseline: ~same Memory: ✅ 42.684MB (SLO: <43.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ bytearray_extend_aspectTime: ✅ 23.844µs (SLO: <30.000µs 📉 -20.5%) vs baseline: -0.2% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ bytearray_extend_noaspectTime: ✅ 2.748µs (SLO: <10.000µs 📉 -72.5%) vs baseline: +0.2% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +5.0% ✅ bytearray_noaspectTime: ✅ 1.468µs (SLO: <10.000µs 📉 -85.3%) vs baseline: -0.9% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +5.0% ✅ bytes_aspectTime: ✅ 16.697µs (SLO: <20.000µs 📉 -16.5%) vs baseline: -0.1% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ bytes_noaspectTime: ✅ 1.426µs (SLO: <10.000µs 📉 -85.7%) vs baseline: +0.1% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ bytesio_aspectTime: ✅ 55.853µs (SLO: <70.000µs 📉 -20.2%) vs baseline: -0.2% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ bytesio_noaspectTime: ✅ 3.267µs (SLO: <10.000µs 📉 -67.3%) vs baseline: -0.8% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.1% ✅ capitalize_aspectTime: ✅ 14.698µs (SLO: <20.000µs 📉 -26.5%) vs baseline: +0.2% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ capitalize_noaspectTime: ✅ 2.572µs (SLO: <10.000µs 📉 -74.3%) vs baseline: -1.3% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.9% ✅ casefold_aspectTime: ✅ 14.701µs (SLO: <20.000µs 📉 -26.5%) vs baseline: -0.5% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.9% ✅ casefold_noaspectTime: ✅ 3.149µs (SLO: <10.000µs 📉 -68.5%) vs baseline: -0.4% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.1% ✅ decode_aspectTime: ✅ 15.641µs (SLO: <30.000µs 📉 -47.9%) vs baseline: +0.3% Memory: ✅ 42.526MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ decode_noaspectTime: ✅ 1.590µs (SLO: <10.000µs 📉 -84.1%) vs baseline: -1.6% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.6% ✅ encode_aspectTime: ✅ 18.190µs (SLO: <30.000µs 📉 -39.4%) vs baseline: 📈 +22.0% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +5.1% ✅ encode_noaspectTime: ✅ 1.512µs (SLO: <10.000µs 📉 -84.9%) vs baseline: +1.5% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.8% ✅ format_aspectTime: ✅ 171.385µs (SLO: <200.000µs 📉 -14.3%) vs baseline: +0.4% Memory: ✅ 42.644MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +5.0% ✅ format_map_aspectTime: ✅ 190.938µs (SLO: <200.000µs -4.5%) vs baseline: ~same Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +4.6% ✅ format_map_noaspectTime: ✅ 3.807µs (SLO: <10.000µs 📉 -61.9%) vs baseline: +1.4% Memory: ✅ 42.684MB (SLO: <43.250MB 🟡 -1.3%) vs baseline: +5.1% ✅ format_noaspectTime: ✅ 3.153µs (SLO: <10.000µs 📉 -68.5%) vs baseline: +0.4% Memory: ✅ 42.526MB (SLO: <43.250MB 🟡 -1.7%) vs baseline: +4.6% ✅ index_aspectTime: ✅ 15.424µs (SLO: <20.000µs 📉 -22.9%) vs baseline: -0.2% Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.9% ✅ index_noaspectTime: ✅ 0.468µs (SLO: <10.000µs 📉 -95.3%) vs baseline: +1.0% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.8% ✅ join_aspectTime: ✅ 17.088µs (SLO: <20.000µs 📉 -14.6%) vs baseline: -1.3% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ join_noaspectTime: ✅ 1.538µs (SLO: <10.000µs 📉 -84.6%) vs baseline: -1.7% Memory: ✅ 42.566MB (SLO: <43.250MB 🟡 -1.6%) vs baseline: +4.8% ✅ ljust_aspectTime: ✅ 20.880µs (SLO: <30.000µs 📉 -30.4%) vs baseline: ~same Memory: ✅ 42.546MB (SLO: <43.250MB 🟡 -1.6%) vs baseline: +4.5% ✅ ljust_noaspectTime: ✅ 2.733µs (SLO: <10.000µs 📉 -72.7%) vs baseline: +1.9% Memory: ✅ 42.625MB (SLO: <43.250MB 🟡 -1.4%) vs baseline: +5.0% ✅ lower_aspectTime: ✅ 17.975µs (SLO: <30.000µs 📉 -40.1%) vs baseline: +0.4% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.0% ✅ lower_noaspectTime: ✅ 2.416µs (SLO: <10.000µs 📉 -75.8%) vs baseline: -0.8% Memory: ✅ 42.546MB (SLO: <43.250MB 🟡 -1.6%) vs baseline: +4.8% ✅ lstrip_aspectTime: ✅ 17.730µs (SLO: <30.000µs 📉 -40.9%) vs baseline: +0.2% Memory: ✅ 42.585MB (SLO: <43.250MB 🟡 -1.5%) vs baseline: +4.8% ✅ lstrip_noaspectTime: ✅ 1.863µs (SLO: <10.000µs 📉 -81.4%) vs baseline: +0.3% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ modulo_aspectTime: ✅ 166.502µs (SLO: <200.000µs 📉 -16.7%) vs baseline: +0.5% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +5.1% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 180.409µs (SLO: <200.000µs -9.8%) vs baseline: +3.4% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.6% ✅ modulo_aspect_for_bytesTime: ✅ 168.761µs (SLO: <200.000µs 📉 -15.6%) vs baseline: +0.1% Memory: ✅ 42.703MB (SLO: <43.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 171.938µs (SLO: <200.000µs 📉 -14.0%) vs baseline: -0.4% Memory: ✅ 42.743MB (SLO: <43.500MB 🟡 -1.7%) vs baseline: +5.1% ✅ modulo_noaspectTime: ✅ 3.690µs (SLO: <10.000µs 📉 -63.1%) vs baseline: ~same Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ replace_aspectTime: ✅ 212.325µs (SLO: <300.000µs 📉 -29.2%) vs baseline: ~same Memory: ✅ 42.644MB (SLO: <44.000MB -3.1%) vs baseline: +4.8% ✅ replace_noaspectTime: ✅ 2.908µs (SLO: <10.000µs 📉 -70.9%) vs baseline: +0.2% Memory: ✅ 42.526MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ repr_aspectTime: ✅ 1.419µs (SLO: <10.000µs 📉 -85.8%) vs baseline: -0.4% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ repr_noaspectTime: ✅ 0.525µs (SLO: <10.000µs 📉 -94.7%) vs baseline: +0.2% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +5.1% ✅ rstrip_aspectTime: ✅ 19.057µs (SLO: <30.000µs 📉 -36.5%) vs baseline: -0.3% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ rstrip_noaspectTime: ✅ 2.027µs (SLO: <10.000µs 📉 -79.7%) vs baseline: +4.4% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ slice_aspectTime: ✅ 15.937µs (SLO: <20.000µs 📉 -20.3%) vs baseline: -0.2% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.8% ✅ slice_noaspectTime: ✅ 0.597µs (SLO: <10.000µs 📉 -94.0%) vs baseline: -1.2% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +5.1% ✅ stringio_aspectTime: ✅ 54.390µs (SLO: <80.000µs 📉 -32.0%) vs baseline: -0.6% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.9% ✅ stringio_noaspectTime: ✅ 3.677µs (SLO: <10.000µs 📉 -63.2%) vs baseline: +0.7% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.9% ✅ strip_aspectTime: ✅ 17.681µs (SLO: <20.000µs 📉 -11.6%) vs baseline: -0.3% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ strip_noaspectTime: ✅ 1.854µs (SLO: <10.000µs 📉 -81.5%) vs baseline: -1.5% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ swapcase_aspectTime: ✅ 18.582µs (SLO: <30.000µs 📉 -38.1%) vs baseline: -0.4% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ swapcase_noaspectTime: ✅ 2.776µs (SLO: <10.000µs 📉 -72.2%) vs baseline: -1.3% Memory: ✅ 42.526MB (SLO: <43.500MB -2.2%) vs baseline: +4.8% ✅ title_aspectTime: ✅ 18.322µs (SLO: <30.000µs 📉 -38.9%) vs baseline: ~same Memory: ✅ 42.644MB (SLO: <43.000MB 🟡 -0.8%) vs baseline: +5.1% ✅ title_noaspectTime: ✅ 2.666µs (SLO: <10.000µs 📉 -73.3%) vs baseline: -0.9% Memory: ✅ 42.507MB (SLO: <43.500MB -2.3%) vs baseline: +4.4% ✅ translate_aspectTime: ✅ 24.552µs (SLO: <30.000µs 📉 -18.2%) vs baseline: 📈 +19.1% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.1% ✅ translate_noaspectTime: ✅ 4.352µs (SLO: <10.000µs 📉 -56.5%) vs baseline: -0.1% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ upper_aspectTime: ✅ 18.039µs (SLO: <30.000µs 📉 -39.9%) vs baseline: -0.3% Memory: ✅ 42.526MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ upper_noaspectTime: ✅ 2.442µs (SLO: <10.000µs 📉 -75.6%) vs baseline: -0.4% Memory: ✅ 42.546MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.159µs (SLO: <10.000µs 📉 -48.4%) vs baseline: 📈 +21.4% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +4.7% ✅ ospathbasename_noaspectTime: ✅ 4.330µs (SLO: <10.000µs 📉 -56.7%) vs baseline: +0.8% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ ospathjoin_aspectTime: ✅ 6.250µs (SLO: <10.000µs 📉 -37.5%) vs baseline: -0.2% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ ospathjoin_noaspectTime: ✅ 6.327µs (SLO: <10.000µs 📉 -36.7%) vs baseline: -0.1% Memory: ✅ 42.644MB (SLO: <43.500MB 🟡 -2.0%) vs baseline: +5.1% ✅ ospathnormcase_aspectTime: ✅ 3.565µs (SLO: <10.000µs 📉 -64.3%) vs baseline: +0.3% Memory: ✅ 42.625MB (SLO: <43.500MB -2.0%) vs baseline: +5.0% ✅ ospathnormcase_noaspectTime: ✅ 3.630µs (SLO: <10.000µs 📉 -63.7%) vs baseline: +0.9% Memory: ✅ 42.566MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ ospathsplit_aspectTime: ✅ 4.886µs (SLO: <10.000µs 📉 -51.1%) vs baseline: -0.5% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +5.0% ✅ ospathsplit_noaspectTime: ✅ 5.017µs (SLO: <10.000µs 📉 -49.8%) vs baseline: +0.7% Memory: ✅ 42.605MB (SLO: <43.500MB -2.1%) vs baseline: +4.9% ✅ ospathsplitdrive_aspectTime: ✅ 3.763µs (SLO: <10.000µs 📉 -62.4%) vs baseline: ~same Memory: ✅ 42.526MB (SLO: <43.500MB -2.2%) vs baseline: +4.7% ✅ ospathsplitdrive_noaspectTime: ✅ 0.746µs (SLO: <10.000µs 📉 -92.5%) vs baseline: -1.0% Memory: ✅ 42.487MB (SLO: <43.500MB -2.3%) vs baseline: +4.5% ✅ ospathsplitext_aspectTime: ✅ 4.593µs (SLO: <10.000µs 📉 -54.1%) vs baseline: -0.6% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +4.8% ✅ ospathsplitext_noaspectTime: ✅ 4.653µs (SLO: <10.000µs 📉 -53.5%) vs baseline: -0.1% Memory: ✅ 42.585MB (SLO: <43.500MB -2.1%) vs baseline: +5.0% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.363µs (SLO: <20.000µs 📉 -83.2%) vs baseline: 📈 +10.9% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.4% ✅ 1-count-metrics-100-timesTime: ✅ 198.962µs (SLO: <220.000µs -9.6%) vs baseline: -2.3% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 1-distribution-metric-1-timesTime: ✅ 3.300µs (SLO: <20.000µs 📉 -83.5%) vs baseline: -2.6% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.6% ✅ 1-distribution-metrics-100-timesTime: ✅ 213.431µs (SLO: <230.000µs -7.2%) vs baseline: -1.9% Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.3% ✅ 1-gauge-metric-1-timesTime: ✅ 2.204µs (SLO: <20.000µs 📉 -89.0%) vs baseline: -1.0% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.6% ✅ 1-gauge-metrics-100-timesTime: ✅ 135.729µs (SLO: <150.000µs -9.5%) vs baseline: -0.4% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.0% ✅ 1-rate-metric-1-timesTime: ✅ 3.138µs (SLO: <20.000µs 📉 -84.3%) vs baseline: -1.9% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.9% ✅ 1-rate-metrics-100-timesTime: ✅ 211.660µs (SLO: <250.000µs 📉 -15.3%) vs baseline: -1.8% Memory: ✅ 34.937MB (SLO: <35.500MB 🟡 -1.6%) vs baseline: +4.9% ✅ 100-count-metrics-100-timesTime: ✅ 19.869ms (SLO: <22.000ms -9.7%) vs baseline: ~same Memory: ✅ 34.780MB (SLO: <35.500MB -2.0%) vs baseline: +4.4% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.206ms (SLO: <2.550ms 📉 -13.5%) vs baseline: -1.1% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +4.8% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.405ms (SLO: <1.550ms -9.3%) vs baseline: +0.8% Memory: ✅ 34.819MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 100-rate-metrics-100-timesTime: ✅ 2.193ms (SLO: <2.550ms 📉 -14.0%) vs baseline: +1.0% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.583µs (SLO: <20.000µs 📉 -77.1%) vs baseline: +0.2% Memory: ✅ 35.193MB (SLO: <35.500MB 🟡 -0.9%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 173.891µs (SLO: <250.000µs 📉 -30.4%) vs baseline: +0.5% Memory: ✅ 35.389MB (SLO: <35.500MB 🟡 -0.3%) vs baseline: +5.1% ✅ flush-1000-metricsTime: ✅ 2.163ms (SLO: <2.500ms 📉 -13.5%) vs baseline: -0.3% Memory: ✅ 35.999MB (SLO: <36.500MB 🟡 -1.4%) vs baseline: +4.9% 🟡 Near SLO Breach (17 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
This change moves the lib injection sitecustomize tests from github actions to gitlab in the interest of reducing this repo's dependence on github actions.