Skip to content

Conversation

@hassila
Copy link
Contributor

@hassila hassila commented Feb 11, 2025

Description

Swift 6 benchmark targets should now be possible with:

let benchmarks: @Sendable () -> Void = {
...
}

Make it possible to specify output units for the text output for a given metric by specifying it in the configuration.

E.g.

    Benchmark.defaultConfiguration.units = [.peakMemoryResident: .mega, .peakMemoryVirtual: .giga]

Also add the ability to override the time units from the command line using --time-units.

E.g.

swift package benchmark --time-units microseconds

This update also displays overflowing numeric values in scientific notation in the text output:

╒═══════════════════════════════════════════════════════════════════════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╤═══════════╕
│ Test                                                                      │        p0 │       p25 │       p50 │       p75 │       p90 │       p99 │      p100 │   Samples │
╞═══════════════════════════════════════════════════════════════════════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╪═══════════╡
│ Samples:All metrics, full concurrency, async (ns) *                       │  8.08e+07 │  8.34e+07 │  8.38e+07 │  8.40e+07 │  8.41e+07 │  8.46e+07 │  8.49e+07 │       183 │
├───────────────────────────────────────────────────────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Samples:Counter, custom metric thresholds (ns) *                          │      2375 │      2417 │      2417 │      2459 │      2501 │      2709 │      4334 │      4753 │
├───────────────────────────────────────────────────────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Samples:Extended + custom metrics (ns) *                                  │      1875 │      2000 │      2000 │      2041 │      2167 │      2667 │      8750 │       707 │
├───────────────────────────────────────────────────────────────────────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┼───────────┤
│ Samples:Extended metrics (ns) *                                           │      1750 │      1875 │      1875 │      1917 │      1958 │      2209 │      3250 │       705 │
╘═══════════════════════════════════════════════════════════════════════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╧═══════════╛

Also updates documentation.

Addresses:
#306
and
#237
#293
#277
#258

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Minimal checklist:

  • I have performed a self-review of my own code
  • I have added DocC code-level documentation for any public interfaces exported by the package
  • I have added unit and/or integration tests that prove my fix is effective or that my feature works

@github-actions
Copy link
Contributor

github-actions bot commented Feb 11, 2025

@codecov
Copy link

codecov bot commented Feb 11, 2025

Codecov Report

Attention: Patch coverage is 26.26263% with 73 lines in your changes missing coverage. Please review.

Project coverage is 70.21%. Comparing base (51fffb6) to head (68fe709).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
Sources/Benchmark/BenchmarkResult.swift 11.76% 60 Missing ⚠️
Sources/Benchmark/BenchmarkExecutor.swift 42.86% 8 Missing ⚠️
Sources/Benchmark/Benchmark.swift 54.55% 5 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #309      +/-   ##
==========================================
- Coverage   71.41%   70.21%   -1.20%     
==========================================
  Files          32       32              
  Lines        3791     3867      +76     
==========================================
+ Hits         2707     2715       +8     
- Misses       1084     1152      +68     
Files with missing lines Coverage Δ
Sources/Benchmark/BenchmarkMetric.swift 62.57% <ø> (ø)
Sources/Benchmark/BenchmarkRunner.swift 60.22% <100.00%> (+1.13%) ⬆️
Sources/Benchmark/Statistics.swift 66.92% <ø> (ø)
Tests/BenchmarkTests/BenchmarkResultTests.swift 100.00% <ø> (ø)
Tests/BenchmarkTests/BenchmarkRunnerTests.swift 92.31% <100.00%> (+0.20%) ⬆️
Sources/Benchmark/Benchmark.swift 69.71% <54.55%> (-1.02%) ⬇️
Sources/Benchmark/BenchmarkExecutor.swift 91.16% <42.86%> (-1.31%) ⬇️
Sources/Benchmark/BenchmarkResult.swift 61.93% <11.76%> (-8.20%) ⬇️
Files with missing lines Coverage Δ
Sources/Benchmark/BenchmarkMetric.swift 62.57% <ø> (ø)
Sources/Benchmark/BenchmarkRunner.swift 60.22% <100.00%> (+1.13%) ⬆️
Sources/Benchmark/Statistics.swift 66.92% <ø> (ø)
Tests/BenchmarkTests/BenchmarkResultTests.swift 100.00% <ø> (ø)
Tests/BenchmarkTests/BenchmarkRunnerTests.swift 92.31% <100.00%> (+0.20%) ⬆️
Sources/Benchmark/Benchmark.swift 69.71% <54.55%> (-1.02%) ⬇️
Sources/Benchmark/BenchmarkExecutor.swift 91.16% <42.86%> (-1.31%) ⬇️
Sources/Benchmark/BenchmarkResult.swift 61.93% <11.76%> (-8.20%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51fffb6...68fe709. Read the comment docs.

@hassila hassila changed the title feat: Support specifying units per metric feat: Support specifying units per metric and enable Swift 6 mode Feb 11, 2025
@hassila hassila changed the title feat: Support specifying units per metric and enable Swift 6 mode feat(minor): Enable Swift 6 mode and support specifying units per metric Feb 11, 2025
@hassila hassila merged commit 2bb8a39 into main Feb 11, 2025
12 of 16 checks passed
@hassila hassila deleted the support-unit-overrides branch February 11, 2025 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants