Description
Currently with the benchmark utility, if a benchmark needs to perform setup and tear down, it can only be done on a per-run basis. A good example would be:
node/benchmark/module/module-require.js
Line 15 in fcf5de0
This benchmark needs to create 10K files per-run, so for 60 runs * 3 configurations, it needs to create and delete 60 * 3 * 10K files, so the majority of the time spent on running the benchmark come from this process instead of the actual benchmarking that's done in between.
If for example we support setup and teardown on a per-suite basis (it doesn't necessarily have to be config-specific, a benchmark can do a setup for all the configs in one setup/teardown pair), then this would be reduced to only creating/deleting 3 * 10K files.
The setup/teardown should probably be done somewhere around here - before starting the queue and after executing the last job in the queue:
Lines 234 to 241 in 7178588