[Test Optimization] Add support for MSTest 3.9.x #7070

tonyredondo · 2025-06-05T14:46:57Z

Summary of changes

This PR adds support to the latest MSTest.TestAdapter / MSTest.TestFramework 3.9.x nuget packages

Reason for change

Currently it was not supported so test were not instrumented in this version.

Implementation details

The changes are mostly on new integrations targeting the new async methods.

Test coverage

Current snapshots should work for the new version.

Other details

datadog-datadog-prod-us1 · 2025-06-05T15:19:10Z

Datadog Report

All test runs 0c409b6 🔗

✅ 2 Total Test Services: 0 Failed, 2 Passed

Test Services

Service Name	Failed	Known Flaky	New Flaky	Passed	Skipped	Total Time	Test Service View
`dd-trace-dotnet`	0	0	0	472107	1607	6h 32m 10.21s	Link
`exploration_tests`	0	0	0	22085	3	2m 12.97s	Link

andrewlock · 2025-06-05T16:03:43Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7070) - mean (74ms)  : 67, 81
     .   : milestone, 74,
    master - mean (74ms)  : 70, 78
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (7070) - mean (1,066ms)  : 1046, 1086
     .   : milestone, 1066,
    master - mean (1,073ms)  : 1040, 1106
     .   : milestone, 1073,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7070) - mean (109ms)  : 106, 113
     .   : milestone, 109,
    master - mean (111ms)  : 106, 116
     .   : milestone, 111,

    section CallTarget+Inlining+NGEN
    This PR (7070) - mean (738ms)  : 716, 760
     .   : milestone, 738,
    master - mean (748ms)  : 726, 769
     .   : milestone, 748,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7070) - mean (95ms)  : 93, 97
     .   : milestone, 95,
    master - mean (99ms)  : 94, 104
     .   : milestone, 99,

    section CallTarget+Inlining+NGEN
    This PR (7070) - mean (699ms)  : 674, 724
     .   : milestone, 699,
    master - mean (712ms)  : 683, 742
     .   : milestone, 712,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7070) - mean (190ms)  : 186, 193
     .   : milestone, 190,
    master - mean (190ms)  : 184, 195
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (7070) - mean (1,166ms)  : 1151, 1180
     .   : milestone, 1166,
    master - mean (1,167ms)  : 1151, 1184
     .   : milestone, 1167,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7070) - mean (269ms)  : 266, 272
     .   : milestone, 269,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7070) - mean (931ms)  : 921, 942
     .   : milestone, 931,
    master - mean (932ms)  : 913, 950
     .   : milestone, 932,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (7070) - mean (262ms)  : 257, 266
     .   : milestone, 262,
    master - mean (262ms)  : 258, 265
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (7070) - mean (925ms)  : 909, 941
     .   : milestone, 925,
    master - mean (929ms)  : 912, 947
     .   : milestone, 929,

pr-commenter · 2025-06-05T16:15:59Z

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7070 compared to master:

2 benchmarks are slower, with geometric mean 1.262
2 benchmarks have fewer allocations
7 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7070

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472	6.09 KB	6.21 KB	127 B	2.09%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	11.2μs	59.7ns	322ns	0	0	0	5.58 KB
master	`StartStopWithChild`	netcoreapp3.1	14.1μs	67.5ns	286ns	0	0	0	5.75 KB
master	`StartStopWithChild`	net472	22.1μs	119ns	651ns	1.03	0.411	0.103	6.09 KB
#7070	`StartStopWithChild`	net6.0	10.6μs	56.8ns	316ns	0	0	0	5.55 KB
#7070	`StartStopWithChild`	netcoreapp3.1	13.6μs	58ns	225ns	0	0	0	5.76 KB
#7070	`StartStopWithChild`	net472	22.3μs	122ns	655ns	0.931	0.31	0	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7070

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	3.33 KB	3.35 KB	23 B	0.69%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	927μs	26.6ns	103ns	2.71 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.04ms	404ns	1.56μs	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.25ms	357ns	1.38μs	3.33 KB
#7070	`WriteAndFlushEnrichedTraces`	net6.0	921μs	270ns	1.04μs	2.71 KB
#7070	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.02ms	73.1ns	264ns	2.7 KB
#7070	`WriteAndFlushEnrichedTraces`	net472	1.23ms	55.5ns	192ns	3.35 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	331μs	1.76μs	8.42μs	0	0	197.06 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	510μs	1.45μs	5.61μs	0	0	204.77 KB
master	`AllCycleSimpleBody`	net472	436μs	119ns	460ns	36.6	2.16	236.35 KB
master	`AllCycleMoreComplexBody`	net6.0	338μs	1.76μs	8.8μs	0	0	200.56 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	495μs	987ns	3.56μs	0	0	208.18 KB
master	`AllCycleMoreComplexBody`	net472	446μs	106ns	412ns	36.6	2.16	239.87 KB
master	`ObjectExtractorSimpleBody`	net6.0	311ns	1.77ns	12.3ns	0	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	409ns	1.93ns	8.2ns	0	0	272 B
master	`ObjectExtractorSimpleBody`	net472	303ns	0.175ns	0.676ns	0.0442	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	6.52μs	29.5ns	110ns	0	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.76μs	36.2ns	140ns	0	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	6.66μs	0.89ns	3.33ns	0.599	0	3.8 KB
#7070	`AllCycleSimpleBody`	net6.0	340μs	169ns	655ns	0	0	197.41 KB
#7070	`AllCycleSimpleBody`	netcoreapp3.1	517μs	351ns	1.36μs	0	0	205.17 KB
#7070	`AllCycleSimpleBody`	net472	441μs	158ns	611ns	36.6	2.16	236.95 KB
#7070	`AllCycleMoreComplexBody`	net6.0	337μs	1.74μs	7.97μs	0	0	200.91 KB
#7070	`AllCycleMoreComplexBody`	netcoreapp3.1	526μs	2.58μs	10.3μs	0	0	208.58 KB
#7070	`AllCycleMoreComplexBody`	net472	447μs	147ns	568ns	37.9	2.23	240.46 KB
#7070	`ObjectExtractorSimpleBody`	net6.0	311ns	0.0435ns	0.168ns	0	0	280 B
#7070	`ObjectExtractorSimpleBody`	netcoreapp3.1	409ns	2.12ns	9.69ns	0	0	272 B
#7070	`ObjectExtractorSimpleBody`	net472	305ns	0.0332ns	0.128ns	0.0445	0	281 B
#7070	`ObjectExtractorMoreComplexBody`	net6.0	6.38μs	2.43ns	9.4ns	0	0	3.78 KB
#7070	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.82μs	15.1ns	58.5ns	0	0	3.69 KB
#7070	`ObjectExtractorMoreComplexBody`	net472	6.85μs	3.62ns	14ns	0.58	0	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7070

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0	2.16 KB	2.15 KB	-11 B	-0.51%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EncodeArgs`	net6.0	73.3μs	263ns	984ns	0	32.41 KB
master	`EncodeArgs`	netcoreapp3.1	95.7μs	36.2ns	130ns	0	32.4 KB
master	`EncodeArgs`	net472	107μs	17.6ns	65.8ns	4.82	32.51 KB
master	`EncodeLegacyArgs`	net6.0	143μs	122ns	472ns	0	2.16 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	197μs	42.5ns	147ns	0	2.14 KB
master	`EncodeLegacyArgs`	net472	261μs	66.8ns	250ns	0	2.16 KB
#7070	`EncodeArgs`	net6.0	73.3μs	23.1ns	89.4ns	0	32.41 KB
#7070	`EncodeArgs`	netcoreapp3.1	94.2μs	233ns	901ns	0	32.4 KB
#7070	`EncodeArgs`	net472	105μs	19.5ns	67.4ns	4.72	32.51 KB
#7070	`EncodeLegacyArgs`	net6.0	146μs	56ns	217ns	0	2.15 KB
#7070	`EncodeLegacyArgs`	netcoreapp3.1	199μs	139ns	522ns	0	2.14 KB
#7070	`EncodeLegacyArgs`	net472	262μs	25.4ns	98.3ns	0	2.17 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`RunWafRealisticBenchmark`	net6.0	271μs	133ns	499ns	4.55 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	294μs	264ns	989ns	4.48 KB
master	`RunWafRealisticBenchmark`	net472	307μs	38.1ns	147ns	4.66 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	181μs	80.1ns	300ns	2.24 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	198μs	104ns	404ns	2.22 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	207μs	45.3ns	169ns	2.28 KB
#7070	`RunWafRealisticBenchmark`	net6.0	269μs	118ns	424ns	4.55 KB
#7070	`RunWafRealisticBenchmark`	netcoreapp3.1	288μs	59.2ns	229ns	4.48 KB
#7070	`RunWafRealisticBenchmark`	net472	307μs	55.8ns	216ns	4.66 KB
#7070	`RunWafRealisticBenchmarkWithAttack`	net6.0	182μs	55.1ns	213ns	2.24 KB
#7070	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	193μs	64.6ns	233ns	2.22 KB
#7070	`RunWafRealisticBenchmarkWithAttack`	net472	206μs	32.2ns	125ns	2.28 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`SendRequest`	net6.0	61.1μs	29ns	104ns	14.53 KB
master	`SendRequest`	netcoreapp3.1	69.9μs	118ns	440ns	17.42 KB
master	`SendRequest`	net472	0.0171ns	0.00147ns	0.00551ns	0 b
#7070	`SendRequest`	net6.0	62.1μs	56.8ns	212ns	14.53 KB
#7070	`SendRequest`	netcoreapp3.1	72μs	217ns	840ns	17.43 KB
#7070	`SendRequest`	net472	0.0124ns	0.00189ns	0.00731ns	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7070

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	1.178	645,405.00	760,059.38	several?

More allocations ⚠️ in #7070

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.73 KB	42.51 KB	779 B	1.87%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	55.75 KB	56.29 KB	533 B	0.96%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	645μs	512ns	1.98μs	0	41.73 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	631μs	1.67μs	6.67μs	0	41.89 KB
master	`WriteAndFlushEnrichedTraces`	net472	923μs	1.93μs	7.2μs	4.46	55.75 KB
#7070	`WriteAndFlushEnrichedTraces`	net6.0	729μs	5.51μs	54.2μs	0	42.51 KB
#7070	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	679μs	3.92μs	30.4μs	0	41.9 KB
#7070	`WriteAndFlushEnrichedTraces`	net472	898μs	1.44μs	5.57μs	8.33	56.29 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.99μs	10.2ns	46.8ns	0	0	1.03 KB
master	`ExecuteNonQuery`	netcoreapp3.1	2.53μs	3.43ns	13.3ns	0	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.7μs	2.41ns	9.34ns	0.147	0.0134	995 B
#7070	`ExecuteNonQuery`	net6.0	1.87μs	9.16ns	37.8ns	0	0	1.03 KB
#7070	`ExecuteNonQuery`	netcoreapp3.1	2.51μs	7.39ns	26.6ns	0	0	1.02 KB
#7070	`ExecuteNonQuery`	net472	2.69μs	3.31ns	12.8ns	0.146	0.0133	995 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.83μs	5.85ns	22.7ns	0	1.04 KB
master	`CallElasticsearch`	netcoreapp3.1	2.3μs	11.8ns	56.6ns	0	1.04 KB
master	`CallElasticsearch`	net472	3.55μs	3.05ns	11.8ns	0.159	1.05 KB
master	`CallElasticsearchAsync`	net6.0	1.81μs	3.89ns	14.5ns	0	1.02 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	2.35μs	7.26ns	28.1ns	0	1.09 KB
master	`CallElasticsearchAsync`	net472	3.84μs	3.19ns	12.4ns	0.169	1.11 KB
#7070	`CallElasticsearch`	net6.0	1.73μs	7.57ns	27.3ns	0	1.04 KB
#7070	`CallElasticsearch`	netcoreapp3.1	2.3μs	1.18ns	4.24ns	0	1.04 KB
#7070	`CallElasticsearch`	net472	3.57μs	4.47ns	16.7ns	0.16	1.05 KB
#7070	`CallElasticsearchAsync`	net6.0	1.84μs	6.64ns	24.9ns	0	1.02 KB
#7070	`CallElasticsearchAsync`	netcoreapp3.1	2.39μs	10.5ns	39.3ns	0	1.09 KB
#7070	`CallElasticsearchAsync`	net472	3.73μs	2.47ns	8.57ns	0.168	1.11 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.78μs	9.27ns	43.5ns	0	960 B
master	`ExecuteAsync`	netcoreapp3.1	2.29μs	7.45ns	25.8ns	0	960 B
master	`ExecuteAsync`	net472	2.58μs	1.78ns	6.88ns	0.143	923 B
#7070	`ExecuteAsync`	net6.0	1.78μs	1.81ns	7.01ns	0	960 B
#7070	`ExecuteAsync`	netcoreapp3.1	2.35μs	5.14ns	19.9ns	0	960 B
#7070	`ExecuteAsync`	net472	2.62μs	2.25ns	8.73ns	0.143	923 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	7.1μs	10.4ns	39ns	0	2.37 KB
master	`SendAsync`	netcoreapp3.1	8.69μs	14.1ns	54.6ns	0	2.9 KB
master	`SendAsync`	net472	12.5μs	10.3ns	38.5ns	0.498	3.19 KB
#7070	`SendAsync`	net6.0	6.84μs	11.4ns	41.2ns	0	2.37 KB
#7070	`SendAsync`	netcoreapp3.1	8.27μs	15.8ns	61.3ns	0	2.9 KB
#7070	`SendAsync`	net472	12.3μs	13.1ns	50.7ns	0.492	3.19 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7070

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	1.353	413,700.00	559,800.00

More allocations ⚠️ in #7070

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1	42.64 KB	43.19 KB	552 B	1.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	274.93 KB	278.22 KB	3.29 KB	1.20%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	43.83 KB	44.3 KB	464 B	1.06%

Fewer allocations 🎉 in #7070

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472	286.72 KB	279.15 KB	-7.57 KB	-2.64%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	44.6μs	212ns	1.6μs	43.83 KB
master	`StringConcatBenchmark`	netcoreapp3.1	47.2μs	224ns	838ns	42.64 KB
master	`StringConcatBenchmark`	net472	56.8μs	259ns	968ns	57.34 KB
master	`StringConcatAspectBenchmark`	net6.0	458μs	1.08μs	3.89μs	259.96 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	447μs	6.44μs	63.7μs	274.93 KB
master	`StringConcatAspectBenchmark`	net472	410μs	2.07μs	9.27μs	286.72 KB
#7070	`StringConcatBenchmark`	net6.0	46.2μs	176ns	634ns	44.3 KB
#7070	`StringConcatBenchmark`	netcoreapp3.1	48.8μs	241ns	1.46μs	43.19 KB
#7070	`StringConcatBenchmark`	net472	57.1μs	259ns	1.07μs	57.34 KB
#7070	`StringConcatAspectBenchmark`	net6.0	513μs	2.07μs	7.73μs	260.27 KB
#7070	`StringConcatAspectBenchmark`	netcoreapp3.1	562μs	2.64μs	10.6μs	278.22 KB
#7070	`StringConcatAspectBenchmark`	net472	407μs	2.34μs	17.5μs	279.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.63μs	2.6ns	10.1ns	0	1.76 KB
master	`EnrichedLog`	netcoreapp3.1	3.44μs	4.38ns	17ns	0	1.76 KB
master	`EnrichedLog`	net472	4.05μs	4.83ns	18.7ns	0.265	1.69 KB
#7070	`EnrichedLog`	net6.0	2.52μs	11.7ns	46.7ns	0	1.76 KB
#7070	`EnrichedLog`	netcoreapp3.1	3.56μs	14.4ns	55.9ns	0	1.76 KB
#7070	`EnrichedLog`	net472	3.81μs	4.97ns	19.2ns	0.267	1.69 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`EnrichedLog`	net6.0	123μs	145ns	563ns	4.37 KB
master	`EnrichedLog`	netcoreapp3.1	126μs	330ns	1.24μs	4.37 KB
master	`EnrichedLog`	net472	167μs	161ns	603ns	4.57 KB
#7070	`EnrichedLog`	net6.0	122μs	90.2ns	325ns	4.37 KB
#7070	`EnrichedLog`	netcoreapp3.1	127μs	61ns	220ns	4.37 KB
#7070	`EnrichedLog`	net472	167μs	162ns	629ns	4.57 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.94μs	19.3ns	74.6ns	0	2.32 KB
master	`EnrichedLog`	netcoreapp3.1	6.77μs	22.2ns	79.9ns	0	2.32 KB
master	`EnrichedLog`	net472	7.45μs	8.04ns	31.1ns	0.335	2.14 KB
#7070	`EnrichedLog`	net6.0	4.92μs	16.6ns	64.2ns	0	2.32 KB
#7070	`EnrichedLog`	netcoreapp3.1	6.62μs	8.56ns	33.1ns	0	2.32 KB
#7070	`EnrichedLog`	net472	7.4μs	7.23ns	28ns	0.333	2.14 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	2.04μs	10.5ns	49.4ns	0	1.21 KB
master	`SendReceive`	netcoreapp3.1	2.53μs	11.8ns	47.4ns	0	1.21 KB
master	`SendReceive`	net472	3.28μs	2.71ns	10.5ns	0.178	1.21 KB
#7070	`SendReceive`	net6.0	1.96μs	9.1ns	37.5ns	0	1.21 KB
#7070	`SendReceive`	netcoreapp3.1	2.59μs	12.2ns	50.2ns	0	1.21 KB
#7070	`SendReceive`	net472	3.11μs	2.03ns	7.59ns	0.186	1.21 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.18μs	0.92ns	3.44ns	0	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	5.64μs	23ns	89.3ns	0	1.69 KB
master	`EnrichedLog`	net472	6.67μs	7.28ns	27.3ns	0.298	2.08 KB
#7070	`EnrichedLog`	net6.0	4.18μs	14.4ns	55.7ns	0	1.64 KB
#7070	`EnrichedLog`	netcoreapp3.1	5.56μs	20.7ns	80.1ns	0	1.69 KB
#7070	`EnrichedLog`	net472	6.96μs	6.43ns	24.1ns	0.312	2.08 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	751ns	3.97ns	22.4ns	0	584 B
master	`StartFinishSpan`	netcoreapp3.1	953ns	4.42ns	17.7ns	0	584 B
master	`StartFinishSpan`	net472	914ns	0.809ns	3.13ns	0.0912	586 B
master	`StartFinishScope`	net6.0	918ns	0.479ns	1.73ns	0	704 B
master	`StartFinishScope`	netcoreapp3.1	1.15μs	6.24ns	34.2ns	0	704 B
master	`StartFinishScope`	net472	1.09μs	0.174ns	0.652ns	0.104	666 B
#7070	`StartFinishSpan`	net6.0	721ns	3.48ns	14.4ns	0	584 B
#7070	`StartFinishSpan`	netcoreapp3.1	947ns	5.13ns	27.1ns	0	584 B
#7070	`StartFinishSpan`	net472	949ns	0.115ns	0.446ns	0.0903	586 B
#7070	`StartFinishScope`	net6.0	882ns	4.61ns	23ns	0	704 B
#7070	`StartFinishScope`	netcoreapp3.1	1.18μs	6.08ns	29.1ns	0	704 B
#7070	`StartFinishScope`	net472	1.12μs	0.207ns	0.774ns	0.1	666 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	1.02μs	5.36ns	26.8ns	0	704 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.38μs	2.28ns	8.82ns	0	704 B
master	`RunOnMethodBegin`	net472	1.36μs	0.141ns	0.544ns	0.102	666 B
#7070	`RunOnMethodBegin`	net6.0	1.02μs	4.85ns	20ns	0	704 B
#7070	`RunOnMethodBegin`	netcoreapp3.1	1.43μs	6.88ns	26.7ns	0	704 B
#7070	`RunOnMethodBegin`	net472	1.35μs	0.0764ns	0.296ns	0.102	666 B

tracer/src/Datadog.Trace/ClrProfiler/AutoInstrumentation/Testing/MsTestV2/MsTestIntegration.cs

...src/Datadog.Trace/ClrProfiler/AutoInstrumentation/Testing/MsTestV2/SkipTestMethodExecutor.cs

...e/ClrProfiler/AutoInstrumentation/Testing/MsTestV2/TestMethodRunnerExecuteTestIntegration.cs

tracer/test/Datadog.Trace.ClrProfiler.IntegrationTests/PackageVersionsLatestMajors.g.cs

tracer/test/Datadog.Trace.ClrProfiler.IntegrationTests/PackageVersionsLatestMinors.g.cs

tracer/test/Datadog.Trace.ClrProfiler.IntegrationTests/PackageVersionsLatestSpecific.g.cs

andrewlock · 2025-06-06T13:40:25Z

tracer/test/test-applications/integrations/Samples.MSTestTests/Samples.MSTestTests.csproj

@@ -3,7 +3,7 @@
  <PropertyGroup>
    <IsPackable>false</IsPackable>
    <OutputType>Library</OutputType>
-    <ApiVersion Condition="'$(ApiVersion)' == ''">3.8.1</ApiVersion>
+    <ApiVersion Condition="'$(ApiVersion)' == ''">3.9.1</ApiVersion>


👍 I like that you did this, we should probably do it for all our Samples tbh, to make sure we're testing latest on .NET FX

Co-authored-by: Andrew Lock <[email protected]>

andrewlock

Thanks!

Initial implementation

6a3d2de

tonyredondo self-assigned this Jun 5, 2025

revert

81eea87

Update package versions

77fb831

tonyredondo marked this pull request as ready for review June 6, 2025 09:20

tonyredondo requested review from a team as code owners June 6, 2025 09:20

tonyredondo added 2 commits June 6, 2025 15:19

fix

471b3b5

Merge branch 'master' into tony/test-optimization-mstest-3.9.x-support

424c34f

andrewlock reviewed Jun 6, 2025

View reviewed changes

tonyredondo and others added 2 commits June 6, 2025 16:42

Apply suggestions from code review

0c409b6

Co-authored-by: Andrew Lock <[email protected]>

changes from review

0db15ca

andrewlock approved these changes Jun 6, 2025

View reviewed changes

tonyredondo merged commit c05ddfb into master Jun 6, 2025
131 of 132 checks passed

tonyredondo deleted the tony/test-optimization-mstest-3.9.x-support branch June 6, 2025 22:51

github-actions bot added this to the vNext-v3 milestone Jun 6, 2025

[Test Optimization] Add support for MSTest 3.9.x #7070

[Test Optimization] Add support for MSTest 3.9.x #7070

Uh oh!

Conversation

tonyredondo commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

datadog-datadog-prod-us1 bot commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Datadog Report

Uh oh!

andrewlock commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

pr-commenter bot commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmark details

More allocations ⚠️ in #7070

Raw results

More allocations ⚠️ in #7070

Raw results

Raw results

Fewer allocations 🎉 in #7070

Raw results

Raw results

Raw results

Slower ⚠️ in #7070

More allocations ⚠️ in #7070

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #7070

More allocations ⚠️ in #7070

Fewer allocations 🎉 in #7070

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrewlock Jun 6, 2025

Choose a reason for hiding this comment

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tonyredondo commented Jun 5, 2025 •

edited

Loading

datadog-datadog-prod-us1 bot commented Jun 5, 2025 •

edited

Loading

andrewlock commented Jun 5, 2025 •

edited

Loading

pr-commenter bot commented Jun 5, 2025 •

edited

Loading