-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchF.MDSqMtx
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 930.53 ms | 770.26 ms | 0.83 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDSqMtx*'
Payloads
Histogram
Benchstone.MDBenchF.MDSqMtx.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 770.2648947692308 < 885.1524572446428.
IsChangePoint: Marked as a change because one of 6/6/2022 10:12:27 PM, 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 32.968357364196336 (T) = (0 -762239601.1260234) / Math.Sqrt((1984588261209949.8 / (299)) + (465130434643554.44 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.1512434488371719 = (898066235.9327027 - 762239601.1260234) / 898066235.9327027 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDGeneralArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 16.63 ms | 11.17 ms | 0.67 | 0.03 | False | |||||
Test2 - Duration of single invocation | 16.57 ms | 11.16 ms | 0.67 | 0.02 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDGeneralArray*'
Payloads
Histogram
Benchstone.MDBenchI.MDGeneralArray.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.173597879166666 < 15.745436724068512.
IsChangePoint: Marked as a change because one of 6/14/2022 10:05:19 AM, 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 34.68017868912036 (T) = (0 -11048083.262695614) / Math.Sqrt((2624996997198.8794 / (299)) + (76785529424.15703 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.24324601434485607 = (14599306.342775278 - 11048083.262695614) / 14599306.342775278 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### Benchstone.MDBenchI.MDGeneralArray.Test2
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.159814841145835 < 15.660590579609377.
IsChangePoint: Marked as a change because one of 6/14/2022 10:05:19 AM, 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 34.915333879275316 (T) = (0 -10751162.077037025) / Math.Sqrt((2590281324396.07 / (299)) + (147570572407.2758 / (44))) is greater than 1.9669451293272056 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (44) - 2, .975) and 0.2625410069884124 = (14578657.496781103 - 10751162.077037025) / 14578657.496781103 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDMidpoint
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 556.00 ms | 413.91 ms | 0.74 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDMidpoint*'
Payloads
Histogram
Benchstone.MDBenchI.MDMidpoint.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 413.9137170714286 < 527.9502956349999.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 160.14970836247554 (T) = (0 -413893152.28634006) / Math.Sqrt((64429995116528.63 / (299)) + (27811119708367.203 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.2610431713948699 = (560104645.1761101 - 413893152.28634006) / 560104645.1761101 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchF.MDInvMt
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 5.97 ms | 2.55 ms | 0.43 | 0.06 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInvMt*'
Payloads
Histogram
Benchstone.MDBenchF.MDInvMt.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5481520705357146 < 5.671169423869621.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 273.83275849387024 (T) = (0 -2543588.855900561) / Math.Sqrt((42794490307.911385 / (299)) + (84033987.2352634 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.5645203390217909 = (5840890.135229161 - 2543588.855900561) / 5840890.135229161 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDArray2
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 1.31 secs | 1.03 secs | 0.78 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDArray2*'
Payloads
Histogram
Benchstone.MDBenchI.MDArray2.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0266930618461538 < 1.242550278826923.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 440.348503206845 (T) = (0 -1025749608.3284578) / Math.Sqrt((2332543136787.7573 / (299)) + (18195856383547.684 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.21605731666833128 = (1308449750.393915 - 1025749608.3284578) / 1308449750.393915 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDPuzzle
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 550.24 ms | 415.93 ms | 0.76 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDPuzzle*'
Payloads
Histogram
Benchstone.MDBenchI.MDPuzzle.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 415.9348021428571 < 522.6139306895834.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 415.8859307031019 (T) = (0 -417048645.4050549) / Math.Sqrt((2380625742831.7383 / (299)) + (4203366618506.9224 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.24098618427737317 = (549461204.4815015 - 417048645.4050549) / 549461204.4815015 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDAddArray2
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 17.51 ms | 11.99 ms | 0.68 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDAddArray2*'
Payloads
Histogram
Benchstone.MDBenchI.MDAddArray2.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.99241771875 < 17.02533243732804.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 92.27283746740929 (T) = (0 -11773072.591718813) / Math.Sqrt((166012199423.00952 / (299)) + (180987961031.2528 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.34651323094103886 = (18015778.052664112 - 11773072.591718813) / 18015778.052664112 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDXposMatrix
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 44.59 μs | 33.72 μs | 0.76 | 0.05 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDXposMatrix*'
Payloads
Histogram
Benchstone.MDBenchI.MDXposMatrix.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.715246545951345 < 42.357845826338014.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 42.27667062710874 (T) = (0 -32693.206483787755) / Math.Sqrt((31161.21119668241 / (299)) + (3620264.3331391825 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.26848114327597233 = (44692.22656842812 - 32693.206483787755) / 44692.22656842812 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in ByteMark
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
BenchNumericSortRectangular - Duration of single invocation | 1.12 secs | 861.35 ms | 0.77 | 0.00 | False | |||||
BenchLUDecomp - Duration of single invocation | 1.01 secs | 916.23 ms | 0.91 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'
Payloads
Histogram
ByteMark.BenchNumericSortRectangular
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 861.3462037857142 < 1.0682378820841345.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 178.22353945623888 (T) = (0 -865166731.555458) / Math.Sqrt((116021972679181.58 / (299)) + (73076325628214.98 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.22612349537013468 = (1117964851.4710438 - 865166731.555458) / 1117964851.4710438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### ByteMark.BenchLUDecomp
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 916.2305363333334 < 0.9526970249516666.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/31/2022 6:06:22 PM, 6/2/2022 5:58:32 PM, 6/7/2022 6:20:49 PM, 6/13/2022 2:27:15 PM, 7/10/2022 2:23:14 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 31.13495311479163 (T) = (0 -915928264.8756002) / Math.Sqrt((2100552268231235 / (299)) + (13102130965775.168 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09007743690586592 = (1006600233.9375387 - 915928264.8756002) / 1006600233.9375387 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchF.MDInProd
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 1.52 secs | 1.33 secs | 0.88 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInProd*'
Payloads
Histogram
Benchstone.MDBenchF.MDInProd.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3346294435384614 < 1.4378234977942306.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 196.76574661647675 (T) = (0 -1341012413.5492716) / Math.Sqrt((59360445186986.29 / (299)) + (26360601836324.215 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.11500212572148966 = (1515271903.4975362 - 1341012413.5492716) / 1515271903.4975362 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchI.MDLogicArray
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 354.44 ms | 294.00 ms | 0.83 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDLogicArray*'
Payloads
Histogram
Benchstone.MDBenchI.MDLogicArray.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 294.0012206666667 < 336.4827557410714.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 113.24481335470121 (T) = (0 -293915767.34662414) / Math.Sqrt((84702558871754.42 / (299)) + (72768919868.9163 / (44))) is greater than 1.9669451293272056 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (44) - 2, .975) and 0.1705860368994194 = (354365588.7439911 - 293915767.34662414) / 354365588.7439911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Architecture | x64 |
---|---|
OS | ubuntu 18.04 |
Baseline | 338db1a1ba498f308797ce7a54c0ed54e7286d50 |
Compare | ffcb0149763663a55a79784be216afe81f6bf76c |
Diff | Diff |
Improvements in Benchstone.MDBenchF.MDRomber
Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
---|---|---|---|---|---|---|---|---|---|---|
Test - Duration of single invocation | 181.94 ms | 163.02 ms | 0.90 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDRomber*'
Payloads
Histogram
Benchstone.MDBenchF.MDRomber.Test
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 163.01615503333332 < 169.1859327204762.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 6:44:07 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 6:44:07 AM.
IsImprovementStdDev: Marked as improvement because 26.523036514383964 (T) = (0 -164940727.7717664) / Math.Sqrt((3685055286333.8438 / (299)) + (11474443400035.455 / (45))) is greater than 1.9669246454804268 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (45) - 2, .975) and 0.07675745411891197 = (178653733.5260657 - 164940727.7717664) / 178653733.5260657 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository