Skip to content

Validate Code Improvements With BenchmarkDotNet

After we found a slow method on the hot path of our application, it is time to improve the code and make sure that the change is indeed faster. Here we must dive into the next challenges and do the hard work of figuring out where exactly we lose time or waste too many resources. Let us find a helpful approach and explore BenchmarkDotNet, a tool that helps us to create benchmarks of our improvements.

Find the bottleneck in the application

In the last post we used dotnet-trace and Visual Studio to find the bottleneck in our application. During the performance test, the application spends 65% of the time in this line:

1
var end = _waste.Waiting(500);

The code to improve is inside the Waiting() method that currently looks like this – deliberately written to waste as much resources as possible:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
/// <summary>
/// https://stackoverflow.com/a/13001661
/// </summary>
/// <param name="miliSeconds"></param>
/// <returns></returns>
public long Waiting(int miliSeconds)
{
    var endTime = DateTime.Now.AddMilliseconds(miliSeconds);

    while (true)
    {
        if (DateTime.Now >= endTime)
            break;
    }

    return endTime.Ticks;
}

How can we improve the code? Maybe we are lucky and find along the hot path an obvious error. Those are the best cases then they allow us to quickly improve our application. However, a more likely scenario is that we need to deep dive into .Net and what happens behind the syntactic sugar.

To get quick results, an iterative approach is often helpful. We go through our code, check the documentation by Microsoft and then go on and look at blog posts and talks by more experienced developers who were kind enough to share their knowledge. To begin your research, I can recommend the following websites:

The list is by far not complete, but it puts you into the right mood to go on your own and search for more closely related improvement tips. As soon as we have an idea on how to improve the method, we can copy the existing method and put our idea into code. To make sure that the change indeed has an impact on the performance, we can now benchmark our old method against the new one and see which one is faster.

Create a benchmark project BenchmarkDotNet is an open-source tool to create micro benchmarks. This allows us to compare the performance of one method against another one so that we can find out which one is faster.

To use BenchmarkDotNet, we create a console project and add the NuGet package:

dotnet new console -n Benchmarks -lang "C#"
cd Benchmarks

dotnet add package BenchmarkDotNet

Add the benchmark

For simplicity, we can add the whole code into the Program.cs file. The main method runs the benchmark, while the definition for our benchmarks goes into the class WaitingBenchmark:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using WebApp.Helper;

namespace Benchmarks
{
    [MemoryDiagnoser]
    public class WaitingBenchmark
    {
        private Waste waste = new();

        [Benchmark(Baseline = true)]
        public long Original() => waste.Waiting(500);

        [Benchmark]
        public long Improved() => waste.WaitingNew(500);
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            var summary = BenchmarkRunner.Run<WaitingBenchmark>();
        }
    }
}

The methods that we want to use as benchmarks get the [Benchmark] attribute. Mark the old method as a baseline, that way we get a ratio column in the report and that saves us the manual calculations to compare the methods. The code for our new method is completely fabricated and has no relevance outside of this blog post – that is the reason why we do not look at it.

If we have more benchmarks, we should split the classes into different files. Then we can change the main method to give us a selector at the start of the application where we can pick the benchmark we want to run:

1
2
3
4
static void Main(string[] args)
{
    BenchmarkSwitcher.FromAssembly(Assembly.GetExecutingAssembly()).Run(args);
}

Run the benchmarks

When we start the project in release mode, BenchmarkDotNet will take over and run our benchmarks: 

dotnet run -c Release

If you follow the output, you see the enormous effort BenchmarkDotNet goes through to warm up everything, run our methods so that it can in the end give us a realistic answer on what method was the fastest.

We see that the original method uses 500ms, while the improved method only takes 461.5ms (0.92). The allocated memory is 6.25 times worser in the improved method compared to the original one.

The ratio of 0.92 for the duration looks good. The improved method is a bit faster than the one we had so far.

However, the memory allocation is 6.25 times worse than what our original method needs. This means the new method requires more memory, and more memory means more work for the garbage collector what could slow down the application.

Next

With BenchmarkDotNet we can write a small amount of code to create realistic benchmarks. That way we figured out that our new method is a bit faster but needs more memory. This could be a worthwhile trade-off, but we need to redo the load tests to verify if the overall performance has improved. Next week we go back to the performance tests and close the loop.