In this chapter we are going to learn the following:

• What microbenchmarking is
• How to apply it to your code
• What activation functions are
• How to plot and benchmark activation functions

Every developer needs to have a good benchmarking tool at their disposal. Qualitative benchmarks are everywhere; you hear everyday, We decreased this by 10% and increased that by 25%. Remember the old adage, When you hear a number thrown out, 98.4 percent of the time that number is false? By the way, I just made up that number as well. When you hear a quote like that, ask that person to prove it and what do you get? Task manager perhaps? As data scientists, we don't need qualitative results; we need quantitative results that can be proven and consistently replicated. Reproducible results are incredibly important, not only for consistency but also for credibility and accuracy. And that's where microbenchmarking comes in to play.

We are going to use the irreplaceable BenchmarkDotNet library, which you can find here: https://github.com/dotnet/BenchmarkDotNet.

If you are not already using this library, you need to drop what you are doing right now and install it. I consider it one of the most irreplaceable frameworks you can use, and I consider it in terms of importance, right up there with unit and integration testing.

To show you just how valuable this tool is, we are going to plot several activation functions and compare their runtimes. As part of this, we will consider warmup, legacy and RyuJIT, cold starting, and more aspects of a program execution. In the end, we will have a quantitative set of results that prove the exact measurements of our functions. If, say in release 2.0, we see that something is running slower, we can rerun the benchmarks and compare.

I would strongly recommend that this be integrated into your continuous integration/continuous build process so that at each release, you have benchmark numbers to compare. And that's not just our code. I have created massive CI/CD systems that encompassed a huge number of programs, microservices, environments, and build and deploy steps. We would also regularly benchmark certain .NET library functions that we use all the time to verify; in between .NET framework releases, nothing has changed.

In this chapter, we are going to have two samples. The first is an activation function viewer; it will plot each activation function so that you can see how it looks. You can find this as part of what I consider one of the most valuable open source programs, SharpNEAT, by Mr. Colin Green. This package is absolutely incredible, and there's not a day that goes by when I don't use it. I have created new UIs on top of it as well as advanced versions to work with my requirements, and it's as flexible a tool as you can find. I work daily with researching the integration of mirror and canonical neurons into extendable substrates, and tools such as SharpNEAT are incredible. A future advanced book will be highlighting SharpNEAT much more, so get familiar with it now! This first sample application is available with the latest SharpNEAT package, which can be found at https://github.com/colgreen/sharpneat.