The SingleBenchmark class within the KelpNetTester class allows for simple benchmarking of various activation, noise, and other functions. If a function has a GPU capability, that is benchmarked, and so are CPU capabilities. The timing is at the microsecond level, as ReLU forward will usually always be below 1 ms in granularity.
With CPU enabled
With GPU enabled
Now let's talk about how we run a single benchmark.