This chapter has covered benchmarking and tuning. We have motivated the need for benchmarking, mostly since having a benchmark framework in place prevents performance regressions, but also because well understood and easy-to-setup sub-problems are simpler to work with than complex applications. We also stressed the importance of having performance goals for commercial software development, another area where benchmarking is of a great help. It can also be useful to maintain a set of third-party applications as part of a benchmark suite.

We have discussed micro benchmarks to some extent and explained why they are useful and when they are not. We also talked about the importance of knowing what you are measuring.

Profiling always needs to be done to understand where the bottlenecks are in an application and to make sure that external benchmarks really address these bottlenecks. Profiling can be done with several levels of complexity, and with several levels of intrusiveness. For JRockit, low cost profiling is easily facilitated by the JRockit Mission Control suite.

We went on to present some industrial standard benchmarks, most famously the SPEC suite, and discussed where they can do some good for everyone from hardware vendors to application developers.

Once an application is well understood, it is time to decide if tuning JVM parameters alone can improve performance or if rewriting certain parts of the program is required. Using JRockit as an example JVM, we covered the parameters that can be used to tune application behavior in the memory system, in the code generator, and elsewhere.

Finally, we presented a section on common bottlenecks or anti-patterns that are frequently seen in Java applications and taught the reader how to avoid them.

This concludes the first part of this book. The next part of the book will go over the versatile and powerful tools in JRockit Mission Control in great detail. Hopefully this chapter contained enough philosophy on benchmarking and tuning to provide an excellent primer for understanding how to apply the necessary tools to solve your performance problems.