This chapter is an introduction to automatic and adaptive memory management in the Java runtime. It provides a background on techniques for garbage collection and looks at the history of automatic memory management. It also discusses the advantages and disadvantages of automatic memory management compared to static solutions.
You will learn the following from this chapter:
- The concepts of automatic and adaptive memory management and understanding the problems and possibilities associated with these
- How a garbage collector works, including algorithms for garbage collection and implementation details
- How a garbage collector must be implemented in order to perform well and be scalable
- About the latency versus throughput equation
- The problems of object allocation in a runtime and algorithms for doing efficient object allocation
- The most important Java APIs for memory management, for example, the
java.lang.refpackage - How the JRockit Real Time product and deterministic garbage collection works
- How to write Java code that plays well with the garbage collector, and common pitfalls and false optimizations
- How to use the most fundamental command-line flags associated with the memory subsystem
Automatic memory management is defined as any garbage collection technique that automatically gets rid of stale references, making a free operator unnecessary. This is quite an old idea—implementations have been with us for almost as long as the history of modern computer science, probably starting out as reference counting methods in early Lisp machines. After that, other heap management strategies were developed. Most are refinements of tracing techniques, which involve traversing live object graphs on the heap in order to determine what can be garbage collected.
Note
We will use the term heap throughout this chapter to mean all the non-thread local memory available for objects in a garbage collected environment.
As we have already seen in the previous chapter, basing JVM behavior on runtime feedback is a good idea. JRockit was the first JVM to recognize that adaptive optimizations based on runtime feedback could be applied to all subsystems in the runtime and not just to code generation. One of these subsystems is memory management.
We will use the term adaptive memory management to describe a memory management system whose behavior is based heavily on runtime feedback. Adaptive memory management is a special case of automatic memory management. Automatic memory management should be taken to mean just that some kind of garbage collection technique is employed. Garbage collection means, of course, that the user does not have to explicitly remove objects that are no longer in use. The system will automatically detect and free those resources.
Adaptive memory management must correctly utilize runtime feedback for optimal performance. This can mean changing GC strategies, automatic heap resizing, getting rid of memory fragmentation at the right intervals, or mainly recognizing when it is most appropriate to "stop the world". Stopping the world means halting the executing Java program, which is a necessary evil for parts of a garbage collection cycle.
The first and foremost advantage of automatic memory management is its contribution to the speed of the software development cycle. Any support organization knows that, with the possible exception of erroneous multi-threaded behavior, some of the most common causes for problems in software are memory allocation bugs, buffer overruns, and memory leaks. All of these are fairly hard to debug. It may not be a trivial matter to spot a one-byte-off allocation that leads to a crash much later in the program lifetime, when, for example, a totally different object is freed.
Both memory allocation bugs and buffer overruns are impossible in Java due to the intrinsic properties of the Java language. Memory allocation bugs can't occur because automatic memory management exists and buffer overruns can't occur because the runtime does not allow them. For example, whenever the program tries to write outside an array, an ArrayIndexOutOfBoundsException is thrown.
While memory leaks are still possible in a garbage collected world, modern JVMs provide ways of detecting them. There are also constructs in the Java language that can help the developer work around them. In the case of JRockit, the JRockit Mission Control suite contains a tool that can, with very low overhead, detect memory leaks in a running application. This is possible, as the garbage collector in the JVM already collects a lot of useful information that can be used for multiple purposes. The Memory Leak Detector tool is covered in detail in Chapter 10 of this book. It is a prime example of a value add, stemming from automatic memory management.
Note
It is the authors opinion that built-in automatic memory management and the shorter development cycles it enabled, was one of the main factors behind today's widespread Java adoption. Complex server applications crash less often with automatic memory management.
An additional advantage is that an adaptive memory manager may pick the appropriate garbage collection strategy for an application based on its current behavior, appropriately changing the number of garbage collecting threads or fine tuning other aspects of garbage collection strategies whenever needed. This might be compared to the adaptive behavior of the code generator, as explained in the previous chapter. The code generator can use runtime feedback to, for example, optimize only hot parts of methods and leave cold parts alone until they become hot at some later stage.
It is often argued that automatic memory management can slow down execution for certain applications to such an extent that it becomes impractical. This is because automatic memory management can introduce a high degree of non-determinism to a program that requires short response times. To avoid this, extensive workarounds may be needed to get enough application performance.
In truth, giving up control of memory management to the runtime may result in slowdowns, but this is rarely the case anymore, at least for well written applications.
The main bottleneck for a garbage collector tends to be the amount of live data on the heap, not the actual heap size. Any garbage collection algorithm will break down given too large an amount of live data. This could indeed be less of a problem in a non-garbage collecting system, but humans are fallible and there is no guarantee that manual memory management would fare any better with a large live data set.
Finally, there may still be memory leaks in a garbage collected environment. If the program erroneously holds on to references that should have been garbage collected, these will be treated as live. A common example is a broken cache implementation, for example in the form of a java.util.HashMap that doesn't throw away all old objects as it should. The system has no way of knowing that a forgotten object, still referenced by a key/value pair in a HashMap that is still in use, should be reclaimed.