To measure the total footprint of a process, you need to use an operating-system specific tool. In UNIX-based systems, programs like top and ps can show you that data at a basic level; on Windows, you can use perfmon or VMMap. No matter which tool and platform are used, you need to look at the actual allocated memory (as opposed to the reserved memory) of the process.

The distinction between allocated and reserved memory comes about as a result of the way the JVM (and all programs) manage memory. Consider a heap is specified with the parameters -Xms512m -Xmx2048m. The heap starts by using 512 MB, and it will be resized as needed to meet the GC goals of the application.

That concept is the essential difference between committed (or allocated) memory and reserved memory (sometimes called the virtual size of a process). The JVM must tell the operating system that it might need as much as 2 GB of memory for the heap, so that memory is reserved—the operating system promises that when the JVM attempts to allocate additional memory when it increases the size of the heap, that memory will be available.

Still, only 512 MB of that memory is actually allocated initially, and that 512 MB is all of the memory that actually is being used (for the heap). That (actually allocated) memory is known as the committed memory. The amount of committed memory will fluctuate as the heap resizes; in particular, as the heap size increases, the committed memory correspondingly increases.

This difference applies to almost all significant memory that the JVM allocates. The code cache grows from an initial to a maximum value as more code gets compiled. Permgen or the class metaspace is allocated separately and grows between its initial (committed) size and its maximum (reserved) size.

One exception to this is thread stacks. Every time the JVM creates a thread, the OS allocates some native memory to hold that thread’s stack, committing more memory to the process (until the thread exits, at least). Threads stacks, though, are fully allocated when they are created.

In Unix systems, the actual footprint of an application can be estimated by the resident set size (RSS) of the process as reported by various OS tools. That value is a good estimate of the amount of committed memory a process is using, though it is inexact in two ways. First, the few pages that are shared at the OS level between JVM and other processes (that is, the text portions of shared libraries) are counted in the RSS of each process. Second, a process may have committed more memory than it actually paged in at any moment. Still, tracking the RSS of a process is a good first-pass way to monitor the total memory use. On more recent Linux kernels, the PSS is a refinement of the RSS that removes the data shared by other programs.

On Windows systems, the equivalent idea is called the working set of an application, which is what is reported by the task manager.

To minimize the footprint used by the JVM, limit the amount of memory used by the following:

Developers can allocate native memory via JNI calls, but NIO byte buffers will also allocate native memory if they are created via the allocateDirect() method. Native byte buffers are quite important from a performance perspective, since they allow native code and Java code to share data without copying it. The most common example here is buffers that are used for filesystem and socket operations. Writing data to a native NIO buffer and then sending that data to the channel (e.g, the file or socket) requires no copying of data between the JVM and the C library used to transmit the data. If a heap byte buffer is used instead, contents of the buffer must be copied by the JVM.

The allocateDirect() method call is quite expensive; direct byte buffers should be reused as much as possible. The ideal situation is when threads are independent and each can keep a direct byte buffer as a thread local variable. That can sometimes use too much native memory if there are many threads that need buffers of variable sizes, since eventually each thread will end up with a buffer at the maximum possible size. For that kind of situation—or when thread local buffers don’t fit the application design—an object pool of direct byte buffers maybe more useful.

Byte buffers can also be managed by slicing them. The application can allocate one very large direct byte buffer, and individual requests can allocate a portion out of that buffer using the slice() method of the ByteBuffer class. This solution can become unwieldy when the slices are not always the same size: the original byte buffer can then become fragmented in the same way the heap becomes fragmented when allocating and freeing objects of different sizes. Unlike the heap, however, the individual slices of a byte buffer cannot be compacted, so this solution really works well only when all the slices are a uniform size.

From a tuning perspective, the one thing to realize with any of these programming models is that the amount of direct byte buffer space that an application can allocate can be limited by the JVM. The total amount of memory that can be allocated for direct byte buffers is specified by setting the -XX:MaxDirectMemorySize=N flag. Starting in Java 7, the default value for this flag is 0, which means there is no limit (subject to the address space size and any operating-system limits on the process). That flag can be set to limit the direct byte buffer use of an application (and to provide compatibility with previous releases of Java, where the limit was 64 MB).

Beginning in Java 8, the JVM allows some visibility into how it allocates native memory when using this option: -XX:NativeMemoryTracking=off|summary|detail. By default, native memory tracking (NMT) tracking is off. If the summary or detail mode is enabled, you can get the native memory information at any time from jcmd:

% jcmd process_id VM.native_memory summary

If the JVM is started with the argument -XX:+PrintNMTStatistics (by default, false), the JVM will print out information about the allocation when the program exits.

Here is the summary output from a JVM running with a 512 MB initial heap size and a 4 GB maximum heap size:

Native Memory Tracking:

 Total:  reserved=4787210KB,  committed=1857677KB

Although the JVM has made memory reservations totaling 4.7 GB, it has used much less than that: only 1.8 GB total. This is fairly typical (and one reason not to pay particular attention to the virtual size of the process displayed in OS tools, since that reflects only the memory reservations).

This memory usage breaks down as follows:

-                 Java Heap (reserved=4296704KB, committed=1470428KB)
                            (mmap: reserved=4296704KB, committed=1470428KB)

The heap itself is (unsurprisingly) the largest part of the reserved memory at 4 GB. But the dynamic sizing of the heap meant it grew only to 1.4 GB.

-                     Class (reserved=65817KB, committed=60065KB)
                            (classes #19378)
                            (malloc=6425KB, #14245)
                            (mmap: reserved=59392KB, committed=53640KB)

This is the native memory used to hold class metadata. Again, note that the JVM has reserved more memory than it actually used to hold the 19,378 classes in the program.

-                    Thread (reserved=84455KB, committed=84455KB)
                            (thread #77)
                            (stack: reserved=79156KB, committed=79156KB)
                            (malloc=243KB, #314)
                            (arena=5056KB, #154)

77 thread stacks were allocated at about 1 MB each.

-                      Code (reserved=102581KB, committed=15221KB)
                            (malloc=2741KB, #4520)
                            (mmap: reserved=99840KB, committed=12480KB)

This is the JIT code cache. 19,378 classes is not very many, so just a small section of the code cache is committed.

-                        GC (reserved=183268KB, committed=173156KB)
                            (malloc=5768KB, #110)
                            (mmap: reserved=177500KB, committed=167388KB)

These are areas outside of the heap that GC algorithms use for their processing.

-                  Compiler (reserved=162KB, committed=162KB)
                            (malloc=63KB, #229)
                            (arena=99KB, #3)

Similarly, this area is used by the compiler for its operations, apart from the resulting code placed in the code cache.

-                    Symbol (reserved=12093KB, committed=12093KB)
                            (malloc=10039KB, #110773)
                            (arena=2054KB, #1)

Interned String references and symbol table references are held here.

-           Memory Tracking (reserved=22466KB, committed=22466KB)
                            (malloc=22466KB, #1872)

Native memory tracking itself needs some space for its operation.

0x00000006c0000000 - 0x00000007c0000000] reserved 4194304KB for Java Heap
        from [ReservedSpace::initialize(unsigned long, unsigned long,
                            bool, char*, unsigned long, bool)+0xc2]
        [0x00000006c0000000 - 0x00000006fb100000] committed 967680KB
            from [PSVirtualSpace::expand_by(unsigned long)+0x53]
        [0x000000076ab00000 - 0x00000007c0000000] committed 1397760KB
            from [PSVirtualSpace::expand_by(unsigned long)+0x53]

NMT provides two keys pieces of information:

% jcmd process_id VM.native_memory baseline

% jcmd process_id VM.native_memory summary.diff
Native Memory Tracking:

Total:  reserved=5896078KB  -3655KB, committed=2358357KB -448047KB

-                 Java Heap (reserved=4194304KB, committed=1920512KB -444927KB)
                            (mmap: reserved=4194304KB, committed=1920512KB -444927KB)
....

Discussions about memory allocation and swapping occur in terms of pages. A page is a unit of memory by which operating systems manage physical memory. A page is the minimum unit of allocation for the operating system: when one byte is allocated, the operating system must allocate an entire page. Further allocations for that program come from that same page until it is filled, at which point a new page is allocated.

The operating system allocates many more pages than can fit in physical memory, which is why there is paging: pages of the address space are moved to and from swap space (or other storage depending on what the page contains). This means there must be some mapping between these pages and where they are currently stored in the computer’s RAM. Those mappings are handled in two different ways. All page mappings are held in global page table (which the OS can scan to find a particular mapping), and the most frequently used mappings are held in translation lookaside buffers (TLBs). TLBs are held in a fast cache, so accessing pages through a TLB entry is much faster than accessing it through the the page table.

Machines have a limited number of TLB entries, so it becomes important to maximize the hit rate on TLB entries (it functions as a least-recently used cache). Since each entry represents a page of memory, it is often advantageous to increase the page size used by an application. If each page represents more memory, fewer TLB entries are required to encompass the entire program, and it is more likely that a page will be found in the TLB when required. This is true in general for any program, and so is also true in specific for things like Java application servers or other Java programs with even a moderately-sized heap.

Java supports this with the -XX:+UseLargePages option. The default value of this flag varies depending on the operating system configuration. On Windows, large pages must be enabled in the OS. In Windows terms, this means giving individual users the ability to lock pages into memory, which is possible only on server versions of Windows. Even so, the JVM on Windows defaults to using regular pages unless the UseLargePages flag is explicitly enabled.

On Linux, the UseLargePages flag is enabled by default, but the OS must also be configured to support large pages. Otherwise, regular pages will be used. Hence, the default depends on the OS configuration.

On Solaris, no OS configuration is required, and large pages are enabled by default.

If the UseLargePages flag is enabled on a system that does not support large pages, no warning is given and the JVM uses regular pages. If the UseLargePages flag is enabled on a system that does support large pages, but for which no large pages are available (either because they are already all in use or because the operating system is misconfigured), the JVM will print a warning.

# java -Xms4G -Xmx4G -XX:+UseLargePages -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Java HotSpot(TM) 64-Bit Server VM warning:
Failed to reserve shared memory (errno = 22).

# cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
# echo always > /sys/kernel/mm/transparent_hugepage/enabled
# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

# java -Xms4G -Xmx4G -XX:+UseLargePages -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Java HotSpot(TM) Server VM warning: JVM cannot use large page memory
because it does not have enough privilege to lock pages in memory.

Chapter 4 mentioned that the performance of a 32-bit JVM is anywhere from 5% to 20% faster than the performance of a 64-bit JVM for the same task. This assumes, of course, that the application can fit in a 32-bit process space, which limits the size of the heap to less than 4 GB.footnote[In practical terms, this often means less than 3.5 GB, since the JVM needs some native memory space, and on certain versions of Windows, the limit is 3 GB.]

This performance gap is because of the 64-bit object references. The main reason for this is 64-bit references take up twice the space (eight bytes) in the heap than do 32-bit references (four bytes). That leads to more GC cycles, since there is now less room in the heap for other data.

The JVM can compensate for that additional memory by using compressed oops. “oop” stands for ordinary object pointer—oops are the handles the JVM uses as object references. When oops are only 32-bits long, they can reference only 4 GB of memory (2**32), which is why a 32-bit JVM is limited to a 4GB heap size.^[48] When oops are 64 bits long, they can reference terabytes of memory.

There is a middle ground here—what if there were 35-bit oops? Then the pointer could reference 32 GB of memory (2**35) and still take up less space in the heap than 64-bit references. The problem is that there aren’t 35-bit registers in which to store such references. Instead, though, the JVM can assume that the last three bits of the reference are all 0. Now every reference can be stored in 32 bits in the heap. When the reference is stored into a 64-bit register, the JVM can shift it left by three bits (adding three zeros at the end). When the reference is saved from a register, the JVM can right-shift it by three bits, discarding the zeros at the end.

This leaves the JVM with pointers that can reference 32 GB of memory while using only 32 bits in the heap. However it also means that the JVM cannot access any object at an address that isn’t divisible by eight, since any address from a compressed oop ends with three zeros. The first possible oop is 0x1, which when shifted becomes 0x8. The next oop is 0x2, which when shifted becomes 0x10 (16). Objects must therefore be located on an 8-byte boundary.

It turns out that objects are already aligned on an 8-byte boundary in the JVM (both the 32- and 64-bit versions); this is the optimal alignment for most processors. So nothing is lost by using compressed oops. If the first object in the JVM is stored at location 0 and occupies 57 bytes, then the next object will be stored at location 64—wasting 7 bytes that cannot be allocated. That memory trade-off is worthwhile (and will occur whether compressed oops are used or not), because the object can be accessed faster given that 8-byte alignment.

But that is the reason that the JVM doesn’t try to emulate a 36-bit reference which could access 64 GB of memory. In that case, objects would have to be aligned on a 16 byte boundary, and the savings from storing the compressed pointer in the heap would be outweighed by the amount of memory that would be wasted in between the memory-aligned objects.

There are two implications of this. First, for heaps that are between 4 GB and 32 GB, use compressed oops. Compressed oops are enabled using the -XX:+UseCompressedOops flag; in Java 7 and later versions, they are enabled by default whenever the maximum heap size is less than 32 GB.^[49]

Second, a program that uses a 31 GB heap and compressed oops will usually be faster than a program that uses a 33 GB heap. Although the 33 GB heap is larger, the extra space used by the pointers in that heap means that the larger heap will perform more frequent GC cycles and have worse performance.

Hence, it is better to use heaps that are less than 32 GB, or heaps that are at least a few GB larger than 32 GB. Once extra memory is added to the heap to make up for the space used by the uncompressed references, the number of GC cycles will be reduced. There is no hard rule there for how much memory is needed before the GC impact of the uncompressed oops is ameliorated—but given that 20% of an average heap might be used for object references, planning on at least 38 GB is a good start.