ART Basics

The ART system does far more than simply compile code and run it in a garbage-collected environment. A closer description might be that it compiles, links, and even starts running the application, and then stores the running image.

The most common invocation of the compiler occurs when a new application is installed on the system. As part of the application installation process, the Android system runs dex2oat, over the DEX code, newly downloaded and installed in the applications directory /data/app/<application package name>. The verified, optimized native code is stored in the directory /data/dalvik-cache as data@<application package name>-<n>[email protected]. Despite the .dex suffix, the file is an OAT file.

The format of the OAT file is complex and not well documented. Although at the time of this writing it appears that the format is still in flux, some things are stable. The file is an ELF file, a standard format for Unix binaries. It contains both the compiled native code and also the entire pre-compilation DEX format “source.” Figure 7.2 is a rough outline of the OAT file format. (The format is described in the file $AOSP/art/dex2oat/linker/oat_writer.h).

The point of embedding the complete, uncompiled DEX code in the OAT file is that doing so allows incremental compilation. Bits in the file header indicate which methods have been compiled and which have not.

The ART runtime is not only compiled to native code but, in the normal uses of the term, it is linked. The dex2oat compiler refers, during the process of compiling a .dex file, to the output of the previously compiled Android system libraries and compiles references to symbols defined in those libraries to their absolute locations. In other words, the compiled OAT file must be loaded into memory at exactly the location for which it was compiled and depends on having an image of the compiled libraries at exactly the locations at which they were found during the compilation process.

The process of loading independently compiled binaries into memory and resolving the references from one to the other is normally called linking. Linking can be an expensive process. Even the earliest versions of Android pre-linked many portions of the system libraries as an optimization: doing it once to save the cost of doing it each time it was necessary.

Of course, the consequence of pre-linking binary libraries is that those libraries cannot, once linked, be versioned independently. For instance, after a symbolic reference to the code that activates the haptic feedback device is linked as a reference to a specific location in memory, updating the haptic feedback library and changing its location would be disastrous.

On current mobile devices, however, this is not a significant issue. System libraries are never updated individually. Updates to the system software always involve updating the entire system as a unit, and all affected components are updated together.

In the Android system, building the pre-compiled, pre-linked image of the ART runtime is part of building the system image. The directory /system/framework contains the .jar files produced by compiling the Android libraries. In the Dalvik era, these .jar files were loaded into an application image and the code within them executed by the DVM.

For ART, those libraries are recompiled with dex2oat into a new subdirectory named for the device architecture (for example, arm). The two most significant files in this directory are boot.art and boot.oat. These two files—the first an image of initialized classes and objects, the second the ELF file containing the code and linking information—are loaded into memory: .art first, and immediately below it, .oat. Together, they comprise the pre-linked, pre-initialized system library image. These two files are soft-linked into /data/dalvik-cache along with the .art and .oat files for installed applications.

And this brings us at last to the second, less common use of dex2oat. A system update will almost certainly mean a new boot.art and boot.oat. These new files cannot be used with the pre-linked .oat files that dex2oat created previously when it compiled new applications as they were installed. A system update requires the recompilation of every single application on the device.

This recompilation is the source of the often seemingly interminable “Android Is Optimizing Applications” dialog that followed system updates on KitKat devices. It was not a great experience; more recent versions of ART do better.

Hybrid ART

Recent versions of ART are smarter and lazier. Why take the time and energy to compile code that might never be used? In what might at first seem like a step backwards, the new hybrid ART contains a Dalvik-like interpreter complete with a JIT.

Most applications are executed at first in the interpreted environment. Although interpreted, this environment is not Dalvik. The interpreted ART environment, for instance, uses the same highly tuned garbage collection system that was introduced with ART in KitKat.

The interpreter collects profile information on the code it executes. It stores the data it collects, per application, in files in the directory /data/dalvik-cache/profiles/. A new system daemon, the BackgroundDexOptService, parses these files occasionally and uses dex2oat.to compile the “hot” 123methods—methods that are 90 percent of the calls in the application. It also recompiles a program if the list of methods that comprise 90 percent of the calls changes by more than 10 percent.

The most immediate effect of this new, hybrid ART is that a system update no longer requires recompilations of every application known to the system and the attending several-minute wait. In addition, though, hybrid ART saves the substantial cost of running dex2oat on applications or parts of applications that are never used.

Zygote Memory Management

Zygote is the solution to one of the oldest problems with Java desktop applications: startup time. Starting a Java desktop application requires each of the following steps:

• The OS resources. The system must find resources for a new, large program. On any machine that has been running for a while, finding those resources might well require swapping some other running program to disk.

• Start the JVM. The JVM, normally a program called java, is a moderately large and complex C++ program. Although it starts fairly quickly, getting it fully initialized does take time.

• Load basic Java libraries. Java loads its libraries lazily from a very large file called rt.jar. Finding and loading the classes required for minimal functionality, though heavily optimized, does take time.

• Initialize the basic libraries. Java classes quite frequently require some initialization (assignments to static variables or static initializer blocks). As each class is loaded, its initialization code must be executed. At this point, on a typical laptop computer, even for a completely trivial application, it may now be several tenths of a second since the user started the application.

• Load the application. This is the first step toward executing the application that the user actually requested. Application classes are, most likely, in another .jar file from which they are, again, loaded lazily. To start executing the program, however, the VM must load the closure of references from the main class: all the things to which it refers; all the things to which any of the referents refer; all the things to which they refer, and so on, until no unresolved references exist. Note, first of all, that this recursive resolution of references may very well require loading many Java system libraries as well as application code. Also note that it does not, by any means, imply the loading of all of them either.

• Initialize the newly loaded classes. Again, some or all of the newly loaded classes may require initialization. Any initialization code in any of the newly loaded classes must be executed.

• At this point, it is entirely possible that the initialization process has used enough memory so that a minor garbage collection (GC) is necessary. Although minor GCs are fast, they do take time.

• After all necessary classes have been loaded and initialized, the VM can begin to execute the application’s main method. For a moderately complex application, it is entirely possible that it has been nearly a minute since the user-initiated startup.

Although this delay is definitely annoying on a laptop computer, it would be devastating on a battery powered mobile device. Wading through the preceding list of initialization procedures each time a new application started would drain a battery in no time. Add to that the idea of waiting for a full minute for a phone application to start up so that you could answer an incoming call. The Android OS had to address this problem to be viable. The answer is the Zygote application.

As mentioned earlier, Zygote is to Android what init is to Linux: the parent of all applications. init starts Zygote as part of bringing up the system. Zygote initializes itself by pre-loading the entire Android framework. Unlike desktop Java, it does not load the libraries lazily. It loads all of them as part of system startup. When completely initialized, it enters a tight loop, waiting for connections to a socket.

Zygote makes use of an important feature of the Linux OS, copy-on-write paging, to eliminate nearly all the items in the Java initialization list. Figure 7.3 is a simplified illustration of how the operating system lays out virtual memory for applications; in this case, Zygote. Memory is organized into uniformly sized pages. When the application refers to memory at a particular address, the device hardware reinterprets the address as an index into a page table and an offset into the associated page to which the index points. It is entirely possible, as shown in Figure 7.3, that an address that points at the top of an application’s virtual memory space refers to a location that is actually near the bottom of physical memory.

When the system needs to create a new application, it connects to the Zygote socket and sends a small packet describing the application to be started. Zygote clones itself, creating a new kernel-level process. Figure 7.4 illustrates the memory layout of a new application cloned from Zygote. The new application has its own page table. Most of the new page table is simply a copy of Zygote’s page table. It points to the exact same pages of physical memory. Only the pages the new application uses for its own purposes are not shared.

The new process is interesting in that it shares memory with Zygote, its parent, in a mode called copy-on-write. Because the two processes are using exactly the same memory, starting the child process is nearly instantaneous. The kernel does not need to allocate much memory for the new process, nor does it need to load the Android framework libraries. Zygote has already loaded everything.

Because both processes have mapped exactly the same physical memory into their virtual address spaces, if either changed the contents of the memory by writing to it, the other would be affected. That would be very bad.

To avoid the problem, the system copies pages on write. The hardware notifies the kernel when either process attempts to write to a shared page. Instead of allowing the write, the kernel allocates a new page of memory and copies the contents of the original page—the page to which a process is writing—into it. After the two processes have separate copies of the page, each can freely modify its own copy. Figure 7.5 represents the state of memory after a new application spawned from Zygote attempts to write to a shared page.

Copy on write is a tremendous savings. In addition to the fast startup, there is only one copy for all processes of any pages that are unaltered by any process (all the library code, for instance). Even if some child process were to write on every single memory page (something that quite literally never happens), the cost of allocating the new memory is amortized over the life of the process. It is not incurred at initialization.

Zygote Startup

As mentioned, Zygote is started by init. Recent Androids typically start multiple copies of Zygote. Depending on the chipset architecture or OEM preference, platform developers use the system variable, ro.zygote, set at platform at build time, to control which of four types of Zygotes are started and which one is “primary.” Most modern Android devices start two zygotes—one for 32-bit applications and one for 64-bit apps—and default to 64-bit version.

The base init script, init.rc, described in Chapter 6, includes the script that starts Zygote:

Click here to view code image

import /init.${ro.zygote}.rc

The sources for the startup scripts are found in the directory:

Click here to view code image

$AOSP/system/core/rootdir

Listing 7.1 shows init.zygote32_64.rc for an older system, on which 32-bit apps were the default. It provides an opportunity to revisit the init scripting language. Note that the service lines have been wrapped to fit the page. They cannot be wrapped this way in the actual script.

Listing 7.1 Zygote init for 32-bit Default Applications

Click here to view code image

service zygote /system/bin/app_process32 
        -Xzygote /system/bin --zygote --start-system-server --socket-name=zygote
    class main
    priority -20
    user root
    group root readproc
    socket zygote stream 660 root system
    onrestart write /sys/android_power/request_state wake
    onrestart write /sys/power/state on
    onrestart restart audioserver
    onrestart restart cameraserver
    onrestart restart media
    onrestart restart netd
    onrestart restart wificond
    writepid /dev/cpuset/foreground/tasks

service zygote_secondary /system/bin/app_process64 
        -Xzygote /system/bin --zygote --socket-name=zygote_secondary
    class main
    priority -20
    user root
    group root readproc
    socket zygote_secondary stream 660 root system
    onrestart restart zygote
    writepid /dev/cpuset/foreground/tasks

Consider the definition of the zygote_secondary service, the 64-bit zygote. The actual application that is started as user root at the very highest priority by init is /system/bin/app_process64. The script requests that init create a stream socket for the process and catalog it as /dev/socket/zygote_secondary: This is the socket that the system will use to start new Android applications. The script also contains an onrestart directive that will restart the primary Zygote if the secondary zygote fails.

The definition of the primary Zygote service, zygote, is nearly identical. It uses a different socket and has a much longer list of other servers that will be restarted if it fails. Note, of course, that if zygote_secondary fails, all of these servers will be restarted as well.

Runtime Initialization

Zygote’s first order of business when it is started is to initialize the ART runtime environment. While reading this, keep in mind that it happens only once; that is, Zygote is started only once, during system startup. Subsequent application startup simply clones the running Zygote.

The applications that init starts, app_process64 and app_process32, are just small applications (source in $AOSP/frameworks/base/cmds/app_process) that parse their command line and then run either Zygote or the class whose name has been passed as an argument. These are the programs used to start all programs (not applications!) compiled to DEX: anything that requires either Dalvik or ART. They serve a purpose very similar to that served by the “java” command, which starts the Java runtime.

The app_process creates an instance of AppRuntime, a subclass of AndroidRuntime. It does a lot of bookkeeping, setting up any runtime parameters (runtime.addOption(...)), the name for the process (runtime.setArgv0(...)), and the name of the class to run (used only when not running Zygote: runtime.setClassNameAndArgs(...)), and then calls AndroidRuntime.start() to invoke the runtime.

The start method (source in $AOSP/frameworks/base/core/jni/AndroidRuntime.cpp) eventually invokes startVM. startVM again is mostly setup.

Starting the runtime has gotten significantly more complex since the advent of ART. It is especially difficult to read the code because nearly all the variables that contain “dalvik” in their names actually apply to ART. Nonetheless, a review of the VM configuration options controlled by this code can prove extremely helpful to a developer. Among the more interesting are -verbose:gc and the compiler option -generate-debug-info. The former logs information about system garbage collection, and the latter causes dex2oat to add debugging information to the system image it builds after a system update.

After processing the options, startVM invokes JNI_CreateJavaVM. Again, the name is confusing: The runtime environment that is about to be initialized has almost nothing to do with Java. To be compatible with code written for Java, though—and that includes the Java JNI API, discussed in the next chapter—this method inherits its name from its JNI counterpart. It is, though, almost certainly not the case that an application that uses the JNI API to start an instance of the Java VM on, say, a desktop Linux system, could successfully initialize the ART runtime on Android without further modification.

The JNI_CreateJavaVM method is defined in art/runtime/jni/java_vm_ext.cc. It, in turn, calls Runtime::Create, defined in art/runtime/runtime.cc. Finally, Runtime::Create initializes the ART runtime, loading the system OAT files and the libraries they contain.

The Android VM is now ready to roll. The argument that app_process passed in its calls to start was "com.android.internal.os.Zygote.Init". Its source is in $AOSP/frameworks/base/core/java/com/android/internal/os/ZygoteInit.java.

System Service Startup

130Most of the code in ZygoteInit’s main method (the method the runtime calls, once initialized) is wrapped in a Java try/catch block. The obvious reason is to allow Zygote to attempt to clean up after a failure. There is a second, very clever purpose to which we will return in a moment.

Zygote has three major tasks, on startup:

• Register the socket to which the system will connect to start new applications

• Preload Android resources and libraries, if necessary

• Start the Android System Server

After it has completed these three tasks, it enters a loop, waiting for connections to the socket.

The first of the tasks, opening the socket, is handled in the registerServerSocket method. The method creates the socket using the name passed in the parameters, probably from the init script originally.

The second task is accomplished with a call to the preload method. Historically, Zygote was initialized aggressively. Modern Androids, however, support a minimal system that postpones the cost of preloading the framework until the first application is spawned.

The preload method loads everything it can think of: classes, libraries, resources, and even a very special case, the WebView (a web browser view that can be embedded in an Android application). Note that, when using the ART runtime, many of the preloads are already in memory because they are in the system OAT file.

At the completion of the preload method, Zygote is fully initialized and ready to clone new applications very quickly, sharing most of its virtual memory with them.

The last of the tasks, starting the System Server, is accomplished in the startSystemServer method. The SystemServer is the first application cloned from Zygote. Following the process is instructive.

The startSystemServer method fakes a connection to the Zygote socket (for subsequent application startup, the connection will be real). It creates a new ZygoteConnection object with the hard-coded content that it would have received over the canonical socket connection were the system trying to start the server. It passes the contents of the connection object to Zygote.forkSystemServer which, in turn, wraps a native method, com_android_internal_os_nativeForkSystemServer (source in $AOSP/frameworks/base/core/jni/com_android_internal_os_Zygote.cpp). The native method first calls ForkAndSpecializeCommon, which does the actual fork system call. There are now two execution paths, one for Zygote, in which the fork system call returned the process id (pid) of the new child process, and one for the newly spawned child process, in which fork returned a pid of 0.

In the parent process (pid > 0) control returns, eventually to ZygoteInit and immediately enters the ZygoteServer method runSelectLoop. This is the endless loop that processes incoming connections with almost the same code that just started the System Server. The call to ZygoteConnection.runOnce eventually calls forkAndSpecialize, the analog of forkSystemServer for the masses.

Perhaps of more interest is what happens to the child process (pid == 0), the process that will become the System Server. First, the method SpecializeCommon is called with a flag indicating that this process will become the System Server. SpecializeCommon in turn calls a series of additional setup methods that set the correct SE Linux context and process capabilities.

After the child process has been set up with the correct process options, it calls Zygote. callPostForkSystemServerHooks and Zygote.callPostForkChildHooks to load the specialized System Server class libraries and to start its application code. Both of these calls trigger calls to the ART runtime native code located in art/runtime/native/dalvik_system_ZygoteHooks.cc.

Next, the child process calls the handleSystemServerProcess method, which, in turn, calls ZygoteInit.zygoteInit, RuntimeInit.applicationInit (source $AOSP/ frameworks/base/core/java/com/android/internal/os.RuntimeInit.java), and, finally, invokeStaticMain, which is where the magic happens.

The invokeStaticMain method finds the class whose name was given in the data transferred to Zygote through the socket (or, in the case of the System Server, faked in hard-coded data), and then obtains a reference to the public static void main method on that class. The very last thing it does is to create a new exception holding the method reference and throw it.

This clever trick is the second reason for wrapping the top-level ZygoteInit code in a try/catch block. This throw, from within invokeStaticMain, clears the entire Java stack back to its very top. At this point, the new instance of Zygote has closed all unneeded file descriptors; set its permissions, group, and user IDs correctly; has the entire Android framework loaded and warmed up; and now has an empty stack. The exception’s run method, called in the catch block, runs the new application’s main method in this fully initialized environment with an empty stack.

When Zygote starts the System Server, it is the class com.android.server.SystemServer (source in $AOSP/frameworks/base/services/java/com/android/server/SystemServer.java) whose main method is run. It is here that all the familiar Android services are started: the PackageManagerService, the ActivityManagerService, the PowerManagerService, and so on.