Appendix C. ProGuard

We need not mention that decent application performance is crucial and difficult to achieve. Though you can spend hours profiling your code, there’s another way to grab some low-hanging performance optimization fruit, and even improve your application’s resistance to being hacked or reverse-engineered along the way. Readers with a strong Java background may already know this useful companion: meet ProGuard.

C.1. Overview

We mentioned ProGuard a few times in chapter 14 when we discussed build management and automation, but didn’t explain in detail what it’s good for or when and how you should use it, so that’s what this section will clarify. In a nutshell, Pro-Guard is a Java class file processor that does two useful things for you:

• Make your application smaller and faster
• Make your application difficult to reverse-engineer

It should be mentioned that ProGuard isn’t Android-specific; it was around long before Android. It ships with the SDK though, and can be found in your ANDROID_HOME/tools/proguard folder. The documentation can be found online at http://proguard.sourceforge.net/.

The desire for small and fast applications is something we don’t need to explain. ProGuard can shrink and optimize your classes by processing bytecode in numerous ways, such as removing unused code, inlining method calls, merging class hierarchies, applying final and static modifiers whenever possible, and applying peephole optimizations for things such as arithmetic operations or flow control simplification.

Though every application could benefit from this, the question about reverse engineering may require some explanation. Unless you’re sensitive about crafty developers being able to inspect your application’s innards, there’s typically no need to prevent it from being reverse engineered—converted back to its source code (this is possible by first converting the Dalvik bytecode back into Java bytecode, and then back into Java source code). There are cases where this can become problematic though:

• You’ve hard-coded sensitive data such as passwords in your Java classes.
• Your source code contains sensitive intellectual property that under no circumstances should be share with the world.
• You want to prevent clever developers from bypassing license or security checks by recompiling the application with the checks disabled (for example, this is important when you’re charging customers for the application download using Android’s licensing service).

ProGuard can assist you by obfuscating class, method, and field names, as well as removing structural information such as filename and line number tables, so that it becomes practically impossible to temper with reversed source code. This sounds all cool and useful, but it doesn’t work at the snap of a finger. Therefore, we’ll now show you how you set up your Android projects to be processible by ProGuard, how to trigger ProGuard in your builds, and most importantly, how to set up proper ProGuard rules for your project and where some common pitfalls lurk.

C.2. Enabling ProGuard

You may have wondered about the proguard.cfg file that you get with every project you create using the ADT project wizard (located in the project root folder). That’s where you put the options and rules for how you want ProGuard to process your application. The wizard, thankfully, doesn’t create an empty file, but predefines some reasonable defaults that you can build upon. We’ll see in the following subsection how these rules work in detail. The only thing you have to do is to tell the ADT where the configuration resides by adding a line like this to default.properties:

proguard.config=proguard.cfg

If the ProGuard configuration file isn’t located in the project root, you must change the path accordingly. This is enough to tell the ADT that you’d like to have your class files processed by ProGuard before building an APK, but note that ProGuard will only be active for release builds. For example, when creating an APK via right-clicking the project, then selecting Android Tools > Export Signed Application Package. This makes sense, because while developing the application, ProGuard only gets in your way. Debugging obfuscated methods is as fun as finding a needle in a haystack while blindfolded. Unfortunately, this problem also applies to analyzing error reports from a ProGuarded application that’s already live, and we’ll explain in section C.4 how to handle that.

We went through great lengths in chapter 14 to explain why build management using Ant or Maven is desirable, so the question now is how to run ProGuard when not using the ADT to create release builds. For Maven, there’s been a ProGuard plugin for a while now, and its use in conjunction with the Maven Android plugin is documented at http://mng.bz/9uKq. These days, ProGuard is also fully integrated into the standard Android tool chain. Android’s Ant tasks define a private task called -obfuscate, which will invoke ProGuard as part of the release target. Here’s what we see when running ant release for our HelloAnt project from chapter 14 (output shortened):

Listing C.1. The ProGuard output as seen for HelloAnt

The first thing you should know is that when invoking ProGuard from the Android tools (Ant target or ADT) is that it’ll write four different log files , of which three are of particular importance:

• mapping.txt— Contains the mapping from obfuscated names to their original names. Make sure to always archive this file for every release build! It’ll become important when processing error reports from obfuscated applications (see section C.4).
• seeds.txt— A list of entry points into your application that ProGuard identified. (We’ll explain in a minute why that’s important.)
• usage.txt— A list of classes, fields, and methods that ProGuard removed because it thinks they’re unused. This is a good reference to check how modifying your ProGuard rules will affect the shrinking phase. If something that’s used is listed here, then your shrinking rules are too aggressive. If no unused code is listed here, then your shrinking rules are too coarse or lax.

 

Locating the Proguard Log Files

Note that though the Ant ProGuard target writes these files to bin/proguard/, when ProGuard is triggered from the ADT (right-click project > Android Tools > Export Signed Application Package) instead they’re written to proguard/.

 

 

How Does Proguard Know Which Files to Process?

How does ProGuard know which files to process, including android.jar and other library JARs? You’d normally have to tell ProGuard using the -injars and -libraryjars options. But both the ADT and the Ant tasks set these fields for you, by inspecting your project’s class path and output and libs/folder respectively.

 

From the code listing, you can also see that ProGuard operates in three steps: a shrinking phase , an optimization phase with potentially multiple iterations , and an obfuscation phase . All phases are optional and can be turned off entirely using the -dontshrink, -dontoptimize, and -dontobfuscate switches, respectively. It’s usually a better idea to run all three phases and tweak your rule set so as to suit your needs. For instance, code optimization is generally a good idea, but optimizing too aggressively may break your application, so be careful.

Let’s look at how ProGuard configuration files are structured, how rules are set up, and what you need to watch for when writing rules.

C.3. Writing ProGuard rules

ProGuard is powerful, and contains a plethora of knobs to turn and switches to flip. One thing you should understand is that there’s no single recipe for success; although some rules and options are common to many Android applications, we recommend not adopting anything in a dogmatic way—including those rules you’ll see in this section—but instead adjusting and tweaking a configuration to match your application’s specific requirements. One rule set may be perfect for a Twitter client, but terrible for a game.

Because ProGuard has so many options, and also uses a powerful pattern syntax to match Java code elements (like class and method names), it can be overwhelming at first. Therefore, we’ve put together a simple example application that we’re going to process with ProGuard. The application doesn’t serve any purpose other than doing a few things that will require special treatment while ProGuarding it. We start with a few basic options, which—surprise—won’t be sufficient and crash our application, because ProGuard will shrink and optimize too aggressively. We’ll then add more rules, bit by bit, until the application works again. We hope this will make writing Pro-Guard rules easier to understand and digest.

 

Grab the Project: Proguarded

You can get the source code for this project at the Android in Practice code website. Because some code listings here are shortened to focus on specific concepts, we recommend that you download the complete source code and follow along within Eclipse (or your favorite IDE or text editor).

Source: http://mng.bz/hxHs

 

The application is almost identical to the HelloAnt application from chapter 14, with a few customizations sprinkled in:

• The Hello text view is implemented using a custom widget class (called MyButton) that inherits from Button.
• The button click handler (myClickHandler) is wired to the view directly in the layout XML, not in Java code (something that’s possible since Android 1.6).

The application is shown in figure C.1. The layout XML is in the next listing.

Listing C.2. The layout of a simple application we’re going to shrink and obfuscate

Figure C.1. Our sample application shows a toast when started. When clicking the button (a custom widget class), we show a Toast using an XML click handler.

There’s nothing special about MyButton; it inherits from Button and doesn’t add any new functionality. We do this to illustrate a problem with custom view classes and ProGuard shrinking rules, as we’ll see in a minute. The click handler also doesn’t do anything interesting, apart from showing a toast. Like MyButton, it merely exists to illustrate a typical problem arising when using ProGuard to obfuscate method names. We’ve also added a method that’s not used anywhere, so we want to strip this useless bloat from the final APK. Here’s the full listing of our application’s main Activity.

Listing C.3. Our application’s main Activity

public class MainActivity extends Activity {

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        String toast = StringUtils.repeat("Hello ProGuard! ", 3);
        Toast.makeText(this, toast, Toast.LENGTH_SHORT).show();
    }

    public void myClickHandler(View customView) {
        Toast.makeText(this, "clicked", Toast.LENGTH_SHORT).show();
    }

    public void unusedMethod() {
        System.out .println("I'm not used anywhere!");
    }
}

Let’s quickly summarize what we want ProGuard to do with this application:

• We want to keep MainActivity, because it’s the entry point into our application.
• We want to keep the StringUtils class and its repeat method.
• We want to keep the myClickHandler method.
• We want to keep our MyButton class.
• We want to scrap the unusedMethod.
• All names of classes that we keep should be obfuscated, except MainActivity and MyButton, because they’re referenced from XML.
• All names of methods that we keep should be obfuscated, except myClickHandler, because it’s referenced from XML.
• We want to apply some common optimizations (we’ll have a closer look at that later).

Let’s start at the top. We’ll set up a minimalistic proguard.cfg file that we’ll extend bit by bit to match these requirements. ProGuard shrinking and obfuscation rules are based on whitelist semantics; that means ProGuard will only refuse to shrink or obfuscate those classes that we explicitly list. That implies that we need to tell it to keep at least one class: the entry point into our application, which is MainActivity, and which will therefore appear in seeds.txt. We can do this by adding the following rule to proguard.cfg:

-keep public class * extends android.app.Activity

As you can see, ProGuard’s rule syntax closely resembles Java syntax, which makes it easy to read. The key difference is that it supports numerous wildcards, which we won’t explain here in detail (they’re well documented on the ProGuard website). This particular rule uses the -keep option, which tells ProGuard to neither remove nor obfuscate the names of classes that inherit from android.app.Activity. It’s important to understand here that keeping activities but still obfuscating their names would not be acceptable. That’s because activities are always declared in the application manifest—in XML—so renaming them only in the class files would break the application.

That was easy! Let’s export the signed application package and start the application. Unfortunately, you’ll see this:

java.lang.RuntimeException: Unable to start activity ComponentInfo{
   com.manning.aip.proguard/com.manning.aip.proguard.MainActivity}:
   android.view.InflateException: Binary XML file line #6: Error inflating
   class com.manning.aip.proguard.MyButton

The application crashed with an exception: it failed inflating our custom button view. What went wrong? We already mentioned it: similar to our MainActivity, the custom view class isn’t being used from Java code, but XML, so ProGuard thinks it’s unused and strips it away. A peek into usage.txt verifies this: it lists com.manning.aip.proguard.MyButton. We could add a rule to keep this specific class, but clearly, this is only one instance of a larger problem that’s symptomatic for Android applications: custom View classes should be preserved, or more precisely, any class that defines a constructor that may be invoked during a layout inflation. We can encode this requirement in ProGuard rules as follows:

-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet);
}

-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet, int);
}

These two rules translate to: don’t remove or obfuscate any classes that define constructors (specified using the <init> wildcard) that may be called from a LayoutInflater (see also the JavaDoc comments for these constructors on android.view.View). You may have noticed that we didn’t use the -keep option, but the -keepclasseswith-members option. The latter is different in that it only keeps classes on the condition that they define all listed members. This is useful if you want to keep classes that may be derived from different base classes, but that all define certain fields or methods. If you were to use -keep instead here, then ProGuard would keep all classes, along with the specified constructors if a class defines them.

Rebuilding and starting the application shows that our application starts again. Good! Let’s click our custom button.

java.lang.IllegalStateException: Could not find a method
   myClickHandler(View) in the activity class
     com.manning.aip.proguard.MainActivity for onClick handler on
   view class com.manning.aip.proguard.MyButton

Apparently, we’re still doing something wrong. Looking again at usage.txt reveals that ProGuard removed myClickHandler:

com.manning.aip.proguard.MainActivity:
    22:23:public void myClickHandler(android.view.View)

We already know that it wasn’t referenced from Java code, only XML, but it’s part of MainActivity, which we preserved in our first rule, so why does ProGuard still remove it? That’s a common misunderstanding of the -keep rule. When we set it to keep all classes that extend from Activity, we didn’t supply a class body as part of the class specification to define which members we want to keep along with the class. If you omit this, then ProGuard will merely keep the class itself and its name, but it’ll still happily remove, rename, and optimize anything inside it. We could do something like this instead:

-keep public class * extends android.app.Activity {
    <methods>;
}

This rule will preserve all activities along with all methods they define. This is wasteful though, and obviously, the problem with the click handler is again part of a larger problem symptomatic for Android applications: being able to reference methods from XML. A better solution would be to add a new rule instead:

-keepclassmembers class * extends android.app.Activity {
    public void *(android.view.View);
}

This rule translates to: if an Activity is not removed during the shrinking phase, then keep those methods (and their names) that are public, return no value, and take a single View parameter. That’s exactly the requirement for click handlers to be usable from layout XML with the android:onClick attribute. Rebuilding the application shows that finally our application works as expected.

The rules we introduced here were the minimum rules to make our example application work. There are more rules and options that make sense in almost every Android application, so we’ll quickly look at those now.

C.4. Useful rules and options

The rules we defined earlier are sufficient for our example application. But that’s only because it doesn’t, for example, define a Service. As is the case with Activity, the Service class is an Android component class that’s referenced from the manifest XML, so if we were to use services, then we’d have to extend proguard.cfg with similar rules. The following rules may prove useful in Android applications.

C.4.1. Useful rules

In general, it’s a good idea to always to keep the following Android framework classes:

-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends android.app.backup.BackupAgentHelper
-keep public class * extends android.preference.Preference
-keep public class com.android.vending.licensing.ILicensingService

Even if your application doesn’t use all these classes, it doesn’t hurt to define these rules, and it may save you from pondering application crashes when you add any of these classes but forget to update the ProGuard rules.

A second rule that you almost always want to define is keeping the static CREATOR field that Android uses to parcel objects (see chapter 5). That field is read at runtime via introspection, so ProGuard thinks it’s unused and will remove it. You can prevent that from happening with the following rule:

-keepclassmembers class * implements android.os.Parcelable {
    static android.os.Parcelable$Creator CREATOR;
}

A similar problem arises from the use of native method invocations—methods implemented in a compiled language such as C. Only the method signatures are present in Java code, not the body, so they must be linked against native code. This means that you must prevent ProGuard from obfuscating method names, or linking them to native code will fail. Keep ProGuard from doing so by adding this rule:

-keepclasseswithmembernames class * {
    native <methods>;
}

We use -keepclasseswithmembernames here because we still want ProGuard to remove those methods if we don’t invoke them, but if we do, then their names should remain intact.

The preceding rules are all fairly obvious with respect to their usage, but the last one we’re going to show you here may not be. Let’s look at it first:

-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}

This rule prevents ProGuard from shrinking and obfuscating the values and valueOf methods of any enumerations we define. These methods are special in that the Java runtime itself invokes them via reflection. This is probably also one reason why Google suggests to use Java enums conservatively. They perform worse than final static class fields. Again, if you don’t use Java enums in your code, then you don’t need this, but it doesn’t hurt to keep this rule either.

That covers rules; let’s look at a few useful options.

C.4.2. Useful options

So far, we’ve been looking at rules that tell ProGuard which classes or class members to keep. ProGuard also defines a host of options that affect the behavior of a ProGuard execution. A few general options you typically want to set are:

-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-dontpreverify
-verbose

Stopping ProGuard from using mixed-case class names during obfuscation will ensure that no two class names will end up being written to the same file on case-insensitive operating systems like Windows (A.class and a.class would end up being the same file, breaking everything horribly). Moreover, ProGuard, by default, skips nonpublic (package-level) library classes from being processed, because it assumes that they’re confined to the library. Some libraries expose public classes that inherit from internal, nonpublic classes, so it makes sense to sacrifice a bit of performance to get better coverage through ProGuard. We also want to skip the entire preverification step, because this is only meaningful for applications that target the Java Micro-Edition platform or Java 6. The last option, -verbose, will make ProGuard output more detailed information while processing classes.

We mentioned before that ProGuard also goes through a code optimization step. Except for a few cases, ProGuard will pull all registers and apply all optimizations it knows by default. Though personally, we never have problems with that, you should keep in mind that these optimizations are sometimes quite aggressive. For instance, ProGuard will merge class hierarchies both horizontally and vertically, so as to reduce the number of class files and hence, the final APK size. It’ll also optimize loops and arithmetic operations. If you find that your code doesn’t work as expected anymore, you can turn off optimizations bit by bit. The default proguard.cfg file generated by the ADT turns off all arithmetic, field, and class merging optimizations (using the ! symbol before an optimization identifier will disable that optimization):

-optimizations !code/simplification/arithmetic,!field/*,!class/merging/*

Unfortunately, Google didn’t document why they decided to add this rule, but I suggest you disable it first, and see if your application still works as expected. Remember that ProGuard configurations aren’t collections of cookie-cutter rules. Find what works best for you and go with that. When optimizing, ProGuard performs several iterations of the optimizations. You can tell it how many passes to do, but know that no matter how high you set this value, it’ll stop by itself if it finds that there’s nothing more to optimize:

-optimizationpasses 5

That covers our discussion of configuring and running ProGuard. Before we wrap up this section, we want to quickly look at one more thing: how to reverse obfuscated stack traces from error reports to their original form.

C.5. Processing error reports

We already mentioned the one thing about obfuscation that will sooner or later get in your way: processing error reports from an obfuscated application that’s out in the wild. The first thing you usually want to do when receiving an error report is look at the stack trace. That’s difficult, if not impossible, when all classes and methods suddenly have cryptic, meaningless names.

In order to demonstrate this, let’s crash our application by introducing a Bomb class:

public class Bomb {

    public void explode() {
        throw new RuntimeException("Boom!");
    }
}

Let’s have it explode in our onCreate method:

public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.main);

    String toast = StringUtils.repeat("Hello ProGuard! ", 3);
    Toast.makeText(this, toast, Toast.LENGTH_SHORT).show();

    new Bomb().explode();
}

If we now start our application, we’ll see this stack trace appear in the device logs (shortened for brevity):

java.lang.RuntimeException: Unable to start activity ...MainActivity}:
     java.lang.RuntimeException: Boom!
...
Caused by: java.lang.RuntimeException: Boom!
   at com.manning.aip.proguard.MainActivity.onCreate(Unknown Source)
   at android.app.Instrumentation.callActivityOnCreate(
   Instrumentation.java:1047)
   at android.app.ActivityThread.performLaunchActivity(
   ActivityThread.java:2627)
   ... 11 more

As you can see, there’s neither a line number nor a source filename next to the line in the stack trace where the error originated. That’s because ProGuard by default removes this information to make reverse engineering even harder. We can prevent it from doing so by adding the following option to our proguard.cfg:

-keepattributes SourceFile,LineNumberTable

This will bring back the source filename and the line numbers in stack traces. This doesn’t solve the problem with obfuscated method names. In our overly simplified example, we don’t have this problem, because ProGuard refused to obfuscate the onCreate method for a simple reason: it overrides a superclass method of the same name. Because android.jar isn’t targeted by ProGuard (it’s merely used for code analysis; after all, android.jar isn’t part of your application, it’s part of the platform), it couldn’t change its name. Still, in more complex scenarios you’ll have to revert the stack trace to its original form, and that’s what the retrace tool does. The retrace tool is a companion tool to proguard, and can be found in the same place in your SDK home directory. The only thing it does is read obfuscated stack traces, matching them against a ProGuard mapping file (mapping.txt, as seen earlier), and putting out the original stack trace:

$ retrace proguard/mapping.txt stacktrace.txt

Now it should become clear why it’s so utterly important that you archive the mapping file with every application release. Without that file, you won’t be able to retrace stack traces.

C.6. Summary

This wraps up our discussion of ProGuard. We’ve shown you that ProGuard can be a powerful companion to make your applications smaller, faster, and harder to reverse engineer, but it also takes a fair amount of time to configure and fine-tune it to suit your needs. It’s a shame that ProGuard doesn’t receive more attention in the Android documentation, but it shares this somewhat shadowy existence with another neat tool, the Monkeyrunner, which we’re going to cover in appendix D.