Observing the Structure of a Package

Android applications are distributed in the form of a zipped archive with the file extension of .apk, which stands for Android Package. The official mime-type of an Android Package is application/vnd.android.package-archive. These packages are nothing more than zip files containing the relevant compiled application code, resources, and application metadata required to define a complete application. According to Google’s documentation at http://developer.android.com/tools/building/index.html, an APK is packaged by performing the following tasks:

• An SDK tool named aapt (Android Asset Packaging Tool) converts all the XML resource files included in the application to a binary form. R.java is also produced by aapt to allow referencing of resources from code.
• A tool named aidl is used to convert any .aidl files (explored in Chapter 7 in “Attacking Insecure Services”) to .java files containing a converted representation of it using a standard Java interface.
• All source code and converted output from aapt and aidl are compiled into .class files by the Java 1.6 compiler. This requires the android.jar file for your desired API version to be in the CLASSPATH environment variable.
• The dx utility is used to convert the produced .class files and any third-party libraries into a single classes.dex file.
• All compiled resources, non-compiled resources (such as images or additional executables), and the application DEX file are used by the apkbuilder tool to package an APK file. More recent versions of the SDK have deprecated the standalone apkbuilder tool and included it as a class inside sdklib.jar. The APK file is signed with a key using the jarsigner utility. It can either be signed by a default debug key or if it is going to production, it can be signed with your generated release key.
• If it is signed with a release key, the APK must be zip-aligned using the zipalign tool, which ensures that the application resources are aligned optimally for the way that they will be loaded into memory. The benefit of this is that the amount of RAM consumed when running the application is reduced.

This compilation process is invisible to you as the developer as these tasks are automatically performed by your IDE but are essential to understanding how code becomes a complete package. When you unzip an APK you see the final product of all steps listed above. Note also that a very strictly defined folder structure is used by every APK. The following is a high-level look at this folder structure:

/assets 
/res 
/lib 
/META-INF 
AndroidManifest.xml 
classes.dex 
resources.asrc 

• Assets—Allows the developer to place files in this directory that they would like bundled with the application.
• Res—Contains all the application activity layouts, images used, and any other files that the developer would like accessed from code in a structured way. These files are placed in the raw/ subdirectory.
• Lib—Contains any native libraries that are bundled with the application. These are split by architecture under this directory and loaded by the application according to the detected CPU architecture; for example, x86, ARM, MIPS.
• META-INF—This folder contains the certificate of the application and files that hold an inventory list of all included files in the zip archive and their hashes.
• classes.dex—this is essentially the executable file containing the Dalvik bytecode of the application. It is the actual code that will run on the Dalvik Virtual Machine.
• AndroidManifest.xml—the manifest file containing all configuration information about the application and defined security parameters. This will be explored in detail later in this chapter.
• Resources.asrc—Resources can be compiled into this file instead of being put into the res folder. Also contains any application strings.

Installing Packages

Behind the scenes, the process of downloading an application from the Play Store and installing it is actually quite a bit more complicated than one would imagine. The simplest way that Google could have implemented this process is to have the Play Store application visit a website and allow the user to browse through the application categories. When the user chooses to install an application Google would provide an “install” link and all that this does is download the APK file over HTTPS from the browser. What is wrong with this approach? Well, considering this method from a security point of view, how does the OS know that the downloaded package came from the Play Store and is safe to install? The APK would be treated like every other download using the browser and therefore no degree of trust can be afforded using this method.

Instead, Google implemented a very modular and robust way to perform installations. When you click the Install button on the Google Play application or website, functionality to deliver and install the application is invoked on the device via the GTalkService. This functionality works from a system application on every Android device and maintains a connection to Google infrastructure via a pinned SSL connection. Various other services such as the Android Device Manager or Google Cloud Messaging (GCM) make use of the GTalkService. The installation process via the GTalkService was explored in an excellent blog post by Jon Oberheide at https://jon.oberheide.org/blog/2010/06/28/a-peek-inside-the-gtalkservice-connection/. The GTalkService gracefully handles cases where the device on which you are installing an application is offline or in a low-signal area. It simply queues the message and delivers it when the device comes online. One of the reasons Android is considered so “open and free” is that so many different ways exist to find and install Android applications. Google does not force users to make use of its Play Store and users can make use of many other application stores instead. Some device vendors and phone carriers like to include their own app stores on devices they sell. A good example of this is the Samsung Apps application that is included on all Samsung devices. Other such examples of popular alternative app stores include Amazon Appstore, GetJar, SlideMe, F-Droid, and a number of big players in the Eastern markets.

In addition to these application stores, multiple ways exist to install new applications onto your device by simply having access to the APK that you would like to install. Making use of an Android SDK tool named ADB (Android Debug Bridge) is one of the simplest ways to do this. Assuming a correct SDK installation, ADB will be on your PATH. Issuing the following command will install an APK onto a connected device or emulator:

$ adb install /path/to/yourapplication.apk 

On Android 4.2.2 and later, making an ADB connection may require you to accept a prompt allowing your computer to connect. The install command of ADB works behind the scenes invoking the package manager on the device (/system/bin/pm). Package Manager can perform a number of actions, including listing all installed packages, disabling an application that came with the device that you consider unnecessary “bloatware,” or obtaining the installed path to a particular application. For all the available options, type the following command and observe the output:

$ adb shell pm 

Another way to install an application could be to host it on a web server. Some application developers choose not to put their application on any app stores and rather serve it from their website. These sites often check for Android browser user agent strings and automatically start the download of their APK. A simple method of hosting the contents of your current folder using Python can be done as follows:

$ python -m SimpleHTTPServer 
Serving HTTP on 0.0.0.0 port 8000 ... 
10.0.0.100 - - [04/May/2014 22:27:14] "GET /agent.apk HTTP/1.1" 200 - 

Browse to http://your_computer_ip:8000 on your device and click on the APK you want to install. You will be prompted with an installation activity.

Other techniques may exist to install applications; however, the ones mentioned here are reliable and work on any device regardless of whether you have root access on it. Other ways may include SSH access to the device or even other installer desktop applications, but these are non-standard ways to perform installations and require additional tools.

ADB

For instance, when installing an application on the device you may see the following output:

$ adb install application.apk 
541 KB/s (156124 bytes in 0.236s) 
     pkg: /data/local/tmp/application.apk 
Success 

This output shows that the user who runs adbd (which is typically “shell” on a normal non-rooted device) has the ability to read, write, and execute files in the /data/local/tmp directory. When exploring a device that is not rooted, you can use this directory but have insufficient privileges to access the /data parent directory.

ADB is the single most useful SDK tool for exploring Android. The following is a list of common tasks that you can perform using ADB:

• List connected devices—$ adb devices
• Get a shell on a device—$ adb shell
• Perform a shell command and return—$ adb shell <command>
• Push a file to a device—$ adb push /path/to/local/file /path/on/android/device
• Retrieve a file from a device—$ adb pull /path/on/android/device /path/to/local/file
• Forward a TCP port on the local host to a port on the device—$ adb forward tcp:<local_port> tcp:<device_port>
• View the device logs—$ adb logcat

If more than one device is connected, prepend the ADB command with -s <device_id>. If you have one connected device and one emulator, instead of providing their device IDs with the -s argument, you can use -d (for device) and -e (for emulator).

Some Android devices may come with a very limited set of utilities installed by default, and having additional tools installed that ease the process of exploring the device is useful.

BusyBox

BusyBox incorporates a large variety of standard Linux utilities into a single binary. A common misconception about running BusyBox on Android is that it requires root. This is incorrect, and users should be aware that executing a BusyBox binary runs it under the same user account and privilege context of the calling process. You can compile BusyBox with the utilities you require or download a pre-compiled binary that includes many utilities. At the time of this writing, the BusyBox website provided pre-compiled binaries for many architectures at http://www.busybox.net/downloads/binaries/. This includes ARM, which is the CPU architecture used by the majority of Android devices. You can download a BusyBox binary for the correct architecture (ARMv7 in this case) from the site and then upload it to the /data/local/tmp directory on your Android device without the need for root access using the following command:

$ adb push busybox-armv7l /data/local/tmp 
77 KB/s (1109128 bytes in 14.041s) 

Get a shell on the device, browse to /data/local/tmp, and mark it executable using the following command:

shell@android:/ $ cd /data/local/tmp 
shell@android:/data/local/tmp $ chmod 755 busybox-armv7l 

Here is an output of the available tools provided by BusyBox:

shell@android:/data/local/tmp $ ./busybox-armv7l 
./busybox-armv7l 
BusyBox v1.21.1 (2013-07-08 10:26:30 CDT) multi-call binary. 
 
... 
acpid, add-shell, addgroup, adduser, adjtimex, arp, arping, ash, 
awk, base64, basename, beep, blkid, blockdev, bootchartd, brctl, 
bunzip2, bzcat, bzip2, cal, cat, catv, chat, chattr, chgrp, chmod, 
chown, chpasswd, chpst, chroot, chrt, chvt, cksum, clear, cmp, comm, 
conspy, cp, cpio, crond, crontab, cryptpw, cttyhack, cut, date, dc, dd, 
deallocvt, delgroup, deluser, depmod, devmem, df, dhcprelay, diff, 
dirname, dmesg, dnsd, dnsdomainname, dos2unix, du, dumpkmap, 
dumpleases, echo, ed, egrep, eject, env, envdir, envuidgid, ether-wake, 
expand, expr, fakeidentd, false, fbset, fbsplash, fdflush, fdformat, 
fdisk, fgconsole, fgrep, find, findfs, flock, fold, free, freeramdisk, 
fsck, fsck.minix, fsync, ftpd, ftpget, ftpput, fuser, getopt, getty, 
grep, groups, gunzip, gzip, halt, hd, hdparm, head, hexdump, hostid, 
hostname, httpd, hush, hwclock, id, ifconfig, ifdown, ifenslave, 
ifplugd, ifup, inetd, init, insmod, install, ionice, iostat, ip, 
ipaddr, ipcalc, ipcrm, ipcs, iplink, iproute, iprule, iptunnel, 
kbd_mode, kill, killall, killall5, klogd, last, less, linux32, linux64, 
linuxrc, ln, loadfont, loadkmap, logger, login, logname, logread, 
losetup, lpd, lpq, lpr, ls, lsattr, lsmod, lsof, lspci, lsusb, lzcat, 
lzma, lzop, lzopcat, makedevs, makemime, man, md5sum, mdev, mesg, 
microcom, mkdir, mkdosfs, mke2fs, mkfifo, mkfs.ext2, mkfs.minix, 
mkfs.vfat, mknod, mkpasswd, mkswap, mktemp, modinfo, modprobe, more, 
mount, mountpoint, mpstat, mt, mv, nameif, nanddump, nandwrite, 
nbd-client, nc, netstat, nice, nmeter, nohup, nslookup, ntpd, od, 
openvt, passwd, patch, pgrep, pidof, ping, ping6, pipe_progress, 
pivot_root, pkill, pmap, popmaildir, poweroff, powertop, printenv, 
printf, ps, pscan, pstree, pwd, pwdx, raidautorun, rdate, rdev, 
readahead, readlink, readprofile, realpath, reboot, reformime, 
remove-shell, renice, reset, resize, rev, rm, rmdir, rmmod, route, rpm, 
rpm2cpio, rtcwake, run-parts, runlevel, runsv, runsvdir, rx, script, 
scriptreplay, sed, sendmail, seq, setarch, setconsole, setfont, 
setkeycodes, setlogcons, setserial, setsid, setuidgid, sh, sha1sum, 
sha256sum, sha3sum, sha512sum, showkey, slattach, sleep, smemcap, 
softlimit, sort, split, start-stop-daemon, stat, strings, stty, su, 
sulogin, sum, sv, svlogd, swapoff, swapon, switch_root, sync, sysctl, 
syslogd, tac, tail, tar, tcpsvd, tee, telnet, telnetd, test, tftp, 
tftpd, time, timeout, top, touch, tr, traceroute, traceroute6, true, 
tty, ttysize, tunctl, udhcpc, udhcpd, udpsvd, umount, uname, unexpand, 
uniq, unix2dos, unlzma, unlzop, unxz, unzip, uptime, users, usleep, 
uudecode, uuencode, vconfig, vi, vlock, volname, wall, watch, watchdog, 
wc, wget, which, who, whoami, whois, xargs, xz, xzcat, yes, zcat, zcip

This is a huge set of tools, many of which do not come as part of the Android image. Some of these tools are common utilities used on a desktop or server version of Linux, such as cp and grep, which the Android image inconveniently left out. Do not expect all the included tools to work fully, because some aspects of Android simply do not work the same as on conventional Linux systems. You can add BusyBox to the shell’s PATH environment temporarily without root by entering the following command:

shell@android:/ $ export PATH=$PATH:/data/local/tmp 

Standard Android Tools

Some useful tools that are present on Android systems in the /system/bin directory include the following:

• pm—This stands for “package manager” and is the command-line package management utility on Android. It performs all tasks relating to installation, uninstallation, disabling, and information retrieval of installed packages. Some useful commands are:
  • List all installed packages—shell@android:/ $ pm list packages
  • Find the stored APK path of an installed application—shell@android:/ $ pm path <package_name>
  • Install a package—shell@android:/ $ pm install /path/to/apk
  • Uninstall a package—shell@android:/ $ pm uninstall <package_name>
  • Disable an installed application (useful for disabling pesky applications that came with your device)—shell@android:/ $ pm disable <package_name>
• logcat—This tool allows you to view system and application logs with flexible filters. This tool can only be invoked by applications or users on the device that have the associated privilege level to do so.
  • If you would like to view all logs, simply run—shell@android:/ $ logcat
  • If you know the name of the tag you are looking for then you can filter by it using—shell@android:/ $ logcat -s tag

• getprop—This tool allows you to retrieve all system properties including verbose hardware and software information.
• dumpsys—This tool displays information about the status of system services. If run without any arguments it iterates through all system services. You can also find these services by running service list.

drozer

drozer is an Android assessment tool that was released in March 2012 at Blackhat EU under the name Mercury. Its original intention was to eliminate the need for writing one-use applications that test for a certain issue, and it has evolved into a full testing suite. It was created because of the need to test each aspect of an Android application in a dynamic way. Put simply, drozer has two distinct use cases:

• Finding vulnerabilities in applications or devices—It allows you to assume the role of an installed Android application and interact with other apps and the underlying operating system in search of vulnerabilities.
• Providing exploits and useful payloads for known vulnerabilities—It does this by building malicious files or web pages that exploit known vulnerabilities to install drozer as a remote administration tool.

Chapter 7 focuses heavily on using drozer to find vulnerabilities, and Chapter 8 delves into the darker side of drozer and ways of using provided exploits to gain access to Android devices as an attacker.

drozer has two different versions: the community and pro editions. The community edition provides the raw power of drozer and gives the user access to a command-line interface only. It is also a fully open-source project that was released under a 3-clause BSD license. The professional version focuses on features that make doing Android security testing easy for people who do it as a part of their job. It provides a graphical user interface that makes visualizing the large amount of information that can be collected during the course of a typical security assessment of an Android device easier. Throughout the following chapters, the community edition of drozer is used for two reasons: It is free, and it facilitates the learning of Android security better than the pro version, mainly because it does not shield you from what it is doing under the hood. For more information about the differences, see the tool’s homepage at https://www.mwrinfosecurity.com/products/drozer/.

$ adb install agent.apk 

Starting a Session

You must first set up suitable port forwarding from your device or emulator to your computer because the embedded server in the drozer agent listens on TCP port (31415 by default). Perform the following command to forward this port to your computer:

$ adb forward tcp:31415 tcp:31415

You can now open the drozer agent on the device and turn on the Embedded Server option as shown in Figure 6.3.

On your computer you can now perform the following command to connect to your agent:

$ drozer console connect 

You should now see a drozer command prompt that confirms your device ID and looks as follows:

Selecting 1f3213a063299199 (unknown sdk 4.4.2) 
 
            ..                    ..:. 
           ..o..                  .r.. 
            ..a..  . ....... .  ..nd 
              ro..idsnemesisand..pr 
              .otectorandroidsneme. 
           .,sisandprotectorandroids+. 
         ..nemesisandprotectorandroidsn:. 
        .emesisandprotectorandroidsnemes.. 
      ..isandp,..,rotectorandro,..,idsnem. 
      .isisandp..rotectorandroid..snemisis. 
      ,andprotectorandroidsnemisisandprotec. 
     .torandroidsnemesisandprotectorandroid. 
     .snemisisandprotectorandroidsnemesisan: 
     .dprotectorandroidsnemesisandprotector. 
 
drozer Console (v2.3.4) 
dz> 

dz> list package 
app.package.attacksurface  Get attack surface of package 
app.package.backup         Lists packages that use backup API (returns 
                           true on FLAG_ALLOW_BACKUP) 
app.package.debuggable     Find debuggable packages 
app.package.info           Get information about installed packages 
app.package.launchintent   Get launch intent of package 
app.package.list           List Packages 
app.package.manifest       Get AndroidManifest.xml of package 
... 

dz> module search -d 
... 
metall0id.root.cmdclient 
   Exploit the setuid-root binary at /system/bin/cmdclient on certain 
   devices to gain a root shell. Command injection vulnerabilities exist 
   in the parsing mechanisms of the various input arguments.       
    This exploit has been reported to work on the Acer Iconia, Motorola 
    XYBoard and Motorola Xoom FE. 
... 
metall0id.tools.setup.nmap 
    Installs Nmap on the Agent.    
    Nmap ("Network Mapper") is a free and open source (license) utility 
    for network discovery and security auditing.
 
mwrlabs.develop 
    Start a Python shell, in the context of a drozer module. 

dz> shell 
u0_a59@android:/data/data/com.mwr.dz $ id 
uid=10059(u0_a59) gid=10059(u0_a59) groups=3003(inet),50059(all_a59) 
 context=u:r:untrusted_app:s0 
u0_a59@android:/data/data/com.mwr.dz $

 
dz> run app.package.info -a com.mwr.dz -h 

dz> run app.package.info -a com.mwr.dz > /path/to/output.txt 

dz> module install mwrlabs.develop 
Processing mwrlabs.develop... Done. 
 
Successfully installed 1 modules, 0 already installed. 

dz> run auxiliary.develop.interactive 
Entering an interactive Python shell. Type 'c' to end. 
 
> /home/tyrone/dz-repo/mwrlabs/develop.py(24)execute() 
-> self.pop_completer() 
(Pdb) context = self.getContext() 
(Pdb) pm = context.getPackageManager() 
(Pdb) name = pm.getNameForUid(10059) 
(Pdb) print name 
com.mwr.dz

Context context = getApplicationContext(); 
PackageManager pm = context.getPackageManager(); 
String name = pm.getNameForUid(10059); 

from drozer.modules import Module 
 
class GetPackageFromUID(Module): 
 
    name = "Get a package's name from the given UID" 
    description = "Get a package's name from the given UID" 
    examples = """ 
dz> run app.package.getpackagefromuid 10059 
UID 10059 is com.mwr.dz 
""" 
    author = "Tyrone" 
    date = "2014-05-30" 
    license = "BSD (3 clause)" 
    path = ["app", "package"] 
    permissions = ["com.mwr.dz.permissions.GET_CONTEXT"]  
    def add_arguments(self, parser): 
        parser.add_argument("uid", help="uid of package") 
 
    def execute(self, arguments): 
        context = self.getContext() 
        pm = context.getPackageManager() 
        name = pm.getNameForUid(int(arguments.uid)) 
        self.stdout.write("UID %s is %s

" % (arguments.uid, name)) 

dz> module repository create /path/to/repository 

dz> run app.package.getpackagefromuid 10059 
UID 10059 is com.mwr.dz

Defining Components

Each Android package contains a file named AndroidManifest.xml in the root of the archive. This file defines the package configuration, application components, and security attributes. Figure 6.6 shows an example manifest.

Only components that are defined in the manifest file are usable inside the application, with the exception of broadcast receivers. One of the most important aspects of securing defined components in the manifest is using strongly configured permissions, which is explored in detail later in this chapter in “Understanding Permissions”.

Interacting with Components

An intent is a defined object used for messaging that is created and communicated to an intended application component. This communication is done through calls to binder. It includes all relevant information passed from the calling application to the desired application component and contains an action and data that is relevant to the request being made. A simple example of an application sending a request to open a particular URL in a browser would look as follows in code:

Intent intent = new Intent(Intent.ACTION_VIEW); 
intent.setData(Uri.parse("http://www.google.com")); 
startActivity(intent); 

The preceding code creates a simple implicit intent to view a URL, and the startActivity() function is called with the intent as a parameter. Any application’s activity that is able to respond to a VIEW action on data that is formatted like a URL will be eligible to receive this intent. If only a single application can handle this intent, the intent is routed to that application by default. Otherwise, an application picker is shown. An application defines “intent filters” in its manifest, which catches the intents that are appropriate for its components. For example, if an activity in your application can handle HTTP links to websites, then an appropriate intent filter looks as follows:

<activity android:name="MyBrowserActivity"> 
    <intent-filter> 
        <action android:name="android.intent.action.VIEW"/> 
        <data android:scheme="http" /> 
    </intent-filter> 
</activity> 

This snippet states that the activity named MyBrowserActivity in this application can handle any intent with an action of android.intent.action.VIEW and has the data scheme of http://.

If you want to make sure that an intent that you send always reaches an application you intend and would not like the system to decide, then you can make use of explicit intents. Explicit intents specify the application and component that the intent should be delivered to. For example, if an application you created needs to explicitly open a URL in the Android browser application, you use the following code:

Intent intent = new Intent(Intent.ACTION_VIEW); 
intent.setData(Uri.parse("http://www.google.com")); 
 
String pack = "com.android.browser"; 
ComponentName comp = new ComponentName(pack, pack + ".BrowserActivity"); 
intent.setComponent(comp); 
 
startActivity(intent); 

You can try this from drozer without having to create a test application as follows:

dz> run app.activity.start --action android.intent.action.VIEW --data-uri 
http://www.google.com --component com.android.browser 
com.android.browser.BrowserActivity 

drozer can be used to interact with all application components in the same easy manner. The following is an example of querying the system settings content provider from drozer that can be queried from any application:

dz> run app.provider.query content://settings/system 
| _id | name                         | value | 
| 1   | volume_music                 | 11    | 
| 2   | volume_ring                  | 5     | 
| 3   | volume_system                | 7     | 
| 4   | volume_voice                 | 4     | 
| 5   | volume_alarm                 | 6     | 
| 6   | volume_notification          | 5     | 
| 7   | volume_bluetooth_sco         | 7     | 
| 9   | mute_streams_affected        | 46    | 
| 10  | vibrate_when_ringing         | 0     | 
| 11  | dim_screen                   | 1     | 
| 12  | screen_off_timeout           | 60000 | 
| 13  | dtmf_tone_type               | 0     | 
| 14  | hearing_aid                  | 0     | 
| 15  | tty_mode                     | 0     | 
| 16  | screen_brightness            | 102   | 
| 17  | screen_brightness_mode       | 0     | 
| 18  | window_animation_scale       | 1.0   | 
| 19  | transition_animation_scale   | 1.0   | 
| 20  | accelerometer_rotation       | 1     | 
| 21  | haptic_feedback_enabled      | 1     | 
| 22  | notification_light_pulse     | 1     | 
| 23  | dtmf_tone                    | 1     | 
| 24  | sound_effects_enabled        | 1     | 
| 26  | lockscreen_sounds_enabled    | 1     | 
| 27  | pointer_speed                | 0     | 
| 28  | mode_ringer_streams_affected | 422   | 
| 29  | media_button_receiver        | 
com.android.music/com.android.music.MediaButtonIntentReceiver | 
| 30  | next_alarm_formatted         |       | 

Chapter 7 shows many more examples of interacting with components using drozer. The ability to find vulnerabilities in application components requires a thorough understanding of their features and how they can be invoked.

Installing an Application

When an application is installed on an Android device, various tasks must be performed by the Package Manager Service and installd to ensure that the OS fully recognizes and knows how to work with it. The following is a high-level view of the steps:

• Determine correct installation location according to package parameters
• Determine if this is a new installation or update
• Store the APK in the appropriate directory
• Determine the application’s UID
• Create the application data directory and set the appropriate permissions
• Extract native libraries and place them in libs directory of application data directory and set appropriate file and folder permissions
• Extract the DEX file from the package and put its optimized equivalent in the cache directory
• Add package particulars to packages.list and packages.xml
• Send a broadcast stating that the package was installed

This installation process was documented in depth by Ketan Parmar in a blog post at http://www.kpbird.com/2012/10/in-depth-android-package-manager-and.html#more. For the purposes of the next discussion, one of the most important points to take away from the previous list is that when an Android package is installed, it is also stored on the device. User-level applications are stored in /data/app/, and applications that came with the system image are under /system/app/.

Here is an example listing of all the APK files present in the /data/app/ folder on an Android 4.4 emulator:

root@android:/data/app # ls -l *.apk 
-rw-r--r-- system   system  ...  ApiDemos.apk 
-rw-r--r-- system   system  ...  CubeLiveWallpapers.apk 
-rw-r--r-- system   system  ...  GestureBuilder.apk 
-rw-r--r-- system   system  ...  SmokeTest.apk 
-rw-r--r-- system   system  ...  SmokeTestApp.apk 
-rw-r--r-- system   system  ...  SoftKeyboard.apk 
-rw-r--r-- system   system  ...  WidgetPreview.apk 

An important point to note is that each of the APK files listed is world readable according to their file permissions. This is the reason downloading them off a device or accessing them without having any particular level of privileges is possible. These same permissions are set on packages stored in the /system/app and /system/priv-app folders.

The Play Store used to have a Copy Protection function that you could enable when publishing an application. Applications that have been installed with this deprecated option reside in /data/app-private/ and are marked with the following file permissions, which do not allow world read access like the other third-party and system applications:

shell@android:/data/app-private # ls -l -a 
-rw-r----- system   app_132    629950 2014-04-18 23:40 com.mwr.dz-1.apk 

These applications have essentially been installed using the FORWARD_LOCK option provided by the Package Manager. You can replicate this installation option by using the following command from an ADB shell on your device:

shell@android:/data/local/tmp $ pm install -l agent.apk 

This installs the package with FORWARD_LOCK enabled, which places its APK in the /data/app-private folder. It should be noted here that this form of “copy protection” is fundamentally broken and relies on users not having privileged access on their device. If users have privileged access they can retrieve the application and redistribute it by other means and install it on other devices without this mechanism having any bearing.

Upon installing an application, in addition to storing the APK on disk, the application attributes are cataloged in files located at /data/system/packages.xml and /data/system/packages.list. These files contain a list of all installed applications as well as other information important to the package. The packages.xml file stores information about each installed application, including the permissions that were requested. This means that any changes made inside this file will directly affect the way that the OS treats the application. For instance, editing this file and adding or removing a permission from an application literally changes the application’s permissions. This fact may be used by application testers on Android to manipulate packages into a desirable state for testing or modification. It has also been used by Android “tinkerers” to build toolkits that allow for the “revocation” of permissions on chosen applications. This, of course, requires privileged access on the device because of the allocated file permissions on packages.xml, which is shown here:

root@android:/ # ls -l /data/system/packages.xml 
-rw-rw----- system   system      57005 2014-04-18 21:38 packages.xml 

Another procedure that takes place at installation time is the optimization and caching of the package’s DEX file. The classes.dex file is extracted from the APK, optimized using the dexopt utility, and then stored in the Dalvik cache folder. This folder exists at $ANDROID_DATA/dalvik-cache on every device (which is normally /data/dalvik-cache). It is optimized so that minimal instruction checking needs to be performed at runtime, and other such pre-execution checks can be performed on the bytecode. For more information about the specific tasks run by dexopt go to https://cells-source.cs.columbia.edu/plugins/gitiles/platform/dalvik/+/android-4.3_r0.9/docs/dexopt.html. The process of creating an ODEX may take time, and this could degrade first-run performance for applications. This is why most system applications on an Android image come pre-”odexed,” or a process of odexing is performed on first startup of the OS. If you explore the filesystem, notice that APKs in the /system/app directory may have an accompanying file with the same name and an extension of .odex. These are the application’s “optimized DEX” files that are stored outside of the package archive.

Pre-optimizing the DEX files means that when applications are run they do not need to be processed and stored in the cache first, which improves the loading time of the application. The processing procedure used by the dexopt utility for converting a DEX to an ODEX is a complex one. It involves parsing each instruction and checking for redundancies that can be replaced and using inline native replacements for methods that are called frequently. This process makes these ODEX files highly dependent on the specific version of the VM in use on the device. As a consequence, it is unlikely that an ODEX file will work on another device, unless the device software type and versions are identical.

Running an Application

Android uses an unusual procedure for starting new applications. It works by having a single application VM started when the OS boots that listens for requests to launch new applications. When it receives a request, it simply fork()’s itself with new application parameters and code to run. The process that listens for new application requests is aptly named zygote. This technique makes the process of creating new application VMs efficient, as core libraries are shared between VMs. When a user clicks on an application icon, an intent is formulated and sent using startActivity(). This is handled by the Activity Manager Service, which sends a message to zygote with all the parameters required to start the application. Zygote listens on a UNIX socket located at /dev/socket/zygote and has the following permissions, which allow only the system UID or root to interact with it:

root@android:/ # ls -l /dev/socket/zygote 
srw-rw---- root     system            2014-05-04 11:05 zygote 

When an application is started, the Dalvik cache is checked to see whether the application’s DEX file has been optimized and stored. If it has not, the system has to perform this optimization, which impacts the application’s loading time.

Discovered Vulnerabilities

A number of vulnerabilities have been discovered in the way that the validation of signatures is performed on APK files. The presented vulnerabilities affect devices up to and including Android 4.3.

$ cat META-INF/MANIFEST.MF 
Manifest-Version: 1.0 
Created-By: 1.0 (Android SignApk) 
 
Name: res/layout-land/activity_main.xml 
SHA1-Digest: tHBSzedjV31QNPH6RbNFbk5BW0g= 
 
Name: res/drawable-xhdpi/ic_launcher.png 
SHA1-Digest: itzF8BBhIB+iXXF/RtrTdHKjd0A= 
... 
Name: AndroidManifest.xml 
SHA1-Digest: HoN6bMMe9RH6KHGajGz3Bn/fWWQ= 
... 
Name: classes.dex 
SHA1-Digest: 6R7zbiNfV8Uxty8bvi4VHpB7A8I= 
... 

Google Bug #9695860

Just two days after bug #8219321 was revealed, a patch was committed (see https://android.googlesource.com/platform/libcore/+/9edf43dfcc35c761d97eb9156ac4254152ddbc55%5E%21/) that revealed another way that could be used to manipulate an APK to the same effect as the Master Key bug. This time, the vulnerability existed in the way that the length of the “extra” field in the local file headers of an entry in the zip archive was calculated in code. Figure 6.8 shows a simplified view of a zip archive containing a single file.

The format provides for a 16-bit length “extra” field, but in the Java code the length was read as a signed 16-bit number. This meant that overflowing this value into a negative length was possible. Exploitation techniques presented by the community were quite involved but putting it simply, the discrepancy between how the Java and C++ implementation parsed these values allowed for the injection of altered files that passed the signature validation. Jay Freeman covers various exploitation techniques in detail at http://www.saurik.com/id/18.

Inspecting the Android Permission Model

Android employs a fine-grained privilege model for applications. Applications have to request “permission” to access certain information and resources on a device. A user who installs an application from the Play Store is presented with an activity displaying the types of information and hardware that the application can access on your device. However, this information is abstracted away from the technical details in newer versions of the Play Store and does not display the details of the actual permission requested. Figure 6.9 shows an example of clicking the “Permission details” option in the Play Store on the Twitter (https://twitter.com/) application.

Each defined permission has a unique name that is used when referring to it in code as well as a friendly label and a more verbose description about what it is for. This means that when an application permission activity shows “Read your text messages (SMS or MMS)” that it actually translates back to the permission with the name android.permission.READ_SMS. If you examine the AndroidManifest .xml file associated with an application, notice the XML describing the defined and requested permissions respectively as <permission> and <uses-permission> tags.

In drozer, to find the permissions that have been requested and defined by a certain application, run the app.package.info module with the package name as the argument (in this case the Android browser):

dz> run app.package.info -a com.android.browser 
Package: com.android.browser 
  Application Label: Browser 
  Process Name: com.android.browser 
  Version: 4.4.2-938007 
  Data Directory: /data/data/com.android.browser 
  APK Path: /system/app/Browser.apk 
  UID: 10014 
  GID: [3003, 1028, 1015] 
  Shared Libraries: null 
  Shared User ID: null 
  Uses Permissions: 
  - android.permission.ACCESS_COARSE_LOCATION 
  - android.permission.ACCESS_DOWNLOAD_MANAGER 
  - android.permission.ACCESS_FINE_LOCATION 
  - android.permission.ACCESS_NETWORK_STATE 
  - android.permission.ACCESS_WIFI_STATE 
  - android.permission.GET_ACCOUNTS 
  - android.permission.USE_CREDENTIALS 
  - android.permission.INTERNET 
  - android.permission.NFC 
  - android.permission.SEND_DOWNLOAD_COMPLETED_INTENTS 
  - android.permission.SET_WALLPAPER 
  - android.permission.WAKE_LOCK 
  - android.permission.WRITE_EXTERNAL_STORAGE 
  - android.permission.WRITE_SETTINGS 
  - android.permission.READ_SYNC_SETTINGS 
  - android.permission.WRITE_SYNC_SETTINGS 
  - android.permission.MANAGE_ACCOUNTS 
  - android.permission.READ_PROFILE 
  - android.permission.READ_CONTACTS 
  - com.android.browser.permission.READ_HISTORY_BOOKMARKS 
  - com.android.browser.permission.WRITE_HISTORY_BOOKMARKS 
  - com.android.launcher.permission.INSTALL_SHORTCUT 
  - android.permission.READ_EXTERNAL_STORAGE 
  Defines Permissions: 
  - com.android.browser.permission.PRELOAD 

Searching for applications that have requested a particular permission using the permission filter is also possible. For verbose package information, make use of the app.package.info module or for a short list, use app.package.list in the following manner, providing the permission of interest as a parameter:

dz> run app.package.list -p android.permission.READ_SMS 
com.android.phone (Phone) 
com.android.mms (Messaging) 
com.android.gallery (Camera) 
com.android.camera (Camera) 

Requesting certain permissions may cause the application’s user identifier to be added to a Linux group. For instance, requesting the permission android.permission.INTERNET puts the application in the inet group. This mapping is shown here:

<permission name="android.permission.INTERNET" > 
    <group gid="inet" /> 
</permission> 

These mappings are defined in /system/etc/permissions/platform.xml. Other permissions may not equate to any group amendments being made and are simply a form of access control. For instance, the READ_SMS permission does not allow the application to read the SMS database directly, but rather allows it to query content://sms and other related content providers. The following drozer command allows a user to query which content providers require the READ_SMS permission:

dz> run app.provider.info -p android.permission.READ_SMS 
Package: com.android.mms 
  Authority: com.android.mms.SuggestionsProvider 
    Read Permission: android.permission.READ_SMS 
    Write Permission: null 
    Content Provider: com.android.mms.SuggestionsProvider 
    Multiprocess Allowed: False 
    Grant Uri Permissions: False 
    Path Permissions: 
      Path: /search_suggest_query 
        Type: PATTERN_PREFIX 
        Read Permission: android.permission.GLOBAL_SEARCH 
        Write Permission: null 
      Path: /search_suggest_shortcut 
        Type: PATTERN_PREFIX 
        Read Permission: android.permission.GLOBAL_SEARCH 
        Write Permission: null 
 
Package: com.android.providers.telephony 
  Authority: mms 
    Read Permission: android.permission.READ_SMS 
    Write Permission: android.permission.WRITE_SMS 
    Content Provider: com.android.providers.telephony.MmsProvider 
    Multiprocess Allowed: False 
    Grant Uri Permissions: True 
    Uri Permission Patterns: 
      Path: /part/ 
        Type: PATTERN_PREFIX 
      Path: /drm/ 
        Type: PATTERN_PREFIX 
  Authority: sms 
    Read Permission: android.permission.READ_SMS 
    Write Permission: android.permission.WRITE_SMS 
    Content Provider: com.android.providers.telephony.SmsProvider 
    Multiprocess Allowed: False 
    Grant Uri Permissions: False 
  Authority: mms-sms 
    Read Permission: android.permission.READ_SMS 
    Write Permission: android.permission.WRITE_SMS 
    Content Provider: com.android.providers.telephony.MmsSmsProvider 
    Multiprocess Allowed: False 
    Grant Uri Permissions: False 
... 

When an application attempts to access one of the content providers listed previously, the operating system will check that the calling application holds the required permission. If it does not hold the appropriate permission, a permission denial results. An example of querying content://sms from drozer, which does not hold the READ_SMS permission by default, is shown here:

dz> run app.provider.query content://sms 
Permission Denial: opening provider 
com.android.providers.telephony.SmsProvider from ProcessRecord{b1ff0638 
 1312:com.mwr.dz:remote/u0a56} (pid=1312, uid=10056) requires 
android.permission.READ_SMS or android.permission.WRITE_SMS

Protection Levels

Each permission that is defined has an associated attribute known as its protection level. Protection levels control the conditions under which other applications can request the permission. Naturally, some permissions are more dangerous than others and this should reflect in the assigned protection level. For instance, third-party applications should never be granted the ability to install new applications (using the android.permission.INSTALL_PACKAGES permission) and the system should not allow it. An author of a number of applications may want to share information or invoke functionality between her applications at runtime in a secure manner. Both of these scenarios can be achieved by selecting the correct protection level on defined permissions. Table 6.2 describes all the available protection levels that can be set on a newly defined permission.

As a practical example of protection levels in action, take a look at what happens when you compile a new drozer agent with the INSTALL_PACKAGES permission and attempt to install it.

$ drozer agent build --permission android.permission.INSTALL_PACKAGES 
Done: /tmp/tmp2RdLTd/agent.apk 
 
$ adb install /tmp/tmp2RdLTd/agent.apk 
2312 KB/s (653054 bytes in 0.275s) 
     pkg: /data/local/tmp/agent.apk 
Success

The package installs successfully but logcat shows a log entry from the Package Manager saying the following:

W/PackageManager(  373): Not granting permission 
android.permission.INSTALL_PACKAGES to package com.mwr.dz 
(protectionLevel=18 flags=0x83e46) 

It refused to grant the INSTALL_PACKAGES permission. This can be confirmed in drozer by displaying the permissions held by the agent:

dz> permissions 
Has ApplicationContext: YES 
Available Permissions: 
 - android.permission.INTERNET 

It is quite obvious that this happened because of the protection level set on the INSTALL_PACKAGES permission, which is signature|system (which equates to an integer protection level of 18. This value comes from performing a Boolean OR operation on 0x02 and 0x10). The drozer agent was not signed by the same certificate as the application that defined the INSTALL_PACKAGES permission (which is usually the package named android) and it did not come as part of the system image. Hence, the request to attain this permission was rejected by the OS. If one application permission request is rejected, the application will still function correctly as long as it handles this rejection gracefully when attempting to use functionality provided by this permission at runtime. If the application does not handle this scenario gracefully it may result in an application crash.

Third-party applications that do not have any intention of sharing data or functionality with applications from other developers should always define permissions with the signature protection level. This ensures that another developer cannot write an application that requests your permission and gains access to your exported components. This may not constitute a direct risk to your application or its data depending on what the permission is used for; however, in most cases this is not desirable from a security perspective. Using the signature protection level does not affect the application’s ability to integrate or communicate with other applications created by the same developer, as these applications would be signed with the same certificate. This is why it is so important that Android packages are signed cryptographically, or else how would Android know which application is fit to hold a particular permission? In fact, Android will not allow you to install an application that is not signed and doing so from ADB will result in an error with the code INSTALL_PARSE_FAILED_NO_CERTIFICATES. The use of permissions with protection levels provides a strong foundation for application security for developers; however, the foundation’s strength depends on the correct configuration of protection levels.

Rooting Explained

On Android, by default no way exists to run an application or some task within it as the root user. This simple fact has led to entire communities of researchers dedicating their time to finding ways to obtain root on various Android devices. There are also very many misconceptions about what rooting your device entails technically and why it is possible (or not) on certain devices. This section sheds light on some of the common rooting methods and gives a technical breakdown of each.

Rooting Objectives

A typical objective of rooting an Android device is so that you can put a su binary in a directory on the PATH (for example, /system/bin or /system/xbin). The job of the su binary is to allow a user to switch security contexts and become another user, including root. The su binary should, however, first determine whether the user should be allowed to impersonate the requested user. The required criteria is different on conventional Linux systems from the methods used on commonly found su packages on Android, but one fact that remains the same is that the su binary needs to be running as root in order to allow the change to another user context. The following shows the file permissions on su on a modern Linux system:

$ ls -l /bin/su 
-rwsr-xr-x 1 root root 36936 Feb 17 04:42 /bin/su 

These permissions tell you that any user can execute this binary and when she does she will be running it as the root user. This is a Set User Identifier (SUID) binary, which sets the user ID to the file’s owner upon execution. You can invoke it from within an application by using code similar to this:

Runtime.getRuntime().exec(new String[]{"su", "-c", "id"}); 

This executes the id command as the root user and works because the su binary is on the PATH, which means that the OS knows where to find it on the system. When using su on a Linux system, it asks for the target user’s password to authenticate the action. However, on Android a different approach is commonly taken because the root user does not have a password. Different root manager application developers use different technical methods but they both come down to the same concept for the user. When an application executes su, an activity is displayed to the user requesting the user’s permission to grant the requesting application root context. These applications usually display information about the application requesting root and what it is attempting to execute. Figure 6.10 shows an example of a prompt from the SuperSU application.

This application works by using a custom version of su that sends a broadcast directly to a broadcast receiver in the SuperSU application. This broadcast contains the requesting application’s information as well as relevant details about which command will be executed as root. After this broadcast is received by the application it displays a prompt to the user with the supplied information. The su binary then polls a file in the private data directory to find out whether permission was granted by the user. According to the user’s decision, su decides to setuid(0) or not.

The information just presented explains how you can allow applications to execute commands as root in a user-controlled manner that in theory is safe. Another objective that an attacker may pursue is gaining persistent root access on a device under his control without the user noticing. For this purpose, a completely unprotected custom version of su is included with drozer as part of the tools.setup.minimalsu module. This su version is meant to be used for post-exploitation on older devices and should not be used for everyday purposes. Here is the code for it:

#include <stdio.h> 
#include <unistd.h>
 
int main(int argc, char **argv) 
{ 
    if (setgid(0) || setuid(0)) 
        fprintf(stderr, "su: permission denied
"); 
    else 
    { 
        char *args[argc + 1]; 
        args[0] = "sh"; 
        args[argc] = NULL; 
 
        int i; 
        for (i = 1; i < argc; i++) 
            args[i] = argv[i]; 
 
        execv("/system/bin/sh", args); 
    } 
} 

This code is simply using setuid(0) and setgid(0) to change to the root user’s context, which means that any application that executes su will receive root context and no checks are performed or prompts shown to the user. An application that has been allowed to run commands as root can control absolutely any aspect of the device and completely breaks the Android security model. This means that it will be able to access any other application’s files or modify their code at rest or at runtime. This is why there are so many warning labels about downloading untrusted applications that require root access. An application that implements poor or malicious code can damage the OS or even ruin it completely.

Retrieving APK Files

If the application you are targeting is on a device that you are able to get ADB access to, you can use this access to retrieve the APK file. Sometimes, finding the package name of a target application can be tricky. For example, look at the twitter application. The following approach lists all installed packages on the device and looks specifically for the word twitter:

$ adb shell pm list packages | grep twitter 
package:com.twitter.android 

This package was easy to find because it had a predictable word in the package name. However, this may not always be the case. For example, to find the package that is started when you click the Terminal Emulator launcher icon, run your search in drozer using the app.packages.list command with a filter for this application’s label.

dz> run app.package.list -f "Terminal Emulator" 
jackpal.androidterm (Terminal Emulator) 

This application would not have been found using the ADB method. To pull this application off the device you first need to find the path where the APK is stored, which you can do using ADB as follows:

$ adb shell pm path jackpal.androidterm 
package:/data/app/jackpal.androidterm-2.apk 

Or using drozer’s app.package.info module and observing the APK Path line in the output:

dz> run app.package.info -a jackpal.androidterm 
Package: jackpal.androidterm 
  Application Label: Terminal Emulator 
  Process Name: jackpal.androidterm 
  Version: 1.0.59 
  Data Directory: /data/data/jackpal.androidterm 
  APK Path: /data/app/jackpal.androidterm-2.apk 
  UID: 10215 
  GID: [3003, 1015, 1023, 1028] 
  Shared Libraries: null 
  Shared User ID: null 
  Uses Permissions: 
  - android.permission.INTERNET 
  - android.permission.WRITE_EXTERNAL_STORAGE 
  - android.permission.ACCESS_SUPERUSER 
  - android.permission.WAKE_LOCK 
  - android.permission.READ_EXTERNAL_STORAGE 
  Defines Permissions: 
  - jackpal.androidterm.permission.RUN_SCRIPT 
  - jackpal.androidterm.permission.APPEND_TO_PATH 
  - jackpal.androidterm.permission.PREPEND_TO_PATH 

To reverse engineer applications from the Play Store, you would need to install them onto a device you own and then use the preceding method. However, sometimes the application you are targeting is not available in the Play Store from your country. You can overcome this issue by using sites to which you provide the package name or Play Store link to your target application, and they provide a direct APK download. Two such sites are

• http://apkleecher.com/
• http://apps.evozi.com/apk-downloader/

Viewing Manifests

A big part of understanding an Android application is obtaining and reviewing the AndroidManifest.xml file associated with the package. A number of tools are available to do this, and this section discusses three of them.

$ aapt dump xmltree /path/to/agent.apk AndroidManifest.xml 
N: android=http://schemas.android.com/apk/res/android 
  E: manifest (line=2) 
    A: android:versionCode(0x0101021b)=(type 0x10)0x5 
    A: android:versionName(0x0101021c)="2.3.4" (Raw: "2.3.4") 
    A: package="com.mwr.dz" (Raw: "com.mwr.dz") 
    E: uses-sdk (line=7) 
      A: android:minSdkVersion(0x0101020c)=(type 0x10)0x7 
      A: android:targetSdkVersion(0x01010270)=(type 0x10)0x12 
    E: uses-permission (line=11) 
      A: android:name(0x01010003)="android.permission.INTERNET" (Raw: 
"android.permission.INTERNET") 
    E: application (line=13) 
      A: android:theme(0x01010000)=@0x7f070001 
      A: android:label(0x01010001)=@0x7f060000 
      A: android:icon(0x01010002)=@0x7f020009 
      A: android:debuggable(0x0101000f)=(type 0x12)0xffffffff 
... 

$ aapt l -a /path/to/agent.apk 

$ unzip agent.apk 
Archive:  agent.apk 
  inflating: res/drawable/ic_stat_connecting.xml 
  inflating: res/layout/activity_about.xml 
  inflating: res/layout/activity_endpoint.xml 
  inflating: res/layout/activity_endpoint_settings.xml 
  inflating: AndroidManifest.xml 
... 
 
$ java -jar AXMLPrinter2.jar AndroidManifest.xml 
<?xml version="1.0" encoding="utf-8"?> 
<manifest 
    xmlns:android="http://schemas.android.com/apk/res/android" 
    android:versionCode="5" 
    android:versionName="2.3.4" 
    package="com.mwr.dz" 
    > 
    <uses-sdk 
        android:minSdkVersion="7" 
        android:targetSdkVersion="18" 
        > 
    </uses-sdk> 
    <uses-permission 
        android:name="android.permission.INTERNET" 
        > 
    </uses-permission> 
    <application 
        android:theme="@7F070001" 
        android:label="@7F060000" 
        android:icon="@7F020009" 
        android:debuggable="true" 
...

dz> run app.package.manifest com.mwr.dz 
<manifest versionCode="5" 
          versionName="2.3.4" 
          package="com.mwr.dz"> 
  <uses-sdk minSdkVersion="7" 
            targetSdkVersion="18"> 
  </uses-sdk> 
  <uses-permission name="android.permission.INTERNET"> 
  </uses-permission> 
  <application theme="@2131165185" 
               label="@2131099648" 
               icon="@2130837513" 
               debuggable="true" 
...

Disassembling DEX Bytecode

Like all other compiled and interpreted code, the Dalvik bytecode contained within DEX files can be disassembled into low-level human-readable assembly.

$ ./dexdump -d /path/to/classes.dex 
... 
    #3              : (in Landroid/support/v4/app/FragmentState$1;) 
      name          : 'newArray' 
      type          : '(I)[Ljava/lang/Object;' 
      access        : 0x1041 (PUBLIC BRIDGE SYNTHETIC) 
      code          - 
      registers     : 3 
      ins           : 2 
      outs          : 2 
      insns size    : 5 16-bit code units 
057050:                                        |[057050] 
android.support.v4.app.FragmentState.1.newArray:(I)[Ljava/lang/Object; 
057060: 6e20 ea03 2100                         |0000: invoke-virtual {v1, 
v2}, 
Landroid/support/v4/app/FragmentState$1;.newArray:(I)[Landroid/support/v4/a 
pp/FragmentState; // method@03ea 
057066: 0c00                                   |0003: move-result-object v0 
057068: 1100                                   |0004: return-object v0 
      catches       : (none) 
      positions     : 
        0x0000 line=137 
      locals        : 
        0x0000 - 0x0005 reg=1 this Landroid/support/v4/app/FragmentState$1; 
        0x0000 - 0x0005 reg=2 x0 I 
 
  source_file_idx   : 1152 (Fragment.java) 
... 

$ java -jar baksmali-x.x.x.jar /path/to/app.apk 

$ java -jar smali-x.x.x.jar -o classes.dex out/ 

IDA

IDA is a very popular disassembler used by reverse engineers all around the world. The power of IDA is its rich user interface and vast support for many different CPU architectures and interpreters. It is a commercial tool sold by Hex-Rays and is available at https://www.hex-rays.com/.

IDA is able to understand the DEX format and provides a user interface with a “graph-view” for understanding the flow of application logic in an intuitive way. Figure 6.12 shows an example of the graph view provided when disassembling a DEX file with IDA.

Decompiling DEX Bytecode

Reading and understanding disassembled code is hard work. The more natural way to review an application would be to obtain the source code. Dalvik bytecode contained within a DEX file is an interpreted language that can be translated back to something that resembles the original source code. This can be performed by tools natively on the DEX file or by first converting the DEX file to standard Java CLASS files.

Dex2jar and JD-GUI

Dex2jar converts Android DEX files to Java Class files. This is useful because many tools are already available that can decompile Java bytecode back to source code. It is open source and you can download it from https://code.google.com/p/dex2jar/. It has grown from just a decompiler into a tool suite that performs many different tasks. However, the focus in this section is on converting Android DEX files to Java files. Here is an example of performing this operation with the d2j-dex2jar utility:

$ ./d2j-dex2jar.sh /path/to/agent.apk -o /output/to/agent.jar 
dex2jar /path/to/agent.apk -> /output/to/agent.jar 

The produced JAR file can now be decompiled back into Java source code using a number of available tools. The most popular choice for decompilation and viewing is JD-GUI. Figure 6.13 shows the converted JAR file open in JD-GUI.

JD-GUI can be downloaded from http://jd.benow.ca/ for all major platforms.

JEB

JEB is a dedicated Android application decompiler that is sold by PNF Software. It comes in two flavors:

• JEB Automation—This command-line decompiler enables you to write scripts and perform bulk analysis of multiple files quicker.
• JEB Full—This includes the command-line decompiler as well as a GUI that allows for easy navigation of the decompiled application. The look and feel of the user interface is very similar to IDA by Hex-Rays.

Figure 6.14 shows an example of decompiling an application in the JEB interface.

JEB works directly on the Android package’s DEX file and does not use any intermediate steps that convert the DEX to a JAR file like other tools. Subtle differences in the Dalvik and Java bytecode sometimes cause other tools to fail to decompile the code. This is what JEB overcomes by performing this decompilation natively on the DEX file. For the casual Android application hacker, this failure may not be a problem. However, if accuracy and quality decompilation is what you are after, JEB offers it at a price. Go to http://www.android-decompiler.com/ for more information about JEB.

Decompiling Optimized DEX Bytecode

DEX files for system applications aren’t usually stored inside their APK. Rather, the code is pre-optimized and stored as an ODEX file. This file is the result of many optimizations that cause it to become reliant on the exact version of the Dalvik VM in use and other framework dependencies. This means that ODEX files cannot be decompiled in the same way as DEX files. In fact, they first need to be converted back to DEX files that have those optimizations and framework dependencies removed.

To perform this conversion from ODEX to DEX you can use smali and baksmali. You download the entire /system/frameworks directory of the device on which the optimization took place, which you can do using ADB:

$ mkdir framework 
$ adb pull /system/framework framework/ 
pull: building file list... 
... 
pull: /system/framework/framework2.odex -> framework/framework2.odex 
pull: /system/framework/framework2.jar -> framework/framework2.jar 
pull: /system/framework/framework.odex -> framework/framework.odex 
pull: /system/framework/framework.jar -> framework/framework.jar 
pull: /system/framework/framework-res.apk -> framework/framework-res.apk 
pull: /system/framework/ext.odex -> framework/ext.odex 
pull: /system/framework/ext.jar -> framework/ext.jar 
pull: /system/framework/core.odex -> framework/core.odex 
pull: /system/framework/core.jar -> framework/core.jar 
pull: /system/framework/core-libart.odex -> framework/core-libart.odex 
pull: /system/framework/core-libart.jar -> framework/core-libart.jar 
pull: /system/framework/core-junit.odex -> framework/core-junit.odex 
pull: /system/framework/core-junit.jar -> framework/core-junit.jar 
... 
123 files pulled. 0 files skipped. 
1470 KB/s (56841549 bytes in 37.738s) 

The target ODEX file can then be disassembled into an assembly-like format that uses the provided framework dependencies and then compiled back into a normal DEX file. For instance, try this on the Settings.odex file that belongs to the settings application.

$ adb pull /system/priv-app/Settings.odex 
2079 KB/s (1557496 bytes in 0.731s) 

You can use the following command to convert the ODEX to smali. By default, it stores the disassembled code in the out/ directory.

$ java -jar baksmali-x.x.x.jar -a 19 -x Settings.odex -d framework/ 

Now the disassembled code can be assembled again into a DEX file.

$ java -jar smali-x.x.x.jar -a 19 -o Settings.dex out/ 

The -a parameter given to smali and baksmali is the API version used by the applications. After you have generated a DEX file you can use your favorite decompilation and viewing tools to analyze the source code.

You can find the API version in use programmatically or by observing which Android version is running on your device and then finding the corresponding API version number. Table 6.5 shows this mapping for all versions available at the time of writing.

This table is going to be useful as a reference for future chapters that will discuss vulnerabilities that were fixed in certain API versions.

Reversing Native Code

The Linux shared object (.so) files that can be included as part of an Android application may also require reverse engineering. This may be a scenario where source code is not available and the code being executed by the native component needs to be understood. Typically, native components run compiled machine code for the ARM architecture; however, Android now runs on multiple other architectures as well. At the time of writing, the supported architectures also included x86 and MIPS.

Disassembly and the understanding of native code in this way is a topic that is beyond the scope of this book. A number of tools are available to disassemble native code, and IDA is one of the most popular choices for this task.

In addition to just disassembling native code, it is possible to decompile it with the Hex-Rays Decompiler. Hex-Rays provides a full decompiler from ARM machine code to pseudo-C output; it is at https://www.hex-rays.com/products/decompiler/ with a hefty price tag attached to it. Multiple open-source attempts have been made at creating a decompiler for ARM machine code, but to date they have not been as successful as commercial counterparts.

Additional Tools

This section lists other tools that may be of interest to an Android reverse engineer.

$ java -jar apktool.jar d /path/to/app.apk output 
I: Baksmaling...  
I: Loading resource table... 
I: Loaded. 
I: Decoding AndroidManifest.xml with resources... 
I: Loading resource table from file: /home/tyrone/apktool/framework/1.apk 
I: Loaded. 
I: Regular manifest package... 
I: Decoding file-resources... 
I: Decoding values */* XMLs... 
I: Done. 
I: Copying assets and libs... 

$ java -jar apktool.jar b output/ new.apk 
I: Checking whether sources has changed... 
I: Smaling... 
I: Checking whether resources has changed... 
I: Building resources... 
I: Copying libs... 
I: Building apk file...

Jadx

Jadx is an open source DEX decompiler project that is in a working state and looks evermore promising each version. It contains command-line tools as well as a GUI to browse decompiled code. Source code and downloads are at https://github.com/skylot/jadx. Figure 6.15 shows the jadx-gui tool that has decompiled an Android application.

Dealing with ART

Android devices making use of the new Android Runtime (ART) convert DEX files into OAT files at installation time. OAT files are essentially ELF dynamic objects that are run on the device and one would assume that they would have to be treated like native code when reverse engineering them. A tool named oatdump performs a similar disassembling function for OAT files as dexdump does for DEX files. Explore the options provided by this tool if you are interested in disassembling an OAT file. However, similarly to dexdump the output is provided in quite a raw manner.

One simple fact that can be used is that the APK file of each installed application is still stored on the device. This means that the DEX file of your target application is still accessible in the normal way even when the converted OAT file is being used on the device. Another interesting detail is that every OAT file contains the original DEX file(s) embedded inside it. Pau Oliva created a script called oat2dex that can extract the DEX file(s) from within a given OAT file. This script relies on radare2 (see http://www.radare.org/) and can be found at https://github.com/poliva/random-scripts/blob/master/android/oat2dex.sh. This can be used if the original APK containing the DEX is no longer available. At the time of writing reverse-engineering tools and techniques for OAT files were still in active research by the security community.