© Sheran Gunasekera 2020
S. GunasekeraAndroid Apps Securityhttps://doi.org/10.1007/978-1-4842-1682-8_8

8. Rooting Your Android Device

Sheran Gunasekera1 
(1)
Singapore, Singapore
 

You may have heard about “rooting” your device. You’ve already probably done it and are enjoying its benefits. In this chapter, we will take a look at what it means to root your device, what benefits you can get out of it, and how to actually root a device. We also take a look at some of the things we normally do on an emulator, but this time on the device proper.

The Internet is full of forums where hapless users bemoan their failed efforts at rooting. The perils of a bad rooting session could mean the infamous “bricking” of your device, essentially rendering it completely useless save for securing loose papers on your desk during a windy day where you’ve left your windows open. I insist that if you wish to follow along with our rooting exercise, you do it on a phone that is not your daily driver where you save all your data. If you have the means and are so inclined, I recommend you pick up a cheap used Android phone and use that instead. I’ll talk you through the steps on how I rooted my Google Pixel XL in this chapter. The process should be the same for other devices as well. Samsung’s will be trickier, and I do not recommend that you use my technique if you have a Samsung. I will expand on the rooting processes as I gather more data. I will make these available on my companion site to this book which is www.aas2book.com.

What Is Root?

Root refers to the Linux or Unix administrator account. Often called the superuser, the root user is able to modify all parts of a Unix system. It has the highest access rights and allows a user to do things like create and delete other users, read and change any part of the filesystem or configuration, install and remove software on the system, and view all network traffic and process-related information on that system. Essentially, if you have root on a Unix system, you can do anything to it as the root user.

As we know, Android is based on Linux and adopts most parts of the Linux kernel. Therefore, it is able to run a host of Linux-based programs and services. The Android shell that you access by typing in adb shell is known as the MirBSD Korn Shell, also called mksh . This continues to be the default shell that was shipped from Android 4.0 onward. Through this shell, you can run Linux commands like ls, cd, cat, echo, and so on. In essence, the Android shell looks and feels as if you are in a Linux machine. By default, the root user and access to it has been restricted by the Android system. Thus, whenever you type in adb shell, you get placed into the default user called shell with user id 2000. You can find this information by typing in the Linux id command:
1|marlin:/ $ id
uid=2000(shell) gid=2000(shell) groups=2000(shell),1004(input),1007(log),1011(adb),1015(sdcard_rw),1028(sdcard_r),3001(net_bt_admin),3002(net_bt),3003(inet),3006(net_bw_stats),3009(readproc),3011(uhid) context=u:r:shell:s0
marlin:/ $

On a Linux system, to switch to the root user, you can invoke the su command which is called the substitute user command. If you know the root password, invoking su will first prompt you for the root password and then drop you into a root shell. You will know you are in a root shell because the prompt usually changes to a #.

Why Root?

There are many reasons you may want to root your device. Some that come to mind include the desire to customize your device how you want to. For example, if you’re not satisfied with how the system fonts look, or how nav buttons look, then you can easily change that when you have root on your device. Similarly, you may want to remove OEM bundled system applications that take up unnecessary space on your device. You may want to take a look at the data that apps are writing to the file system, or you may want to debug your system. There are also many malicious reasons why someone would want to root an Android device. Some of the most popular reasons are for downloading and running cracked software without paying for it or watching content that has DRM or digital rights management encryption on it – pirating.

We will use root for debugging and security testing. Because Android is based on Linux, it can use many of the Android debugging tools. We will also use root to take a deeper look into the data that each app stores in its own space. Android separates apps by user account. This means that each app will get its own user id and group id and storage directory within the /data partition. As is common with Linux, one user may not see another user’s files without explicitly being granted permission to. If you do a long listing in Linux (ls -al), you will see the file permissions clearly visible.
drwxrwx--x 3 shell shell    4096 2020-05-22 00:46 .
drwxr-x--x 4 root  root     4096 1970-01-01 13:06 ..
drwxrwxrwx 4 shell shell    4096 2020-05-22 00:46 .studio
-rwxrwxrwx 1 shell shell  190464 2017-03-30 22:24 jtrace64
-rwxrwxrwx 1 shell shell   10432 2017-03-30 22:45 plugin.so
-rwxrwxrwx 1 shell shell 5614728 2020-05-16 19:53 strace
-rw-r--r-- 1 root  root  5221990 2020-05-16 23:07 symbols
-rw-r--r-- 1 root  root  1002367 2020-05-16 19:54 xxx
-rw-rw-rw- 1 root  root   506465 2020-05-16 17:45 xxxx
The first column in the preceding directory listing shows you the permissions of each file. The format is like this:
- r w x r w x r w x

The first character indicates if the entry is a directory. If it is, it will have a d in the first character space. The next three characters tell you if the owner and creator (in the third column) of the file can read (r), write (w), or execute (x) the file. The next three tell you the access permissions of the group (in the fourth column). The last three characters tell you what permissions others or “the world” has. If you look at the preceding entry called symbols, you will see that the owner of the file, root, has read and write access to the file. The root group has read, and the world also has read access to the file.

By setting only permissions of the owner of the file to read and write and disabling all other permissions, it is impossible for another user to view or access that file. This is exactly what Android has done to keep app-specific data separate. Each app will have a user and group id and have its data owned only by that user id. Other apps will have their own permissions and will not be able to access each other’s data. Here is what it looks like when we view the /data/data partition (where Android stores all its app-related data) for the two apps that we wrote:
marlin:/data/data # ls -al com.redteamlife*
com.redteamlife.aas2.aas2client:
total 80
drwx------   8 u0_a178 u0_a178        4096 2020-05-20 18:33 .
drwxrwx--x 229 system  system        20480 2020-05-22 00:46 ..
drwxrws--x   2 u0_a178 u0_a178_cache  4096 2020-05-17 15:36 cache
drwxrws--x   2 u0_a178 u0_a178_cache  4096 2020-05-17 15:36 code_cache
drwxrwx--x   2 u0_a178 u0_a178        4096 2020-05-20 18:33 databases
drwxrwx--x   2 u0_a178 u0_a178        4096 2020-05-20 18:33 files
drwxrwx--x   2 u0_a178 u0_a178        4096 2020-05-20 18:33 no_backup
drwxrwx--x   2 u0_a178 u0_a178        4096 2020-05-20 18:33 shared_prefs
com.redteamlife.aas2.aas2obfuscate:
total 80
drwx------   8 u0_a187 u0_a187        4096 2020-05-25 23:47 .
drwxrwx--x 229 system  system        20480 2020-05-22 00:46 ..
drwxrws--x   2 u0_a187 u0_a187_cache  4096 2020-05-22 00:46 cache
drwxrws--x   2 u0_a187 u0_a187_cache  4096 2020-05-22 00:46 code_cache
drwxrwx--x   2 u0_a187 u0_a187        4096 2020-05-25 23:47 databases
drwxrwx--x   2 u0_a187 u0_a187        4096 2020-05-25 23:47 files
drwxrwx--x   2 u0_a187 u0_a187        4096 2020-05-25 23:47 no_backup
drwxrwx--x   2 u0_a187 u0_a187        4096 2020-05-25 23:47 shared_prefs
marlin:/data/data #

The two app bundle names are com.redteamlife.aas2.aas2client and com.redteamlife.aas2.aas2obfuscate. You will notice that aas2client has user and group id u0_a178, and aas2obfuscate has user and group id u0_a187. Notice that the world or other users can only execute (x) or list the directory of that app. They cannot actually read or write to those directories and files. As root, this restriction is lifted, and you can look in every app’s files and directories. We should pause here and understand why it is a bad idea to store any sensitive data like API keys and back-end system passwords on the filesystem of your app. If the owner of the phone roots his device, then he can go into your app data directory and read all the files that you have written.

Sometimes, you may want to run, test, and debug apps that do not run on the emulator. By rooting your device, you can run and debug these apps on your device. You may also want to do malware analysis, and in cases where malware can detect and refuse to run in an emulator, you can run them on a rooted device. I don’t recommend doing this unless you have set the device up properly where the malware cannot escape your device and network and proceed to wreak havoc elsewhere.

Rooting Safely

Unless you are very well aware of Android internals, you will have to rely on a third party to help root your phone. Here, again, it is important to know how your third-party app will root your device. Downloading root apps willy-nilly can get you into hot water, especially if the author has embedded some malicious backdoor code that allows him to return to your device whenever he wants and gain full control of it. The most popular mechanism of rooting that is out there presently is called Magisk. I will take you through the rooting process that Magisk uses. Magisk is an open source project which you can build yourself. You have the freedom to inspect the code to see exactly what it is doing and, therefore, can gain some reasonable confidence that no malicious intent exists. You can find Magisk here: https://github.com/topjohnwu/Magisk.

As I stressed earlier, I highly recommend getting a separate device to test your root first. You can of course use an Android emulator, but in some cases, there are apps that will verify you are running on an emulator by doing simple things like fetching the default Bluetooth adapter which will always be undefined or null if run on an emulator. If you absolutely have to root on your own device, then it goes without saying that you need to take a backup of all your data. The rooting method I outline will wipe your data partition anyway, so a backup is essential.

The Rooting Process

The rooting process with Magisk is quite straightforward. Magisk was developed by John Wu who ironically now works for Apple. He still actively maintains Magisk, though, and it is quite an elegant set of tools he has put into place. Here is an overview of what we will do to root our device:
  1. 1.

    Download the matching AOSP (Android Open Source Project) images that match our device

     
  2. 2.

    Install Magisk Manager on the device via adb

     
  3. 3.

    Extract the boot image from the AOSP image

     
  4. 4.

    Patch the boot image through Magisk on the device

     
  5. 5.

    Unlock the bootloader

     
  6. 6.

    Flash the boot image that we patched onto the device

     

For the purposes of this chapter, the device I will use is the Google Pixel XL. Its codename is marlin, and it is running the latest OS version 10.0.0 (QP1A.191005.007.A3, Dec 2019). I will not be able to cover other devices’ rooting in this chapter. I do intend to test other devices with root and upload the write-ups to a companion website for this book [https://aas2book.com].

Getting the Factory Image

Android factory images can be found here: https://developers.google.com/android/images. The images range from the Nexus S 4G all the way up to the most recent Pixel 4 and Pixel 4 XL. It is important that we match the factory image to the device. You can always check the phone to see what factory image you would need to download. Go into Settings ➤ About Phone. Then look for both the Model and the Build Number (Figure 8-1). If your phone is another make or model, you will have to check with that phone manufacturer’s website for factory images to download. Additionally, there are different mechanisms when it comes to rooting Samsung and Huawei phones. Both are covered on Magisk’s website [https://topjohnwu.github.io/Magisk/install.html].
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig1_HTML.png
Figure 8-1

Getting the build number and the device model

It is definitely worth noting that the factory images for the device are over 1 GB in size so you will have to spend some time downloading them and also need the space to store them. Hopefully, this isn’t an issue in the present day, but you have been warned – especially when you realize that you only need one small file from the image. For my device, I downloaded the appropriate version of my image which amounted to 1.4 GB. It came as a ZIP file which I decompressed as shown in Figure 8-2.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig2_HTML.jpg
Figure 8-2

The downloaded image and location of the files when decompressed

Inside the folder, you will see one more ZIP file. Using my specific case, I see the file named image-marlin-qp1a.191005.007.a3.zip. Decompress this file as well, and you should then see another folder that contains the file called boot.img. This is the file that we’re interested in. The structure of the folders is shown in Figure 8-3.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig3_HTML.jpg
Figure 8-3

The decompressed structure of the factory image

Installing Magisk Manager

Magisk Manager is no longer available on the Google Play Store. Instead, you have to get it from the GitHub site [https://github.com/topjohnwu/Magisk/releases]. Look for the release named Magisk Manager and download that. At the time of writing this chapter, version 7.51 was released. I will download the APK file now in Figure 8-4.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig4_HTML.jpg
Figure 8-4

Downloading Magisk Manager v 7.51

Once downloaded, you can go ahead and install it by running adb install:
➜  adb install MagiskManager-v7.5.1.apk
Performing Streamed Install
Success

Patching the boot.img File

When installed, you should see the icon in your app screen. Before running it, let’s copy over the boot.img to the device so that we can patch it. Go to the directory where you decompressed your boot.img file and then copy it across to the device. We will copy it straight into the Download directory of the device:
➜  adb push boot.img /storage/emulated/0/Download
boot.img: 1 file pushed, 0 skipped. 29.8 MB/s (31712486 bytes in 1.013s)
Verify that the boot.img is now visible on your device (Figure 8-5).
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig5_HTML.jpg
Figure 8-5

The uploaded boot.img file on the device

Let’s now patch the boot.img file using Magisk Manager. Fire it up and you should see something like Figure 8-6 greet you. Under the Magisk entry, click Install, then click Install again when the dialog window opens, and lastly choose Select and Patch a File (Figure 8-7).
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig6_HTML.jpg
Figure 8-6

The Magisk Manager main screen when Magisk isn’t installed

../images/273312_2_En_8_Chapter/273312_2_En_8_Fig7_HTML.png
Figure 8-7

Selecting which method to use when installing Magisk

The standard file browser should pop up allowing you to select the boot.img file to patch. If you get prompted for permissions to access the storage, click Allow. Find the boot.img file that you copied over previously and select it. Magisk should then patch the boot.img, and when it is done, you should see something like Figure 8-8.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig8_HTML.png
Figure 8-8

boot.img has been successfully patched

Now, if you navigate back to your Download folder on your device, you should see a new file called magisk_patched.img . This is the patched image file that we will flash onto the device in order to get root. First copy it over from the device back to your workstation using adb:
➜  adb pull /storage/emulated/0/Download/magisk_patched.img
/storage/emulated/0/Download/magisk_patched.img: 1 file pulled, 0 skipped. 33.8 MB/s (31937832 bytes in 0.902s)

Unlock the Device Bootloader

Android devices have several partitions on their filesystems. Each partition will perform a specific function. The Android bootloader contains a set of instructions that help the device find and boot up the Android kernel. It contains a very minimal user interface and USB interface that allows you to interact with it (as you will soon see). It also permits users to flash further or additional partitions onto the device. Generally, the bootloader is always locked by the vendor of a device. This is to prevent users from arbitrarily flashing additional or different partitions onto the device to alter their behavior (typically rooting). The bootloader effectively offers up some level of protection (almost like write protection) where you can’t either accidentally or purposely flash images onto the device. Some vendors allow you to unlock the bootloader (which you need to flash our modified boot.img file); some do not. You may thus read about specific brands of phones being ideal candidates for rooting. These ideal candidates typically allow easy unlocking of their bootloader. In the absence of bootloader unlocking, the next available mechanism of gaining root access is to rely on a known vulnerability that gets you root by exploiting a weakness in the code to elevate privileges. These do not stay active for very long, however, as vendors typically find and patch these flaws in subsequent versions of their Android updates. On the Google Pixel range of phones, the bootloader is very easy to unlock as you will see. First, we have to reboot our device into the bootloader mode. Then we issue a command to unlock the bootloader itself.

As a matter of security, since an unlocked bootloader effectively gives an attacker much easier access to compromise a device by bypassing a security pin, whenever a bootloader is locked or unlocked, it wipes the entire data partition to safeguard the user’s private data.

PROCEEDING WITH THE BOOTLOADER UNLOCK IN THIS NEXT SECTION WILL WIPE ALL YOUR DATA!!! MAKE BACKUPS!

With your device still connected to your workstation via USB, issue this adb command:
➜  adb reboot bootloader
Your device should now restart, and you should see a screen similar to Figure 8-9.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig9_HTML.png
Figure 8-9

The Android bootloader screen

You will notice that the last line says Device is LOCKED. Let’s unlock it. With your device still connected, go to your workstation command line and issue the command (remember issuing this command wipes all your data):
➜  fastboot flashing unlock
                                                   OKAY [  0.034s]
Finished. Total time: 0.035s
On your device, you should then see a prompt asking you to confirm. This screen is shown in Figure 8-10. Use the volume up and down keys to select the Yes option and then press the power button to confirm.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig10_HTML.png
Figure 8-10

Confirmation on whether to unlock the bootloader

Flashing the Modified boot.img

After this, the device goes back to the bootloader screen, and now the screen should say Device is UNLOCKED. At this time, we are ready to flash our Magisk patched boot.img. Go to the directory where you downloaded the magisk_patched.img file previously, and then issue the command:
➜  fastboot flash boot magisk_patched.img
Sending 'boot_b' (31189 KB)                        OKAY [  0.851s]
Writing 'boot_b'                                   OKAY [  0.297s]
Finished. Total time: 1.301s
This should flash the new boot.img onto the device. At this point, either the phone will restart by itself or you will have to restart it. On the bootloader screen, press the volume up or down keys until you see the green arrow with the word Start like in Figure 8-9. Then press the power button to restart the device. You will see now that the startup screens have been altered somewhat. First, in Figure 8-11, you will see this warning telling you that the bootloader has been unlocked and that Android cannot check the integrity of your device software. Then, you will notice in Figure 8-12 an unlocked padlock icon on the Google startup screen.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig11_HTML.png
Figure 8-11

The warning that the bootloader has been unlocked

../images/273312_2_En_8_Chapter/273312_2_En_8_Fig12_HTML.jpg
Figure 8-12

The Google startup screen with unlocked padlock icon

Completing the Rooting Process

After this, allow your device to reboot and then configure it as you need to by adding your Wi-Fi, Google account, and others. Also you need to reenable Developer Options and turn on USB Debugging. Then, reinstall Magisk Manager like before and start it up. Magisk should now prompt you to complete your setup. Click OK and make sure that you’re connected to the Internet. Magisk will continue the installation and then automatically reboot once again. When the reboot completes, open Magisk once again, and you should now see that Magisk Manager says Magisk is installed (Figure 8-13).
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig13_HTML.png
Figure 8-13

Magisk now shows up as installed

Let’s test out root now. On adb open up an adb shell and then see if you are able to su to the root user:
➜  adb shell
marlin:/ $ su
marlin:/ #
When you switch to root by typing su, you should receive a prompt on your device from Magisk that looks like Figure 8-14. Pick for how long you want to grant root privileges and then click the Grant button.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig14_HTML.png
Figure 8-14

The Magisk superuser permission request window

You will notice that Magisk tells you that if you’re not sure, you should deny permission. In case you were rooted without your knowledge using Magisk, then you will get this notification each time a program wants root access. It’s kind of a safety mechanism. That’s it, you now have root on your device!

Looking a Little Bit Deeper

So what does Magisk actually do under the hood? I took a closer look at what Magisk does for the Google Pixel, and essentially, it boils down to a few main things.

Magisk will patch the ramdisk and copy over the magiskinit binary to the ramdisk. It will extract the magisk binary from within the magiskinit file. It then configures the boot process of the device so that it runs itself first, does some setup tasks required for staying active as a service, and then calls the normal boot process.

Magisk will patch the kernel at offset 0x017853f6 to require the loading of the ramdisk from the kernel (Figure 8-15 shows the Magisk patch when applied to the kernel). This step is essential so that the patches to the ramdisk can take effect.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig15_HTML.jpg
Figure 8-15

The patch that Magisk made from skip_initramfs to want_initramfs

Further details can be found on the Magisk website (https://topjohnwu.github.io/Magisk/details.html and https://topjohnwu.github.io/Magisk/tools.html).

Other Ways of Rooting

If we are purely after rooting for the purposes of being able to debug our applications and not so much for customizing the look and feel of the device, then it is possible to build an Android image from scratch using the AOSP source code. I state it here as an option, but will not be going into depth in this book.

Testing Frida

Now that we have root on our device, let’s do a test with Frida but this time using our device instead of the emulator. There is one change that you will have to make and that is in uploading the server. When we used Frida on our emulator, we had to download the Frida x86 server. For my Google Pixel XL, I will use the Frida ARM64 server. To get this, head down to the Frida releases [https://github.com/frida/frida/releases] page and pick up the server that says frida-server-12.9.4-android-arm64.xz. Then, as before, you will have to decompress the image using unxz and push it to your device:
(p3) ➜  unxz frida-server-12.9.4-android-arm64.xz
(p3) ➜  adb push frida-server-12.9.4-android-arm64 /data/local/tmp/frida-server
Then get a shell into your device, switch to the root user, and run the server:
1: (p3) ➜  adb shell
2: marlin:/ $ su
3: marlin:/ # cd /data/local/tmp
4: marlin:/data/local/tmp # chmod u+x frida-server
5: marlin:/data/local/tmp # chown root:root frida-server
6: marlin:/data/local/tmp # ./frida-server &
Now that we have the Frida server running, let’s try to do a simple experiment with breaking SSL Pinning. I have tested and know that the Google Play Store and pretty much all Google apps will use SSL Pinning. Let’s try to bypass that so that we can take a look at what traffic goes back and forth between the device and the Play Store whenever we’re browsing the store. First, get Burp Proxy up and running and start the proxy on port 8888 by going to Proxy ➤ Options ➤ Proxy Listeners ➤ Add. Enter your port and host IP address for your workstation running Burp. Next, configure your proxy to point to the host running Burp. Figure 8-16 shows a screenshot of my Pixel XL.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig16_HTML.png
Figure 8-16

Configuring the proxy for your device’s Wi-Fi connection

We will use the frida-multiple-unpinning script to defeat SSL Pinning of the Google Play Store. On your workstation or host, run the following:
(p3) ➜  frida -U --codeshare akabe1/frida-multiple-unpinning -f com.android.vending --no-pause
     ____
    / _  |   Frida 12.9.4 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://www.frida.re/docs/home/
Spawned `com.android.vending`. Resuming main thread!
[Pixel XL::com.android.vending]->
======
[#] Android Bypass for various Certificate Pinning methods [#]
======
[-] OkHTTPv3 pinner not found
[-] Trustkit pinner not found
[-] Appcelerator PinningTrustManager pinner not found
[-] OpenSSLSocketImpl Conscrypt pinner not found
[-] OpenSSLEngineSocketImpl Conscrypt pinner not found
[-] OpenSSLSocketImpl Apache Harmony pinner not found
[-] PhoneGap sslCertificateChecker pinner not found
[-] IBM MobileFirst pinTrustedCertificatePublicKey pinner not found
[-] IBM WorkLight HostNameVerifierWithCertificatePinning pinner not found
[-] Conscrypt CertPinManager pinner not found
[-] CWAC-Netsecurity CertPinManager pinner not found
[-] Worklight Androidgap WLCertificatePinningPlugin pinner not found
[-] Netty FingerprintTrustManagerFactory pinner not found
[-] Squareup CertificatePinner pinner not found
[-] Squareup OkHostnameVerifier pinner not found
[-] Apache Cordova WebViewClient pinner not found
[-] Boye AbstractVerifier pinner not found
[+] Bypassing TrustManagerImpl (Android > 7): connectivitycheck.gstatic.com
[+] Bypassing TrustManagerImpl (Android > 7): android.clients.google.com
[+] Bypassing TrustManagerImpl (Android > 7): lh3.googleusercontent.com
In the last three lines, you can see that the script has already detected SSL Pinning using the TrustManager and has bypassed it. To see all the requests, you will have to change the filter on Burp Proxy. Under the Proxy ➤ HTTP history tab, click the section that says “Filter:…” as shown in Figure 8-17. Then click the Show all button and close the window.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig17_HTML.jpg
Figure 8-17

Changing Burp Proxy’s filter settings to show all traffic

Then select some apps on your Google Play Store on your device and take a look at your Burp HTTP history tab. You should now be able to see the requests and responses going back and forth between the device and the server as shown in Figures 8-18 and 8-19, respectively.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig18_HTML.jpg
Figure 8-18

HTTPS traffic requests going from the device and Google Play Store

../images/273312_2_En_8_Chapter/273312_2_En_8_Fig19_HTML.jpg
Figure 8-19

HTTPS traffic responses returned from Google Play Store

Examining the Filesystem

One of the perks of having root is unfettered access throughout the Android filesystem. I want to spend this section diving into the /data partition. The /system partition contains the operating system and most of that remains static, and for our purposes of debugging or reverse engineering, we don’t necessarily need to look there. The /data partition, on the other hand, is where all user and app data reside. Let’s look at two of our apps: aas2client and aas2obfuscate. Here are their directory layouts:
com.redteamlife.aas2.aas2client/
|-- cache
|-- code_cache
|-- databases
|-- files
|-- no_backup
`-- shared_prefs
com.redteamlife.aas2.aas2obfuscate/
|-- cache
|-- code_cache
|-- databases
|-- files
|-- no_backup
`-- shared_prefs
The app data is stored in the /data/user/0 directory, and then underneath this directory, there are the subdirectories that correspond to each app. Here’s a snippet:
marlin:/data/user/0 # ls -al |tail
drwx------   8 u0_a187        u0_a187         4096 2020-05-25 23:47 com.redteamlife.aas2.aas2obfuscate
drwx------   8 u0_a174        u0_a174         4096 2020-05-16 23:50 com.topjohnwu.magisk
drwx------   4 u0_a89         u0_a89          4096 2019-10-29 07:37 com.ustwo.lwp
drwx------   4 u0_a61         u0_a61          4096 2019-10-29 07:37 com.verizon.llkagent
drwx------   8 u0_a118        u0_a118         4096 2020-05-16 23:46 com.verizon.obdm
drwx------   4 u0_a120        u0_a120         4096 2019-10-29 07:37 com.verizon.obdm_permissions
drwx------   8 u0_a55         u0_a55          4096 2020-05-16 23:46 com.verizon.services
drwx------   8 u0_a145        u0_a145         4096 2020-05-16 23:46 com.vzw.apnlib
drwx------   4 radio          radio           4096 2019-10-29 07:37 org.codeaurora.ims
drwx------   8 u0_a92         u0_a92          4096 2020-05-16 23:46 qualcomm.com.vzw_msdc_api
marlin:/data/user/0 #
Notice how each directory is owned by a different user with no permissions being assigned to any other users. I took a look inside our app directories, and since they are such simple apps, none of the directories in each app data directory is populated. As a better example, let’s look at Google Chrome. Here’s what the Chrome directory structure looks like:
com.android.chrome/
|-- app_chrome
|   |-- BrowserMetrics
|   |-- BrowserMetrics-spare.pma
|   |-- CertificateRevocation
|   |-- Crowd Deny
|   |-- Default
|   |-- Default.previews_hint_cache_store
|   |-- FileTypePolicies
|   |-- GrShaderCache
|   |-- Local State
|   |-- OnDeviceHeadSuggestModel
|   |-- OriginTrials
|   |-- SSLErrorAssistant
|   |-- SafetyTips
|   |-- ShaderCache
|   |-- Subresource Filter
|   `-- TLSDeprecationConfig
|-- app_dex
|   |-- oat
|   `-- webapk6.dex
|-- app_tabs
|   `-- 0
|-- app_textures
|   `-- 0
|-- cache
|   |-- Cache
|   |-- Code Cache
|   |-- Crash Reports
|   |-- Crashpad
|   |-- Offline Pages
|   |-- font_unique_name_table.pb
|   `-- image_cache
|-- code_cache
|-- databases
|-- files
|   |-- images
|   `-- splitcompat
|-- lib -> /data/app/com.android.chrome-uekNic08OMP4vIsHfP_jag==/lib/arm
|-- no_backup
|   |-- com.google.InstanceId_Y29tLmdvb2dsZS5jaHJvbWUuT2ZmbGluZVBhZ2VQcmVmZXRjaA.properties
|   `-- com.google.android.gms.appid-no-backup
`-- shared_prefs
    |-- com.android.chrome_preferences.xml
    |-- com.google.android.apps.chrome.omaha.xml
    |-- com.google.android.gms.appid.xml
    |-- com.google.android.libraries.hats20.xml
    |-- icing_firebase_pref.xml
    |-- org.chromium.components.background_task_scheduler.xml
    |-- org.chromium.components.gcm_driver.subscription_flags.xml
    `-- webapp_registry.xml
Now you will notice more directories in the Chrome directory (I’m only going two levels down). Let’s first talk about the main directories created – starting with the cache directory. As the name implies, this directory holds the app’s cached set of files. In this example, it holds files like offline files, images, fonts, and crash reports. For example, if we go into the image_cache directory , we see that it stores some image files in PNG format:
marlin:/data/user/0/com.android.chrome/cache # cd image_cache/image_data_storage/
marlin:/data/user/0/com.android.chrome/cache/image_cache/image_data_storage # ls -al
total 3052
drwx--S--- 2 u0_a138 u0_a138_cache    4096 2020-05-26 15:58 .
drwx--S--- 4 u0_a138 u0_a138_cache    4096 2020-05-26 15:21 ..
-rw------- 1 u0_a138 u0_a138_cache  165638 2020-05-26 15:58 3BMOCQPJKWUIAPFKUCBRFWKGDFGOQKKG
-rw------- 1 u0_a138 u0_a138_cache  184090 2020-05-26 15:58 4WPXVZNDEHD7GETEAWWHTRYJVQYCMXZ2
-rw------- 1 u0_a138 u0_a138_cache    1438 2020-05-26 15:58 FCHAZF57RCYRNFMRXPZSLFMQIURR2PM7
-rw------- 1 u0_a138 u0_a138_cache     746 2020-05-26 15:58 FUY2CBED2WAUJDJHG4TFNOFDMEYQBI5U
-rw------- 1 u0_a138 u0_a138_cache  101556 2020-05-26 15:58 GMKJWLRURBIRM3RS5SMLJCAQI3D6GBHI
-rw------- 1 u0_a138 u0_a138_cache  112219 2020-05-26 15:58 HXCYNXFALMMGEEYSYGYKQAQYWZQTTH53
-rw------- 1 u0_a138 u0_a138_cache 1821170 2020-05-26 15:58 L345GKEPDJNJKHMFXHIPHGM7Q2IKIZW2
-rw------- 1 u0_a138 u0_a138_cache  131643 2020-05-26 15:58 LXHJ376LY7HJWCMKSYPANTXBY7DB3WQG
-rw------- 1 u0_a138 u0_a138_cache    2281 2020-05-26 15:58 M5GHVE2O64LCK2KD4RYPBTR3DZ2ZQKIR
-rw------- 1 u0_a138 u0_a138_cache  175446 2020-05-26 15:21 RVPMIXPZFAH3GITNBPSCZ6T3JXOXMGYH
-rw------- 1 u0_a138 u0_a138_cache    2703 2020-05-26 15:58 SSEZ7WZFKL6VBFQMCJ6UA3PMOHGB75WW
-rw------- 1 u0_a138 u0_a138_cache  192894 2020-05-26 15:58 UF34JJ7XOK6AXTMTARBMWZRG353H4G4H
-rw------- 1 u0_a138 u0_a138_cache    2344 2020-05-26 15:21 WE4GEZR5PCHUO5N3KB3RKCLJP4AD2BGO
-rw------- 1 u0_a138 u0_a138_cache    2883 2020-05-26 15:58 XC25K6GUFO5KCCOLBVGG6RBHFPUGUVAB
-rw------- 1 u0_a138 u0_a138_cache  119730 2020-05-26 15:58 XL4QHG75GDRZHFXBH4DH4N5FEVS6KLY3
marlin:/data/user/0/com.android.chrome/cache/image_cache/image_data_storage # file XL4QHG75GDRZHFXBH4DH4N5FEVS6KLY3
XL4QHG75GDRZHFXBH4DH4N5FEVS6KLY3: PNG image data, 322 x 322, 8-bit/color RGBA, non-interlaced
marlin:/data/user/0/com.android.chrome/cache/image_cache/image_data_storage #
When you select “Clear Cache” on your App Info screen, this is the directory that it clears. Generally, the data in this directory can be cleared as it is transient and is used to speed things up by either prefetching data or storing frequently accessed data. Chrome stores its open tabs and the state of those tabs in the directory called app_tabs. Inside this directory, you will see the entries for all the tabs. If you look at my tabs, you can see that I have one tab open that is pointing to Wikipedia. The structure of each file is not something we will go into, but by running the strings command on the file, you can see what printable strings there are. Based on those results, you can quickly determine what the tab contents are.
2|marlin:/data/user/0/com.android.chrome # cd app_tabs/0
marlin:/data/user/0/com.android.chrome/app_tabs/0 # ls -al
total 32
drwx------ 2 u0_a138 u0_a138 4096 2020-05-26 16:33 .
drwxrwx--x 3 u0_a138 u0_a138 4096 2020-05-26 15:21 ..
-rw------- 1 u0_a138 u0_a138 1771 2020-05-26 16:48 tab0
-rw------- 1 u0_a138 u0_a138   67 2020-05-26 16:33 tab_state0
127|marlin:/data/user/0/com.android.chrome/app_tabs/0 # strings tab0
rP+s7
chrome-native://newtab/
chrome-native://newtab/
https://en.m.wikipedia.org/wiki/Main_Page
https://en.m.wikipedia.org/
marlin:/data/user/0/com.android.chrome/app_tabs/0 #

A note about the no_backup directory: You will notice this directory inside all app data directories. Files that are placed in this directory do not get backed up. During an app or full backup of data, these files will be skipped over and ignored. Therefore, if there is sensitive data that you don’t want to go into a user’s backup, then you would place those files here. In Kotlin, you access the directory by using Context.getNoBackupDir [https://developer.android.com/reference/android/content/Context#getNoBackupFilesDir()] like this:

val noBackupDir = applicationContext.noBackupFilesDir

Then you can write or read files from this directory as you would normally using getFilesDir() [https://developer.android.com/reference/android/content/Context#getFilesDir()].

The databases directory typically contains your structured data. You can do this by using SQLite or by using Room [https://developer.android.com/training/data-storage/room] which is an abstraction layer that takes care of reading and writing your data to SQLite databases. Google recommends that you use Room instead of SQLite but doesn’t limit your use of SQLite. Let’s look at a concrete example of data in the databases directory. Remember our old friend the NYTimes – Crossword app? Let’s take a look at the databases directory in that app.
marlin:/ # cd /data/user/0/com.nytimes.crossword/databases
marlin:/data/user/0/com.nytimes.crossword/databases # ls -al
total 808
drwxrwx--x  2 u0_a180 u0_a180   4096 2020-05-27 13:02 .
drwx------ 11 u0_a180 u0_a180   4096 2020-05-27 13:02 ..
-rw-rw----  1 u0_a180 u0_a180 458752 2020-05-27 13:02 Crosswords.db
-rw-rw----  1 u0_a180 u0_a180      0 2020-05-27 13:02 Crosswords.db-journal
-rw-rw----  1 u0_a180 u0_a180   4096 2020-05-27 13:02 androidx.work.workdb
-rw-------  1 u0_a180 u0_a180  32768 2020-05-27 13:02 androidx.work.workdb-shm
-rw-------  1 u0_a180 u0_a180  90672 2020-05-27 13:02 androidx.work.workdb-wal
-rw-rw----  1 u0_a180 u0_a180  24576 2020-05-27 13:02 com.microsoft.appcenter.persistence
-rw-rw----  1 u0_a180 u0_a180      0 2020-05-27 13:02 com.microsoft.appcenter.persistence-journal
-rw-rw----  1 u0_a180 u0_a180   4096 2020-05-27 13:02 event-buffer.db
-rw-------  1 u0_a180 u0_a180  32768 2020-05-27 13:02 event-buffer.db-shm
-rw-------  1 u0_a180 u0_a180  86552 2020-05-27 13:02 event-buffer.db-wal
-rw-rw----  1 u0_a180 u0_a180  16384 2020-05-27 13:02 google_app_measurement_local.db
-rw-rw----  1 u0_a180 u0_a180      0 2020-05-27 13:02 google_app_measurement_local.db-journal
marlin:/data/user/0/com.nytimes.crossword/databases #

Hmm, that Crosswords.db looks interesting. Let’s take a look at that. Some device manufacturers will include the sqlite3 binary in your device. Some do not. My Google Pixel doesn’t come with it so I will copy the database file to my Mac and work on it there. I will first copy it over to /data/local/tmp and then use adb pull to download it to my Mac:

On the device
marlin:/data/user/0/com.nytimes.crossword/databases # ls -al
total 808
drwxrwx--x  2 u0_a180 u0_a180   4096 2020-05-27 13:02 .
drwx------ 11 u0_a180 u0_a180   4096 2020-05-27 13:02 ..
-rw-rw----  1 u0_a180 u0_a180 458752 2020-05-27 13:02 Crosswords.db
-rw-rw----  1 u0_a180 u0_a180      0 2020-05-27 13:02 Crosswords.db-journal
-rw-rw----  1 u0_a180 u0_a180   4096 2020-05-27 13:02 androidx.work.workdb
-rw-------  1 u0_a180 u0_a180  32768 2020-05-27 13:02 androidx.work.workdb-shm
-rw-------  1 u0_a180 u0_a180  90672 2020-05-27 13:02 androidx.work.workdb-wal
-rw-rw----  1 u0_a180 u0_a180  24576 2020-05-27 13:02 com.microsoft.appcenter.persistence
-rw-rw----  1 u0_a180 u0_a180      0 2020-05-27 13:02 com.microsoft.appcenter.persistence-journal
-rw-rw----  1 u0_a180 u0_a180   4096 2020-05-27 13:02 event-buffer.db
-rw-------  1 u0_a180 u0_a180  32768 2020-05-27 13:02 event-buffer.db-shm
-rw-------  1 u0_a180 u0_a180  86552 2020-05-27 13:02 event-buffer.db-wal
-rw-rw----  1 u0_a180 u0_a180  16384 2020-05-27 13:02 google_app_measurement_local.db
-rw-rw----  1 u0_a180 u0_a180      0 2020-05-27 13:02 google_app_measurement_local.db-journal
marlin:/data/user/0/com.nytimes.crossword/databases # cp Crosswords.db /data/local/tmp
marlin:/data/user/0/com.nytimes.crossword/databases # chown shell:shell /data/local/tmp/Crosswords.db
On the Mac
➜  adb pull /data/local/tmp/Crosswords.db
/data/local/tmp/Crosswords.db: 1 file pulled, 0 skipped. 13.7 MB/s (458752 bytes in 0.032s)
Now that the file is on my Mac, we can open it in sqlite3 and look at its schema:
➜  sqlite3 Crosswords.db
SQLite version 3.28.0 2019-04-15 14:49:49
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE android_metadata (locale TEXT);
CREATE TABLE `GameData`(`puzzleId` INTEGER,`puzzlePackIap` TEXT,`formatType` TEXT,`publishType` TEXT,`printDate` TEXT,`jsonData` TEXT,`author` TEXT,`editor` TEXT,`title` TEXT, PRIMARY KEY(`puzzleId`));
CREATE TABLE `CommitLog`(`id` INTEGER,`puzzleId` INTEGER,`committed` INTEGER,`commitId` TEXT,`resetTimer` INTEGER,`timestamp` INTEGER,`board` TEXT,`timerDiff` INTEGER, PRIMARY KEY(`id`));
CREATE TABLE `GameState`(`puzzleId` INTEGER,`status` TEXT,`firstOpened` INTEGER,`firstSolved` INTEGER,`firstCleared` INTEGER,`firstRevealed` INTEGER,`firstTimerReset` INTEGER,`timeSpentInPuzzle` INTEGER,`percentComplete` INTEGER,`lastUpdateTime` INTEGER,`solved` INTEGER, PRIMARY KEY(`puzzleId`));
CREATE TABLE `Streak`(`id` INTEGER,`startDate` INTEGER,`endDate` INTEGER, PRIMARY KEY(`id`));
CREATE TABLE `LegacyGameProgress`(`id` INTEGER,`puzzleId` INTEGER,`value` TEXT,`timestamp` INTEGER,`cellIndex` INTEGER,`mode` INTEGER, PRIMARY KEY(`id`));
CREATE TABLE `GamesHubData`(`id` INTEGER,`puzzleId` INTEGER,`title` TEXT,`author` TEXT,`editor` TEXT,`formatType` TEXT,`publishType` TEXT,`printDate` TEXT, PRIMARY KEY(`id`));
CREATE TABLE `PuzzlePack`(`productId` INTEGER,`platform` TEXT,`iapProductId` TEXT,`productType` TEXT,`createdDate` TEXT,`lastModifiedTimestamp` TEXT,`liveStartDateTimestamp` TEXT,`liveEndDateTimestamp` TEXT,`packName` TEXT,`status` TEXT,`iconUrl` TEXT,`numberOfPuzzlesInPack` INTEGER, PRIMARY KEY(`productId`));
CREATE TABLE `GoldStarDay`(`id` INTEGER,`date` INTEGER,`printDate` TEXT,`dayOffset` INTEGER,`dayOfWeek` INTEGER, PRIMARY KEY(`id`));
CREATE TABLE `GameProgress`(`id` INTEGER,`puzzleId` INTEGER,`value` TEXT,`timestamp` INTEGER,`cellIndex` INTEGER,`mode` INTEGER,`puzzleOpenedTimestamp` INTEGER,`puzzleSolvedTimestamp` INTEGER, PRIMARY KEY(`id`));
CREATE TABLE `PuzzleForPack`(`packID` INTEGER,`puzzleID` INTEGER, PRIMARY KEY(`packID`,`puzzleID`));
sqlite>
Look at the GameData table . It has a field called jsonData; let’s take a look at that:
sqlite> select jsonData from GameData;
...
...
... Rows of other data
...
...
{"status":"OK","entitlement":"premium","results":[{"puzzle_id":18293,"version":0,"puzzle_meta":{"formatType":"Normal","publishType":"Mini","title":"","printDate":"2020-05-27","printDotw":3,"editor":"","copyright":"2020, The New York Times","height":5,"width":5,"links":[],"layoutExtra":[],"author":"Joel Fagliano","notes":[]},"puzzle_data":{"answers":[null,"H","O","T",null,"N","O","B","E","L","F","R","A","M","E","L","U","M","P","S",null,"S","A","T",null],"clues":{"A":[{"formatted":"Like summer, or a word before u0026#34;springu0026#34;","squares":[],"clueNum":"1","clueStart":1,"value":"Like summer, or a word before "spring"","clueEnd":3,"index":0},{"squares":[],"clueNum":"4","clueStart":5,"value":"Prize for Malala Yousafzai","clueEnd":9,"index":0},{"squares":[],"clueNum":"6","clueStart":10,"value":"Falsely pin for a crime","clueEnd":14,"index":0},{"squares":[],"clueNum":"7","clueStart":15,"value":"Problems with mashed potatoes","clueEnd":19,"index":0},{"squares":[],"clueNum":"8","clueStart":21,"value":"Test no longer required for University of California admission","clueEnd":23,"index":0}],"D":[{"squares":[],"clueNum":"1","clueStart":1,"value":"Egyptian god with the head of a falcon","clueEnd":21,"index":0},{"formatted":"Subject of the 2020 documentary u0026#34;Becomingu0026#34;","squares":[],"clueNum":"2","clueStart":2,"value":"Subject of the 2020 documentary "Becoming"","clueEnd":22,"index":0},{"squares":[],"clueNum":"3","clueStart":3,"value":"Entice","clueEnd":23,"index":0},{"squares":[],"clueNum":"4","clueStart":5,"value":"Org. for Falcons, Eagles and Ravens","clueEnd":15,"index":0},{"formatted":""Allez ___ Bleus!" (French soccer cheer)","squares":[],"clueNum":"5","clueStart":9,"value":""Allez ___ Bleus!" (French soccer cheer)","clueEnd":19,"index":0}]},"layout":[0,1,1,1,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,0]}}]}
sqlite>

Look at that! All our clues and answers are stored in this database. In our previous hacking chapter, we pulled this data out the hard way even though I knew all along that you could pull it from the database, but the point I was trying to make is that there are other ways in which you can fetch data that an app uses.

The shared_prefs directory will usually be the place that apps store their key-value data. The way Android writes these preferences to the filesystem is via XML files. Again, using our same Crossword app, the shared_prefs looks like this:
marlin:/data/user/0/com.nytimes.crossword # cd shared_prefs/
marlin:/data/user/0/com.nytimes.crossword/shared_prefs # ls -al
total 200
drwxrwx--x  2 u0_a180 u0_a180 4096 2020-05-27 13:02 .
drwx------ 11 u0_a180 u0_a180 4096 2020-05-27 13:02 ..
-rw-rw----  1 u0_a180 u0_a180  490 2020-05-27 13:02 AppCenter.xml
-rw-rw----  1 u0_a180 u0_a180  436 2020-05-27 13:02 EntitlementsAndPurchase.xml
-rw-rw----  1 u0_a180 u0_a180  240 2020-05-27 13:02 NYTIMES_PREFS.xml
-rw-rw----  1 u0_a180 u0_a180  213 2020-05-27 13:02 TwitterAdvertisingInfoPreferences.xml
-rw-rw----  1 u0_a180 u0_a180  127 2020-05-27 13:02 WebViewChromiumPrefs.xml
-rw-rw----  1 u0_a180 u0_a180  862 2020-05-27 13:02 appsflyer-data.xml
-rw-rw----  1 u0_a180 u0_a180  251 2020-05-27 13:02 com.crashlytics.prefs.xml
-rw-rw----  1 u0_a180 u0_a180  125 2020-05-27 13:02 com.crashlytics.sdk.android:answers:settings.xml
-rw-rw----  1 u0_a180 u0_a180   65 2020-05-27 13:02 com.facebook.AccessTokenManager.SharedPreferences.xml
-rw-rw----  1 u0_a180 u0_a180  127 2020-05-27 13:02 com.facebook.internal.SKU_DETAILS.xml
-rw-rw----  1 u0_a180 u0_a180 2042 2020-05-27 13:02 com.facebook.internal.preferences.APP_GATEKEEPERS.xml
-rw-rw----  1 u0_a180 u0_a180 1453 2020-05-27 13:02 com.facebook.internal.preferences.APP_SETTINGS.xml
-rw-rw----  1 u0_a180 u0_a180  129 2020-05-27 13:02 com.facebook.loginManager.xml
-rw-rw----  1 u0_a180 u0_a180  138 2020-05-27 13:02 com.facebook.sdk.USER_SETTINGS.xml
-rw-rw----  1 u0_a180 u0_a180  160 2020-05-27 13:02 com.facebook.sdk.appEventPreferences.xml
-rw-rw----  1 u0_a180 u0_a180 2512 2020-05-27 13:02 com.google.android.gms.appid.xml
-rw-rw----  1 u0_a180 u0_a180  999 2020-05-27 13:02 com.google.android.gms.measurement.prefs.xml
-rw-rw----  1 u0_a180 u0_a180  127 2020-05-27 13:02 com.google.firebase.remoteconfig_legacy_settings.xml
-rw-rw----  1 u0_a180 u0_a180  192 2020-05-27 13:02 com.mobileapptracking.xml
-rw-rw----  1 u0_a180 u0_a180  339 2020-05-27 13:02 com.nytimes.android.eventtracker.CLOCK_CACHE.xml
-rw-rw----  1 u0_a180 u0_a180 2848 2020-05-27 13:02 com.nytimes.crossword_preferences.xml
-rw-rw----  1 u0_a180 u0_a180  181 2020-05-27 13:02 com.tune.ma.profile.xml
-rw-rw----  1 u0_a180 u0_a180  381 2020-05-27 13:02 frc_1:945293061443:android:9db0c9c54c4f0fad_firebase_settings.xml
marlin:/data/user/0/com.nytimes.crossword/shared_prefs #
Let’s dump the contents of com.nytimes.crossword_preferences.xml to see what’s in there:
marlin:/data/user/0/com.nytimes.crossword/shared_prefs # cat com.nytimes.crossword_preferences.xml
<?xml version='1.0' encoding='utf-8' standalone="yes" ?>
<map>
    <boolean name="HAPTIC_ENABLED" value="true" />
    <long name="LAST_ONBOARDING_TIMESTAMP" value="1590555743495" />
    <boolean name="JUMP_TO_NEXT_CLUE" value="true" />
    <int name="LAST_PUSH_REGISTERED_VERSION" value="668" />
    <boolean name="DAILY_MINI_SWITCH_PREF" value="true" />
    <boolean name="HAS_WRITTEN_DEFAULTS" value="true" />
    <boolean name="SHOW_TIMER" value="true" />
    <boolean name="SHOW_OVERLAY" value="true" />
    <long name="PUZZLE_REFRESH_TIMESTAMP" value="1590555758807" />
    <string name="kst">trial</string>
    <boolean name="DARK_MODE_KEY" value="false" />
    <long name="GDPR_LAST_TS" value="1590555744112" />
    <long name="com.nytimes.android.eventtracker.KEY_SESSION_INDEX" value="1" />
    <set name="SUBSCRIBED_PUSH_TAGS" />
    <int name="12881-SELECTED_CELL_KEY" value="1" />
    <string name="MINI_LAST_SELECTED_MONTH_PREF">May 2020</string>
    <string name="12881-CURRENT_DIRECTION_STR">Across</string>
    <boolean name="com.nytimes.android.eventtracker.KEY_SESSION_TIME_TYPE" value="false" />
    <boolean name="LIGHT_MODE_KEY" value="true" />
    <string name="THEME_MODE">LIGHT</string>
    <string name="Purr.Directives">{&quot;adConfiguration&quot;:{&quot;value&quot;:&quot;FULL&quot;},&quot;acceptableTrackers&quot;:{&quot;value&quot;:&quot;CONTROLLERS&quot;},&quot;showDataSaleOptOutDirective&quot;:{&quot;show&quot;:false,&quot;preference&quot;:&quot;NYT_SELL_PERSONAL_INFORMATION_CCPA&quot;}}</string>
    <string name="Purr.Directives.LastTS">2020-05-27T05:02:24.361Z</string>
    <boolean name="PROGRESS_MILESTONES_DEFAULT" value="true" />
    <boolean name="SKIP_FILLED_PENCILLED" value="false" />
    <string name="SmartLockTask.KEY_LAST_CHECK">3.0.1</string>
    <string name="DeviceID">17dbb2ee01d6ab06cf1237331a9c162f</string>
    <boolean name="ENTITLED_TO_DAILY_IN_TRIAL_PREF" value="true" />
    <string name="ktr">9319a3ea0c48e16e</string>
    <long name="com.nytimes.android.eventtracker.KEY_LAST_EVENT_TIME_VALUE" value="1590555758999" />
    <boolean name="SHOULD_PROMO_NEW_SETTING" value="false" />
    <boolean name="JUMP_BACK" value="true" />
    <boolean name="IS_GDPR" value="false" />
    <int name="CURRENT_STREAK" value="0" />
    <boolean name="LEADERBOARD_UPGRADED" value="true" />
    <long name="ktrts" value="1591117342000" />
    <boolean name="com.nytimes.android.eventtracker.KEY_LAST_EVENT_TIME_TYPE" value="false" />
    <long name="com.nytimes.android.eventtracker.KEY_SESSION_TIME_VALUE" value="1590555752362" />
    <int name="LONGEST_STREAK" value="0" />
    <boolean name="show_ab_group_one_key" value="false" />
    <boolean name="FIRST_LAUNCH" value="true" />
    <boolean name="SKIP_FILLED_SQUARES" value="true" />
    <boolean name="PLAY_VICTORY_JINGLE" value="true" />
</map>
marlin:/data/user/0/com.nytimes.crossword/shared_prefs #

You will see how Android designates the type of each of the key-value pairs as well. You can see strings, Booleans, long, and int values in the XML file. By altering these key-value pairs, you can, in most cases, affect the way the app works.

The databases and shared_prefs directories are usually places that I go to frequently because they contain a wealth of data that not only tells you about the inner workings of the app but also gives you an opportunity to change how an app behaves.

Detecting and Hiding Root

Rooting your device is all well and good, but you must keep in mind that there are others as well who will be rooting their devices. Then, when your apps run on those devices, they remain at the mercy of whatever the rooted device’s owner throws at it. Therefore, it makes sense to learn about detecting if a device is rooted or not. Banking applications often rely on root detection of Android devices. If a rooted device is detected, then the app will quit before any further communication or interaction takes place.

The mechanisms of detecting root are not exact. It comes down, once again, to a cat and mouse game where the app developer has to account for and be aware of all the signatures of a rooted device. These can be broadly divided into the following categories:
  • Looking for specific files that have been installed, for example, looking for the su command or even attempting to execute it. Looking for directories that may not originally exist on the Android system partition. The SuperSU root package would create an xbin directory.

  • Looking for installed packages by listing all the packages and looking for known names like com.topjohnwu.magisk for Magisk or other known root packages. Another thing that is done is a substring comparison where the detector will search for certain words in packages or in the filesystem.

  • Looking for test keys that Google ships with their testing or developer units. This is normally done by checking build tags on Android.

  • Looking for a writeable /system partition. The /system partition is not meant to be writeable and is always mounted read-only on non-rooted devices.

In this section, I won’t go into too much depth of the many exhaustive ways of checking for root. I will, however, take a closer look at a package called RootBeer [https://github.com/scottyab/rootbeer] which you can add to your app and call its root detection functions. RootBeer was developed by Scott Alexander-Brown and Mat Rollings. Let’s go ahead and add that to our aas2obfuscate app. Add this line to the app’s build.gradle file :
implementation 'com.scottyab:rootbeer-lib:0.0.8'
Mine looks like this:
dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'androidx.core:core-ktx:1.2.0'
    implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    implementation 'com.scottyab:rootbeer-lib:0.0.8'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
}
Then, edit your MainActivity.kt and add this bit of code to the end of the file and run it:
val rootBeer = RootBeer(applicationContext)
if (rootBeer.isRooted()) {
    Log.d("aas2obfuscate","Device has been rooted!")
} else {
    Log.d("aas2obfuscate", "No root detected")
}
You should be able to see in your logcat that RootBeer instantly detects that Magisk is installed, and then our logging line is called:
E/RootBeer: RootBeer: isAnyPackageFromListInstalled() [249] - com.topjohnwu.magisk ROOT management app detected!
E/QLog: RootBeer: isAnyPackageFromListInstalled() [249] - com.topjohnwu.magisk ROOT management app detected!
D/aas2obfuscate: Device has been rooted!

Defeating Root Detection

As is the norm with apps that root Android devices, they typically ship with a root cloaking or hiding app. Magisk is no exception and ships with a program called magiskhide . Magiskhide is disabled by default, and you have to enable it through the command line. Let’s examine its options first:
marlin:/system # magiskhide
MagiskHide 20.4(20400)
Usage: magiskhide [action [arguments...] ]
Actions:
   status          Return the status of magiskhide
   enable          Start magiskhide
   disable         Stop magiskhide
   add PKG [PROC]  Add a new target to the hide list
   rm PKG [PROC]   Remove target(s) from the hide list
   ls              Print the current hide list
   exec CMDs...    Execute commands in isolated mount
                   namespace and do all hide unmounts
1|marlin:/system #
To check if magiskhide is enabled, you can do a magiskhide status. Similarly, execute magiskhide enable to enable it. Magiskhide also maintains a list of packages that you can add to. This is known as the hide list and will take some extra measures to hide apps that specifically do root detection. Let’s now hide our device’s rooted status. If we look at the logs, we can see that RootBeer is detecting the Magisk Manager. This is likely by looking at the installed packages. So let’s tackle that first by hiding the Magisk Manager. On your device, open Magisk Manager and go into settings. You will then see an option called Hide Magisk Manager as shown in Figure 8-20. Click that and it will prompt you to give Magisk Manager an alternate name (Figure 8-21). I chose Policy. After I click the OK button, Magisk will repackage itself and name itself by the name that you chose.
../images/273312_2_En_8_Chapter/273312_2_En_8_Fig20_HTML.png
Figure 8-20

The setting to hide Magisk Manager

../images/273312_2_En_8_Chapter/273312_2_En_8_Fig21_HTML.png
Figure 8-21

Renaming the Magisk Manager from Manager to a name of your choosing

Now let’s see if our app still continues to detect Magisk Manager. Run it again and look at the logs:
V/RootBeer: RootBeer: checkForBinary() [194] - /sbin/su binary detected!
W/2.aas2obfuscate: type=1400 audit(0.0:3058): avc: denied { read } for name="cache" dev="sda34" ino=16 scontext=u:r:untrusted_app:s0:c187,c256,c512,c768 tcontext=u:object_r:cache_file:s0 tclass=lnk_file permissive=0
D/aas2obfuscate: Device has been rooted!
Hmm, looks like it’s a different message now. Right now, RootBeer doesn’t detect the Magisk Manager which is great, but it does detect that we have the /sbin/su binary file on our filesystem. Let’s see if adding our app to the magiskhide list takes care of it. As we know, we named our app com.redteamlife.aas2.aas2obfuscate. So, let’s go ahead and add that to magiskhide:
1|marlin:/data/local/tmp # magiskhide enable
marlin:/data/local/tmp # magiskhide ls
com.google.android.gms|com.google.android.gms.unstable
org.microg.gms.droidguard|com.google.android.gms.unstable
marlin:/data/local/tmp # magiskhide add com.redteamlife.aas2.aas2obfuscate
marlin:/data/local/tmp # magiskhide ls
com.google.android.gms|com.google.android.gms.unstable
com.redteamlife.aas2.aas2obfuscate|com.redteamlife.aas2.aas2obfuscate
org.microg.gms.droidguard|com.google.android.gms.unstable
marlin:/data/local/tmp #
Here, I first enable magiskhide and then add our app’s bundle id to the hide list. I do a quick magiskhide ls to list the packages on the list to verify that our package has been added. Now, let’s test one more time:
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /data/local/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /data/local/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /data/local/xbin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /sbin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /su/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/bin/.ext/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/bin/failsafe/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/sd/xbin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/usr/we-need-root/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/xbin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /cache/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /data/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /dev/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /system/sbin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /product/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /apex/com.android.runtime/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /odm/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /vendor/bin/su Absent :(
2020-05-27 17:56:10.038 11218-11218/? I/RootBeer: LOOKING FOR BINARY: /vendor/xbin/su Absent :(
2020-05-27 17:56:10.041 11218-11218/? D/aas2obfuscate: No root detected

Boom! No root detected! Our magiskhide cloaking did the trick. So, what have we learned? Well, once again, as the root user, you are all powerful. You can craft the entire look and feel of the device masking processes that you don’t want visible in however way you want to. The apps that run on the device that is rooted usually don’t stand a chance – yet another example of why you want to rethink what data you store on the device through your app.

Next, I want to write a script using Frida to see if we can fool RootBeer into thinking we are not rooted. To do that, we should first check the source code for RootBeer. We already know that all we have to do is call the isRooted() method. Let’s see what that method looks like:
public boolean isRooted() {
        return detectRootManagementApps() || detectPotentiallyDangerousApps() || checkForBinary(BINARY_SU)
                || checkForDangerousProps() || checkForRWPaths()
                || detectTestKeys() || checkSuExists() || checkForRootNative() || checkForMagiskBinary();
    }
This function will call all other functions which return either a true or false. Any one of them returning false will make the entire method return a false. So for us to intercept and neutralize this, we have to reimplement the isRooted() method. Here is my JavaScript code to do that:
1: setTimeout(function() {
2:     Java.perform(function () {
3:          const RootBeer = Java.use('com.scottyab.rootbeer.RootBeer');
4:          RootBeer.isRooted.implementation = function(){
5:               return false;
6:          }
7:     });
8: });
You can see it is very simple to reimplement the isRooted() method to always return false. Save this file in your Frida directory as rootbeer.js. First reverse the magiskhide changes that you made above so that it does not interfere with the process using Frida. Now making sure Frida server is running on your device, launch the aas2obfuscate app through Frida by executing this command on your workstation:
(p3) ➜  frida -U -f com.redteamlife.aas2.aas2obfuscate -l rootbeer.js --no-pause
     ____
    / _  |   Frida 12.9.4 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://www.frida.re/docs/home/
Spawned `com.redteamlife.aas2.aas2obfuscate`. Resuming main thread!
[Pixel XL::com.redteamlife.aas2.aas2obfuscate]->
The aas2obfuscate app should start. Now run logcat and filter for the tag aas2obfuscate so that you can see what is written out to the log. On your device shell, run logcat aas2obfuscate:
05-27 18:13:13.516 11862 11862 W 2.aas2obfuscat: Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (greylist, reflection, allowed)
05-27 18:13:13.516 11862 11862 W 2.aas2obfuscat: Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (greylist, reflection, allowed)
05-27 18:13:13.584 11862 11862 D aas2obfuscate: BT Adapter address is 02:00:00:00:00:00
05-27 18:13:13.584 11862 11862 D aas2obfuscate: No root detected
05-27 18:13:13.642 11862 11907 I Adreno  : QUALCOMM build                   : 4a00b69, I4e7e888065
05-27 18:13:13.642 11862 11907 I Adreno  : Build Date                       : 04/09/19
05-27 18:13:13.642 11862 11907 I Adreno  : OpenGL ES Shader Compiler Version: EV031.26.06.00

There you have it. The line in bold shows that RootBeer has not detected that the device has been rooted.

Further Tools to Help Debugging

I want to wrap this chapter up by talking about a few more tools that you can use when you want to see the inner workings of apps. One of them is strace and the other is jtrace. Now strace is a popular Linux debugging program that allows you to see all the syscalls that a program makes. For example, remember that Crossword app we downloaded and installed? If we wanted to see which files it opened during its execution, we could see that with strace . Here’s the bad news though. strace does not come bundled with my Google Pixel XL, and I think it may also be left out of other vendor devices. All is not lot though, because we can build it and copy it across to use as we like. First, let’s build strace. We are going to use Docker for this, and since you already installed and used it in a previous chapter, you should not be too deep in the water with this. Remember, since we’re building strace for the ARM64 platform, we will have to cross-compile it. We can use the Ubuntu Docker image for our example. I am working in a separate directory that I use for third-party code. I will be mounting this directory and mapping it to my Docker container later. Let’s start by pulling the correct image:
➜  docker pull ubuntu
Using default tag: latest
latest: Pulling from library/ubuntu
Digest: sha256:747d2dbbaaee995098c9792d99bd333c6783ce56150d1b11e333bbceed5c54d7
Status: Image is up to date for ubuntu:latest
docker.io/library/ubuntu:latest
With the image downloaded, let’s next go and get the strace source code. strace has a home at https://strace.io/. You can find and download the source code from here: https://strace.io/files/5.6/strace-5.6.tar.xz. Download it into a directory of your choosing and untar it as shown. Here, I’m doing it in my code directory:
➜  tar xf strace-5.6.tar.xz
➜  cd strace-5.6
Now let’s fire up our Docker container and mount the strace directory to a directory on the container. Run this command:
docker run --rm -it -v "<replace with your directory name>":"/strace" ubuntu
root@468d83dd9d34:/#
Check that the strace directory was mounted correctly:
root@468d83dd9d34:/# ls /strace
AUTHORS                                          ioprio.c                     ptp.c
COPYING                                          ipc.c                        ptrace.h
CREDITS.in                                       ipc_defs.h                   ptrace_syscall_info.c
...
...
Now we will need to update our Ubuntu image and install the tools necessary to build programs on it. To do this, run the following two commands:
root@468d83dd9d34:/#  apt-get update
Get:1 http://ports.ubuntu.com/ubuntu-ports focal InRelease [265 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-updates InRelease [107 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-backports InRelease [98.3 kB]
. . .
. . .
. . .
and
root@468d83dd9d34:/# apt-get install -y build-essential gcc-aarch64-linux-gnu binutils-aarch64-linux-gnu
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  binutils binutils-aarch64-linux-gnu binutils-common cpp cpp-9 dirmngr dpkg-dev fakeroot g++ g++-9 gcc gcc-9
  gcc-9-base gnupg gnupg-l10n gnupg-utils gpg . . .
. . .
. . .
. . .
After this completes, it’s time to build strace. To do this, we will first have to configure the environment using the configure command. Change directory to the strace directory and then run this command:
root@468d83dd9d34:/strace# ./configure --enable-mpers=no --host aarch64-linux-gnu LDFLAGS="-static -pthread"
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
. . .
. . .
. . .
This will build the relevant configuration files necessary for the next step which is to build it. To do this, we simply run make:
root@468d83dd9d34:/strace# make
This will build the strace binary in the current directory. You can now copy this across to your device from your workstation:
➜  adb push strace /data/local/tmp
We’re now ready to see our shiny new strace in action. We were planning to see what the Crossword app did as far as files were concerned. To do this, let’s trace the “openat” syscall (typically you may want to start with the “open” syscall, but from prior knowledge of this app, I know it uses “openat”). Start the Crossword app, but then close it by pressing the back button until you exit the app. This will leave it running in the background. Let’s get its process id first. On your Android shell, run
marlin:/data/local/tmp # ps -ef|grep crossword
root         16026 12548 2 20:43:23 pts/0 00:00:00 grep crossword
u0_a180      16374   649 0 13:58:50 ?     00:00:26 com.nytimes.crossword
marlin:/data/local/tmp #
In my case, the process id is 16374. From your /data/local/tmp directory, start strace with the following parameters:
marlin:/data/local/tmp # ./strace -e trace=openat -f -p 16374
./strace: Process 16374 attached with 89 threads
pid 16859] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/databases/google_app_measurement_local.db", O_RDWR|O_CREAT|O_LARGEFILE|O_CLOEXEC, 0600) = 116
[pid 16430] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/databases", O_RDONLY|O_CLOEXEC) = 119
[pid 16592] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/no_backup/.localytics/com.localytics.android.6f88eb5437a02c5a32a000dcd15ab6f01e46668255c960679c11961f455e5a5e.analytics.sqlite-journal", O_RDWR|O_CREAT|O_LARGEFILE|O_CLOEXEC, 0660) = 119
[pid 16859] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/databases/google_app_measurement_local.db-journal", O_RDWR|O_CREAT|O_LARGEFILE|O_CLOEXEC, 0660) = 117
[pid 16859] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/databases", O_RDONLY|O_CLOEXEC) = 236
[pid 16398] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/shared_prefs/com.nytimes.crossword_preferences.xml", O_WRONLY|O_CREAT|O_TRUNC, 0600) = 116
[pid 16398] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/shared_prefs/androidx.work.util.id.xml", O_WRONLY|O_CREAT|O_TRUNC, 0600) = 116
[pid 16398] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/shared_prefs/EntitlementsAndPurchase.xml", O_WRONLY|O_CREAT|O_TRUNC, 0600) = 116
[pid 16398] openat(AT_FDCWD, "/data/user/0/com.nytimes.crossword/shared_prefs/com.google.android.gms.measurement.prefs.xml", O_WRONLY|O_CREAT|O_TRUNC, 0600 <unfinished ...>
. . .
. . .

Breaking down the parameters we used for strace, the -e trace=openat means we want to only trace the openat syscall, the -f means follow any child processes or threads that the original app creates, and the -p is the PID or process id of the app. Using this approach, you can examine what apps installed on the device are doing, where they read from or write to, and which network hosts they communicate with.

One more app that’s similar to strace but is more Android aware is jtrace. jtrace [http://newandroidbook.com/tools/jtrace.html] is an app that is written by Jonathan Levin. Jonathan Levin is well known for his research in iPhone and Android internals, and his books are an absolute must read for anyone interested in the inner workings of Android. He developed jtrace to be more Android aware. It works in much the same way as strace. To get it running on your device, just download the archive, decompress it, and copy over the 64 bit ARM version of jtrace to your /data/local/tmp directory and run it from there. It takes the same -f and -p flags.

Summary

This chapter was a bit of a heavy one, but I feel like I have only scratched the surface of the rooting topic. For the purposes of this book, I will not go into further depth, but I will look toward covering a bit more about rooting in this book’s companion my site [https://aas2book.com].

The rooting process is an essential technique to learn if you are to truly move to testing your applications in depth. With a rooted device, you can really explore the full reaches of your device and then subject your app to stresses and attacks that can help to further strengthen it. With device rooting being made almost point and click, and the tools for breaking SSL Pinning or breaking encryption are again very much within reach of many, it makes sense to go this far in your security testing.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
3.144.244.44