Chapter 4. Acquiring Data
Although the prime directive of this chapter is to teach you how to acquire data from a remote source, this is really just a sneaky way for me to teach you about Android and the main thread. For the sake of simplicity, all the examples in this chapter will deal with downloading and rendering image data. In the next chapter, on adapters and lists, I’ll introduce you to parsing complex data and displaying it to users. Image data, as a general rule, is larger and more cumbersome, so you’ll run into more interesting and demonstrative timing issues in dealing with it.
The Main Thread
The Android operation system has exactly one blessed thread authorized to change anything that will be seen by the user. This alleviates what could be a concurrency nightmare, such as view locations and data changing in one thread while a different one is trying to lay them out onscreen. If only one thread is allowed to touch the user interface, Android can guarantee that nothing vital is changed while it’s measuring views and rendering them to the screen. This has, unfortunately, serious repercussions for how you’ll need to acquire and process data. Let me start with a simple example.
You There, Fetch Me That Data!
Were I to ask you, right now, to download an image and display it to the screen, you’d probably write code that looks a lot like this:
public void onCreate(Bundle extra){
try{
URL url = new URL("http://wanderingoak.net/bridge.png");
HttpURLConnection httpCon =
(HttpURLConnection)url.openConnection();
if(httpCon.getResponseCode() != 200) {
throw new Exception("Failed to connect");
}
InputStream is = httpCon.getInputStream();
Bitmap bitmap = BitmapFactory.decodeStream(is);
ImageView iv = (ImageView)findViewById(R.id.main_image);
iv.setImageBitmap(bitmap);
}catch(Exception e){
Log.e("ImageFetching","Didn't work!",e);
}
}
This is exactly what I did when initially faced with the same problem. While this code will fetch and display the required bitmap, there is a very sinister issue lurking in the code—namely, the code itself is running on the main thread. Why is this a problem? Consider that there can be only one main thread and that the main thread is the only one that can interact with the screen in any capacity. This means that while the example code is waiting for the network to come back with image data, nothing whatsoever can be rendered to the screen.
This image-fetching code will block any action from taking place anywhere on the device. If you hold the main thread hostage, buttons will not be processed, phone calls cannot be answered, and nothing can be drawn to the screen until you release it.
Watchdogs
Given that a simple programmer error (like the one in the example code) could effectively cripple any Android device, Google has gone to great lengths to make sure no single application can control the main thread for any length of time. Starting in Android Honeycomb (3.0), if you open any network connections on the main thread, your application will crash. If you’re hogging too much of the main thread’s time with long-running operations, such as calculating pi or finding the next prime number, your application will produce this disastrous dialog box (Figure 4.1) on top of your application.
This dialog box is unaffectionately referred to by developers as an ANR (App Not Responding) crash. Although operations will continue in the background, and the user can press the Wait button to return to whatever’s going on within your application, this is catastrophic for most users, and you should avoid it at all costs.
What Not to Do
What kind of things should you avoid on the main thread?
Anything involving the network
Any task requiring a read from or write to the file system
Heavy processing of any kind (such as image or movie modification)
Any task that blocks a thread while you wait for something to complete
Excluding this list, there isn’t much left, so as a general rule, if it doesn’t involve setup or modification of the user interface, don’t do it on the main thread.
When Am I on the Main Thread?
Anytime a method is called from the system (unless explicitly otherwise stated), you can be sure you’re on the main thread. Again, as a general rule, if you’re not in a thread created by you, it’s safe to assume you’re probably on the main one, so be careful.
Getting Off the Main Thread
You can see why holding the main thread hostage while grabbing a silly picture of the Golden Gate Bridge is a bad idea. But how, you might be wondering, do I get off the main thread? An inventive hacker might simply move all the offending code into a separate thread. This imaginary hacker might produce code looking something like this:
public void onCreate(Bundle extra){
new Thread(){
public void run(){
try{
URL url = new URL("http://wanderingoak.net/bridge.png");
HttpURLConnection httpCon =
(HttpURLConnection) url.openConnection();
if(httpCon.getResponseCode() != 200){
throw new Exception("Failed to connect");
}
InputStream is = httpCon.getInputStream();
Bitmap bitmap = BitmapFactory.decodeStream(is);
ImageView iv = (ImageView)findViewById(R.id.remote_image);
iv.setImageBitmap(bt);
}catch(Exception e){
//handle failure here
}
}
}.start();
}
“There,” your enterprising hacker friend might say, “I’ve fixed your problem. The main thread can continue to run unimpeded by the silly PNG downloading code.” There is, however, another problem with this new code. If you run the method on your own emulator, you’ll see that it throws an exception and cannot display the image onscreen.
Why, you might now ask, is this new failure happening? Well, remember that the main thread is the only one allowed to make changes to the user interface. Calling setImageBitmap is very much in the realm of one of those changes and, thus, can be done only while on the main thread.
Getting Back to Main Land
Android provides, through the Activity class, a way to get back on the main thread as long as you have access to an activity. Let me fix the hacker’s code to do this correctly. I don’t want to indent the code into the following page, so I’ll show the code beginning from the line on which the bitmap is created (remember, we’re still inside the Activity class, within the onCreate method, inside an inline thread declaration) (why do I hear the music from Inception playing in my head?).
If you’re confused, check the sample code for this chapter.
final Bitmap bt = BitmapFactory.decodeStream(is);
ImageActivity.this.runOnUiThread(new Runnable() {
public void run() {
ImageView iv = (ImageView)findViewById(R.id.remote_image);
iv.setImageBitmap(bt);
}
});
//All the close brackets omitted to save space
Remember, we’re already running in a thread, so accessing just this will refer to the thread itself. I, on the other hand, need to invoke a method on the activity. Calling ImageActivity.this provides a reference to the outer Activity class in which we’ve spun up this hacky code and will thus allow us to call runOnUiThread. Further, because I want to access the recently created bitmap in a different thread, I’ll need to make the bitmap declaration final or the compiler will get cranky with us.
When you call runOnUiThread, Android will schedule this work to be done as soon as the main thread is free from other tasks. Once back on the main thread, all the same “don’t be a hog” rules again apply.
There Must Be a Better Way!
If you’re looking at this jumbled, confusing, un-cancelable code and thinking to yourself, “Self. There must be a cleaner way to do this,” you’d be right. There are many ways to handle long-running tasks; I’ll show you what I think are the two most useful. One is the AsyncTask, a simple way to do an easy action within an activity. The other, IntentService, is more complicated but much better at handling repetitive work that can span multiple activities.
The AsyncTask
At its core, the AsyncTask is an abstract class that you extend and that provides the basic framework for a time-consuming asynchronous task.
The best way to describe the AsyncTask is to call it a working thread sandwich. That is to say, it has three major methods for which you can provide implementation.
onPreExecute takes place on the main thread and is the first slice of bread. It sets up the task, prepares a loading dialog, and warns the user that something is about to happen.
doInBackground is the meat of this little task sandwich (and is also required). This method is guaranteed by Android to run on a separate background thread. This is where the majority of your work takes place.
onPostExecute will be called once your work is finished (again, on the main thread), and the results produced by the background method will be passed to it. This is the other slice of bread.
That’s the gist of the asynchronous task. There are more-complicated factors that I’ll touch on in just a minute, but this is one of the fundamental building blocks of the Android platform (given that all hard work must be taken off the main thread).
Take a look at one in action, and then we’ll go over the specifics of it:
private class ImageDownloader extends AsyncTask<String, Integer, Bitmap>{
Override
protected void onPreExecute(){
//Setup is done here
}
@Override
protected Bitmap doInBackground(String... params) {
try{
URL url = new URL(params[0]);
HttpURLConnection httpCon =
(HttpURLConnection) url.openConnection();
if(httpCon.getResponseCode() != 200)
throw new Exception("Failed to connect");
}
InputStream is = httpCon.getInputStream();
return BitmapFactory.decodeStream(is);
}catch(Exception e){
Log.e("Image","Failed to load image",e);
}
return null;
}
@Override
protected void onProgressUpdate(Integer... params){
//Update a progress bar here, or ignore it, it's up to you
}
@Override
protected void onPostExecute(Bitmap img){
ImageView iv = (ImageView) findViewById(R.id.remote_image);
if(iv!=null && img!=null){
iv.setImageBitmap(img);
}
}
@Override
protected void onCancelled(){
// Handle what you want to do if you cancel this task
}
}
That, dear readers, is an asynchronous task that will download an image at the end of any URL and display it for your pleasure (provided you have an image view onscreen with the ID remote_image). Here is how you’d kick off such a task from the onCreate method of your activity.
public void onCreate(Bundle extras){
super.onCreate(extras);
setContentView(R.layout.image_layout);
ImageDownloader imageDownloader = new ImageDownloader();
imageDownloader.execute("http://wanderingoak.net/bridge.png");
}
Once you call execute on the ImageDownloader, it will download the image, process it into a bitmap, and display it to the screen. That is, assuming your image_layout.xml file contains an ImageView with the ID remote_image.
How to Make It Work for You
The AsyncTask requires that you specify three generic type arguments (if you’re unsure about Java and generics, do a little Googling before you press on) as you declare your extension of the task.
The type of parameter that will be passed into the class. In this example AsyncTask code, I’m passing one string that will be the URL, but I could pass several of them. The parameters will always be referenced as an array no matter how many of them you pass in. Notice that I reference the single URL string as params[0].
The object passed between the doInBackground method (off the main thread) and the onProgressUpdate method (which will be called on the main thread). It doesn’t matter in the example, because I’m not doing any progress updates in this demo, but it’d probably be an integer, which would be either the percentage of completion of the transaction or the number of bytes transferred.
The object that will be returned by the doInBackground method to be handled by the onPostExecute call. In this little example, it’s the bitmap we set out to download.
Here’s the line in which all three objects are declared:
private class ImageDownloader extends
AsyncTask<String, Integer, Bitmap>{
In this example, these are the classes that will be passed to your three major methods.
onPreExecute is usually when you’ll want to set up a loading dialog or a loading spinner in the corner of the screen (I’ll discuss dialogs in depth later). Remember, onPreExecute is called on the main thread, so don’t touch the file system or network at all in this method.
This is your chance to make as many network connections, file system accesses, or other lengthy operations as you like without holding up the phone. The class of object passed to this method will be determined by the first generic object in your AsyncTask’s class declaration. Although I’m using only one parameter in the code sample, you can actually pass any number of parameters (as long as they derive from the saved class), and you’ll have them at your fingertips when doInBackground is called. Once your long-running task has been completed, you’ll need to return the result at the end of your function. This final value will be passed into another method called back on the main UI thread.
Showing your progress
There’s another aspect of the AsyncTask that you should be aware of even though I haven’t demonstrated it. From within doInBackground, you can send progress updates to the user interface. doInBackground isn’t on the main thread, so if you’d like to update a progress bar or change the state of something on the screen, you’ll have to get back on the main thread to make the change.
Within the AsyncTask, you can do this during the doInBackground method by calling publishProgress and passing in any number of objects deriving from the second class in the AsyncTask declaration (in the case of this example, an integer). Android will then, on the main thread, call your declared onProgressUpdate method and hand over any classes you passed to publishProgress. Here’s what the method looks like in the AsyncTask example:
protected void onProgressUpdate(Integer... params){
//Update a progress bar here, or ignore it, it's up to you
}
As always, be careful when doing UI updates, because if the activity isn’t currently onscreen or has been destroyed, you could run into some trouble. The section “A Few Important Caveats” discusses the “bad things” that can happen.
onPostExecute
The work has been finished, or, as in the example, the image has been downloaded. It’s time to update the screen with what I’ve acquired. At the end of doInBackground, if successful, I return a loaded bitmap to the AsyncTask. Now Android will switch to the main thread and call onPostExecute, passing the class I returned at the end of doInBackground. Here’s what the code for that method looks like:
protected void onPostExecute(Bitmap img){
ImageView iv = (ImageView)findViewById(R.id.remote_image);
if(iv!=null && img!=null){
iv.setImageBitmap(img);
}
}
I take the bitmap downloaded from the website, retrieve the image view into which it’s going to be loaded, and set it as that view’s bitmap to be rendered. There’s an error case I haven’t correctly handled here. Take a second to look back at the original code and see if you can spot it.
A Few Important Caveats
Typically, an AsyncTask is started from within an activity. However, you must remember that activities can have short life spans. Recall that, by default, Android destroys and re-creates any activity each time you rotate the screen. Android will also destroy your activity when the user backs out of it. You might reasonably ask, “If I start an AsyncTask from within an activity and then that activity is destroyed, what happens?” You guessed it: very bad things. Trying to draw to an activity that’s already been removed from the screen can cause all manner of havoc (usually in the form of unhandled exceptions).
It’s a good idea to keep track of any AsyncTasks you’ve started, and when the activity’s onDestroy method is called, make sure to call cancel on any lingering AsyncTask.
There are two cases in which the AsyncTask is perfect for the job:
Downloading small amounts of data specific to one particular activity
Loading files from an external storage drive (usually an SD card)
Make sure that the data you’re moving with the AsyncTask pertains to only one activity, because your task generally shouldn’t span more than one. You can pass it between activities if the screen has been rotated, but this can be tricky.
There are a few cases when it’s not a good idea to use an AsyncTask:
Any acquired data that may pertain to more than one activity shouldn’t be acquired through an AsyncTask. Both an image that might be shown on more than one screen and a list of messages in a Twitter application, for example, would have relevance outside a single activity.
Data to be posted to a web service is also a bad idea to put on an AsyncTask for the following reason: Users will want to fire off a post (posting a photo, blog, tweet, or other data) and do something else, rather than waiting for a progress bar to clear. By using an AsyncTask, you’re forcing them to wait around for the posting activity to finish.
Last, be aware that there is some overhead for the system in setting up the AsyncTask. This is fine if you use a few of them, but it may start to slow down your main thread if you’re firing off hundreds of them.
You might be curious as to exactly what you should use in these cases. I’m glad you are, because that’s exactly what I’d like to show you next.
The IntentService
The IntentService is an excellent way to move large amounts of data around without relying on any specific activity or even application. The AsyncTask will always take over the main thread at least twice (with its pre- and post-execute methods), and it must be owned by an activity that is able to draw to the screen. The IntentService has no such restriction. To demonstrate, I’ll show you how to download the same image, this time from the IntentService rather than the AsyncTask.
Declaring a Service
Services are, essentially, classes that run in the background with no access to the screen. In order for the system to find your service when required, you’ll need to declare it in your manifest, like so:
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.peachpit.Example"
android:versionCode="1"
android:versionName="1.0">
<application
android:name="MyApplication"
android:icon="@drawable/icon"
android:label="@string/app_name">
<!—Rest of the application declarations go here -->
<service android:name=".ImageIntentService"/>
</application>
</manifest>
At a minimum, you’ll need to have this simple declaration. It will then allow you to (as I showed you earlier with activities) explicitly launch your service. Here’s the code to do exactly that:
Intent i = new Intent(this, ImageIntentService.class);
i.putExtra("url", getIntent().getExtras().getString("url"));
startService(i);
At this point, the system will construct a new instance of your service, call its onCreate method, and then start firing data at the IntentService’s handleIntent method. The intent service is specifically constructed to handle large amounts of work and processing off the main thread. The service’s onCreate method will be called on the main thread, but subsequent calls to handleIntent are guaranteed by Android to be on a background thread (and this is where you should put your long-running code in any case).
Right, enough gabbing. Let me introduce you to the ImageIntentService. The first thing you’ll need to pay attention to is the constructor:
public class ImageIntentService extends IntentService{
public ImageIntentService() {
super("ImageIntentService");
}
Notice that the constructor you must declare has no string as a parameter. The parent’s constructor that you must call, however, must be passed a string. Your IDE will let you know that you must declare a constructor with a string, when in reality, you must declare it without one. This simple mistake can cause you several hours of intense face-to-desk debugging.
Once your service exists, and before anything else runs, the system will call your onCreate method. onCreate is an excellent time to run any housekeeping chores you’ll need for the rest of the service’s tasks (more on this when I show you the image downloader).
At last, the service can get down to doing some heavy lifting. Once it has been constructed and has had its onCreate method called, it will then receive a call to handleIntent for each time any other activity has called startService.
Fetching Images
The main difference between fetching images and fetching smaller, manageable data is that larger data sets (such as images or larger data retrievals) should not be bundled into a final broadcast intent (another major difference to the AsyncTask). Also, keep in mind that the service has no direct access to any activity, so it cannot ever access the screen on its own. Instead of modifying the screen, the IntentService will send a broadcast intent alerting all listeners that the image download is complete. Further, since the service cannot pass the actual image data along with that intent, you’ll need to save the image to the SD card and include the path to that file in the final completion broadcast.
The setup
Before you can use the external storage to cache the data, you’ll need to create a cache folder for your application. A good place to check is when the IntentService’s onCreate method is called:
public void onCreate(){
super.onCreate();
String tmpLocation = Environment.getExternalStorageDirectory().getPath() + CACHE_FOLDER;
cacheDir = new File(tmpLocation);
if(!cacheDir.exists()){
cacheDir.mkdirs();
}
}
Using Android’s environment, you can determine the correct prefix for the external file system. Once you know the path to the eventual cache folder, you can then make sure the directory is in place. Yes, I know I told you to avoid file-system contact while on the main thread (and onCreate is called on the main thread), but checking and creating a directory is a small enough task that it should be all right. I’ll leave this as an open question for you as you read through the rest of this chapter: Where might be a better place to put this code?
The fetch
Now that you’ve got a place to save images as you download them, it’s time to implement the image fetcher. Here’s the onHandleIntent method:
protected void onHandleIntent(Intent intent) {
String remoteUrl = intent.getExtras().getString("url");
String location;
String filename = remoteUrl.substring(
remoteUrl.lastIndexOf(File.separator) + 1);
File tmp = new File(
cacheDir.getPath() + File.separator + filename);
if (tmp.exists()) {
location = tmp.getAbsolutePath();
notifyFinished(location, remoteUrl);
stopSelf();
return;
}
try {
URL url = new URL(remoteUrl);
HttpURLConnection httpCon = (HttpURLConnection) url.openConnection();
if (httpCon.getResponseCode() != 200) {
throw new Exception("Failed to connect");
}
InputStream is = httpCon.getInputStream();
FileOutputStream fos = new FileOutputStream(tmp);
writeStream(is, fos);
fos.flush();
fos.close();
is.close();
location = tmp.getAbsolutePath();
notifyFinished(location, remoteUrl);
} catch (Exception e) {
Log.e("Service", "Failed!", e);
}
}
This is a lot of code. Fortunately, most of it is stuff you’ve seen before.
First, you retrieve the URL to be downloaded from the Extras bundle on the intent. Next, you determine a cache file name by taking the last part of the URL. Once you know what the file will eventually be called, you can check to see if it’s already in the cache. If it is, you’re finished, and you can notify the system that the image is available to load into the UI.
If the file isn’t cached, you’ll need to download it. By now you’ve seen the HttpUrlConnection code used to download an image at least once, so I won’t bore you by covering it. Also, if you’ve written any Java code before, you probably know how to write an input stream to disk.
The cleanup
At this point, you’ve created the cache file, retrieved it from the web, and written it to the aforementioned cache file. It’s time to notify anyone who might be listening that the image is available. Here’s the contents of the notifyFinished method that will tell the system both that the image is finished and where to get it.
public static final String TRANSACTION_DONE =
"com.peachpit.TRANSACTION_DONE";
private void notifyFinished(String location, String remoteUrl){
Intent i = new Intent(TRANSACTION_DONE);
i.putExtra("location", location);
i.putExtra("url", remoteUrl);
ImageIntentService.this.sendBroadcast(i);
}
Anyone listening for the broadcast intent com.peachpit.TRANSACTION_DONE will be notified that an image download has finished. They will be able to pull both the URL (so they can tell if it was an image it actually requested) and the location of the cached file.
Rendering the download
In order to interact with the downloading service, there are two steps you’ll need to take. You’ll need to start the service (with the URL you want it to fetch). Before it starts, however, you’ll need to register a listener for the result broadcast. You can see these two steps in the following code:
public void onCreate(Bundle extras){
super.onCreate(extras);
setContentView(R.layout.image_layout);
IntentFilter intentFilter = new IntentFilter();
intentFilter.addAction(ImageIntentService.TRANSACTION_DONE);
registerReceiver(imageReceiver, intentFilter);
Intent i = new Intent(this, ImageIntentService.class);
i.putExtra("url", getIntent().getExtras().getString("url"));
startService(i);
pd = ProgressDialog.show(this,
"Fetching Image",
"Go intent service go!");
}
This code registered a receiver (so you can take action once the download is finished), started the service, and, finally, showed a loading dialog box to the user.
Now take a look at what the imageReceiver class looks like:
private BroadcastReceiver imageReceiver = new BroadcastReceiver() {
@Override
public void onReceive(Context context, Intent intent) {
String location = intent.getExtras().getString("location");
if(TextUtils.isEmpty(location){
String failedString = "Failed to download image";
Toast.makeText(context, failedString , Toast.LENGTH_LONG).show();
}
File imageFile = new File(location);
if(!imageFile.exists()){
pd.dismiss();
String downloadFail = "Unable to Download file :-(";
Toast.makeText(context, downloadFail, Toast.LENGTH_LONG);
return;
}
Bitmap b = BitmapFactory.decodeFile(location);
ImageView iv = (ImageView)findViewById(R.id.remote_image);
iv.setImageBitmap(b);
pd.dismiss();
}
};
This is a custom extension of the BroadcastReceiver class. This is what you’ll need to declare inside your activity to correctly process events from the IntentService. Right now, there are two problems with this code. See if you can recognize them.
First, you’ll need to extract the file location from the intent. You do this by looking for the “location” extra. Once you’ve verified that this is indeed a valid file, you’ll pass it over to the BitmapFactory, which will create the image for you. This bitmap can then be passed off to the ImageView for rendering.
Now, to the things done wrong (stop reading if you haven’t found them yet—no cheating!). First, the code is not checking to see if the intent service is broadcasting a completion intent for exactly the image originally asked for (keep in mind that one service can service requests from any number of activities).
Second, the bitmap is loading from the SD card... on the main thread! Exactly one of the things I’ve been warning you NOT to do.
Checking Your Work
Android, in later versions of the SDK tools, has provided a way to check if your application is breaking the rules and running slow tasks on the main thread. The easiest way to accomplish this is by enabling the setting in your developer options (Figure 4.2). If you want more fine-grained control of when it’s enabled (or you’re on a Gingerbread phone), you can, in any activity, call StrictMode.enableDefaults(). This will begin to throw warnings when the system spots main thread violations. StrictMode has many different configurations and settings, but enabling the defaults and cleaning up as many errors as you can will work wonders for the speed of your application.
Wrapping Up
That about covers how to load data. Remember, loading from the SD card, network transactions, and longer processing tasks MUST be performed off the main thread, or your application, and users, will suffer. You can, as I’ve shown you in this chapter, use a simple thread, an AsyncTask, or an IntentService to retrieve and process your data. But remember, too, that any action modifying any view or object onscreen must be carried out on the main thread (or Android will throw angry exceptions at you).
Further, keep in mind that these three methods are only a few of many possible background data fetching patterns. Loaders, Workers, and ThreadPools are all other alternatives that might suit your application better than the examples I’ve given.
Follow the simple rules I’ve outlined here, and your app will be fast, it will be responsive to your users, it shouldn’t crash (ha!), and it will avoid the dreaded App Not Responding notification of doom. Correct use and avoidance of the main thread is critical to producing a successful application.
If you’re interested in building lists out of complex data from remote sources, the next chapter should give you exactly what you’re looking for. I’ll be showing you how to render a list of Twitter messages to a menu onscreen.
I’ll leave you with a final challenge: Enable Android’s strict mode and move the little file accesses I’ve left in this chapter’s sample code off the main thread. It should be a good way to familiarize yourself with the process before you undertake it on your own.