In this chapter, we cover tips and tricks from our years in the trenches of mobile software design, development, and testing. We also warn you—the project managers, software developers, and testers of mobile applications—of the various and sundry pitfalls to do your best to avoid.
Reading this chapter all at one time when you’re new to mobile development might be a bit overwhelming. Instead, consider reading specific sections when planning the specific parts of the overall process. Not all our advice is appropriate for your specific project, and processes can always be improved, but hopefully this information about how mobile development projects succeed (or fail) can give you some insight into how you might improve the chances of success for your own mobile development projects.
The “rules” of mobile application design are straightforward and apply across all mobile platforms. These rules were crafted to remind us that our applications play a secondary role on the device, which is, at the end of the day, a phone. They also make it clear that we do operate, to some extent, because of the infrastructure managed by the carriers and handset manufacturers. These rules are echoed throughout the Android Software Development Kit (SDK) License Agreement and those of third-party application marketplace Terms and Conditions.
These “rules” are
• Don’t interfere with handset voice and messaging service.
• Don’t break the phone hardware, firmware, software, or OEM components.
• Don’t abuse or cause problems on operator networks.
• Don’t abuse the user’s trust (no malware).
Now perhaps these rules sound like no-brainers, but even the most well-intentioned developer can accidentally fall into some of these categories on occasion if they aren’t careful and don’t test the application thoroughly before distribution. This is especially true for applications that leverage networking support and low-level hardware APIs on the device, and those that store private user data such as names, locations, and contact information.
Mobile users also have their own set of demands for applications they install on their handsets. Applications are expected to.
• Be responsive, stable, and secure.
• Have straightforward user interfaces, easy to get up and running.
• Get the job done with minimal frustration to the user.
• Be available 24 hours a day, 7 days a week (that is, server uptime).
• Include a Help and/or About Screen for feedback and support contact information.
Designing effective user interfaces (UIs) for mobile devices, especially applications that run on a number of different devices, is something of a black art. We’ve all seen bad mobile application UIs and clunky UIs. A frustrating UI can turn a user off, and a good UIs can win a user’s loyalty to your brand long term. It can also give your application an edge over the competition, even if your functionality is similar. A great UIs can win over users even when the functionality is behind the competition. Great UI is that important on mobile.
Here are some tips for great mobile UIs:
• Fill screens sparingly; too much information on one screen overwhelms the user.
• Be consistent with user interface workflows, menu types, and buttons. Consider the handset norms with this consistency, as well.
• Make Touch Mode “hit areas” large enough and spaced appropriately.
• Use big, readable fonts and large icons.
• Keep localization in mind when designing text-heavy user interfaces. Some languages are lengthier than others.
• Reduce keys or clicks needed as much as possible.
• Do not assume specific input mechanisms (such as specific buttons or the existence of a keyboard) will be available on all handsets.
• Try to design the default use case of each screen to require only the user’s thumb. Special cases might require other buttons, input methods, but encourage “thumbing” by default.
• Size graphics appropriately for phone. Do not include oversized resources and assets because they use valuable device resources and load more slowly, even if they resize appropriately. Also consider stripping out unnecessary information, such as EXIF or IPTC metadata, using tools such as ImageMagick or PNGOptimizer. The Draw 9 Patch tool can also help optimize your Android graphics files. (Read more about this in Chapter 5, “Managing Application Resources.”)
• In terms of “friendly” user interfaces, assume users do not read the application permissions when they approve them to install your application. If your application does anything that could cause the user to incur significant fees or shares private information, consider informing them again (as appropriate) when your application performs such actions.
One way to design for different orientations is to try to keep a “working square” area where most of your application activity takes place. This area remains unchanged (or changes little) when the screen orientation changes. Only functionality displayed outside of the “working square” changes substantially when screen orientation changes (see Figure 19.1).
An example of a “working square” is the Desktop area of the T-Mobile G1 and the emulator (see Figure 19.2). In Portrait mode (see Figure 19.2, left), the pull-out Application tray is on the bottom of the screen; in Landscape mode (see Figure 19.2, right), the pull-out Application tray moves to the right side, but the bulk of the Desktop changes little (spreads or shrinks only slightly in different orientations).
Mobile device hardware has come a long way in the past few years, but developers must still work with limited resources. Users do not have the luxury of upgrading the RAM in their phones as they might their laptops. Removable storage devices such as SD cards provide some “extra” space for media storage but cannot be relied upon on all devices.
Mobile developers find that spending some time upfront to design a stable and responsive application results in gains for the long term. The following are some tips for designing robust and responsive mobile applications:
• Use efficient data structures and algorithms; these choices manifest themselves in app responsiveness and happy users.
• Use recursion with care; these functional areas should be code reviewed and performance tested.
• Keep application state at all times. Android activity stack makes this work well, but you should take extra care to go above and beyond.
• Save your state and assume your application will be suspended or stopped at any moment. If your application is suspended or closed, you cannot expect a user to verify anything (click a button, and so on). If your application resumes gracefully, your users will be grateful.
• Start up fast and resume fast. You cannot afford to have the user twiddling thumbs waiting for your application to start. Instead, you need to strike a delicate balance between preloading and on-demand data because your application may be suspended (or closed) with no notice.
• Inform users during long operations by using progress bars. Consider offloading heavy processing to a server instead of performing these operations on the handset because these operations might drain battery life beyond the limits users might accept.
• Ensure long operations are likely to succeed before embarking upon them. For example, if your application downloads large files, check for network connectivity, file size, and available space before attempting the download.
• Verify that your application resource consumption model matches your target audience. Gamers might anticipate shorter battery life on graphics-intensive games, but productivity applications should not drain the battery unnecessarily and be lightweight for people “on the go” who do not always have their phone charging.
We also talk about some of the different design strategies for mobile applications, such as the benefits and drawbacks of networked applications versus stand-alone applications, in Chapter 18, “The Mobile Software Development Process.”
Tip
The Android Developers blog (http://android-developers.blogspot.com/) is another fantastic resource for Android developers. Run by the Google Android team, this Web site provides detailed insight into the Android platform that goes beyond the typical documentation. Here you can find tips, tricks, best practices, and shortcuts on relevant Android development topics such as memory management (like Context management), view optimization (avoiding deep view hierarchies), and layout tricks to improve UI speed. Savvy Android developers visit this blog regularly and incorporate these practices and tips into their Android coding standards.
Many mobile applications integrate with existing handset features such as the phone and Contact database. Make sure you take all the proper precautions necessary to secure and protect private data such as names and contact information used by your application.
Tip
If your application accesses or uses private data, especially usernames, passwords, or contact information, it’s a good idea to include an End User License Agreement (EULA) and a Privacy Policy with your application.
To begin with, limit the private or sensitive data your application stores as much as possible. Don’t store this information in plain text, and don’t transmit it as such. Do not try to work around any security mechanisms imposed by the handset operating system or Android Framework. Consider using the javax.crypto package provided with the Android framework for your encyrption needs.
The same applies to any server-side data storage or transmission. Make sure any servers your application relies on are properly secured against identity theft and invasion of privacy. Treat any servers your application uses like any other part of the system—test them thoroughly. Any private data transmitted should be secured using typical security mechanisms such as SSL.
For billing and revenue generation, mobile applications generally fall into one of four categories:
• Free applications (including those who leverage advertising revenue)
• Single payment (pay once, often seen with games)
• Subscription payments (pay on a schedule, often seen with productivity and service applications)
• On-demand payment for content (pay for specific content, such as a ringtone)
Applications can use multiple types of billing, depending on which marketplaces and billing APIs they use. Not all methods might be available for all applications. With Android, the billing methods can be provided by third parties, so the sky is the limit.
When designing your mobile application, consider the functional areas where billing can come into play and factor this into your application design. Consider the transactional integrity of specific workflow areas of the application that can be charged for. For example, if your application has the capability to deliver data to the handset, make sure this process is transactional in nature so that if you decided to charge for this type of service, the billing features can be dropped in, and when the user pays, the delivery occurs, or the entire transaction is rolled back.
You learn more about the different methods currently available to market your application in Chapter 20, “Selling Your Android Application.”
There are currently no certification programs for Android applications. However, as more applications are developed, third-party standards might be developed to differentiate quality applications from the masses. For example, mobile marketplaces might impose quality requirements. Developers with an eye on financial applications would do well to consider conformance requirements.
Caution
With Android, the market is expected to manage itself. Do not make the mistake of interpreting that as “no rules” when it really means “few rules imposed by Android.”
It can be highly beneficial to examine the certification programs available in other mobile platforms and adjust them for Android. You might also want to examine the certification programs for desktop and server applications (again, we’re thinking of banking and financial applications, but this can apply to any vertical market application) and consider how the requirements can be applied within Android. For example, if a specific type of encryption is required, and it’s available within Android, you can consider using it within your application. Planning for conformance in advance can result in a quality application now and make porting easier in the future.
Generally speaking, it’s best to make as few assumptions about the handset configurations as possible when developing a mobile application. You’ll be rewarded later when you want to port your application or provide an easy upgrade. What assumptions you do make should be carefully considered.
In addition to providing adequate documentation and easy-to-decipher code, you can leverage some tricks to help maintain and monitor mobile applications in the field. Most of these tricks apply only to mobile applications leveraging an application server, but you can sometimes gather information from third-party reports, such as application sales figures from mobile marketplaces.
For networked applications, it can be highly useful to build in some lightweight auditing, logging, and reporting on the application server side to keep your own statistics and not rely solely on third-party numbers. For example, you can easily keep track of
• How many users launch the application for the first time?
• How many users regularly use the application?
• What are the most popular usage patterns and trends?
• What are the least popular usage patterns and features?
• What handsets (determined by application versioning) are the most popular?
Often these figures can be translated into rough estimates of expected sales, which can later be compared with actual sales figures from third-party marketplaces. The most popular usage patterns can be streamlined and made more efficient. The least popular features can be reviewed. Sometimes you can even identify potential bugs, such as features that are not working at all, just by noting that a feature has never been used in the field. Finally, you can begin to determine which handsets are most appropriate for your specific application and user base.
Tip
Gathering anonymous diagnostics is fairly common in these sorts of situations, but avoid keeping any data that can be considered private. Make sure your sample sizes are large enough to obfuscate any personal user details, and make sure to factor out any live QA testing data from your results (especially when considering sales figures).
Mobile application upgrades pose some challenges to developers. It’s best to consider how you will drive users from one version of your application to the next early in the development process. This might be as simple as providing a method to inform users that a new application is available, or as complicated as walking them through the entire upgrade process including migrating their data. There are generally two types of upgrade situations for mobile applications:
• A minor upgrade of content, provided by the application itself from a server
• A major upgrade of the application software, generally purchased through an application store
Minor upgrades are actually built into the application functionality as a feature. This includes the capability to deploy fresh content or features to the handset, and the process is entirely controlled by the developer. Building this sort of functionality into your application can keep your users happy longer, and content is less likely to become dated.
Until recently, major upgrades on mobile applications were unusual. Generally speaking, the software developer was not responsible for helping users migrate data to a new version of the software. However, this is changing. Users now expect that their data, which can be difficult to input into a phone, should follow them when they upgrade.
If your application stores important user data on a server, upgrades can be quite straightforward. However, if application data is available only on the handset, you want to consider how it might be accessed at a future time for upgrade purposes. Perhaps consider an “export data” feature or implement a Content Provider interface for an upgraded application that can be leveraged to extract data from the old version.
Almost inevitably, a developer must adjust to handset changes in the field. Perhaps you develop for a handset that is retired (“sun setting”) soon after you deploy your application, but a new version of that handset is available with minor changes. It helps to design your application with these sorts of potential future problems in mind.
The Android SDK and developer community provide a number of useful tools and resources for application design. You might want to leverage the following tools during this phase of your development project:
• The Android emulator is a good place to start for rapid proof of concept, before you have specific handsets. Remember to test on real handsets for true feasibility analysis.
• The Hierarchy Viewer in Pixel Perfect View allows for accurate user interface design.
• The Draw Nine Patch tool can create stretchable graphics for mobile use.
• Physical handsets, when available.
• Technical specifications for handsets, available from manufacturers and carriers.
Last but not least, here is a list of some of the silly mistakes Android designers should generally do their best to steer clear of.
• Designing or developing for months without performing feasibility testing on the handset
• Designing for a single phone, in a single language, for a single carrier
• Designing as if your device has a large amount of storage and processing power and is always plugged in to a power source
• Developing for the wrong version of the Android SDK (verify handset SDK version)
• Trying to adapt applications to smaller screens after the fact by having the phone “scale”
• Deploying oversized graphics and media assets with an application instead of sizing them appropriately
As we previously discussed, developing applications for mobile is not that different from traditional desktop development. However, developers might find developing mobile applications more restrictive, especially resource constrained. Again, let’s start with some best practices or “rules” for mobile application development.
• Test assumptions regarding feasibility early and often on the target handsets.
• Keep application size as small and efficient as possible.
• Choose efficient data structures and algorithms appropriate to mobile.
• Exercise prudent memory management.
• Assume handsets are running primarily on battery power.
A successful project’s backbone is a good software process. It ensures standards, good communication, and reduces risks. We talked about the overall mobile development process in Chapter 18. Again, here are a few general tips of successful mobile development processes:
• Use an iterative development process.
• Use a regular, reproducible build process with adequate versioning.
• Communicate scope changes to all parties—changes often affect testing most of all.
It cannot be said enough:You must test developer assumptions on the handset. There is nothing worse than designing and developing an application for a few months only to find it needs serious redesign to work on the handset. Just because your application works on the emulator does not, in any way, guarantee that it will work on the handset. Some functional areas to examine carefully for feasibility include
• Functionality that interacts with peripherals and device hardware
• Network speed and latency
• Memory footprint and usage
• Algorithm efficiency
• User interface suitability for small screens
• Device input restrictions when compared to required input methods
• File size and storage usage
We know, we sound like a broken record but, truly, we’ve seen this mistake happen over and over again. Projects are especially vulnerable to this when target handsets aren’t yet available. What happens is that engineers are forced closer to the waterfall method of software development with a big, bad surprise after weeks or months of development on the emulator.
We don’t need to explain why waterfall approaches are dangerous again, do we? You can never be too cautious about this stuff. Think of it as the preflight airline safety speech of mobile software development.
Developers who spend the time and effort necessary to develop efficient mobile application will be rewarded by their users. The following is a representative list of some of the efforts that can be taken:
• Centralizing core features in shared Java packages
• Developing to the specific version of the Android SDK compatible with target handsets
• Using built-in controls and widgets appropriate to the application, customizing only where needed to keep code size small
Use system settings to derive default data, such as language (built-into Android) and local time, and so on. If you change system settings in your app, change them back when your app exits/pauses, as necessarily.
Developing a set of well-communicated coding standards for the development team can help drive home some of the important requirements of mobile applications. Some standards might include
• Implementing robust error handling and handle exceptions gracefully.
• Moving lengthy or process-intensive operations off the main UI thread.
• Releasing objects and resources you aren’t actively using.
• Practicing prudent memory management. Memory leaks can render your application useless.
• Using resources appropriately for future localization. Don’t hardcode strings and other assets in code or layout files.
• Avoiding obfuscation; comments are worthwhile.
• Considering using standard document generation tools, such as Javadoc.
• Instituting and enforce naming conventions—in code and in database schema design.
Performing code inspections can improve the quality of project code, help enforce coding standards, and identify problems before QA gets their hands on a build and spends time and resources testing it.
It can also be helpful to pair developers with the individual QA personnel who test specific functional areas to build a closer relationship between the teams. If testers understand (even less-technical personnel) how the system functions internally, they can test it more thoroughly. This might or might not be done as a formal code review process. For example, a tester can identify defects related to type-safety just by noting the type of input expected (but not validated) on a form input field of a layout or by reviewing Submit or Save button handling function with the developer.
The Android SDK provides a number of packages related to code diagnostics. By building a framework for logging, unit testing, and exercising your application to gather important diagnostic information, such as the frequency of method calls and performance of algorithms, can help you develop a solid, efficient, and effective mobile application.
It should be noted that these diagnostic utilities are generally not used in production, after the application is distributed, because they might impose significant performance reductions and greatly reduce application responsiveness.
In Chapter 3, “Writing Your First Android Application,” we discuss how to leverage the built-in logging class android.util.Log to implement many levels of diagnostic logging, which can be monitored via a number of Android tools, such as the LogCat utility (available within DDMS, ADB, and Android Plug-in for Eclipse [ADT]).
Unit testing can help developers move one step closer to the elusive 100 percent of code coverage testing. The Android SDK includes basic unit testing classes and a more robust package for Android-specific testing mechanisms.
Basic JUnit support is provided through the junit.framework and junit.runner packages. Here you find the familiar framework for running basic unit tests with helper classes for individual test cases. These test cases can be combined into test suites. There are utility classes for your standard assertions and test result logic.
The Android SDK also includes the android.test package, which includes an extensive array of testing tools designed specifically for Android applications. This package builds upon the JUnit framework and adds many interesting features, such as the following:
• Simplified Hooking of Test Instrumentation (android.app.Instrumentation) with android.test.InstrumentationTestRunner, which can be run via ADB shell commands
• Performance Testing (android.test.PerformanceTestCase)
• Single Activity (or Context) Testing (android.test.ActivityUnitTestCase)
• Full Application Testing (android.test.ApplicationTestCase)
• Services Testing (android.test.ServiceTestCase)
• Utilities for generating events such as Touch events (android.test.TouchUtils)
• Many more specialized assertions (android.test.MoreAsserts)
• View validation (android.test.ViewAsserts)
Occasionally, you have a situation in which you need to provide code for a specific handset. Handling bugs that occur only on a single handset can be tricky. You don’t want to branch code unnecessarily, so here are some of your choices:
• If possible, keep the client generic, and use the server to serve up handset-specific items.
• If the conditions can be determined programmatically on the client, try to craft a generic solution that allows developers to continue to develop under one source code tree, without branching.
• If the handset is not a high-priority target, consider dropping it from your requirements if the cost-benefit ratio suggests that a workaround is not cost effective.
• If required, branch the code.
The Android SDK and developer community provide a number of useful tools and resources for application development. You might want to leverage the following tools during this phase of your development project:
• The Eclipse development environment with the ADT
• The Android emulator and physical handsets for unit testing and bug reproduction
• The Android Dalvik Debug Monitor Service (DDMS) tool for debugging and interaction with the emulator or handset
• The Android Debug Bridge (ADB) tool for logging, debugging, and shell access tools
• The sqlite3 command-line tool for application database access (available via ADB shell)
• The Hierarchy Viewer for user interface debugging of views
• The Traceview tool for graphical logging functionality
• The dx tool for generating .dex files and Android bytecode from class files
Here are some of the frustrating and silly mistakes Android developers should try to avoid:
• Forgetting to add new application Activities and necessarily permissions to the AndroidManifest.xml file
• Forgetting to display Toasts using the show() method
• Hard-coding information like network information, test user information, and other data into the application
• Forgetting to disable diagnostic logging before release
• Distributing live applications with debug mode enabled
Like all QA processes, mobile development projects benefit from a well-designed defect tracking system, regularly scheduled builds, and planned, systematic testing. There are also plentiful opportunities for white box (or gray box) testing and some limited opportunities for automation.
Most defect tracking systems can be customized to work for the testing of mobile applications. The defect tracking system must encompass tracking of issues for specific handset defects and problems related to any centralized application servers (if applicable).
A good mobile defect tracking system includes the following information about a typical handset defect:
• Build version information, language, and so on.
• Handset configuration and state information including handset type, firmware version, screen orientation, network state, and carrier information.
• Steps to reproduce the problem using specific details about exactly which input methods were used (touch versus click).
• Device screenshots that can be taken using the Hierarchy Viewer tool provided with the Android SDK.
It’s also important to consider the larger definition of the term defect. Defects can occur on all handsets, on only some handsets, and on the application server (for client/server applications). Some types of defects typical on mobile applications include
• Application crashes and unexpected terminations.
• Features that do not function properly.
• Application uses too much disk space/memory on the handset.
• Inadequate input validation (typically, button mashing).
• Application state management problems (startup, shutdown, suspend, resume, power off).
• Application responsiveness issues (slow startup, shutdown, suspend, resume).
• Usability issues related to input methods, font sizes, and cluttered screen real estate.
• Pausing or “freezing” on the main UI thread (failure to implement asynchronous threading).
• Application indicators missing (failure to indicate progress).
• Application integration with other applications on the handset causing problems.
• Application “not playing nicely” on the handset (draining battery, disabling power-saving mode, overusing networking resources, incurring extensive user charges, obnoxious notifications).
• Application not conforming to third-party agreements, such as Android SDK License Agreement, Google Maps API terms, marketplace terms, or any other terms that apply to the application.
• Application client or server not handling protected/private data securely. This includes ensuring that the application server has adequate uptime and has adequate security measures taken to ensure it won’t be hacked, and so on.
Testing mobile applications poses a unique challenge to the QA team, especially in terms of configuration management. The difficulty of such testing is often underestimated. Don’t make the mistake of thinking that mobile applications are easier to test because they have fewer features than desktop applications and are, therefore, simpler to validate.
Caution
Ensure that all changes in project scope are reviewed by the quality assurance team. Adding new handsets sometimes have little impact on the development schedule but can have significant consequences in terms of testing schedules.
Handset fragmentation is perhaps the biggest challenge the mobile developer faces. Handsets come in various form-factors different resolution screens and different underlying hardware. They come with a variety of input methods such as buttons and touch screens. They come with optional features, such as cameras, WiFi, enhanced graphics, and different location-based service mechanisms. Keeping track of all the handsets, their functional abilities, and so on is a big job and much of the work falls on the test team.
QA personnel must have a detailed understanding of the functionality available of each target handset, including familiarity with what features are available and any handset-specific idiosyncrasies that exist. They must test each handset as it will be used in the field, which might not be the handset’s default configuration or language. This means changing input modes, screen orientation, and locale settings.
Tip
Be aware of how carrier-related firmware can affect how your application works on the handset. For example, let’s assume you’ve gotten your hands on an unbranded version of a target handset and testing has gone well. However, if certain carriers take that same handset, remove some default applications and load up others, this is valuable information to the developer. Just because your application runs flawlessly on the “vanilla” handset doesn’t mean that this is how most users’ devices will be configured by default. Do your best to get test handsets that closely resemble the ones in the field.
One hundred percent testing coverage is impossible, so QA must develop priorities thoughtfully. As we discuss in Chapter 18, developing a handset database can greatly reduce the confusion of mobile configuration management, help determine testing priorities, and keep track of physical hardware available for testing.
Tip
If you have trouble configuring handsets for real-life situations, you might want to look into the handset “labs” available through some carriers. Instead of loaner programs, the developer visits the carrier’s onsite lab where they can rent time on specific handsets. Here, the developer installs their application and tests it—not ideal for recurring testing but much better than no testing, and some labs are staffed with experts to help out with handset specific issues.
There is currently no way to “image” a handset so that you can return to the same starting state again and again. The QA test team needs to define what a “clean” handset is for the purposes of test cases. This can involve a specific uninstall process and some manual clean-up.
Tip
Using the Android SDK tools such as DDMS and ADB enables developers and testers access to the Android file system, including application SQLite databases. These tools can be used to monitor and manipulate data on the handset and the emulator. For example, testers might use the sqlite3 command-line interface to “wipe” an application database or fill it with test data for specific test scenarios.
It is nearly impossible (and certainly not cost-effective) to set up a complete isolated environment for mobile application testing. Although it’s fairly common for networked applications to be tested against test (mock) application servers and then go “live” on a separate server with a similar configuration. However, in terms of handset configuration, mobile software testers must use real handsets with real service to test mobile applications properly. They need to properly make and receive phone calls, send and receive text messages, and do anything a phone would normally do.
Testing an application involves more than just making sure the application works properly. In the real world, your application does not exist in a vacuum but is one of many installed on the handset. Testing a mobile application involves ensuring that the software integrates well with other handset functions and applications. For example, let’s say you were developing a game. Testers must verify that calls received while playing the game caused the game to automatically pause (keep state) and allow calls to be answered or ignored without issue.
Sometimes testers need to be creative when it comes to reproducing certain types of events. For example, testers must ensure that their application behaves appropriately during events such as losing phone service.
Tip
To test loss of signal, you could go out and test your application under a highway overpass, or you could just place the handset in the refrigerator. Don’t leave it in the cold too long, though, or it will drain the battery. Tin cans work great, too, especially those that have cookies in them: First, eat the cookies; then place the phone in the can to seal off the signal.
All test teams strive for 100 percent testing coverage, but most also realize such a goal is not reasonable or cost-effective. Testers must do their best to cover a wide range of scenarios, the depth and breadth of which can be daunting—especially for those new to mobile. Let’s look at several specific types of testing and how QA teams have found ways—some tried-and-true and others new and innovative—to maximize coverage.
In addition to a regular build process, it can be helpful to institute a build acceptance test policy (also sometimes called build validation, smoke testing, sanity testing). Build acceptance tests are short and targeted at key functionality to determine if the build is good enough for more through testing to be completed. This is also an opportunity to quickly verify bug fixes expected to be in the build before a complete retesting cycle occurs.
Mobile build acceptance testing is typically done manually on the highest priority target handset; however, this is also an ideal situation for an automated “sanity” test. By creating a bare-bones functional test for the emulator which, as desktop software, can be used with typical QA automation platforms such as Borland SilkTest (www.borland.com/us/products/silk/silktest/index.html), the team can increase its level of confidence that a build is worth further testing, and the number of bad builds delivered to QA can be minimized.
There is a well-established quality assurance mantra within the mobile development community.
Test early, test often, test on the device.
When you can get your hands on the actual handset your users will have, focus your testing there. Handsets and the service contracts that generally come with them are expensive. Your test team cannot be expected to set up test environments on every carrier or every country where your users will use your application. There are times when the Android emulator can reduce costs and improve testing coverage. Some of the benefits of using the emulator include
• Ability to simulate handsets when they are not available or in short supply
• Ability to test difficult test scenarios not feasible on live handsets
• Ability to be automated like any other desktop software
Developers often target up-and-coming handsets not available to the general public. These handsets are often highly anticipated and developers who are ready with applications for these handsets on Day 1 of release often experience a sales bump because fewer applications are available to these users—less competition, more sales.
Often, developers can gain access to preproduction phones through carrier and manufacturer developer programs. However, developers and testers should be aware of the dangers of testing on preproduction phones:These phones are beta-quality. The final technical specifications and firmware can change without notice. These phone release dates can slip, and the phone might never reach production.
When preproduction phones cannot be acquired, testers can do some functional testing using emulator configurations that attempt to closely match the target platform, lessening the risks for a compact testing cycle when these handsets go live, allowing developers to release applications faster.
How we wish we had better news, but we know of few automated device testing methods available to the mobile community.
Tip
There are some third-party solutions such as Mobile Complete’s DeviceAnywhere (www.deviceanywhere.com/), which enable developers remote access to physical handsets, but the number of Android devices on the market at the time of this writing makes this kind of solution less enticing.
It’s certainly possible to rig up automated testing software to exercise the software emulator. we can certainly imagine someone coming up with a solution—in our minds, the device looks a lot like the automated signature machine U.S. presidents use to sign pictures and Christmas cards. The catch is that every handset looks and acts differently, so any animatronic hand would need to be recalibrated for each device. The other problem is how to determine when the application has failed or succeeded. Again, there are no easy methods to determine this programmatically. If anyone is developing mobile software automated testing tools, it’s likely a mobile software testing consultancy company. For the typical mobile software developer, the costs are likely prohibitive.
Unfortunately, the emulator is more of a “generic” Android device that pretends at many of the device internals. It does not represent the specific implementation of the Android platform that is unique to a given handset. It does not use the same hardware internals to determine signal, networking, or location information. The emulator can pretend to make and receive calls and messages, but this is a simulation and requires some setup (making virtual SD Cards, using multiple emulator instances, and so on). At the end of the day, it doesn’t matter if the application works on the emulator if it doesn’t work on the actual device.
The Android tools provide ample tools for black box and white box testing.
Black box testers might require only testing handsets and test documentation. For black box testing, it is even more important that the testers have a working knowledge of the specific handsets, so providing handset manuals and technical specifications will also aid in more thorough testing.
White box testing has never been easier on mobile. White box testers can leverage the many affordable tools including the Eclipse development environment, which is free, and the many debugging tools available as part of the Android SDK. White box testers want to use the Android Emulator, DDMS, and ADB especially. For this, testers require a computer set up with a development environment, much like the developer’s. The Android tools generally do not impose any substantial burden on the resources of your average laptop.
Although testers often focus on the client portion of the application, they sometimes neglect to thoroughly test the server portion. Many mobile applications rely on networking and leverage an application server. If your application depends on a server to operate, testing the server side of your application is vital.
Caution
In the past, certain carriers tried to impose fees on mobile developers whose application servers go down or don’t respond. We haven’t seen this with Android applications, but you should understand that users expect applications to be available any time, day or night, 24/7. Minimize server down times and make sure the application notifies the users appropriately (and doesn’t crash and burn) if the server is unavailable.
Server rollouts should be managed like any other part of the build process. The server should be versioned and rolled out in a reproducible way.
Often, QA tests against a mock server in a controlled environment. This is especially true if the live server is already operational with real users.
Most server testing should focus on the primary method the server will access—generally through the handset client. It’s also important to test the server in other ways, such as
• Testing the server under load, including stress testing (many users, simulated clients)
• Testing the server security (hacking, SQL injection, and such)
• Testing server upgrades and rollbacks
These types of testing offer yet another opportunity for automated testing to be employed.
Testing a mobile application is not only about finding dysfunctional features, but also about evaluating the usability of the application. Report areas of the application that lack visual appeal or are difficult to navigate or use.
We like to use the walking-and-chewing-gum analogy when it comes to mobile user interfaces. Mobile users frequently do not give the application their full attention. Instead, they walk or do something else while they use it. Applications should be as easy for the user as chewing gum.
Make a habit to try to adapt traditional software testing principles to mobile. For example, veteran testers such as Vijay at Software Testing Help (www.softwaretestinghelp.com) have provided helpful online testing tips that can be easily applied to mobile software projects. Encourage quality assurance personnel to develop and share these practices within your company.
Again, no certification programs are specifically designed for Android applications at this time; however, nothing is stopping the mobile marketplaces from developing them. Consider looking over the certification programs available in other mobile platforms, such as the extensive testing scripts used for TRUE BREW testing and adjusting them for your Android applications. Whether you plan to apply for a specific certification, making an attempt to conform to well-recognized quality guidelines can improve your application’s quality.
In addition to functional testing, there are a few other specialized testing scenarios that any QA team should consider.
It’s necessary to test how the application behaves with other parts of the Android operating system. For example:
• Proper handling of interruptions from the operating system (incoming messages, calls and powering off)
• Validating Content Provider data exposed by your application
• Validating functionality triggered in other applications via an Intent
• Validating any known functionality triggered in your application via an Intent
• Validating any secondary entry points to your application as defined in the AndroidManifest.xml, such as application shortcuts
• Validating service-related features, if applicable
When possible, perform upgrade tests of both the client and the server. If upgrade support is planned, have development create a mock upgraded Android application so that QA can validate that data migration occurs properly, even if the upgraded application does nothing with the data.
It’s a good idea to test internationalization support early in the development process—both the client and the server. You’re likely to run into some problems in this area related to screen real-estate problems and issues with strings, dates, and times.
Make sure to review any policies, agreements, and terms that your application must conform to and make sure your application complies. For example, Android applications must by default conform to the Android Developer Agreement and the Google Maps terms of service (if applicable).
Generally speaking, installation of Android applications is straightforward; however, you need to test installations on handsets with low resources and low memory and test installation from the specific marketplaces when your application “goes live.”
Application performance matters in the mobile world. The Android SDK has support for calculating performance benchmarks within an application and monitoring memory and resource usage. Familiarize the QA team with these utilities and use them often to help identify performance bottlenecks and dangerous memory leaks and misused resources.
Billing is too important to leave to guesswork. Test it. You’ll notice a lot of test applications on the Android marketplace—we wouldn’t be surprised to see a sandbox test market area for this soon. As the Android platform acceptance grows, you might find carriers that have sandbox networks you can use for testing. If not, do what you can and always test the deployed application as soon as possible.
Regardless of the workflow you design, understand that users will do random, unexpected things—on purposes and by accident. Some users are “button mashers,” whereas others forget to set the keypad lock before putting the phone in their pocket, resulting in a weird set of key presses. A phone call or text message will inevitably come in during the farthest, most-remote edge cases. Your application must be robust enough to handle this. The Exerciser Monkey command-line tool is a good way to test for this type of event.
Every mobile developer wants to develop a “killer app”—those applications that go viral, rocket to the top of the charts, make millions a month. Most people think that if they just find the right idea, they’ll have a killer app on their hands. Developers are always scouring the top-ten lists, trying to figure out how to develop the next big thing.
But let us tell you a little secret: If there’s one thing that all “killer apps” share, it’s a higher-than-average quality standard. No clunky, slow, obnoxious, or difficult-to-use application ever makes it to the big leagues. Testing and enforcing quality standards can mean the difference between a mediocre application and a killer app.
If you spend any time examining the mobile marketplace, you notice a number of larger mobile development companies publish a variety of high-quality applications with a shared look and feel. These companies leverage user interface consistency, shared, and above-average quality standards to build brand loyalty and increases market share, while hedging their bets that perhaps just one of their many applications will have that magical combination of great idea and quality design. Other, smaller companies often have the great ideas but struggle with the quality aspects of mobile software development. The inevitable result is that the mobile marketplace is full of fantastic application ideas badly executed with poor user interfaces and crippling defects.
The Android SDK and developer community provide a number of useful tools and resources for application testing and quality assurance. You might want to leverage these tools during this phase of your development project.
• The physical handsets for testing and bug reproduction
• The Android emulator for automated testing and testing of builds when handsets are not available
• The Android DDMS tool for debugging and interaction with the emulator or handset
• The ADB tool for logging, debugging, and shell access tools
• The Exerciser Monkey command-line tool for stress testing of input (available via ADB shell)
• The sqlite3 command-line tool for application database access (available via ADB shell)
• The Hierarchy Viewer for user interface navigation and verification and for screenshots of the handset
• The Eclipse development environment with the ADT and related logging and debugging tools for white box testing
It should be noted that although we have used the Android tools such as the Android emulator and DDMS debugging tools with Eclipse, these are stand-alone tools that can be used by quality assurance personnel without the need for source code or a development environment.
Here are some of the frustrating and silly mistakes Android testers should try to avoid:
• Not testing the server side of an application as thoroughly as the client side.
• Not testing with the appropriate version of the Android SDK (handset versus development build versions).
• Not testing enough on the handset and assuming the emulator is acceptable.
• Not testing the live application (billing, installation, and such).
• Neglecting to test all entry points to the application.
• Neglecting to test using battery power only. Don’t always have the device plugged in.
Mobile quality assurance can be outsourced. Remember, though, that the success of outsourcing your QA responsibilities depends on the quality and detail of the documentation you can provide. Outsourcing makes it more difficult to form the close relationships between QA and developers that help ensure thorough and comprehensive testing.
In this chapter, we armed you—the software designers, developers, and testers—with real-world knowledge and experience from veteran mobile developers. Feel free to pick and choose which information works well for your specific project, and keep in mind that software process, especially the mobile software process, is always open to improvement.
Software Testing Help: www.softwaretestinghelp.com
Wikipedia on Software Testing: http://en.wikipedia.org/wiki/Software_testing