Use Your Product

The best way to learn about your product is to use it (if it already exists). In the case of a travel app, you should search for a hotel and flight, make a reservation, cancel your reservation, ask for help, etc. Stretch the product to its limits. Be sure to note possible pain points so you spot them if your participants encounter them as well. It will be easier to identify patterns if you have some ideas of what to look for.

Networking

You are surrounded by people who know about your product and domain; you just have to get to know them. If you work at a company, start by finding out if it has ever conducted user research in the past (in-house or hired a vendor). Read any existing research reports to see if there are known issues and/or existing user requirements. Meet the content writers for the company. These are the folks who create the user manuals and online help (for websites and web applications). Ask them what is difficult to document. Is it because it is difficult to articulate, or is the product itself too complicated to explain?

If you work in academia, or if your company offers training courses, attend classes and speak with the folks who teach the courses. What is hard to teach? What types of question are users asking? What tips (not documented) are the instructors offering?

Customer Support Comments

If you are working on a product that already has an existing version and your company has a help desk or customer support group, you can learn a great deal about your product by visiting that department. If you work independently, you can often find customer comments available online.

People rarely call or e-mail with compliments to tell a company how wonderful their product is, so customer calls pertain to issues you should become familiar with. If you can access logs of customer questions, problems, and complaints over time, you will see trends of difficulties users are encountering. Perhaps the user’s manual or help is not helpful enough. Perhaps the product has a bug that was not captured during quality assurance (QA), or perhaps users do not like or cannot figure out a feature the product team thought every user had to have. This can give you a sense of where you will need to focus your efforts to improve the product.

Users may not be able to accurately describe the problem they are having or may misdiagnose the cause. Likewise, customer support may not have experience with the product under consideration. Although this should never be the case, it often is. If your support staff is not very familiar with the product in question, they may also misdiagnose the cause of the customer’s problem or may not capture the issue correctly. This means that once you have a log of customer feedback, you may need to conduct interviews or field studies with users to get a full understanding of the issues.

Social Sentiment Analysis

People could be talking about your product and brand right now! According to the Pew Research Center’s Internet and American Life Project, 73% of American online adults use social media (as of September 2013). Ninety-five percent of people say they share bad product experiences online and 45% share bad customer service experiences via Facebook, Twitter, and other platforms (Costa et al., 2013). However, 87% say they share good customer service interactions online, too. So whether you know it or not, you have a presence on social media.

Does your company have a Facebook or Google Plus page? A Twitter feed? If so, speak with the people at your company whose job it is to respond to user comments, often called “community managers” or the like. Whether you have a formal presence on a social networking site or not, do a search of the various sites and see what people have said. Using a tool like Radian6, Crimson Hexagon, Sysomos, or Clarabridge, you can analyze the sentiment of those comments. Using these tools, you can:

■ Learn what users are saying right now

■ Learn what users say they like and do not like about your product/service

■ Learn emerging trends or topics

■ See where your users are that are talking about you

■ Track how your customer base is changing over time and in response to campaigns

Figure 2.1 shows a photo of the Clemson University Social Media Listening Center as an example of what you might see. These tools can take a first pass at detecting positive and negative sentiment, but many will be labeled as “unknown.” You will have to manually review those and mark them as positive or negative. Good tools will learn based on your feedback, but be wary of sarcasm and slang, as many tools are unable to accurately categorize them.

Log Files and Web Analytics

If you are responsible for the usability of a website, web server log files may provide an interesting perspective for you. When a file is retrieved from a website, server software keeps a record of it. The server stores this information in the form of text files.

The information contained in a log file varies but will typically include the source of a request, the file requested, the date and time of the request, the content type and length of the transferred file, the referring page, the user’s browser and platform, and error messages. Figure 2.2 shows a sample of a server log file.

Services like Google Analytics are available that capture user behavior across your site by implanting code on the website. There are also analytics tools to give more insight into what happens within pages (e.g., Crazy Egg, ClickTale) by recording user actions within a page in a more fine-grained way. However, you may need to work with your organization’s IT department to collect log data, store it, and extract it in order to conduct user research or usability analysis. Information that can be captured includes the following:

■ Unique ID for each visitor so you can see who is returning.

■ Click path (i.e., the pages users visit and the sequence in which they visit them).

■ Time spent per page.

■ Where visitors leave the site (what page).

■ Actions taken (e.g., purchases made, downloads completed, information viewed).

There are some issues or limitations that you should be aware of when interpreting the data you collect from log files:

■ The log files often contain an Internet Protocol (IP) address temporarily assigned by an Internet service provider (ISP) or a corporate proxy server. This prevents unique identification of each user. Be aware that under some circumstances (e.g., children, certain countries), IP addresses are considered personally identifiable information (PII). It is good practice to remove any potentially identifiable information from log files prior to analysis.

■ Browser caching leaves gaps in the recorded click stream. If a page is cached, the log files do not record the second, third, or hundredth time it is visited during the user’s stay at your site. Depending on your tools, you may or may not be able to capture the pages called when the back browser button is used. Consequently, the last page recorded in the log file may not be the last page viewed, if the user exited on a previously cached page. As a result, you cannot be sure what the “exit page” was for the user.

■ Log files record when a request was made for a page but not when the transfer was completed. Also, if a user spends 30 minutes on a single page, you do not know why. Did the user walk away or look at another site or program, or was the user viewing the page the entire time?

■ You cannot be sure whether the user completed his or her goal (i.e., purchasing, downloading, looking for information) if you do not know the user’s goal. Perhaps the user wanted to buy three CDs but could find only one. Perhaps the user was looking for information and never found it.

You can work with your IT department to address some of these issues, but it takes time and knowledge of what you need versus what can be captured. External companies like WebTrends (www.netiq.com/webtrends/default.asp) can be hired to help if you are unfamiliar with this. These companies are a great source of basic usage data once you have a website running (e.g., number of page views per hour or per day, what page the user came from, how the user got there, time spent on the homepage, number of excursions to other pages, time spent on each page, ads clicked on).

Examining the time users spend per page is a more meaningful measure than the number of page views per page. When analyzing time data in a log file, it is best to use median or trimmed mean rather than average times because they are less susceptible to outliers. You must look at your distribution and decide if throwing out clear high and low outliers will make a mean useful or if you really need a median. Unfortunately, if you are using web analytics tools, you probably will not get to see the variance in values. However, there is typically a cap on high outlying time values because if a user spends more than half an hour inactive, it will get counted as an exit and excluded from the time-on-page calculation.

Probably the most interesting data for you will be click-path analysis, segmenting users based on behavior on your site and then comparing those user types, seeing how users got to your site, and analyzing what they searched for. However, it is best to study log files and web analytics over a long period to discover data that complement or spur studies in greater depth. For example, you can identify areas of a site for further testing, look for seasonal trends, and see how redesigns in your site affect user behavior.

Your Marketing Department

Frequently, your company’s marketing department will conduct focus groups or competitive analysis (see next section) to determine the need for a product and how best to promote and place it. Although this information is meant to drive sales, it is an excellent source to learn about the product, as well as potential customers and competitors.

Marketing information should not be confused with user requirements. The data from the marketing department can reflect the end users’ needs, but not always. Marketing collects information about the value and perceived worth of a product, whereas user research professionals collect information about how something is used and how the product’s worth is realized.

Additionally, the information you collect from the marketing department is only part of the information needed when creating a user profile. It is often missing contextual information about circumstances and environment that can affect a user’s decision to use and how to use your product. This is often the case for products that are to be used by corporations rather than an individual (e.g., human resources applications versus a website designed to sell books to the public). In the case of corporate products, the marketing department is typically interested in the business needs of the marketplace (i.e., companies that could be potential buyers) rather than the end users. Regardless, this information can be very helpful to you when you are starting out.

When you contact the marketing department, you may want to ask them questions, such as:

■ Where are you collecting your data?

■ Who are our competitors?

■ What is the profile of the people you have spoken with, and how did you find them?

■ What activities have you conducted (e.g., focus groups, surveys)?

■ When is your next activity scheduled?

■ Have you conducted a competitive analysis?

■ What are the requirements you have collected?

Competitors

You can learn a lot from your competitors. A competitive analysis lists the features, strengths, weaknesses, user base, and price point for your competitors. It should include not only first-hand experience with the product(s) but also user reviews and analysis from external experts or trade publications.

This can be an effective way to gain an advantage over your competitors. It is beneficial to conduct a competitive analysis when you are creating an entirely new product or simply entering a new product domain. It can also be a great strategy if your product is failing, but your competitor’s product is thriving. It is wise to periodically check out your competition to see where you stand with the rest of the pack. Some companies have product analysts whose primary job is to keep tabs on the competitors and the market. Get to know this person and learn everything he or she knows. If the product does not have a product analyst, this is something that you can do.

Do not limit yourself to direct competitors. You should also examine surrogate products. These products may or may not compete directly with your product, but they have similar features to your product and should be studied to learn about your strengths and weaknesses. For example, if you are adding a shopping cart to your travel app, check out companies that have shopping carts, even if they do not compete with you (e.g., online book stores). Some people make the mistake of thinking their product or service is so innovative that no one else does it; therefore, they skip competitive analysis. No product is so revolutionary that there is not someone out there from which to learn.

Traditional competitive analysis focuses more on cost, buying trends, and advertising. A usability competitive analysis is more concerned with the user experience (e.g., user interface, features, user satisfaction, overall usability). The goals of both types of competitive analyses are to learn from your competitors and to snag a strategic advantage. Below, we will concentrate on conducting a usability competitive analysis.

To identify your competitors, speak with the product team and sales or marketing department, conduct a web search, read trade magazines, and conduct user surveys, interviews, or focus groups. Market analysts and researchers (e.g., CNET, ZDNet, Gartner, Anderson, Forrester Research, IDC) can be a great way to collect information about your product’s market space and competitors. These companies identify and analyze emerging trends in products and their impact on business.

Keep in mind primary competitors as well as secondary competitors. A secondary competitor can be a smaller company with a less threatening product offering, one having only a few features in common with your product, or one competing indirectly with your product. For example, the brick-and-mortar travel agency does not compete directly with your online travel company, but it is an alternative that should not be ignored.

Once you have identified your competitors, you should ascertain their:

■ Strengths

■ Weaknesses

■ Customer base (profile of users, size of customer base, loyalty, etc.)

■ Availability

■ Functionality and unique features

■ Reputation

■ Requirements (hardware, software, etc.)

If a product can be bought off the shelf or is an easily accessible website, this should not be a problem. However, some products or services are sold only directly to companies. Many major software companies state in their licensing agreement that you are not allowed to conduct competitive tests against their product. They also state that you cannot show the installation of the product (or the product in use) to a competitor company. Check with your legal department for advice before proceeding.

If you are able to evaluate the competitor product yourself, you should identify a set of core tasks with which to compare your product (if you have one at this stage) and the competitor’s. This is particularly important if you plan to include functionality from the other product, because you may learn that a function does not work well. Take numerous screenshots or photos and record your interaction as you conduct the evaluation.

Whether or not you are able to access the product yourself, interviews (Chapter 9, page 220), surveys (Chapter 10, page 266), focus groups (Chapter 12, page 340), and evaluations (Chapter 14, page 432) will be valuable ways to learn about users’ perceptions of the product. By conducting these kinds of activities with users of your competitor’s products, you can learn about the strengths, weaknesses, and key features of these products. In a competitive analysis, the majority of your effort should be spent mining the competitor’s product for ideas (e.g., functionality, user interface style, widgets, task structure, terminology).

There are a number of companies available to measure the usability or user satisfaction of a website (e.g., Bizrate Insights, OpinionLab, User Focus). They can do this for your site or for your competitor’s site. These companies send actual or target customers to any website (i.e., yours or a competitor’s) and then collect qualitative, quantitative, and behavioral data as users pursue actual tasks in a natural environment, such as their own homes and offices. Most companies allow clients to easily analyze both the quantitative and qualitative data gathered during an evaluation. This approach can be quite beneficial, but it is often expensive and requires that users can easily access the website. If the web-based product must be purchased or is behind a firewall, you will have to provide the users with access. In the case of a competitor that sells licensed software, this option is more difficult and more expensive. For example, you may need to send participants to a dummy page where they get assigned a dummy product key, and proceed from there.

As you conduct your competitive analysis, it is helpful to create a grid comparing your product against the competitor’s (see Table 2.1). List the key features, design strengths and weaknesses, usability scores or issues, or anything you can learn. Tracking this information over time will show you how your product compares and how the market may be changing.

Early Adopter or Partner Feedback

Frequently, companies will align themselves with a small number of customers, sometimes referred to as “trusted testers,” in the early stages of development. These customers may play an active role in the design of the product and become partners in the process. They may implement early versions of the product on a small scale in their own companies. This relationship is beneficial to all parties. The customers get to help design the product to meet the needs of their own companies. On the other side, the product team gets early feedback to “fail fast” and iterate, as well as ask early adopters for references when selling the product to others.

The feedback early adopters provide can be enlightening because the product can be implemented and used in a real-world setting (as opposed to testing in the lab). You can leverage these existing relationships to learn about the product space and some of the product’s known issues. However, when you are ready to move on to the next stage and collect user requirements, be wary of basing your user requirements on the few early adopters as they may not be representative of all of your user base. You should obtain user requirements from a variety of users (refer to Section “Learn About Your Users” below).

Step 1: User Profile

The first step in understanding your users is to create a user profile.

Creating a User Profile

There are several characteristics you need to consider to develop a thorough user profile. We provide an ideal list of characteristics below, but you will likely not have access to all of this information. As you do further research and conduct additional user requirement activities, you will fill in these blanks, but you may never find the answers to some of the questions. Ideally, you should determine not only the typical or most frequent level for each of the characteristics but also the range and the percentage of users who fall along that range. As a final note, some of the characteristics on page 39 are more important than others in regard to your product and situation. Prioritize the characteristics and spend the majority of resources capturing information on those key characteristics for your product. For example, if a human resources administrator enters the wrong social security number into a financial application, an employee might not get paid. This is terrible, but if a medical professional enters the wrong social security number in an electronic chart, a patient might get the wrong medication. This is much more serious, so it is important to understand not only the tasks a user does but also the consequences of a possible error. Figure 2.4 shows a sample user profile. The US Census (census.gov) and Pew Research Center (www.pewinternet.org/) have well-tested questions and options for capturing demographic data, so we recommend referring to their surveys when designing yours:

■ Demographic characteristics—age, gender, location, socioeconomic status

■ Occupation experience—current job title, years at the company, years of experience at that position, responsibilities, previous jobs and job titles

■ Company information—company size, industry

■ Education—degree, major, courses taken

■ Computer experience—computer skills, years of experience

■ Specific product experience—experience with competitors’ products or other domain-specific products, usage trends

■ Tasks—primary tasks, secondary tasks

■ Domain knowledge—the users’ understanding of the product area

■ Technology available—computer hardware (monitor size, computing speed, etc.), software, other tools typically used

■ Attitudes and values—product preferences, fear of technology, etc.

■ Learning style—visual learner, audio learner, etc.

■ Criticality of errors—in general, the possible consequences of a user’s error

Once you determine the range of responses for each of the characteristics and the percentage of users along that range, you will want to categorize your users into groups based on their similarities. Some groupings you may use are the following:

■ Age (e.g., child, young adult, adult, older adult)

■ Experience (e.g., novice, expert)

■ Attitudes (e.g., first adopters, technophobe)

■ Primary task(s) (e.g., buyer, seller)

You can use an affinity diagram to organize the characteristics into groups (see Chapter 12, Focus Groups, page 363). The groups should be significantly different from each other in order to justify them as different user types. As with many things, this is more of an art than a science, and there are rarely clearly marked boundaries that put every user in one group or another. Having multiple stakeholders take part in the affinity diagram exercise can help when creating these groups and also assures stakeholder buy-in from the very beginning (refer to Chapter 1, “Getting Stakeholder Buy-in for Your Activity” section, page 16).

Now that you have a handle on your user population, you can develop personas, scenarios, and a recruitment screener (refer to Chapter 6, “Recruiting Participants” section, page 126).

Step 2: Personas

© 1993 The New Yorker Collection 1993 Roz Chast from cartoonbank.com. All rights reserved.

Alan Cooper developed a method called “Goal-Directed Design” in which personas are a key part. Personas were first introduced to the world in Cooper’s 1999 book The Inmates are Running the Asylum.

Creating a Persona

There are several components to a persona. You can add as much detail to each of these areas as you have, but you may not be able to fill in all areas at first. The details will come from the information in your user profile. Just as developing a user profile is an iterative process, so is persona development. As you conduct user requirement activities, you should take what you learn to validate and beef up your personas. When creating a persona, it should be fictional but describe attributes from real users. Provide details and maintain authenticity. The list below is an idealized list—you may not have all the information below (fill in what you can):

■ Identity. Give this user a first and last name. Provide an age and other demographic data that would be representative of the user profile.

■ Status. Is this a primary user, secondary user, tertiary user, or antiuser of your system?

■ Goals. What are these user’s goals, particularly those related to your specific product or competitor products?

■ Skill set. What is the background and expertise of your user? This includes education, training, and specialized skills. Again, do not limit yourself to details related to your specific product.

■ Tasks. What are the basic or critical tasks the user conducts? What are the frequency, importance, and duration of those tasks? More detailed task information is included in scenarios (see below).

■ Relationships. Understanding with whom the user associates is important. Including relationships in the persona keeps you thinking about secondary and tertiary stakeholders.

■ Requirements. What does your user need in order to use your product or be successful using it (e.g., high-speed Internet connection, specific mobile phone OS, specific training or education)? Including quotes will really drive those needs home.

■ Expectations. How does the user think the product works? How does the user organize the information in his or her domain/job?

■ Photograph. Include a photo in your persona to put a human face to your end user.

Tip

It is helpful to give your personas disabilities of one kind or another. Even though a minority of your users may have disabilities at any given time, designing for people with disabilities will tend to help everyone, and building accessibility into a product starts with thinking about users.

Just as there are several types of user profiles, there are several types of personas: primary user, secondary user, tertiary user, and the antiuser (or negative users). The primary, secondary, and tertiary users have been described. An antiuser is one who would not buy or use your product in any way. You want to keep this persona in mind to warn you when you are getting off track. For example, if you are designing a product for an expert user but find more and more instruction text, tutorials, and help dialogues creeping into the product, you should check your antiuser persona (a novice user in need of a “walk up and use” product) to see whether this product would now work for him or her. If so, you are on the wrong track. You want to be sure that you are designing for the primary user while considering the secondary and tertiary users. Figure 2.5 shows a persona for a travel agent.

Step 3: Guiding Principles/Antiprinciples

Most products begin with some kind of design document or spec that states the goal or purpose of the product and planned functionality to meet that goal. Despite this, teams can still end up with different understandings about what the product should do. To address this, it is helpful to list the guiding principles (GPs) for the product. This is a qualitative description of the principles the product stands by. Similarly, antiprinciples (APs) should be documented to explicitly state what this product does not intend to address. If you find your product can be described by any of the APs, you know you are off track. Although the APs can be the opposite of the GPs, they oftentimes include unique considerations. For example, an AP might be “ad-generating revenue.” You are unlikely to have “negative ad revenue” or similar in your GPs. See Table 2.3 for an example.

Step 4: Scenarios

Scenarios, often referred to as “use cases,” are stories about the personas you created and should fit into the guiding principles you identified. A good scenario begins with a persona and then adds more detail based on your user requirements activities. The story describes how a particular persona completes a task or behaves in a given situation. It provides a setting; has actors, objectives or goals, and a sequence of events; and closes with a result.

Many global companies based in the United States develop products first for the United States and then think about how they might need to change that product for outside of the United States. At a minimum, this involves internationalization and localization or globalization (a combination of internationalization and localization). Internationalization is the process of developing the infrastructure in your product so that it can potentially be adapted for different languages and regions without requiring engineering changes each time. Localization means using the infrastructure created during internationalization to adapt your product to a specific language and/or region by adding local-specific components and translating text. This means adapting your product to support different languages, regional differences, and technical requirements. But it is not enough to simply translate the content and localize things like currency, time, measurements, holidays, titles, and standards (e.g., battery size, power source). If you rely on third-party sources for content or functionality, you need to adapt it or find alternative sources if they are not available for the country of interest. You also must be aware of any regulatory compliance that applies to your product/domain (e.g., taxes, laws, privacy, accessibility, censorship).

But even if you get all of that covered, you have not focused on the user. It is just as important to understand the user needs of your population and any cultural differences as it is to get the language, technical platform, and legal issues covered. It is unlikely that your product is going to work for every single person in the United States because the population in the United States is so diverse. The differences in context, background, technology, income, needs, etc., mean that a one-size-fits-all product will not support every person in the United States. The same is true for countries in the rest of the world. You cannot make assumptions about needs of users in China as a whole, much less a region as broad as Asia, and yet, many companies do. Once a product is localized, works on the major platforms for the region, and is in compliance with all laws and regulations, many companies think they are done. To truly support the users in a particular country or region, you must use the same techniques in this book to identify user profiles, scenarios, and needs of those users. You should spend as much time and effort developing your product for any country/region outside of the United States as you did developing it for the United States.

You will likely need to work with a researcher or at least recruiting vendor in the country of interest in order to get access to participants, incentives, and facilities to conduct the research. A vendor in the country can help not only with conducting the research (or conduct it for you), but they can also inform you about cultural issues you may not be aware of prior to the study. Being culturally aware in advance of the study will not only make your study go much more smoothly and help with the reliability and validity of the data collection, but can also help you avoid offending your participants.

Accessibility

The best products and services adhere to the guidelines of universal design or inclusive design. The term “universal design” was coined by Ronald L. Mace (1985). It means that your product or service enables everyone to access and use regardless of one’s age, abilities, or status in life. Universal design is important because everyone has limitations at some point or another that makes accessing products or services difficult, and design that is inclusive is easier for everyone to use (Story, Mace, & Mueller, 1998):

■ Hearing impaired: People working in loud environments (e.g., construction sites) or those listening to music cranked up are going to have a difficult time hearing any audio signal from your product.

■ Visually impaired: Glare from the screens on mobile devices can make seeing anything, particularly low-contrast content, extremely difficult.

■ Color-blind: If your content depends on color-coding for interpretation, it will be lost when your users view it on displays with limited colors or print it out in black and white.

■ Limited dexterity: In cold winter climates, manipulating smartphones and tablets with gloves can be tricky unless you have purchased special gloves for that purpose.

■ Physically challenged: Few people have gone their whole lives without an injury that limits their movements. Parents carrying a child or heavy groceries tie up the use of their arms. Pushing a stroller, bike, walker, or rolling luggage prevents the use of stairs and pose a challenge with curbs.

■ Mentally challenged: Anyone that has suffered from sleep deprivation, is under the influence of medication, or dealt with short-term memory issues may not be able to comprehend your product or information in the same way you intended.

■ Illiterate: You may be able to read content in your language fluently, but the moment you step into a store, restaurant, or country where information is displayed in a different language, you become illiterate.

When you are conducting user research, you need to think about the full range of users, recruiting a broad range of ages, abilities, and statuses. The designer(s) for your product might be using the latest computers and monitors with the highest speed Internet connection, but there will be a segment of your user population that does not. What does your product look like for them? Can they even access it?

The Center for Universal Design offers seven principles for universal design (Copyright © 1997 NC State University, The Center for Universal Design):

1. Equitable use: The design is useful and marketable to people with diverse abilities.

2. Flexibility in use: The design accommodates a wide range of individual preferences and abilities.

3. Simple and intuitive use: Use of the design is easy to understand, regardless of the user’s experience, knowledge, language skills, or current concentration level.

4. Perceptible information: The design communicates necessary information effectively to the user, regardless of ambient conditions or the user’s sensory abilities.

5. Tolerance for error: The design minimizes hazards and the adverse consequences of accidental or unintended actions.

6. Low physical effort: The design can be used efficiently and comfortably with minimum fatigue.

7. Size and space for approach and use: Appropriate size and space are provided for approach, reach, manipulation, and use regardless of user’s body size, posture, or mobility.

Children

Increasingly, children are being exposed to tablets before they are able to read, and some elementary school children are getting their parents’ hand-me-down smartphones. If you are creating consumer products, there is a good chance children are using your products whether you intend them to or not. These are your future power users and a huge opportunity for you, so you would do well to at least consider children in your product development process. However, there are a number of things you need to be aware of when conducting research with children:

■ Nondisclosure agreements: Anyone under the age of 18 in the United States cannot sign legal documents. A legal guardian must sign NDAs on behalf of the children. You will also need to explain what confidentiality means in terms the child can understand.

■ Informed consent: Not only must parents be informed about the details of the study to consent on behalf of their child(ren), but also, you are ethically obligated to explain to children their rights prior to any study (see Chapter 3, “Ethical and Legal Considerations”) in words that the child can understand. Although parents must consent to the study, it is up to the child to decide whether or not he or she wants to participate and when he or she wishes to withdraw. Parents and children must be informed of this prior to the start of the study.

■ Incentives: Identify appropriate incentives for the age of your participants. A Visa gift card may not be as enticing for a child as a gift card to a toy store. Discuss the incentive with the parents in advance to ensure that the parents will not object to it. Nothing is worse than handing a gift to a child at the end of a study only to have a parent take it away.

■ Cognitive ability: Children will not have the same cognitive abilities or focus as the adults you typically conduct research with. Keep this in mind when designing the length of your study and tasks.

■ Regulations: Depending on the country, there are stricter regulations around personally identifiable information (PII) than for adults. This is certainly true in the United States.

■ Instructions: Children should be informed that this is not a test and that there are no right or wrong answers. They are not being evaluated; the product is. Just as adult participants may need to be reminded of this during the study, so must children.

■ Eye tracking: If you are conducting an evaluative study using an eye tracker, you must ensure that it is safe for children with no long-term exposure concerns, inform the parents during the screening process that an eye tracker will be used, and get both the parents’ and child’s consent. Be aware that some mobile eye trackers can actually cause nausea so you are obligated to advise both child and parent of the risk during recruitment.

Older Users

As we age, we experience changes in our cognitive and physical capabilities and limitations. While some abilities increase (e.g., verbal, wisdom), other abilities tend to decline (e.g., visual acuity). These changes mean you should consider the age of your target users and of your participants. Specifically, you should include older adults in your participant sample if you plan for older people to use your product. And with older adults (65 years and above) representing 13% of the population in 2010 and estimated to represent 19% in the United States by 2030 based on trends in the data (Department of Health and Services, 2010), it is likely that your product will be used by older people.

Just like other populations (e.g., children), there is a huge variation in what it means to be an older adult. “To compare a person who is 60 to a person who is 90 is not unlike comparing a 13-year-old to a 45-year-old” (Fisk, Rogers, Charness, Czaja, & Sharit, 2009). In aging research, there are three categories of “older adult:”

■ The “young old”—65-74

■ The “old”—74-84

■ The “oldest old”—85 +

When selecting older adult research participants for a study, you should carefully consider what inclusion/exclusion criteria are appropriate. Criteria could include the following:

■ Cognitive status (and impairments)—older adults, especially those in the “old” and “oldest old,” are more likely to have a cognitive impairment than younger adults. You can screen for cognitive impairment status using the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975).

■ Health status—the majority of older adults have at least one chronic health condition (AARP, 2009). You may be interested in perceived health status (how older adults feel about their health) or the number and specific types of health conditions your participants live with.

■ Living situation—while many older adults are “community dwelling,” meaning they live in their own home or apartment, many live in communities or institutions that provide various levels of care. While retirement communities typically offer convenience services such as yard maintenance, assisted living facilities may offer more extensive services, such as meal preparation and transportation assistance. Nursing homes offer skilled medical care on a daily basis.

Declines in speed of processing and working memory associated with aging mean that older adults will likely take longer to complete a study (Fisk et al., 2009). A guideline is to plan for a study protocol to take about one and a half times as long with an older adult as with a younger adult. If a study takes an hour for a younger adult, it will likely take an hour and a half for an older adult to complete.

Rasas: India

Rasas are the essences of our emotions that exist in both the body and the mind. The central objective of classical Indian art and drama is to create the appropriate “rasa” for the spectators to communicate or suggest a kind of knowledge that cannot be clearly expressed in words.

We incorporated the nine rasas that are most well known in India (Figure 2.6) in the form of cultural probes for a project in India to explore the emotions people felt when interacting with ATMs for the first time. These rasas were love, joy, anger, peace, courage, sadness, disgust, wonder, and fear.

This culture probe was designed as a set of cinema “emotion tickets,” carrying the omnipresent Bollywood theme forward (see Figure 2.7). Each rasa was expressed (in the emotion ticket) through images and dialogues from familiar Bollywood films (Chavan & Munshi, 2004). The “emotion tickets” were designed in a way that participants could identify a rasa (emotion) when interacting with an ATM and also note some details about the context of the interaction. Each user was given a booklet with a set of emotion tickets and asked to carry these around with them for 2 weeks (see Figure 2.8).

Participants were expected to articulate their feelings when interacting with any financial service and/or technology, by recording it using the appropriate rasa. We asked two questions: When? How? For example, “You laughed at money or technology. When? How?” Participants were instructed to answer these two questions every time they experienced one of the nine rasas.

At the end of two weeks, we collected the emotion tickets and analyzed them. The analysis led to the creation of an emotion map (see Figure 2.9) that helped us identify the dominant emotions people felt when interacting with ATMs and hence what innovations would make sense for a new design for these localized ATMs.

The Auto Rickshaw Radio: A New Representation for the Semantic Differential Scale

The semantic differential scale is used for measuring the meaning of things and concepts (Snider & Osgood, 1969). Participants indicate where they feel a product or experience lies on a 7-point scale between polar adjectives (e.g., strong-weak, good-bad). We noticed, over several projects done in India, that the semantic differential scale seemed to be a major stumbling block for participants, especially for semiliterate users. Whenever the semantic differential scale was presented, there was a tendency to select one of the two extreme options and nothing in between. This was very different from the experience these participants had expressed while using the product in question. So, in spite of an experience that was not black and white, everyone seemed to want to pick one of the two extremes.

After much probing and reflection, we realized that for these users, the concept of a difference in degree (moving from negative to positive) being represented by a horizontal straight line did not make sense. The feeling was that if the different points in the scale represented different degrees of an attribute, then they could not appear to be on the same (horizontal) level. Hence, a control was devised that resembled the volume control knob on a radio, with which all users were familiar. Transitioning from a low to high volume on a radio was very similar to what they needed to do when expressing the degree of an attribute (positive or negative) via the semantic differential scale. This alternative, nonlinear representation significantly improved comprehension among the participants. The stark difference in how the participants went from selecting one of the options at the two ends of the scale to wanting to fine-tune to a granular level was very insightful (Figure 2.10).

The Animal Picture Board: Kenya

We conducted research on the adoption and usage of financial institutions in rural Kenya to advise our client about innovative, localized strategies for various banking channels such as the bank branch, the ATM, mobile banking, and the contact center. See Figure 2.11.

We needed people to talk about their deep associations with different financial institutions/mechanisms, whether formal or informal. Since Kenya, like most of Asia, is a hierarchical and collectivist culture (Geert & Jan, 1991), we realized there was a strong likelihood they would have the same inhibitions we observed in Asia, where talking openly with a stranger would be a problem. Even though we would have a local researcher moderating the sessions, the fact that we (an Indian and South African research team) were going to be present would make it difficult for them to articulate what they really wanted to say. So we wondered, “What would resonate with these participants, triggering instant associations and hence instant responses without any ‘interference’ by the analytical part of the brain?” We observed that Kenya had a long and deep connection with wildlife. Wildlife was an area of shared knowledge in Kenyan culture.

We designed a method using images of animals common in Kenya on a board and asking questions around that picture board (see Figure 2.12). We tested this (and other methods) with our two local researchers as the pilot participants. The pilot sessions helped clarify the local association with each animal and tweak some of the pictures of the animals.

In the study, we asked participants about the different kinds of financial mechanisms they used (e.g., a money lender, an informal neighborhood group where they lend money to each other, the formal bank). We then asked which animal they associated with each financial mechanism and why.

The responses came without hesitation. “The cheetah is like this particular service, and the elephant reminds me of this institution.” Each person would make particular associations quickly. Responses reflected deep-seated patterns rather than being manufactured for us.

There were several interesting patterns that emerged from this exercise. For example, the elephant was repeatedly associated with a specific local bank. The reason was that participants felt this bank was strong and reliable like an elephant, and hence, they felt a sense of security and comfort in engaging with this bank.

On the other hand, many participants associated the cheetah with a new mobile transaction service. They felt this service was fast (in terms of time taken to complete a transaction) like the cheetah, but was also “cunning” like the cheetah. The cheetah, while appearing less intimidating than the lion or the leopard, had the speed to attack and kill its prey while seeming to appear out of nowhere. They also stated that the cheetah was always supposed to attack sneakily, from behind. So for the new mobile transaction service, while they greatly appreciated the speed with which they could make payments and receive money from across large distances, there was a hidden side to using this service. This was the gradual realization that they were, as a result of using this mobile transaction service, spending their money faster and on things that perhaps they did not really need. They felt that this was the “cunningness” of this service where and did not realize when they signed up that there was a flip side to the convenience of instant access. This duality made them apprehensive and not totally comfortable with the service.

Bizarre Bazaar: South Africa

This is a variation of an “informance” or informative performance method (i.e., a set of techniques in which actors and/or researchers study what is known about consumers and role-play with potential consumers) that has tested well with Asian users in our 15 years of ecosystem research and usability testing across Asia.

We used this recently in South Africa, when working on a banking project. We needed to understand low-income users’ motivational drives and blocks regarding mobile banking. The bank had introduced mobile banking, but adoption was minimal. We had to figure out why.

We were in the (supposedly) dangerous township of Umlazi near Durban. We knew that just asking the participants why they were not using mobile banking or specific mobile banking features/functions would be a challenge. Once again, since this specific segment was relatively collectivist and was also very uncomfortable being asked questions by “outsiders,” the chances of participants feeling comfortable enough to share the real reasons of their low adoption of mobile banking were slim.

So we decided to use the Bizarre Bazaar. Participants were asked to compare a range of options and specify which ones they would select. When asked directly, participants usually responded by saying they would select all options, even though on further questioning, it became clear that reaction was far from reality! Hence, the Bizarre Bazaar method was designed to make participants experience a situation that is common in their everyday lives: bargaining with a vendor when buying a product or service (in cultures where bargaining during buying/selling is normal practice). This converts the seemingly difficult task of having to compare options and indicate preference into an experience that mimics everyday life and helps to activate involuntary reflexes based on a familiar situation. The method also uses the strong intrinsic block most of us feel to part with cash. This block trumps the participants’ drive to say what the researchers want to hear and surfaces in collective and hierarchical contexts.

In this study, we wrote the name of every mobile banking function available to the participants (via the bank), on small pieces of paper. Then we put these pieces of paper in a basket/bowl used by market vendors to display their wares to customers. We placed this basket/bowl with the pieces of paper in front of the participant and told them to imagine that they were in their neighborhood bazaar and had chanced upon a vendor selling mobile banking features/functions (see Figure 2.13). Which features would they buy? To make the simulation complete, we handed each participant 300 rand in cash (just for the exercise). They would need to make the first move by offering the amount of cash they felt was appropriate for any feature/function they liked and wanted to buy. As the participant offered cash for a feature/function, the research moderator acted like the vendor and asked for a higher price. In the course of the “bargaining,” we were able to probe and understand which features were considered valuable and useful and why.

The cash acted as the anchor for the conversation about what was preventing participants from using mobile banking in general. More specifically, however, it helped them talk about which features they would actually use every day and in what situations they would use them. The insights from this exercise helped us come up with ideas for new concepts and improvements, and also helped us determine which features could be dropped.

Lessons Learned

I have learned several lessons using these fun and innovative techniques:

1. It is possible to be so engrossed in the thrill of creating a “clever” method that one overlooks the “human” angle.
When we were conducting ecosystem research in Kenya, the Bizarre Bazaar method was part of our repertoire. However, as we started the session with our first very low-income participant, we realized that the act of handing over cash as a mere artifact for research when that entire amount could have fed the family for a week would be cruel and heartless on our part. So we decided to change course, even though that meant we could not create a “clever” technique as replacement for the first day’s research.

2. One needs to be very careful when constructing a hypothesis about a culture, based on secondary data on the Internet.
A decade ago, when I was in China for my first ethnographic research study in Shanghai, several of the research methods I had constructed were all based on the assumption that China was a very hierarchical culture. None of the methods, therefore, involved activities with the children in the family (since the assumption was that they do not play any significant role in decision-making). However, when conducting the pilot sessions with our local researchers, it emerged that due to the one-child policy, China was not as hierarchical in 2003 (at least within the family) as it had been in the 1960s, when most of the data regarding cultural dimensions were gathered. Since much of the cultural data on the Internet are based on old research, the data did not always reflect current cultural values.
Even more important was the fact that since there was only one child in each family, the child was actually a very important decision-maker! Based on this insight, we had to change our research methods to include activities with the children. Subsequently, we decided that, in addition to piloting our methods with local researchers, we also needed to include local researchers in our early discussions before actually creating any customized method(s).

3. When using customized methods, include multiple methods that explore the topics/issues being researched.
It is only by analyzing participant response to all the methods that one can come to a reliable conclusion about the insights gathered from the research. Multiple methods help validate the insights of each method.

To Conclude

In today’s globalized world, where product and service ideas surface in one part of the world (e.g., mobile banking) but are expected to be adopted globally, the local voice is more important than ever (Chavan & Prabhu, 2010). Without clarity about the deep motivators and barriers that exist within the local ecosystem, providing the local “soul” to a global idea becomes impossible.

An important question to ask is whether the insights that one gets from the use of these interactive, culturally customized methods actually lead to any advantages over the conventional research methods. Research insights obviously depend upon the skill and experience of the research team. No amount of customized methods can substitute for skill and experience! However, what has worked distinctly better about these methods than the conventional methods are the following:

1. Reduction in the time needed to get insights
The time it takes to get participants to the level of open sharing using the right triggers in a customized technique versus a more conventional question and answer approach is usually 30-40% shorter.

2. Qualitatively better insights
The quality of insights is deeper with these customized methods. For example, the duality represented by the “cheetah perspective” with the mobile payment was not something that any of the previous qualitative research conducted by the client had unearthed. Would we have been able to uncover this without having used the animal picture board? It is not likely.

3. Actionable insights
As with any insight, regardless of the method, if it cannot lead to any specific action, it is of little use. Once again, in the “cheetah” example, we were able to ask detailed probing questions to understand what problems speed solved, why it was important, and how it made them feel. Similarly, the discomfort with the cheetah’s cunning unlocked a veritable Pandora’s box of issues around perceived disempowerment and discriminatory treatment by foreign banks and the resulting distrust. The insights obtained from the use of the animal picture board led to conceptualizing specific features (ranging from incremental to disruptive innovation) for the client’s offerings to this segment of users.
And last, but not least, it is a great advantage to use these methods because the participants enjoy them as much as the interviewer. It becomes a fun hour or two they spend together!