CHAPTER 6

,

BEST PRACTICES FOR ADS IN SEARCH RESULTS

COAUTHORED WITH FRANK GUO

Don't Kill Your Golden Goose PAGE 86

Integrate Ad Displays with the Rest of Your Site PAGE 86

Make Sure Customers Can Easily Distinguish Ads from Content PAGE 88

Keep Ads Relevant and Appropriate PAGE 94

Understand How Your Customers Interact with Ads PAGE 95

Understand What Makes a Good Ad PAGE 97

Limit Cannibalization PAGE 100

Provide Ads for Internal Merchandise Instead of Third-Party Advertising PAGE 102

Pay Special Attention to Ads if There Are No Search Results PAGE 103

In Conclusion PAGE 106

Eyetracking Tips and Tricks, by Frank Guo PAGE 107

Conflicting demands have left UX professionals concluding online advertising is a necessary evil. Customers frequently go out of their way to say they detest ads, whereas marketers always seem to try their hardest to stuff as many of them as they can on every page of a Web site, including the search results page. This leaves many UX design professionals stuck trying to balance the ad equation—and frequently failing to fully satisfy either customers or marketers. This chapter describes real-world strategies for successfully integrating ads into your search results.

DON'T KILL YOUR GOLDEN GOOSE

Online advertising is increasingly seen as a legitimate way to boost revenues on an ecommerce site—in part, because of the currently rocky economy. Every penny counts, and every stream of potential revenue demands careful consideration. However, research and experience show that, for an online business to succeed and thrive, it is important to balance any temptation to boost short-term revenues through advertising with the long-term goals of boosting UX, customer loyalty, and brand attributes. If you are not careful, trying to squeeze out a few more eggs can kill your golden goose.

However, following a few simple guidelines—while listening carefully to your customers—can help your business make money with ads, without compromising either long-term customer loyalty or your brand image.

INTEGRATE AD DISPLAYS WITH THE REST OF YOUR SITE

In recent years, Google has emerged as the Web's leading supplier of ads. Indeed, Google AdSense makes it easy—and often profitable—for you to sign up and host ads on your Web site. Most of the time, the ad content is fairly well targeted, so many retailers, bloggers, and developers of social networking sites have taken advantage of the opportunity to boost revenue through this service. Unfortunately, few have made the effort to customize the boxed ads to make them look like the rest of their site. Figure 6.1 shows the social networking site Fishing.net, which carries Google ads just as they come out-of-the-box, without any customization.

FIGURE 6.1 On Fishing.net, Google ads have no customization

Neglecting to customize third-party ads is, of course, easier, but there are consequences to this approach:

• Customers frequently perceive your entire site as cluttered and disorganized.
• Customers mostly ignore the ads because they look different from the rest of your content.

This is a situation that negatively impacts both the marketers and your customers. Your customers lose out because their experience of your site is degraded. Your marketers lose out because customers click their ads less often—if at all.

Experience shows that a small amount of visual design and programming effort that makes your ads look like the rest of your site can yield tremendously positive responses from your customers. They stop seeing ads as clutter and instead perceive them as content. Google, of course, provides a superb example of this strategy in practice, as shown in Figure 6.2.

FIGURE 6.2 Google sets the standard for ad placement and integration

Google has carefully crafted its customer experience, paying strict attention to everything from page balance and spacing to tweaking even the smallest visual design elements. On January 26^th, 2007, Search Engine Land magazine interviewed Marissa Mayer, Google's VP of Search Products & User Experience, on the subject of Google ads. In this interview, Marissa Mayer, described how replacing the box that used to contain ads on the right side of search results pages with a vertical line separator improved ad traffic because of the ads' closer integration with the content.

MAKE SURE CUSTOMERS CAN EASILY DISTINGUISH ADS FROM CONTENT

When taken to the extreme, the guideline to better integrate ads and content becomes a design antipattern. Customers' inability to distinguish ads from content becomes especially painful and disruptive when a Web site carries a large number of ads. For example, on Fishing.com, as pictured in Figure 6.3, there are so many closely integrated ads that they become virtually indistinguishable from the content—to the extent that it becomes difficult to understand what service the site actually provides.

FIGURE 6.3 Ads on Fishing.com are overwhelming and confusing

However, even if the number of ads on your site is not overwhelming, customers can have difficulty distinguishing them from content. One common problem is providing different types of search results—some of which stay on the same site, whereas others take customers elsewhere. (Surprising customers about where links go is never a problem for Google or other Web search engines because both their ads and their search results take searchers to other sites.)

On the other hand, for destination sites that sell their own merchandise or provide a branded service while also hosting third-party ads, it is important to positively differentiate between ads, legitimate content, and featured—that is, paid—results. Autotrader.com pictured in Figure 6.4, mixes many different types of search results together, so it's hard to tell what clicking each result might do. Orange results are actual, “organic” search results, whereas the slightly padded results in blue are paid advertisements.

FIGURE 6.4 Autotrader.com mixes many different types of search results

When clicking a search results link takes searchers somewhere surprising or, by mistake, they click a featured result, their reaction is often quite negative. Yet, to the extent that a link satisfies a customer's need, clicking it can be a positive experience—finding exactly what the customer was looking for. The key to solving this problem is grouping the different kinds of content, while making it clear what are paid advertisements or featured results and clearly differentiating paid content from organic search results through subtle, but telling visual cues.

Colin Ware, in his book Information Visualization: Perception for Design describes so-called preattentive attributes, which involve the early stage of visual perception that occurs mostly below the level of conscious thought, at a high speed. He distinguishes four categories of preattentive attributes: form, color, spatial position, and motion. You can apply the grouping strategies for all four categories of these attributes to ads, typically using line size, shape, hue, and enclosure to subtly differentiate ads from content. Motion applies mainly to animated ads, which can be an appropriate differentiation strategy, depending on their content, as we discuss later in this chapter.

Google's search results provide a great example of subtle preattentive differentiation. As shown in Figure 6.2, Google displays three kinds of results on search results pages:

• Ads at the top—Their subtle, yellow background hue differentiates them from the organic results.
• Organic search results in the middle—These are the actual content.
• Ads in the column on the right—Their placement and narrow column format differentiate them from the organic results, plus a vertical line sets them off.

Overall, the results on this page fit together and flow well, whereas the ads' formats subtly, but unequivocally differentiate them from the content.

At some point, a search results page simply reaches its point of saturation, and it becomes impossible for customers to tell the different types of search results apart. When doing usability studies for a major retailer that provided ten different types of search results, Greg found that most test participants could not distinguish one type of result from another when scrolling through search results pages. Thus, the experience became overwhelming for people using the site, who were often frustrated, because they never knew where they were about to go when they clicked a search result.

Buy.com is another site oversaturated with different types of search results and ad content. As called out in Figure 6.5, Buy.com hosts at least 13 different formats for third-party ads on a single page! No surprise; the site rating service Internet Retailer commented, “The site all but overwhelms its visitors with information and options.”

For your search results pages to be effective, they must display only a limited number of different types of content and ad formats. Otherwise, with no clear guidance, your main content gets lost and customers become confused because they are overwhelmed by the number of things competing for their attention. The key to integrating ads into your search results, without destroying the search experience, involves being clear about what generates the most site revenue. Is it the ads? Or the content? Does it depend on the query? Have you made the costly mistake of serving as many ads to your top customers as you do to window-shoppers and people who just happen to drop by? After you answer these key questions, you must remain disciplined and stay focused on your core earning potential. Though you can provide occasional, helpful third-party content on the side—particularly, if it helps a customer make a decision.

For example, can you tell whether Buy.com makes more money if people buy the headphones shown on the page—or instead investigate how to make their sandwiches moist and juicy with Best Foods Mayonnaise? Or maybe Buy.com really rolls in the dough when people buy its headphones from eBay or Dell instead? From its page content and layout, it is impossible to tell. You get the impression that Buy.com may be somewhat confused about where the bulk of its revenue comes from.

FIGURE 6.5 Buy.com's 13 different types of ads and third-party content are overwhelming

KEEP ADS RELEVANT AND APPROPRIATE

Although everyone, no doubt, has their own list of the annoying advertisements, one particularly educational example is the infamous animated pop-up ads featuring the X10 spy cam that became popular in the early 2000s. The ad was, at once, so annoying and so ubiquitous that X10 bears the dubious distinction of having been one of the first companies to get people to register on the company Web site just to opt out from seeing its ads for 30 days. Figure 6.6 shows one version of the X10 ad.

FIGURE 6.6 The infamous X10 pop-up ads were some of the most annoying ever

Even though most ads may not be entirely obnoxious, they can nevertheless be completely inappropriate. Ad placement is often the key. FoxNews.com, which is pictured in Figure 6.7, shows just how insensitive and inappropriate some ads can be.

The news story on the page describes the accidental death of a Marine, yet ads for yellow teeth litter the page. Putting politics aside, how do you think the dead person's family felt when viewing this story? How about all the other families who have fathers, husbands, sons, or daughters serving overseas? Please, make sure your ads are appropriate to the content on a page. One way of doing that is to conduct user research and learn how your customers react to particular ads.

FIGURE 6.7 The insensitivity of inappropriate ads on FoxNews.com is striking

UNDERSTAND HOW YOUR CUSTOMERS INTERACT WITH ADS

Although usability studies and field research can give you important clues about how people interact with ads, eyetracking research can be especially helpful. Eyetracking studies help you to understand how customers perceive and interact with ads, and then depending on your objectives, design ads intelligently—to either catch or not catch their attention.

Thus, it's hard to ask them about whether they paid attention to ads because they probably won't be 100% sure. Eyetracking can fill this gap in your knowledge.

By meticulously tracking all eye movement during a test, you can tell whether and how a participant pays attention to the ads on a Web page, what visual search pattern he uses, and how he either skips or focuses on particular information on the page. When running eyetracking studies, spontaneity is key. If you interrupt a participant, ask questions, or have a broken prototype that doesn't let a participant interact with it naturally, your eyetracking data gets contaminated with all sorts of noise. For more information on eyetracking, see Frank Guo's perspective at the end of this chapter.

So, what does eyetracking methodology tell you about ads? You can observe the well-documented banner blindness—that is, customers typically ignore static banners, which are often located at the top of a Web page—even if they are of a ridiculously large size such as those on the Tutorialized.com frame that “captured” another site, HTMLGoodies.com, as shown in Figure 6.8. Though research clearly shows people usually ignore banner ads, the banner ads on the Tutorialized.com frame combined with the ads on HTMLGoodies.com site take up almost the entire area of the page above the fold, negatively impacting the UX and forcing customers to scroll to see any of the actual content.

FIGURE 6.8 Banner ads on Tutorialized.com take up the entire window

As Jakob Nielsen wrote in his August 20, 2007 Alertbox, “The most prominent result from the new eyetracking studies is not actually new. We simply confirmed for the umpteenth time that banner blindness is real.”

UNDERSTAND WHAT MAKES A GOOD AD

Based on the findings of eyetracking studies, users spontaneously pay attention to things that are

• Concrete
• Actionable
• Not like marketing

These findings apply to both typical search results and third-party ads. Thus, a good ad is concrete and to the point rather than full of generic, marketing jargon. For example, it might present a picture of the actual item being advertised or indicate clearly that there's free shipping rather than saying something like “We have thousands of cheap items in stock!” As shown in Figure 6.9, the ads on Cars.com are actually quite good.

FIGURE 6.9 Good ads with appropriate content on Cars.com

This ad on Cars.com follows an important principle: It is highly relevant to the search results on the page, calling customers' attention to the specific item in the search results. Even though the ad at the top of the page is a banner ad, it doesn't have the drawbacks of typical banner ads—such as presenting irrelevant content or being positioned too far away from the main content. The price quote, showing good value, is prominent and provides a good call to attention. The presentation of the ad is straightforward, with concrete information and little marketing language or visual noise.

Is it enough that an ad's content be appropriate? Although content is important, it is just one aspect of the marketing message. Another important principle of ad appropriateness is keeping an ad's style appropriate to the topic. Contrast the style of the Cars.com ad to the ad on the Yahoo! Movies page, pictured in Figure 6.10. Even without seeing the flashy animation, it is clear that the ad is quite vibrant.

FIGURE 6.10 An appropriate ad style on Yahoo! Movies is eye-catching

An ad with this animated style is highly appropriate if you expect customers to be getting ready for an action-adventure movie experience complete with special effects. In this case, tasteful, yet somewhat loud movie ads, providing links to previews, are not only appropriate, but also expected. This example shows how carefully chosen ads can form a large part of a Web page's useful content, making it unnecessary for Yahoo! to work so hard at keeping the page exciting.

For hosting ads, it definitely pays to get creative. The way an ad is presented matters a great deal in how customers perceive both the ad and the site hosting it. Creative, interesting presentations get extra points, especially if they are unobtrusive. For example, the peel-the-corner ad on EddieBauer.com, shown in Figure 6.11, is a great example of an ad that draws attention and invites participation, without obnoxiously dominating an experience. Peel corners work because they are unusual, visually interesting, and invite interaction; yet require very little space and do not draw undue attention in the same way loud animated banner ads do.

FIGURE 6.11 Getting creative with an interactive, peel-the-corner ad

Anecdotal evidence suggests that most people do not mind interacting with this kind of advertising and consider peel-the-corner style ads acceptable or even positive on a wide range of sites—if the ads contain appropriate content.

What kind of customer action should an ad invite? In Figure 6.9, the ad on Cars.com asks customers to purchase or consume a specific item on the same Web site—for example, purchase a Maxima. The ad in Figure 6.10 invites customers to participate in an offline interaction they can perform after viewing the ad—see a movie after reading a movie review and referring to a theater schedule on Yahoo! Movies. These are both examples of passive ads—that is, they do not invite customers to click, so much as capture eyeballs, with the purpose of instilling a subliminal desire to select a specific item among many that a site offers. This mirrors a classic bricks-and-mortar advertising strategy, in which certain companies pay stores to feature their brands on banners and place their products in prominent locations.

However, the Web is a dynamic medium, leading to all sorts of novel advertising models. One advertising model involves carrying your competitors' ads and advertising similar products for sale on a different Web site or in another store. On the one hand, customers' clicking these ads generates marketing revenue. On the other hand, this marketing revenue literally eats into the sales revenue from products and services your Web site provides. Thus, competitors' ads cause cannibalization.

LIMIT CANNIBALIZATION

Generally, carrying competitors' ads is a dangerous and losing proposition because cannibalization involves many hidden costs that are often hard to quantify. Most companies spend a lot of money on advertising to bring customers to their own Web sites.

In addition to cannibalization threatening your bottom line, the experience of your customers' clicking your competitors' ads is likely to be disconcerting to them, leaving them less satisfied with your site as a result. Most competitors' ads just dump customers on the home page of their site or, at best, on a search results page, leaving them to figure out how to navigate to the one item they want—for instance, to compare a competitor's price for an item they were looking at on your site. Thus, clicking an ad almost always involves extra work for your customers. For this reason, if your customers get the results they want on your site, most do not click a competitor's ad—unless the ad makes an attractive offer or the competitor's brand is stronger.

Some sites understand their customers' reluctance to do extra work and consciously decide to exploit this fact by hosting competitors' ads—boosting their revenue without much cannibalization of their own sales. The key to doing this successfully is making rational judgments regarding which ads to host, based on your overall site revenue strategy and hard site traffic statistics. Some successful strategies for carrying competitors' ads include the following:

• Making competitors' ads look like advertising—Customers deliberately ignore such ads, and they get few clicks. This strategy works great if you get paid for ad impressions, not by the number of clicks.
• Carrying only ads for weaker brands—This limits the number of customers who click away from your site.
• Not allowing ads to display competitors' prices—Displaying the actual prices for specific competing goods or services encourages customers to click ads.

The competitor ad on Yahoo! Cars, shown in Figure 6.12, provides a good example for all three of these strategies.

FIGURE 6.12 Competitor advertising on Yahoo! Cars

Another interesting strategy involves completely embracing the experience of offering the best marketplace price from whatever source and being fully committed to always carrying ads from various competitors. You can see one example of this strategy on Buy.com, which is shown in Figure 6.5 above. Buy.com shows prices from eBay and Dell, and the price for the product from its internal search engine. This strategy works well in any of the following cases:

• Your site has the best price.
• Differences in price are not significant.
• Competitors' prices do not give a complete picture—for example, items for sale are used or have high shipping costs, but their ads do not communicate these details.

PROVIDE ADS FOR INTERNAL MERCHANDISE INSTEAD OF THIRD-PARTY ADVERTISING

Amazon.com uses an interesting variation of this strategy, carrying the same items from both its store and online marketplace, as pictured in Figure 6.13.

FIGURE 6.13 Ad for the Amazon.com marketplace

This is a scenario that allows for both Amazon.com and its customers to win. Amazon.com collects rich fees from sellers in its marketplace, so it makes money regardless of which items customers purchase. Having a marketplace also benefits consumers by providing multiple, highly relevant shopping options without presenting anything that looks or feels like an ad. This strategy is tricky because Amazon.com is essentially competing with its own sellers. However, despite the challenges, this strategy has been quite successful—in large part because Amazon.com has been proactive, constantly adjusting its price points and the inventory of items it carries to keep its marketplace thriving and provide customers with the comfort of knowing they are getting good prices from a brand they trust.

Amazon.com's strategy reflects a more general principle: Whenever possible, providing and upselling merchandise to customers is almost always better than hosting third-party ads. There are many good reasons to host your own merchandising rather than third-party ads, including the following:

• Control—No matter how carefully you screen third-party ads, undesirable content sometimes slips through, harming your brand and sending your hard-earned customers to your competition.
• Visual design—Despite your best efforts at ad customization, third-party ads look like, well, ads. On the other hand, you can fully integrate merchandising into your site, making the best use of your available space.
• Specificity—Third-party ads are often not specific to a site's own products—nor would you want them to be because that would drive your customers to the competition. On the other hand, you can make merchandising as specific as necessary because in this case all ads ultimately drive traffic back to your own site, where transactions convert directly to your bottom line.

Merchandising simply offers a better UX. People like specific, targeted content that does not look like an ad but helpfully provides ideas and choices.

PAY SPECIAL ATTENTION TO ADS IF THERE ARE NO SEARCH RESULTS

All the negatives of displaying competitors' ads discussed earlier become further exacerbated on a no search results page. Such a page is generally devoid of your own content, making third-party ads their primary calls to action.

As described in Chapter 1, “Starting from Zero: Winning Strategies for No Search Results Pages,” no search results pages indicate to customers they've over-constrained their query and need to reformulate or broaden their search. Displaying third-party ads on a no search results page invites customers to abandon their natural behavior of iterative search in favor of following a fresh information scent. This interrupts the finding flow on your site and dumps your customers directly into your competitors' greedy hands.

Unfortunately, marketing folks often simply fail to grasp the iterative nature of the finding process and the critical role the no search results page plays in helping customers reformulate their queries. Some of them like to put ads on no search results pages, justifying their viewpoint by saying something like this: “We are simply meeting the customers' needs with our ads. The customers would leave our site anyway. We are just helping them and making some money along the way.”

This thinking demonstrates a dangerous and fundamental lack of understanding of the differences between how search engines and ad-hosting services work.

Now contrast search engine results with hosted advertising results. Search engines providing hosted ads have no requirements for fidelity or specificity—the rules by which a site's own search engine usually abides. By using an OR operator instead of AND, hosted ad servers can cherry-pick ads, displaying only the most relevant ads or those matching the most profitable keywords in a customer's query, and ensure the ad server never fails to produce results.

For example, in one particularly memorable usability study Greg conducted, a task asked participants to find a ticket to a performance of the “Nutcracker” in San Francisco, during the holidays. After reading the task, one participant typed the query “Nutcracker Ballet in San Francisco December 1-31st,” which, of course, matched no results on the site. However, the ad hosting engine produced a page similar to that shown in Figure 6.14.

FIGURE 6.14 Ads matching the query Nutcracker Ballet in San Francisco December 1–31st

Upon seeing the no search results page displaying these ads, the participant promptly clicked an ad and bought a ticket from a competing site, even though cheaper tickets were readily available on the site we were testing. If the participant had relaxed her query, she would have discovered this, but instead, the third-party advertising appearing on the no search results page led her down a different path. When there are no search results, that is the wrong place and time for displaying third-party ads.

As this story demonstrates, it is easy to lose customers on no search results pages, which represent a critical juncture in a conversation between a customer and a Web site—an opportunity for the customer to reformulate a query and gain a deeper connection with your brand. Host third-party ads on your no search results pages only as a last resort. Carefully calculate the cost of doing so. Which click would generate more long-term revenue: the ad or the tools for query reformulation? Which click would ultimately engender more long-term loyalty and a better relationship with your customers?

IN CONCLUSION

In his keynote speech at the 2008 Business Of Software conference, Seth Godin famously quipped, “Marketing is too important to be left to the marketing department.” The same goes for hosting advertising in your search results pages. UX professionals should be actively involved in ad-hosting decisions to make sure your company's golden goose continues to thrive, well beyond the current quarter.

Use these ideas as a starting point for developing your own comprehensive merchandising and ad-hosting strategies. Making the tough choices that are necessary and getting creative with ad hosting is not easy, but it is an absolute must if your company is to survive and thrive in the current economic environment.

EYETRACKING TIPS AND TRICKS

Eyetracking is an increasingly popular research technique used by web design professionals to see how users interact with designs. It is traditionally used as a somewhat more “scientific” user research method than other research techniques. And therefore many practitioners ignore the “soft” or qualitative way of applying eyetracking. I'd like to offer a qualitative perspective of effective eyetracking research. To begin with, eyetracking is a very valuable technique to help us understand users' visual attention, answering questions such as what kind of content and visual treatment users attend to, the steps they take to find an object or explore the content, how long they stay focused on an object, and how effective their visual search is, and so on. Web design areas such as search, browsing, online ads, and Web content writing can greatly benefit from eyetracking research because in these contexts a big part of the design is to draw user attention and keep them engaged. Conventional usability testing techniques fall short of addressing this design objective. This is because during a conventional usability study, we understand user behavior by observing their mouse and keyboard movement and asking them to talk aloud as they go along. But oftentimes users interact with Web pages by just looking at the screen; therefore we can't tell what they are doing without the aid of an eyetracker. Also, it is very difficult for the user to accurately recall his or her own visual behavior, which consists of many random, unconscious spurts of eye movement, so the think-aloud protocol can't give us in-depth insight about the user's visual behavior. Eyetracking, in contrast, provides an ideal venue for understanding users' visual behavior.

As of today in the field of human computer interaction, a widely accepted approach of conducting eyetracking research is quantitative in nature. By quantitative, I mean, researchers look at eyetracking data at an aggregate level, typically in the form of a heat map and derive conclusions about where users pay attention to on a screen. I guess many of us in the human computer interaction field have heard of this type of eyetracking research, and some are great fans of it because it is perceived as cool and very “scientific.” The problem with this approach is, eyetracking data is notoriously hard to interpret. For instance, users fixate on an area for a long period of time and therefore the heat map would look very “hot.” This could mean that the area is well designed and attracts lots of user attention, or it could mean that it is ill designed and generates lots of user confusion. So a heat map in itself can't tell the entire story.

Based on my experience with conducting eyetracking research, I'd recommend an alternative approach of conducting eyetracking research, the qualitative way. To put it briefly, this means we observe users' eye movement on an individual basis and then cross reference this with other types of user data, such as user feedback collected by think-aloud protocol to produce a qualitative and in-depth understanding of user behavior. For instance, from one of the studies I conducted, the eye movement of a user suggested that her eyes traveled long distances, jumping from the top of the screen to the bottom and ignored all the page content in between. To understand what drove this kind of behavior, it was not just enough to look at the eyetracking data alone. From her verbal comments, I learned that she explored this Web page based on such an approach—she tried to calibrate how much content was there on the page before diving into any particular details; therefore she looked at the top of the screen and then immediately jumped to the bottom. In so doing she got a sense of what was there on the screen. That feedback, as combined with the eyetracking data, gave me a clear understanding of what she was trying to do when exposed to such a page the first time.

The same visual search pattern also emerged from other participants in the study. Here, the eyetracking data did not come from a large sample size and did not present itself as a heat map, which is only meaningful if the data comes from at least 20 plus participants. On the other hand, by synthesizing eyetracking and think-aloud protocol data across just 10 participants, I had a rather robust understanding of the typical visual search pattern for this type of Web design. This approach, by looking into the eye movement and comments of individual users on a case-by-case basis, allows us to tell a complete story of why users do what they do, as opposed to using quantitative techniques such as heat map, which doesn't provide this kind of in-depth understanding. The advantage of this approach relative to the quantitative approach also resides in the fact that to analyze eyetracking data quantitatively requires a very clean research design, some sort of experimental control, a very robust eye tracker hardware setup, and a large sample size, and only eyetracking experts with careful planning can pull off a quantitative eyetracking research study successfully.

On the other hand, the qualitative approach uses eyetracking as just another supplemental technique to be used on top of a conventional user evaluation study; therefore, it is more practical and cost effective. Of course, both methods have their respective areas of application. To generate clean and robust quantitative data in order to support executive decision making, for example, almost always requires a quantitative research approach. On the other hand, to generate in-depth learning for designers to uncover the secrets of effective Web design, qualitative eyetracking could provide richer insights.

Finally, let's talk more about the validity of qualitative analysis in the context of eyetracking. We typically think of eyetracking as a quantitative, scientific method of understanding human behavior—it captures eye movement in an accurate and quantifiable manner. And therefore we might be uncomfortable with hearing “eyetracking” in the same sentence as “qualitative” because this might suggest that the research is less valid. But let's think about a conventional user evaluation study, in which we interview a number of participants around their thoughts about some Web designs, and synthesize their feedback into a meaningful and coherent account of how well the design performs and how to improve the design. All of that is also qualitative in nature. The qualitative application of eyetracking is similar to that—we closely examine users' eye movement on an individual basis and perform synthesis across a number of users to get in-depth insights and uncover patterns. Therefore the qualitative approach of eyetracking research is informative and robust, just like the traditional qualitative user evaluation method.

—FRANK GUO