Compelling reasons exist to keep reputations hidden from users. In fact, in some circumstances, you may want to obscure the fact that you’re tracking them at all. It may sound rather Machiavellian, but the truth of the matter is this: a community under public scrutiny behaves differently (and, in many ways, less honestly) than one in blissful ignorance.

Several trade-offs are involved. Displaying reputations takes up significant page real estate, requires user interface design and testing, and can compete with your content for the user’s attention and understanding. Quickly, show Digg.com (Figure 7-1) to 10 of your friends and ask them, “What kind of site is this? News? Entertainment? Community?” Odds are good that at least a few of them will answer: “This appears to be some sort of contest.”

Figure 7-1. Digg’s site design puts overt reputation scores front and center.

The impression that Digg makes is not a bad thing; it just demonstrates that Digg made a conscious decision to display content reputation prominently. In fact, the display of reputation is the central interaction mechanism on the site. It’s practically impossible to interact with Digg, or get any use out of it, without some understanding of how community voting affects the selection and display of popular items on the site. (Digg is perhaps the most well-known example of a site that employs the Vote-to-Promote pattern. See Chapter 6.)

Juxtapose Digg’s approach with that of Flickr. The popular photo-sharing and discovery service also makes use of reputation to surface quality content, but it does not display explicit reputations, rather it prominently displays items that achieve a certain reputation and that can be browsed (daily, weekly, or monthly) in the “Explore” gallery (at http://www.flickr.com/explore); see Figure 7-2. The result is a very consistent and impressive display of high-quality photos with very little indication of how those photos were selected.

Figure 7-2. Flickr’s “Explore” gallery is also based on reputation, but you never see a score associated with a photo.

Flickr’s interestingness algorithm determines which photos make it into the “Explore” gallery and which don’t. The same algorithm lets users sort their own photos by interestingness.

Digg and Flickr represent two very different approaches to reputation display, but the results are very much the same. Theoretically, you can always glance at the front page of Digg or Flickr’s “Explore” gallery to see where the good stuff is—what people are watching, commenting on, or interacting with the most on the site.

How do you decide whether to display reputations on your site? And how prominently? Generally, follow the rule of least disclosure: do not display a reputation that doesn’t add specific value to the objects being evaluated.

Likewise, don’t bother asking users for reputation input (see Chapter 6) that you’ll never use; you’ll confuse users and encourage undesired patterns of “invented significance,” including abuse.

Irrelevant reputations are meaningless and consume valuable resources. If you don’t have a relevant use for a reputation, beware of sticking yourself later with the tough choice of either awkwardly removing a failed feature or having to support it as a costly legacy element.

Are you tracking a reputation primarily to keep users informed about how well they or their creations are performing in the community? Consider displaying that reputation only to its owner, as a personal communication between site and user.

Personal reputations are used extensively for applications such as social bookmarking, lists of favorites, training recommendation systems, sorting and filtering news feeds, providing content quality and feedback, fine-grained experience point tracking, and other performance metrics. Most of the same user interface patterns for displaying public reputation apply to personal ones, too, but take care to ensure that each user knows when her reputations will and will not be displayed to others.

Google Analytics (see Figure 7-3) is an example of rich personal reputation information. It provides detailed information about the performance of your website, across a known range of score types, and it is available only to you, the site owner (or others to whom you grant access). While that information is invaluable to you in gauging the response of a community (in this case, the entire Web) to your content, exposing it to everyone would offer very little practical benefit. In fact, it would be a horrible idea.

Figure 7-3. Google’s Analytics interface shows information that is clearly best kept between you and Google. It’s personal.

Some reputation display patterns provide both a personal and a public representation. In the named-levels display pattern Named levels the personal representation of the reputation score often is numeric, showing the exact score, whereas the public representation obscures exactly where in the level the target’s score actually is. Online games usually report only the level to other users and the exact experience points to the player.

When the whole community would benefit from knowing the reputations of either people or content, consider displaying public reputations. Public reputations may be displayed to everyone in the community or only to users who are members of a group, are connected through a social network, or have achieved status as senior, trusted members of the community by surpassing some reputation threshold.

When is it a good idea to display public reputations? Remember our original definition: reputation is information used to make a value judgment about a person or an object in a given context for a specific time. Consider the following questions:

Public reputations are used for hundreds of purposes on the Web: to compare items in a list on the basis of community member feedback, evaluate particular targets for online transaction trustworthiness, filter and display top-rated message board posts, rank the best local Indonesian restaurants, show today’s gallery of the most interesting photos, to display leaderboards of the top-scoring reputation targets, and much more.

Over time, public reputations can evolve to represent your community’s understanding of its own zeitgeist. And there’s the rub: depending on how you use public reputation, you can alienate users who aren’t part of the in crowd. For example, Yelp is all about public ratings and reviews of local restaurants, but it isn’t used extensively by people over 50. Most of the reviews are written by twentysomethings (most “Yelpers” are between the ages of 26 and 35) who seem to be mostly interested in a restaurant’s potential as a dating hangout.

Almost every website with a large volume of user-generated content is using hidden reputation scores internally—as a means of tracking exactly who is saying what about a content item or another user:

And internally used reputation scores need not always be acted on immediately by scripts or bots; they can also be a very helpful tool for human decision making. Community managers often use corporate reputation reports on the most active, connected, and highest-quality user contributions and creators. They might use the information to generate high-quality best-of galleries to promote a site, or they might invite top contributors to participate in early testing of new designs, products, or features. Finally, user actions often are aggregated into reputations for behavioral targeting of advertising, customer care planning and budgeting, product feature needs assessment, and even legal compliance.

At its simplest, filtering consists of sorting by one or more reputation dimensions and looking only at the first or last entries in the list to identify the highest and lowest scoring entities for further, even automatic, action. In reality, many reputations used for filtering are often made of more numerous and complex inputs than reputations built for public display in rankings or sorted lists.

Consider Flickr’s interestingness filter reputation: it is corporate (used internally and not displayed to any user), it is complex (made up of many inputs: views, favorites, comments, and more), and it is used to automatically and continuously generate a public gallery. But the score is never displayed to users; you cannot query a photo to get its interestingness score. Perhaps the easiest way to think about a filter reputation is that, if it is not ever displayed to users, they don’t have to understand what it’s made up of. If users can see a reputation indicator, they’ll want to know what it means and how it’s calculated.

The reputation usage patterns related to filtering are: user threshold, public gallery, guided learning, recommendations, bookmarks/favorites, similar items, content by author karma, and friends filtering.

By far the most common displays of reputation are in the form of explicit lists of reputable entities, such as the restaurants in the local neighborhood with the highest average overall rating, or the list of players with the highest Elo ranks for chess or even which keyword search marketing terms are generating the most clicks per dollar spent.

Typically, the reputation score is used alone or in conjunction with other metadata filters, such as geographic location, to make it easy for users to sort between multiple entities at a glance. For example, to list top-rated hotels in a five mile radius of a zip code, one would combine the distance and reputation into a rank-score before displaying the list.

The primary purpose of allowing such sorting is to enable users to select an item to examine in more detail. Note that the reputation score need not be displayed to allow sorting or ranking entities. For example, to avoid encouraging abuse, public search engines typically hide search ranking scores.

Recommender systems use reputation to make suggestions about similarities between user tastes (“People who like the same things as you do you also like…”) and discover taste similarities between items (“People who liked this item also like…”). They use reputation in the form of confidence scores and typically display multiple entities in rank order when making recommendations. When the user selects a suggested item, that selection itself is also entered in the reputation system to further improve the quality of future results.

The specific reputation usage patterns related to ranking and sorting are quality-sort search results, leaderboards, related items, recommendations, search relevance (such as Google’s PageRank), corporate community health metrics, and advertising performance metrics.

This entire class of use patterns often is overlooked because it typically happens behind the scenes, out of users’ sight. Though you may not be aware of it, more hidden decisions are made on the basis of reputation than are actually reflected directly to users, either with filtering or ranking.

Billions of email messages are processed daily across the world. ISPs secretly track the IP addresses of the senders; they use this reputation to decide whether the item should be dropped, put in a bulk folder, or sent on to another content-based reputation check before being delivered to your inbox. This is only one example of many patterns used by Web 2.0 site operators around the world to manage user-generated content without exposing the scores or the methods for their calculations. When used for abuse mitigation, the value of the reputation score can be directly correlated with cost savings from increased efficiency in customer care and community management, as well as in hardware and other operational costs. Each year, the IP reputation system for Yahoo! Mail saves tens of millions of dollars in real costs for servers, storage, and overhead.

When a reputation score is complex, such as karma (see the next section), it may be suitable for public display as a standalone score so that others can make specific, context-sensitive decisions. eBay’s feedback and other reputation scores are a good example of a publicly shared karma. Since the transactions for items are often one of a kind, content filtering and ranking don’t provide enough information for anyone to make a decision about whether to trust the seller or buyer.

Of course, some reputation is nonnumeric and can’t be ranked at all—for example, comments, reviews, video responses, and personal metadata associated with source users who evaluate your entities. These forms of input must be displayed so that users can interpret the input directly. For instance, a 20-year-old single woman in Los Angeles who is looking for a new sweater might want to discount the ratings given by a 50-year-old married man living in Alaska. Nonnumeric reputation often provides just enough additional context for people to make more informed judgments about entities.

Here are the specific reputation usage patterns related to decisions: critical threshold, automatic rejection, flag for moderation, flag for promotion, and reviews and comments.

Content reputation scores may be simple or complex. The simpler the score is—that is, the more it directly reflects the opinions or values of users—the more ways you can consider using and presenting it. You can use them for filters, sorting, ranking, and in many kinds of corporate and personalization applications. On most sites, content reputation does the heavy lifting of helping you to find the best and worst items for appropriate attention.

Content reputation is about things—typically inanimate objects without emotions or the ability to directly respond in any way to its reputation.

But karma represents the reputation of users, and users are people. They are alive, they have feelings, and they are the engine that powers your site. Karma is significantly more personal and therefore sensitive and meaningful. If a manufacturer gets a single bad product review on a website, it probably won’t even notice. But if a user gets a bad rating from a friend—or feels slighted or alienated by the way your karma system works—she might abandon an identity that has become valuable to your business. Worse yet, she might abandon your site altogether and take her content with her. (Worst of all, she might take others with her.)

Take extreme care in creating a karma system. User reputation on the Web has undergone many experiments, and the primary lesson from that research is that karma should be a complex reputation and it should be displayed rarely.

A normalized score ranges from 0.0 to 1.0 and represents a reputation that can be compared to other reputations no matter what forms were used for input. When displaying normalized scores to users, convert them to percentages (multiply by 100.0), the numeric form most widely understood around the world. From here on, we assume this transformation when we discuss display of a percentage or normalized score to users.

The percentage may be displayed as a whole number or with fixed decimal places, depending on the statistical significance of your reputation and user interface and layout considerations. Remember to include the percent symbol (%) to avoid confusion with the display of either points or numbered levels.

Things to consider before displaying percentages:

Figure 7-4. Content example: normalized percentages with summary count.

Figure 7-5. Karma example: percentage bars with named levels.

Points are a specific example of an accumulator reputation display pattern: the score simply increases or decreases in value over time, either monotonically (one at a time) or by arbitrary amounts. Accumulator values are almost always displayed as digits, usually alongside a units designation, for example, 10,000XP or Posts: 1,429. The aggregation of the Vote-to-Promote input pattern is an accumulator.

If an accumulator has a maximum value that is understood by the reputation system, an alternative is to display it using any of the display patterns for normalized scores, such as percentages and levels.

Using points and accumulators:

Figure 7-6. Content example: Digg shows the number of times an item has been “Dugg.” Another example is the count of comments for an item.

Figure 7-7. Karma example: Yahoo! Answers awards points mostly for participation.

One very useful strategy for reputation display is to use statistical evidence: simply include as many of the inputs in a content item’s reputation as possible, without attempting to aggregate them in visible scores. Statistical evidence lets users zero in on the aspects of a content item that they consider the most telling. The evidence might consist of a series of simple accumulator scores:

Using statistical evidence:

Figure 7-8. Content Example: with YouTube’s very powerful “Statistics and Data” you can track a video’s rise in popularity on the site. (Sociologist and researcher Cameron Marlow calls it an “Epidemiology Interface.”)

Figure 7-9. Karma example: answers enhanced point and level information with statistical detail.

Figure 7-10. Yahoo! Answers displays the sources for statistical evidence.

Levels are reputation display patterns that remove insignificant precision from the score. Each level is a bucket holding all the scores in a range. Levels allow you to round off the results and simplify the display. Notice that the range of scores in each level need not be evenly distributed, as long as the users understand the relative difficulty of reaching each level.

Common display patterns for levels include numbered levels and named levels.

When using levels:

Numbered levels

Numbered levels are the most basic form of level display. This display pattern consists of a simple numeric value or a list of repeated icons representing the level that the reputation score falls into. Usually levels are 0 or 1 to n, though arbitrary ranges are possible as long as they make sense to users. The score may be an integer or a rounded fraction, such as 3½ stars. If the representation is unfamiliar to users, consider adding an element to the interface to explain the score and how it was calculated. Such an element is mandatory for reputations with nonlinear advancement rates.

Using numbered levels:

• Assign numbered levels if the reputation will be displayed in a rank-ordered sort a list of entities.

  Figure 7-11 shows a typical Stars-and-Bars display pattern for ratings and reviews. Stars and Bars are numbered levels, which happen to be displayed as graphics. In this example, each has a numbered level of 0 to 5. Though each review’s ratings are useful when displayed alongside the entity, the average of the overall score is used to rank-order results on search results pages.

• It is typical to use numbered levels to display aggregate reputation if the inputs were also numbered levels. Did you input stars? Then output stars.

  Figure 7-12 shows the karma ratings from Orkut.com. The Fans indicator is an accumulator (see Points and Accumulators), and the Trusty, Cool, and Sexy ratings are numeric levels. The users simply click on the smiling faces, ice cubes, and hearts next to their friends’ profiles to influence their scores. Many sites don’t allow direct karma ratings such as these with good reason (see Karma).

• If you need to display more than 10 levels, use numbered levels. Consider using numbered levels instead of named levels if you display more than five levels.

  Figure 7-13 displays two forms, out of many, of numbered levels for the game World of Warcraft. The user controls a character whose name is shown in the Members column. The first numbered level is labeled “Level” and ranges from 1 to 80, representing the amount of time and skill the user has dedicated to this character. The Guild Rank is a reverse-rank numbered level that represents the status of the user in the guild. This score is assigned by the guild master, who has the lowest guild rank.

  Pros	Cons
  Is easy to read. Accommodates unlimited values. You can always add more levels at the top. In ranked lists, relative value is easy to see.	Numeric format doesn’t convey limits or global value. Is level 20 good? What about 40? Often requires “What’s this?” user interface elements to explain levels to new users. Lots of numbers on a page can seem impersonal, especially when they’re associated with people. For karma, numbered levels can be perceived as fostering an undesirable competitive spirit.

Figure 7-11. Content example: stars and bars (iconic numbered levels).

Figure 7-12. Karma example: Orkut profile with an accumulator and iconic number levels.

Figure 7-13. Karma example: Experience levels and guild rank (sortable).

Named levels

In a named levels display pattern, a short, readable string of characters is substituted for a level number.

The name adds semantic meaning to each level so that users can more easily recognize the entity’s reputation when the reputation is displayed separately. Is the user a “silver contributor” or is the beef prime, choice, select, or standard?

Using named levels:

• Named levels are useful when the number of labels is five or less, so that each level can have a name that accurately expresses its meaning.

  Table 7-1 and Figure 7-14 show the meat grading levels used by the United States Department of Agriculture. The labels are descriptive, representing existing industry terms, and several are shared across different animal species—providing consumers a consistent standard for comparison.

  Table 7-1. Content example: USDA meat grades

  Species	Quality grades
  Beef	Prime, choice, select, standard, utility, cutter, canner
  Lamb and yearling mutton	Prime, choice, good, utility, cull
  Mutton	Choice, good, utility, cull
  Veal and calf	Prime, choice, good, standard, utility

Figure 7-14. Content example: USDA prime, choice, and select stamps.

• Named levels are particularly useful when numeric levels are too impersonal or encourage undesired competition.

• If you’re considering using numeric levels but find that the top and bottom levels should feel closer together than the numeric distance between them would otherwise indicate—consider using named levels instead. This is especially useful with karma scores so that new participants don’t get stuck with a demeaning level indicator, like “Level 1 of 10.”

  Figure 7-15 displays the current named levels used by WikiAnswers.com for user contributions. The original three categories were Bronze, Silver, and Gold—named after competitive medals. They are granted when nonlinearly increasing thresholds are met. Over time, the system has been expanded on three separate occasions to reward the nearly compulsive contributions of a handful of users.

  Pros

  Cons

  Hiding level numbers allows for more expressiveness. Level names can be thematically appropriate to, and vary by, your application(s). Common hierarchies work well—for example, poor, average, good, and excellent. This pattern is usually stronger when the named levels are displayed alongside other ratings, such as stars, points, and raw scores, to clarify them.

  Care must be taken when setting up the level names if you ever expect to add more to either end of the scale. Something else for your user to learn. Cultural bias can be a problem, especially if your site has an international audience. For example, the letter grading system of F, D, C, B, A is not internationally understood. Ambiguous names are more confusing than simple level numbers. Is the Ruby level better than Gold?

Figure 7-15. Karma example: The contributor levels on WikiAnswers have seen several awkward expansions.

A ranked list is based on highest or lowest reputation scores. Ranking systems are by their very nature comparative, and—human nature being what it is—the online community is likely to perceive this design choice as an encouragement of competition between users.

Leaderboard ranking

A leaderboard is a rank-ordered listing of reputable entities within your community or content pool. Leaderboards may be displayed in a grid, with rows representing the entities and columns describing those entities across one or more characteristics (name, number of views, and so on). Leaderboards provide an easy and approachable way to display the best performers in your community.

• Use leaderboards for content liberally. Provide filtered views of the boards to slice and dice by time (“Popular Today/This Week/All Time”) or by reputation type (“Most Viewed/Top Rated”).

  Figure 7-16 shows YouTube’s leaderboard ranking for most viewed videos as a grid. With numbers this high, it’s hard for potential reputation abusers to push inappropriate content onto the first page. Note that there are several leaderboards, one each for Today, This Week, This Month, and All Time.

• Use leaderboards for people sparingly, and only in contexts that are competitive by nature. Consider giving people leaderboards narrow scope (for example, only ranking me against my friends, to keep the comparisons fun and the stakes low).

  Figure 7-17 displays Yahoo! Answer’s leaderboard. The original version of this page was based solely on the number of points accumulated by participation, and users quickly figured out which actions produced the most points for the least effort. When the user’s best-answer percentage was eventually added to the profile display, it was discovered that the top-ranked users all had quality scores of less than 10%!

  Pros	Cons
  Clear and browsable way to compare items for specific qualities Data-intensive display: leaderboards satiate demand from information junkie users	May incite unhealthy competition to reach (or stay at) the top of the leaderboard. When used with accumulators, leaderboards can get stale as a few popular items move to the top and get stuck there, since nothing makes something more popular than its appearance on the list of most popular things.

Figure 7-16. Content example: YouTube’s most viewed videos.

Figure 7-17. Karma example: Yahoo! Answers leaderboard.

Top-X ranking

This is a specialized type of leaderboard where top-ranking entities are grouped into numerical categories of performance. Achieving top-10 status (or even top-100) should be a rare and celebrated feat.

When using Top-X ranking:

• Use top-X leaderboards for content to highlight only the best of the best contributions in your community.

  Figure 7-18 shows a Top-X display for content: Billboard’s Hot 100’s list of top recordings. The artists themselves have very little, if any, direct influence over their song’s rank on this list.

• Use top-X designations for people sparingly, and only in contexts that are competitive by nature. Because available categories in a top-X system are bounded, they will have greater perceived value in the community.

  Figure 7-19 displays the new index of Top-X karma for Amazon.com review writers. The very high number of reviews written by each of these leaders creates value both for Amazon and the reviewers themselves. Authors and publishers seek them out to review/endorse their book—sometimes for a nominal fee. The original version of this reputation system, now known as “Classic Reviewer Rank,” suffered deeply from first-mover effects (see First-mover effects) and other problems detailed in this book. This eventually lead to the creation of the new model, as pictured.

  Pros

  Cons

  Highly motivating for top performers. The prestige of earning a top-10 or top-100 designation may make contributors work twice as hard to keep it. Yields a small, bounded set of entities to promote as high quality.

  May incite unhealthy competition to reach (or stay at) the top of the ranks. For top-X karma based on accumulators, if a user’s reputation falls just below a category dividing line and the user knows his score, these categories often lead to minimum/maximum gaming, in which the user engages in a flurry of low-quality activity just to advance his top-X category. Top-X karma badges are unfamiliar to users who don’t contribute content. Don’t expect passive users to understand or even notice a top-X badge displayed alongside content reputation. Top-X badges are for content producers, not consumers.

Figure 7-18. Content example: Billboard’s Hot 100.

Figure 7-19. Karma example: Amazon’s top reviewer rankings.

It’s still too early to speak in absolutes about the design of social-media sites, but one fact is becoming abundantly clear: ranking the members of your community—and pitting them against one another in a competitive fashion—is typically a bad idea. Like the fabled djinni of yore, leaderboards on your site promise riches (comparisons! incentives! user engagement!!) but often lead to undesired consequences.

The thought process involved in creating leaderboards typically goes something like this: there’s an activity on your site that you’d like to promote; a number of people are engaged in that activity who should be recognized; and a whole bunch of other people won’t jump in without a kick in the pants. Leaderboards seem like the perfect solution. Active contributors will get their recognition: placement at the top of the ranks. The also-rans will find incentive: to emulate leaders and climb the boards.

And that activity you’re trying to promote? Site usage should swell with all those earnest, motivated users plugging away, right? It’s the classic win-win-win scenario. In practice, employing this pattern has rarely been this straightforward. Here are just a few reasons why leaderboards are hard to get right.

If it looks like a leaderboard and quacks like a leaderboard…

Even sites that don’t display overt leaderboards may veer too closely into the realm of comparative statistics. Consider Twitter and its prominent display of community members’ stats.

The problem may not lie with the existence of the stats but in the prominence of their display (see Figure 7-20). They give Twitter the appearance of a community that values popularity and the sheer size of a participant’s social network. Is it any wonder, then, that a whole host of community-created leaderboards have sprung up to automate just such comparisons? Twitterholic, Twitterank, Favrd, and a whole host of others are the natural extension of this value-by-numbers approach.

Figure 7-20. You’d be completely forgiven if you signed into Twitter and mistook this dashboard for a scoreboard!