Home Page Icon
Home Page
Table of Contents for
III. Building Web Reputation Systems
Close
III. Building Web Reputation Systems
by F. Randall Farmer, Bryce Glass
Building Web Reputation Systems
Preface
What Is This Book About?
Why Write a Book About Reputation?
Who Should Read This Book
Organization of This Book
Part I: Reputation Defined and Illustrated
Part II: Extended Elements and Applied Examples
Part III: Building Web Reputation Systems
Role-Based Reading (for Those in a Hurry)
Conventions Used in This Book
Safari® Books Online
How to Contact Us
Acknowledgments
From Randy
From Bryce
I. Reputation Defined and Illustrated
1. Reputation Systems Are Everywhere
An Opinionated Conversation
People Have Reputations, but So Do Things
Reputation Takes Place Within a Context
We Use Reputation to Make Better Decisions
The Reputation Statement
Explicit: Talk the Talk
Implicit: Walk the Walk
The Minimum Reputation Statement
Reputation Systems Bring Structure to Chaos
Reputation Systems Deeply Affect Our Lives
Local Reputation: It Takes a Village
Global Reputation: Collective Intelligence
FICO: A Study in Global Reputation and Its Challenges
Web FICO?
Reputation on the Web
Attention Doesn’t Scale
There’s a Whole Lotta Crap Out There
People Are Good. Basically.
Know thy user
Honor creators, synthesizers, and consumers
Throw the bums out
The Reputation Virtuous Circle
Who’s Using Reputation Systems?
Challenges in Building Reputation Systems
Related Subjects
Conceptualizing Reputation Systems
2. A (Graphical) Grammar for Reputation
The Reputation Statement and Its Components
Reputation Sources: Who or What Is Making a Claim?
Reputation Claims: What Is the Target’s Value to the Source? On What Scale?
Reputation Targets: What (or Who) Is the Focus of a Claim?
Molecules: Constructing Reputation Models Using Messages and Processes
Messages and Processes
Reputation Model Explained: Vote to Promote
Building on the Simplest Model
Complex Behavior: Containers and Reputation Statements As Targets
Solutions: Mixing Models to Make Systems
From Reputation Grammar to…
II. Extended Elements and Applied Examples
3. Building Blocks and Reputation Tips
Extending the Grammar: Building Blocks
The Data: Claim Types
Qualitative claim types
Text comments
Media uploads
Relevant external objects
Quantitative claim types
Normalized value
Rank value
Scalar value
Processes: Computing Reputation
Roll-ups: Counters, accumulators, averages, mixers, and ratios
Simple Counter
Reversible Counter
Simple Accumulator
Reversible Accumulator
Simple Average
Reversible Average
Mixer
Simple Ratio
Reversible Ratio
Transformers: Data normalization
Simple normalization (and weighted transform)
Scalar denormalization
External data transform
Routers: Messages, Decisions, and Termination
Common decision process patterns
Simple Terminator
Simple Evaluator
Terminating Evaluator
Message Splitter
Conjoint Message Delivery
Input
Typical inputs
Reputation statements as input
Periodic inputs
Output
Return values
Signals: Breaking out of the reputation framework
Logging
Practitioner’s Tips: Reputation Is Tricky
The Power and Costs of Normalization
Liquidity: You Won’t Get Enough Input
Bias, Freshness, and Decay
Ratings bias effects
First-mover effects
Freshness and decay
Implementer’s Notes
Making Buildings from Blocks
4. Common Reputation Models
Simple Models
Favorites and Flags
Vote to promote
Favorites
Report abuse
This-or-That Voting
Ratings
Reviews
Points
Karma
Participation karma
Quality karma
Robust karma
Combining the Simple Models
User Reviews with Karma
eBay Seller Feedback Karma
Flickr Interestingness Scores for Content Quality
When and Why Simple Models Fail
Party Crashers
Keep Your Barn Door Closed (but Expect Peeking)
Decay and delay
Provide a moving target
Reputation from Theory to Practice
III. Building Web Reputation Systems
5. Planning Your System’s Design
Asking the Right Questions
What Are Your Goals?
User engagement
Establishing loyalty
Coaxing out shy advertisers
Improving content quality
Content Control Patterns
Web 1.0: Staff creates, evaluates, and removes
Bug report: Staff creates and evaluates, users remove
Reviews: Staff creates and removes, users evaluate
Surveys: Staff creates, users evaluate and remove
Submit-publish: Users create, staff evaluates and removes
Agents: Users create and remove, staff evaluates
Basic social media: Users create and evaluate, staff removes
The Full Monty: Users create, evaluate, and remove
Incentives for User Participation, Quality, and Moderation
Predictably irrational
Incentives and reputation
Altruistic or sharing incentives
Tit-for-tat and pay-it-forward incentives
Friendship incentives
Crusader, opinionated incentives, and know-it-all
Commercial incentives
Direct revenue incentives
Incentives through branding: Professional promotion
Egocentric incentives
Fulfillment incentives
Recognition incentives
Personal or private incentives: The quest for mastery
Consider Your Community
What are people there to do?
Is this a new community? Or an established one?
The competitive spectrum
Better Questions
6. Objects, Inputs, Scope, and Mechanism
The Objects in Your System
Architect, Understand Thyself
So…what does your application do?
Perform an application audit
What Makes for a Good Reputable Entity?
People are interested in it
The decision investment is high
The entity has some intrinsic value worth enhancing
The entity should persist for some length of time
Determining Inputs
User Actions Make Good Inputs
Explicit claims
Implicit claims
But Other Types of Inputs Are Important, Too
Good Inputs
Emphasize quality, not simple activity
Rate the thing, not the person
Reward firsts, but not repetition
Use the right scale for the job
Match user expectations
Common Explicit Inputs
The ratings life cycle
The interface design of reputation inputs
Stars, bars, and letter grades
The schizophrenic nature of stars
Do I like you, or do I “like” like you
Two-state votes (thumb ratings)
Vote to promote: Digging, liking, and endorsing
User reviews
Common Implicit Inputs
Favorites, forwarding, and adding to a collection
Favorites
Forwarding
Adding to a collection
Greater disclosure
Reactions: Comments, photos, and media
Constraining Scope
Context Is King
Limit Scope: The Rule of Email
Applying Scope to Yahoo! EuroSport Message Board Reputation
Generating Reputation: Selecting the Right Mechanisms
The Heart of the Machine: Reputation Does Not Stand Alone
Common Reputation Generation Mechanisms and Patterns
Generating personalization reputation
Generating aggregated community ratings
Ranking large target sets (preference orders)
Generating participation points
Points as currency
Generating compound community claims
Generating inferred karma
Practitioner’s Tips: Negative Public Karma
Draw Your Diagram
7. Displaying Reputation
How to Use a Reputation: Three Questions
Who Will See a Reputation?
To Show or Not to Show?
Personal Reputations: For the Owner’s Eyes Only
Personal and Public Reputations Combined
Public Reputations: Widely Visible
Corporate Reputations Are Internal Use Only: Keep Them Hush-hush
How Will You Use Reputation to Modify Your Site’s Output?
Reputation Filtering
Reputation Ranking and Sorting
Reputation Decisions
Content Reputation Is Very Different from Karma
Content Reputation
Karma
Karma is complex, built of indirect inputs
Karma calculations are often opaque
Display karma sparingly
Karma caveats
Reputation Display Formats
Reputation Display Patterns
Normalized Score to Percentage
Points and Accumulators
Statistical Evidence
Levels
Numbered levels
Named levels
Ranked Lists
Leaderboard ranking
Top-X ranking
Practitioner’s Tips
Leaderboards Considered Harmful
What do you measure?
Whatever you do measure will be taken way too seriously
If it looks like a leaderboard and quacks like a leaderboard…
Leaderboards are powerful and capricious
Who benefits?
Going Beyond Displaying Reputation
8. Using Reputation: The Good, The Bad, and the Ugly
Up with the Good
Rank-Order Items in Lists and Search Results
Content Showcases
The human touch
Down with the Bad
Configurable Quality Thresholds
Expressing Dissatisfaction
Out with the Ugly
Reporting Abuse
Who watches the watchers?
Teach Your Users How to Fish
Inferred Reputation for Content Submissions
Just-in-time reputation calculation
A Private Conversation
Course-Correcting Feedback
Reputation Is Identity
On the User Profile
My Affiliations
My History
My Achievements
At the Point of Attribution
To Differentiate Within Listings
Putting It All Together
9. Application Integration, Testing, and Tuning
Integrating with Your Application
Implementing Your Reputation Model
Rigging Inputs
Applied Outputs
Beware Feedback Loops!
Plan for Change
Testing Your System
Bench Testing Reputation Models
Environmental (Alpha) Testing Reputation Models
Predeployment (Beta) Testing Reputation Models
Performance: Testing scale
Confidence: Testing computation accuracy
Application optimization: Measuring use patterns
Feedback: Evaluating customer’s satisfaction
Value: Measuring ROI
Tuning Your System
Tuning for ROI: Metrics
Model tuning
Application tuning
Tuning for Behavior
Emergent effects and emergent defects
Defending against emergent defects
Keep great reputations scarce
Tuning for the Future
Learning by Example
10. Case Study: Yahoo! Answers Community Content Moderation
What Is Yahoo! Answers?
A Marketplace for Questions and Yahoo! Answers
Attack of the Trolls
Time was a factor
Location, location, location
Built with Reputation
Avengers Assemble!
Initial Project Planning
Setting Goals
Cutting costs
Cleaning up the neighborhood
Who Controls the Content?
Incentives
The High-Level Project Model
Objects, Inputs, Scope, and Mechanism
The Objects
Limiting Scope
An Evolving Model
Iteration 1: Abuse reporting
Inputs
Mechanism and diagram
Analysis
Iteration 2: Karma for abuse reporters
Inputs
Mechanism and diagram
Analysis
Iteration 3: Karma for authors
Inputs
Mechanism and diagram
Analysis
Final design: Adding inferred karma
Inputs
Mechanism and diagram
Analysis
Displaying Reputation
Who Will See the Reputation?
How Will the Reputation Be Used to Modify Your Site’s Output?
Is This Reputation for a Content Item or a Person?
Using Reputation: The…Ugly
Application Integration, Testing, and Tuning
Application Integration
Testing Is Harder Than You Think
Lessons in Tuning: Users Protecting Their Power
Deployment and Results
Operational and Community Adjustments
Adieu
A. The Reputation Framework
Reputation Framework Requirements
Calculations: Static Versus Dynamic
Static: Performance, performance, performance
Dynamic: Reputation within social networks
Scale: Large Versus Small
Reliability: Transactional Versus Best-Effort
Model Complexity: Complex Versus Simple
Data Portability: Shared Versus Integrated
Optimistic Messaging Versus Request-Reply
Performance at scale
Framework Designs
The Invisible Reputation Framework: Fast, Cheap, and Out of Control
Requirements
Implementation details
Lessons learned
The Yahoo! Reputation Platform: Shared, Reliable Reputation at Scale
Yahoo! requirements
Yahoo! implementation details
High-level architecture
Messaging dispatcher
Model execution engine
External signaling interface
Reputation repository
Reputation query interface
Yahoo! lessons learned
Your Mileage May Vary
Recommendations for All Reputation Frameworks
B. Related Resources
Further Reading
Recommender Systems
Social Incentives
Patents
Index
About the Authors
Colophon
Copyright
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Prev
Previous Chapter
4. Common Reputation Models
Next
Next Chapter
5. Planning Your System’s Design
Part III. Building Web Reputation Systems
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset