Recommender systems

You’ve probably already encountered one of the major outgrowths of predictive analytics: recommender systems. These systems try to predict your interests (for example, what you want to buy or watch) and give you recommendations. They do this by matching your preferences with items or other like-minded people, using statistics and machine learning algorithms.

If you're an online cruiser, you often see prompts like these on web pages:

• People You May Know …
• People Who Viewed This Item Also Viewed …
• People Who Viewed This Item Bought …
• Recommended Based on Your Browsing History …
• Customers Who Bought This Item Also Bought …

These are examples of recommendation systems that were made mainstream by companies like Amazon, Netflix, and LinkedIn.

Obviously, these systems weren't created only for the user’s convenience — although that reason is definitely one part of the picture. No, recommender systems were created to maximize company profits. They attempt to personalize shopping on the Internet, with an algorithm serving as the salesperson. They were designed to sell, up-sell, cross-sell, keep you engaged, and keep you coming back. The goal is to turn each personalized shopper into a repeat customer. (The sidebar “The personal touch” explores one of the successful techniques.)

Personalized shopping on the Internet

A software recommender system is like an online salesperson who tries to replicate the personal process we experienced at the trade shows. What’s different about a recommender system is that it’s data-driven. It makes recommendations in volume, with some subtlety (even stealth), with a dash of unconventional wisdom and without a feeling of bias. When a customer buys a product — or shows interest in a product (say, by viewing it), the system recommends a product or service that it considers highly relevant to that customer — automatically. The goal is to generate more sales — sales that wouldn’t happen if the recommendation(s) weren’t given.

Amazon is a very successful example of implementing a recommender system; their success story highlights its importance. When you browse for an item on the Amazon website, you always find some variation on the theme of related items — “Customers who viewed this also viewed” or “Customers who bought items in your recent history also bought.”

This highly effective technique is considered one of Amazon’s “killer” features — and a big reason for their huge success as the dominant online marketplace. Amazon brilliantly adapted a successful offline technique practiced by salespeople — and perfected it for the online world.

Amazon popularized recommender systems for e-commerce. Their successful example has made recommender systems so popular and important in e-commerce that other companies are following suit.

Collaborative filtering

Collaborative filtering focuses on user and item characteristics based on the actions of the community. It can group users with similar interests or tastes, using classification algorithms such as k-nearest neighbor — k-NN for short (see Chapter 6 for more on k-NN). It can compute the similarity between items or users, using similarity measures such as cosine similarity (discussed in the next section).

The general concept is to find groups of people who like the same things: If person A likes X, then person B will also like X. For example: If Tiffany likes watching Frozen, then her neighbor (person with similar taste) Victoria will also like watching Frozen.

Collaborative filtering algorithms generally require

• A community of users to generate data
• Creating a database of interests for items by users
• Formulas that can compute the similarity between items or users
• Algorithms that can match users with similar interests

Collaborative filtering uses two approaches: item-based and user-based.

Content-based filtering

Content-based recommender systems mostly match features (tagged keywords) among similar items and the user’s profile to make recommendations. When a user purchases an item that has tagged features, items with features that match those of the original item will be recommended. The more features match, the higher the probability the user will like the recommendation. This degree of probability is called precision.

Content-based filtering uses various techniques to match the attributes of a particular item with a user's profile. These techniques include machine-learning algorithms to determine a user's profile without having to ask. This technique is called implicit data gathering. A more direct approach is to use explicit data gathering: Use a questionnaire to ask the users what features they like in an item. An example of that would be asking what genre of movie or which actresses they like when they first sign up for a movie subscription.

Hybrid recommender systems

The best implementation may be a hybrid approach to creating a recommender system. By combining content-based and collaborative filtering into a single approach, the system can try to overcome each one’s shortcomings.

A major problem with the content-based approach is its accuracy and narrow focus. The recommendations may not be very interesting or unique. Many of the recommendations may have already been known to the user, so the system isn’t providing anything new from the user’s perspective. However, the implementation is much simpler than that of collaborative filtering. The content-based approach requires only that a profile be created for the items (keyword tagging). The user’s profile can be implicitly or explicitly created. This system can start working right away.

A major problem with collaborative filtering is that it suffers from the cold-start problem. Many users who are just starting out won’t receive accurate recommendations — or any recommendations at all — until enough data is gathered from the community of users. The data collection will require time to complete — and that collection depends on how active the website is. It may also require users to create accounts in their systems (in order to create a profile) before they can start receiving recommendations.

A hybrid recommender system can try to solve both these problems. It can start by using the content-based approach to avoid the cold-start problem. After enough data is collected from the community of users, the system can then use the collaborative filtering approach to produce more interesting and personalized recommendations.

Targeting using predictive modeling

Traditional marketing targets a group of customers without applying such modern techniques as predictive modeling using data-mining, and machine-learning algorithms to the dataset. Predictive modeling, in the area of direct marketing is called response modeling using predictive analytics (or simply response modeling from here on). Sometimes analysts create filters to apply to the dataset, thereby creating a select group to target. But that select group may not be optimally configured. Response modeling, on the other hand, seeks to discover patterns in the data that are present but not immediately apparent (in part because the number of variables being considered is nearly always greater than the number of variables that would be present in a marketer’s segment or “filter”); the result is an optimized group to target.

Table 2-5 uses a small sample to compare the profit generated by direct mailings — traditional marketing versus response modeling.

In Table 2-5, response modeling has targeted 10 percent of the traditional number of customers (100 instead of 1000) to an optimized subset. The response rate should be higher with response modeling — 10 percent instead of the 2 percent that is typical for traditional marketing. The net result is a profit of $800 under response modeling; traditional marketing breaks even. Also, as per-customer targeting costs increase, response modeling's value gets even better — without even taking into account the implicit benefits of not targeting unqualified customers.

When you make constant contact with a customer without providing any benefit, you run the risk of being ignored in the future.

Table 2-6 is an example that shows the profit comparison between direct mailings using traditional marketing and response modeling with a larger sample size.

In Table 2-6, response modeling has (again) targeted only 10 percent of the 10,000 prospective customers traditionally targeted. In an optimized subset of 1,000, the response rate should be higher. We assumed a response rate of 2 percent for a traditional direct-mailing marketing campaign; with response modeling, the response rate is 10 percent because the customers are likelier to buy in the first place.

Response modeling creates a profit of $8,000 under this scenario; traditional marketing breaks even. As in the preceding scenario, any revenue earned using traditional marketing is consumed by marketing costs. Thus, as the accuracy of customers targeted, increases, the value of response modeling also increases.

Response modeling also applies to email marketing campaigns, even though the production costs are much cheaper than other channels. Fatigue from constant email contact will diminish the value of future marketing campaigns for all other channels.

Uplift modeling

So how do you know that the customer you targeted wouldn’t have purchased anyway? To clarify this question, you can restate it in a couple different ways:

• How do you know the customer wouldn’t have purchased even when she didn’t get the marketing contact from you?
• How do you know that what you sent to the customer influenced her to make the purchase?

Some modelers claim that the problems with response modeling are as follows:

• You’re taking a subset of your customers whom you’ve predicted will have some interest in the product or service already.
• You’re wasting marketing dollars on customers who don’t need the extra influence to convert.
• You may be decreasing your net margins because the discounts you’re using to entice the customer to buy may be unnecessary.
• You may be reducing your customer satisfaction because some customers don’t want to be (constantly) contacted.
• You’re incorrectly taking credit for the response in your evaluation of the model.

Uplift modeling, also called true lift modeling and net modeling among other terms, aims to answer those criticisms by predicting which customers will only convert when contacted.

Uplift modeling works by generating predictive scores that aim at ranking individuals by their ability to be influenced. Abstractly, there are four possible groupings of customers:

• Persuadables: Customers who can be persuaded to purchase — but will only buy when contacted.
• Sure Things: Customers who will buy, regardless of contact.
• Lost Causes: Customers who will not buy, regardless of contact.
• Do Not Disturbs: Customers whom you shouldn't contact. Contacting them may cause a negative response like provoking them to cancel a subscription, return a product, or ask for a price adjustment.

Uplift modeling is designed to target the Persuadables. The Persuadables get a high predictive score, while the Do Not Disturbs get a low predictive score.

In the middle of the range are the Sure Things and Lost Causes.

Uplift models have proven to be effective, but they are more difficult to create than a response model. Here’s why:

• It generally is more complex to find a group of Persuadables than Sure Things. Sure Things usually are easier to identify, because they are customers who may have already purchased before or have shown interest in the product. Persuadables may have purchased a similar product or exhibit similar features as customers that have purchased. With a potentially smaller target size of Persuadables, complexity of building the uplift model, the operating effort, and cost, companies may not justify the use over response modeling.

  In the case of voters, many undecided voters may fall into the group of Persuadables and can be switched to your candidate and motivated to vote when given a particular treatment.

• It’s more difficult to measure the success of the model because it’s attempting to measure the influence that the treatment caused to change a customer’s behavior, not the concrete action of whether the customer purchased after receiving contact. A customer may have been influenced by other factors between the time they received contact and took action.

  To truly measure whether a customer can be or has been influenced accurately, you would (in effect) have to clone her and split the identical clones into a separate group. The first (treated/cloned group) would receive the advertisement; the second (control/original group) would not. Setting aside such sci-fi scenarios, you have to make some concessions to reality and employ an alternative method to get a useful measurement of the model’s success.

Measuring uplift modeling requires having two test sets: a randomized control group and a treatment group. The treatment group gets the specially designed treatment (or contact), while the control group gets the default experience (no treatment or no contact). The positive difference in response rate between the treatment group and the control group is the uplift.

Even with these difficulties, some modelers argue that uplift modeling provides true marketing impact. They consider it more efficient than response modeling because it doesn’t include the Sure Things in the targeting (which artificially inflates response rates). For that reason, uplift modeling is the choice for target marketing using predictive analytics.

Uplift modeling is still a relatively new technique in target marketing. More companies are starting to use it and have found success using it in their customer retention, marketing campaigns, and even presidential campaigns. For companies with large customer lists, uplift modeling is worth exploring.

Some pundits are crediting uplift modeling for President Obama’s 2012 presidential campaign win. The campaign’s data analyst used uplift modeling to heavily target voters who were most likely to be influenced by contact. They used personalized messages via several channels of contact: social media, television, direct mail, and telephone. They concentrated their efforts to influence the group of Persuadables. They invested heavily in this strategy; apparently it paid off.

Online customer experience

These websites are attempting to personalize your online experience to influence you and make it easier for you to take the action the companies that operate the websites want. The desired outcomes or goals these companies are usually looking for include

• Filling out a registration or an appointment form
• Clicking on a product link
• Reading another news article
• Watching a video
• Buying a product

After the company has collected enough data for customers that successfully meet the goal, they can try to learn from the patterns those customers did before the action was taken, then apply the patterns to similar customers. Finding customers that are similar is by no means a simple task. Sure, an analyst can easily identify a single to a few customer attributes to manually segment the data by and see whether there are any patterns. But this process becomes exponentially more difficult as the number of transactions gets bigger and the customer attributes get wider.

Using machine learning algorithms, the machine can find micro-segments of similar customers so companies can target a group of customers more precisely with personalized content and offers.

Retargeting

Your web surfing behavior tracking isn't limited to the current website you're on. It can be tracked across multiple websites. This technology is possible through browser cookies and advertising networks that share data with its affiliates. When you visit one website and view a product page, then move on to another new website and see an advertisement for the product on the banner or side rail, this is retargeting.

By itself, retargeting isn’t predictive analytics. But this gives you the knowledge that the data is out there and can be shared across websites. The ad networks share the data with their affiliates, and much of this data is available through data management platforms. By combining this third-party data with the company's own data, they can create more advanced predictive models.

Implementation

Personalized websites can be created in several ways. It really depends on which types of data that are available and whether the customer is logged in or not. Having profile data on the customer adds many more attributes to create a predictive model.

These are some data types and sources that can be used to create personalization models:

• Customer's profile data, when she is logged in.

  Profile data can include such attributes as age and gender.

• Content on the page.

  Using text mining techniques like TF-IDF, we can find important keywords.

• The referring webpage.

  The referring webpage may have keywords in its website address (URL).

• The websites and pages you visited before.

  When this data is available, it can show interest in a product or subject.

• The geolocation of the web browser.

  The physical location of the browser accessing the website, using its Internet Protocol address.

• Temporal data, such as time of day.

  Time of day and day of week are common temporal segmentation attributes.

Optimizing using personalization

Optimizing an e-commerce site using personalization is a great example of improving customer experience and satisfaction. By providing individualized content, customers see relevant offers, encounter fewer distractions, and build trust in the system, which will ultimately drive sales. Here’s an illustration of how personalization increases sales.

Suppose we are operating a travel booking website. We want to optimize the experience for our two biggest markets: California and Florida travelers. Our analysts know that Californians like to travel to Nevada and Floridians like to travel to Georgia. So we use a simple rule to personalize the home page by geolocation.

We show a hero image of Las Vegas to IP addresses belonging to California; for Floridians, we show a hero image of downtown Atlanta. (A hero image is a big banner placed on the top or center of the web page.) We're using the hero image to personalize, inspire, and influence the site visitor to make a booking. Some visitors will make the booking regardless of the personalized image, because that was their original intention. Others may be inspired and influenced into further researching and planning a trip to Las Vegas or Atlanta, thus increasing your booking rate.

Using predictive modeling, we can use every available attribute to create sophisticated models that target specific segments of the California and Florida markets. For example, the model could have discovered that northern Californians with children like to travel to Lake Tahoe, while the ones without children like to fly to Waikiki Beach in Hawaii. Meanwhile, southern Californians like to go to Las Vegas. Northern Floridians may like Atlanta, while southerners may prefer the Caribbean Islands.

Predictive modeling allows you to be as granular as you like in segmenting data to make personalized offers. It can detect patterns in micro-segments that would normally be very difficult. A rule-based system may start with a few simple rules, but eventually turn into a complex and convoluted set of rules. The constant manual updating and deploying may make a rule-based system unmanageable. Using machine learning to algorithmically produce content presentation for personalization is a scalable and cost-effective solution. Personalization has great potential to increase ROI.

Similarities of Personalization and Recommendations

Personalization is similar to recommendations, and they are often used together. In addition, some websites use the term personalized recommendations. Recommendation implementations are easy to identify because they are often called out specifically with “Recommended for you” or “Customers that bought this also bought that.” When you read an article on a news site, the recommendation engine will recommend other relevant articles for you to read. When you buy a product, the recommendation engine will recommend other products that may interest you to buy.

One type of personalization can refer to how you like the content of the website to be presented to you:

• The website’s arrangement:
  • Main navigation on the top or the left panel?
  • Content groupings by vertical columns or horizontal rows?
  • Text lists or thumbnail images?
• The colors of CTA (call to action) buttons and links:
  • Colors matter, and they often mean different things and exude different feelings to different people. Some research has shown orange to be the color that converts the best, because orange contrasts well with typical websites.
  • Most people are accustomed to Google search links being blue for unvisited and red for visited. But the default colors on browsers could be different. People with color blindness may prefer different colors for links.

• The flow of the checkout process.

  High-frequency customers may prefer one-click ordering from the product page, while standard customers may prefer a one-page checkout process after clicking the product to review their order.

  Another type of personalization refers to which type of content on the website should be presented to the customer according to their profile. This can be in the form of personalized recommendations.

• Relevant ads and offerings.

  For example, it may not make sense for an apparel company to show ads for men’s clothes when the customer is female and have no history of purchasing men’s clothes. They should be showing ads for women’s clothes in similar price point categories as their purchase history.

• Relevant items.

  Based on your profile, purchase history, reading history, and similarity to other like-minded customers, show relevant products or articles on your personalized home page.