Item-to-User recommendations

We’ll start by describing the architecture of the system we’ve been building in the book thus far. As proposed in Chapter 4, we built the collector offline to ingest and process our recommendations. We utilize representations to encode relationships between items, users, or user-item pairs. The online collector takes the request, usually in the form of a user-id, and finds a neighborhood of items in this representation space to pass along to the ranker. Those items are filtered when appropriate and sent for scoring. The offline ranker learns the relevant features for scoring and ranking, training on the historical data. It then uses this model and, in some cases, item features as well for inference. In the case of recommendation systems this inference computes the scores associated to each item in the set of potential recs. We usually sort by this score Part III. Finally, we integrate a final round of ordering based on some business logic Chapter 14. This last step is part of the serving where we impose things like test-criteria or recommendation diversity requirements.

Figure 7-1. This image by Karl Higley and Even Oldridge is an excellent overview of the Retrieval, Ranking, Serving Structure, although they instead use four stages and slightly different terminology. In this text, we combine Filtering into Retrieval. (used with permission)

Query-based recommendations

To start off our process, we want to make a query. The most obvious example of a query is a text query like in text-based search engines; however, queries may be more general! For example you may wish to allow search-by-image, or search-by-tag. Note that an important type of query-based recommender is where the query is implicit; i.e. where the user is providing a search query via UI choices, or by behaviors. While these systems are quite similar in overall structure to the item-to-user systems, let’s discover how to modify them to fit our use-case.

We wish to integrate more context about the query into the first step of the request. Note that we don’t want to throw out the user-item matching components of this system – even though the user is performing a search, there’s utility in personalizing the recommendations based on their taste. Instead we need to utilize the query in addition; later we will discuss various technical strategies, but a simple summary for now is to also generate an embedding for the query! Note that the query is like an item or a user, but sufficiently different. Some strategies might include similarity between the query and items, or co-occurrence of the query and items. Either way, we now have a query-representation and user-representation, and we want to utilize both for our recommendation. One simple strategy here is to use the query representation for retrieval, but during the scoring, score via both query-item and user-item, combining them via a multi-objective loss. Another strategy is to use the user for retrieval, and then the query for filtering.

Context-based recommendations

A context is actually quite similar to a query, but tends to be more obviously feature based and frequently less similar to the items/users distributions.. Context is usually the term use to represent some exogenous features to the system, that may have an effect on the system, i.e. auxiliary information such as time, weather, location, and more. The ways in which it is similar to query-based recommendation are that it’s an additional signal that the system needs to consider during recommendation, but more often than not: query should often dominate the signal for recommendation, whereas context should not.

Let’s take a simple example: when ordering food. A query for a food-delivery recommendation system would look like Mexican food; an extremely important signal from the user looking for burritos or quesadilla of how the recs should look. A context for a food-delivery recommendation system would look like it’s almost lunch time. This is an useful signal, but may not out-weigh user-personalization. It’s hard to put hard and fast rules on this weighting, so usually we don’t, and instead learn parameters via experimentation.

How context features fit into the architecture is very similar to queries, via learned weightings as part of the objective function. Your model will learn a representation between context features and items, and add that affinity into the rest of the pipeline. Again you can make use of this early in the retrieval, later in the ranking, or even during the serving step.

Sequence-based recommendations

Sequence based recommendations build on context-based recommendations, but with a specific type of context. Sequential recommendations use the idea that the recent items the user has been exposed to should have a significant influence on the recommendations. A common example here is a music streaming service, where the last few songs that have been played can significantly inform what the user might want to hear next. To ensure this auto-regressive, or sequentially predictive, set of features has an influence on recommendations, we can treat each item in the sequence as a weighted context for the recommendation.

Usually, the item-item representation similarities are weighted to provide a collection of recommendations and various strategies are used for combining these. In this case, we normally expect the user to be of high importance in the recommendations but the sequence is also of high importance. One simple model is to think of the sequence of items like a sequence of tokens, and form a single embedding for that sequence – like in NLP applications. This embedding can be used as the context in a context-based recommendation architecture.

Why bother with extra features

Sometimes it is useful to step back and ask if a new technology is actually worth caring about. So far in this section we’ve introduce four new paradigms for how to think about a recommender problem. That level of detail may seem surprising and potentially even unnecessary.

One of the core reasons that things like context and query based recommendations become relevant is to deal with some of the issues mentioned before around sparsity and cold starting. Sparsity makes things that aren’t cold seem cold via the learner’s under-exposure to them, but true cold starting also exists due to new items being added to catalogs with high frequency in most applications. We will address cold-starting in detail, but for now, suffice to say that one strategy for warm starting is to use other features which are available even in this regime.

In applications of machine learning that are explicitly feature based, we rarely battle the cold-start problem to such a degree, because at inference time we’re confident that the model parameters useful for prediction are well aligned with those features that are available. In this way, feature-included recommendation systems are bootstrapping from a potentially weaker learner, that has more guaranteed performance via always available features.

The second analogy that the previous architectures are reflecting is that of boosting. Boosted models operate via the observation that ensembles of weaker learners can reach better performance. Here we are asking for some additional features to help these networks ensemble with weak learners, to boost their performance.

Models as APIs

Let’s discuss two systems architectures that might be appropriate for serving your models in production: micro-service and monolith.

In web applications this dichotomy is well covered from many perspective and special use-cases. As ML engineers, Data Scientists, and potentially Data Platform engineers, it’s not necessary to dig deep into this area, but it’s essential to know the basics. Simply: * micro-service architectures state that each component of the pipeline should be it’s own small program with clear API and output schema. Composing these API calls allows for flexible and predictable pipelines. * In contrast, monolithic architectures suggest that one application should contain all the necessary logic and components for model predictions. Keeping the application self-contained means less interfaces that need to be kept aligned and fewer rabbit holes to hunt around in when some location in your pipeline is being starved.

Whatever you choose as your strategy, you’ll need to make a few decisions:

• How large is the necessary application? If your application will need fast access to large datasets at inference time, you’ll need to think carefully about memory requirements.

• What access does your application need? We’ve previously discussed making use of things like Bloom filters, and Feature stores; these resources may be tightly coupled to your application – by building them in memory in the application – or they might be an API call away. Make sure your deployment accounts for these relationships.

• Should your model be deployed to a single node or a cluster? For some types of model, even at the inference step we wish to utilize distributed computing. This will require additional configuration to allow for fast parallelization.

• How much replication do you need? Horizontal scaling allows you to have multiple copies of the same service running simultaneously to reduce the demand on any particular instance. This is important for ensuring availability and performance. As we horizontally scale, each service can operate independently and there are different strategies for coordinating these services and an API request. Each replica is usually it’s own containerized application and things like CoreOS and Kubernetes are used to manage these. The requests themselves must also be balanced to the different replicas via something like NGINX.

• What are the relevant API’s exposed? Each application in the stack should have a clear set of exposed schemas, and an explicit communication about what types of other applications may call to them.

import wandb, torch
run = wandb.init(project=Prod_model, job_type="inference")

model_dir = run.use_artifact(
		'bryan-wandb/recsys-torch/model:latest',
		type='model'
).download()

model = torch.load(model_dir)
model.eval(user_id)

from fastapi import FastAPI # FastAPI code

import wandb, torch

app = FastAPI() # FastAPI code

run = wandb.init(project=Prod_model, job_type="inference")

model_dir = run.use_artifact(
	'bryan-wandb/recsys-torch/model:latest',
	type='model'
).download()

model = torch.load(model_dir)

@app.get("/recommendations/{user_id}") # FastAPI code
def make_recs_for_user(user_id: int): # FastAPI code
		endpoint_name = 'make_recs_for_user_v0'
		logger.info(
			"{'type': 'recommendation_request',"
			f"'arguments': {'user_id': {user_id}},"
			f"'response': {None}},",
			f"'endpoint_name': {endpoint_name}"
		)
		recommendation = model.eval(user_id)
		logger.log(
			"{'type': 'model_inference',"
			f"'arguments': {'user_id': {user_id}},"
			f"'response': {recommendation}},"
			f"'endpoint_name': {endpoint_name}"
		)
    return { # FastAPI code
			"user_id": user_id,
			"endpoint_name": endpoint_name,
			"recommendation": recommendation
		}

Workflow orchestration

The other component necessary for your deployed system, is workflow orchestration. The model service is responsible for receiving requests, and serving results, but there are many components to the system that need to be in place for this service to do anything of use. There are several components to these workflows, so we will discuss them in sequence: containerization, scheduling, and CI/CD.

Schemas and Priors

When designing software systems, you almost always have expectations of how the components fit together. Much like in writing code where you anticipate the input and output to functions, in software systems you anticipate these at each interface. This is relevant not only for micro-service architectures; even in a monolith architecture components of the system need to work together and often will have boundaries between their defining responsibilities.

Let’s make this more concrete via an example: you’ve built a user-item latent space, a feature store for user features, a bloom filter for client avoids, and an experiment index which defines which of two models should be used for scoring. First let’s examine the latent space: when provided a user_id we need to look up its representation, and we have some assumptions already:

1. user_id provided will be of the correct type

2. user_id will have a representation in our space

3. The representation returned will be of the correct type and shape

4. The component values of the representation vector will be in the appropriate domain. (Note: the support of representations in your latent space may vary day-to-day)

From here, we need look up the $k$ Approximate Nearest Neighbors, which incurs more assumptions:

1. There are $\ge k$ vectors in our latent space

2. Those vectors adhere to the expected distributional behavior of the latent space

While these seem relatively straightforward application of unit-tests, canonizing these assumptions is important. Take the last assumption in both of the two services: how can you know the appropriate domain for the representation vectors? As part of your training procedure, you’ll need to calculate this, then store it for access during the inference pipeline.

In the second case, when finding nearest neighbors in high dimensional spaces, there are well discussed difficulties in distributional uniformity, but this can mean particularly poor performance for recommendations. In practice, we have observed a spiky nature to the behavior of $k$-NN in latent spaces, leading to difficult challenges downstream in ensuring diversity of recommendations. These distributions can be estimated as priors and simple checks like KL-divergence can be used online – one can estimate the average behavior of the embeddings and the difference between local geometries.

In both cases, collecting the output of this information and logging it can provide a rich history of what is going on with your system. This can shorten debugging loops later on if model performance is low in production.

Returning to the possibility of user_id lacking a representation in our space: this is precisely the cold-start problem! In that case we need to transition over to a different prediction pipeline: perhaps user-feature-based, explore-exploit, or even hard-coded recommendations. In this setting, we need understand next steps when a schema condition is not met and gracefully move forward.

Observability

There are quite a number of tools in the software engineering to assist with observability, understanding the why’s of what’s going on in the software stack. Because the systems we are building become quite distributed, the interfaces become critical monitoring points, but the paths also become complex.

Common terms in this area are spans and traces, which refer to two dimensions of a call stack. Given some collection of connected services, like in our above examples, an individual inference request will pass through some or all of those services in a sequence. The sequence of service requests is the trace. The, potentially parallel, time-delays of each of these services, is the span.

Figure 7-3. TODO: Need to remake from scratch

The graphical representation of spans usually demonstrates how the time for one service to respond is comprised of several other delays from other calls.

Observability enables you to see traces, spans, and logs in conjunction to appropriately diagnose the behavior of your system. In our example above where we utilize a call-back from the filter step to get more neighbors from the collector, we might see a slow response and wonder “what has happened?”. By viewing the spans and traces, we’d be able to see that the first call to the collector was as expected, then the filter step made a call to the collector, then another call to the collector, and so on, which built up a huge span for the filter step. Combining that view with logging would help us rapidly diagnose what might be happening.

Slow feedback

Recommendation systems fundamentally are trying to lead to item selection, and in many cases, purchases. But if we step back and think more holistically about how recommendation systems integrate into businesses, it’s to drive revenue. If you’re an e-commerce shop, item selection and revenue may seem easily associated: a purchase leads to revenue, so good item recommendation leads to revenue. However, what about returns? Or even a harder question: are these revenue incremental? One challenge with recommendation systems, is there may sometimes be a very difficult to draw causal arrow between any metric used to measure the performance of your models to the business-oriented key performance indicators.

We call this slow feedback because sometime the loop to get from a recommendation, to a meaningful metric, and back to the recommender can take weeks or even longer. This is especially challenging when you want to run experiments to understand if a new model should be rolled out. The length of the test may need to stretch quite a bit more to get meaningful results.

Usually a proxy metric is aligned on that the data scientists believe is a good estimator for the KPI, and that proxy metric is measured live. There are a huge variety of challenges with this approach, but it often suffices and provides motivation for more testing. Well correlated proxies are often a great start to get directional information on where to take further iterations.

Model metrics

So what are the key metrics to track for your model in prod? Given that we’re looking at recommendation systems at inference time, we should seek to understand:

1. distribution of recommendation across categorical features

2. distribution of affinity scores

3. number of candidates

4. distribution of other ranking scores

As we discussed before, during the training process, we should be calculating broadly what the ranges of our similarity scores are in our latent space. Whether we are looking at high level estimations, or finer ones, we can use these distributions to get warning signals something might be strange. Simply comparing the output of our model during inference, or over a set of inference requests, to these pre-compute distributions can be extremely helpful.

Comparing distributions can be a long topic, but one standard approach is KL-Divergence between the observed distribution, and the expected distribution from training. By computing KL-Divergence between these, we can understand how surprising the model’s predictions are on a given day.

What we’d really like is to understand the ROC(Receiver Operating Curve of your model predictions with respect to one of your conversion types. However, this involves yet another integration to tie back to logging. Since your model API only produces the recommendation, you’ll still need to tie into logging from the web application to understand outcomes! To tie back in outcomes one must join the model predictions with the logging output to get the evaluation labels, which can be done via log parsing technologies (like graphana, ELK, or Prometheus). We’ll see more of this in the next chapter.

Deployment topologies

Let’s consider a few different structures for how you want to deploy models to not only keep your models well in tune, but to accommodate iteration, experimentation, and optimization.

Daily warm-starts

As we’ve now discussed several times, there needs be a connection between the continuous output of our model and retraining. The first simplest example of this is daily warm-starts, which essentially asks us to utilize the new data seen each day in our system.

As might already be obvious, some of the recommendation models that show great success are quite large. Retraining some of them can be a massive undertaking, and simply rerunning everything each day is often infeasible. So what can be done?

Let’s ground this conversation in the user-user collaborative filtering example that we’ve been sketching out; we have seen that the first step was to build an embedding via our similarity definition. Let’s recall:

here we remember that the similarity between two users is dependent on the shared ratings, and by each user’s average rating.

On a given day, let’s consider $\tilde{X}=\left.\tilde{x}\mid x\text{was}\text{rated}\text{since}\text{yesterday}\text{by}\text{some}\text{user}\right\}$, then we need to update our user similarities, but ideally we’d leave everything else the same. To update the above, we see that all $\tilde{x}$ rated by two users, $A$ and $B$${\overline{r}}_A$ and ${\overline{r}}_B$ would need to change, but we could probably skip these updates in many cases where the number of ratings by those users was large. All in all, this means for each $\tilde{x}$ we should look up which users previously rated $x$ and update the user similarity between them and the new rater.

This is a bit ad-hoc, but for many methods you can utilize these tricks to reduce a full retraining. This would avoid a full batch retraining, via a fast-layer. There are other approaches to this like building a separate model that can approximate recs for low-signal items. This can be done via feature models, and can significantly reduce the complexity of these quick re-trainings.

Logging

Logging has come up several times already. In model monitoring and our schema assumptions sections we saw that logging was very important for insuring that our system was behaving as expected. Let’s discuss some best practices for logging, and how they fit into our plans.

When we discussed traces and spans earlier, we were able to get a snapshot of the entire call-stack of the services involved in responding to a request. This concept of linking together the services to see the larger picture is incredibly useful, and when it comes to logging gives us a hint as to how we should be orienting our thinking. Returning to our favorite RecSys architecture we have:

1. Collector receiving the request and looking up the embedding relevant to the user

2. Computing ANN on items for that vector

3. Applying filters via blooms to eliminate potential bad recs

4. Feature augmentation to the candidate items and user via the feature stores

5. Scoring of candidates via the ranking model, and potential confidence estimation

6. Ordering and application of business logic or experimentation

Each of the above have potential applications of logging, but let’s now think about how to link them together. The relevant concept from microservices is that of correlation IDs which is simply an identifier that’s passed along the call stack to ensure the ability to link everything later. As is likely obvious at this point, each of these services will be responsible for their own logging, but they’re almost always more useful in aggregate.

These days, Kafka is often used as the log-stream processor to listen for logs from all of the services in your pipeline, and manage their processing and storing. Kafka relies on a message-based architecture where each service is a producer and Kafka helps manage those messages to consumer channels. In terms of log management, the Kafka cluster receives all of the logs in the relevant formats, hopefully augmented with correlation IDs, and sends them off to what’s called an ELK stack. The ELK stack–Elastic Search, LogStash, Kibana–consists of a Logstash component to handle incoming log streams and apply structured processing, Elastic Search which builds search indices to the log store, and Kibana which adds a UI and high level dashboarding to the logging.

This stack of technologies is focused at ensuring you have access and observability from your logs, and there are others that focus on other aspects, but what should you be logging?

Active Learning

So far we have discussed using updating data to train on a much more frequent schedule, and we’ve discussed how to provide good recommendations, even when the model hasn’t seen enough data for those entities. However, there’s a further opportunity for the feedback loop of recommendation and rating, and that’s active learning.

We wont be able to go deep into the topic, which is a large and active field of research, but we will discuss the core ideas in relation to recommendation systems. Active learning changes the learning paradigm a bit by suggesting that the learner should not only be passively collecting labeled–maybe implicit–observations, but also be attempting to mine relations and preferences from them. Active learning determines what data and observations would be most useful in improving model performance, and then seeks out those labels. In the context of RecSys, we know that the Matthew Effect is one of our biggest challenges, in that many of potentially good matches for a user may be lacking enough or appropriate ratings to bubble to the top during the recommendations.

What if we employed a simple policy: every new item to the store gets recommended as a second option to the first 100 customers. The outcome would be two things:

1. we would quickly establish some data for our new item to help cold-start it

2. we would likely decrease the performance of our recommender.

In many cases, the second outcome is worth enduring to achieve the first, but when? And is this the right way to approach this problem? Active learning provides a methodical approach to these problems.

Another more specific advantage of active learning schemes is that you can broaden the distribution of observed data. In addition to just cold-start items, one can use active learning to target broadening users interests. This is usually framed as an uncertainty reduction technique, as it can be used to improve the confidence in recommendations in a broader range of item categories. A simple example here would be if a user only ever shops Sci-Fi books, so one day you show them a few extremely well liked Westerns to ascertain if that user might be open to getting Westerns recommended to them some times. “Propensity Weighting for Recommendation System Evaluation”

An active learning system is instrumented as a loss function inherited from the model it’s trying to enhance–usually tied to uncertainty in some capacity–and it’s attempting to minimize that loss. Given a model $ℳ$ trained on a set of observations and labels $\left.x_i,y_i\right\}$, with loss $ℒ$, then an active learner seeks to find a new observation, $\overline{x}$, such that if a label was obtained, $\overline{y}$, the loss would decrease via the model’s training including this new pair. In particular, the goal is to approximate the marginal reduction in loss due to each possible new observation, and find the observation which maximizes that reduction in the loss function:

The structure of an active learning system roughly follows the following steps:

1. Estimate marginal decrease in loss due to obtaining one of a set of observations

2. Select the one with the largest effect

3. Query the user; i.e. provide the recommendation to obtain a label

4. Update the model.

It’s probably clear that this paradigm requires a much faster training loop than our previous fast retraining schemes. Active learning can be instrumented in the same infrastructure as our other setups, or have its own mechanisms for integration into the pipeline.