In the last chapter, you learned how to assess the risk of a particular algorithm being biased. The conclusion was that in many situations, we may find that a certain risk of algorithmic bias is present but that based on a cost-benefit analysis, the algorithm will still make better decisions than other approaches (such as even more biased humans). This situation can be compared to a life-saving medicine with serious side-effects. Just as the doctor will try to find ways to alleviate the side-effects of a medication, in this chapter we will discuss what steps you can take to protect yourself from algorithmic bias.

How should laymen deal with algorithms in order to avoid problems from algorithmic biases? The most important line of defense for laymen is to use algorithms in an informed fashion.

By analogy, consider how an informed consumer thinks about buying groceries and consuming food: few of us are professional nutritionists or have the necessary medical and biological knowledge to understand the health risks of specific food items. Even worse, if you follow the news flow on the “latest” insights on what is good or bad for you, you sometimes cannot resist the impression that even science is still figuring out what specific food items do to our bodies. However, there are broad and basic rules of thumbs that help us to nevertheless make informed decisions. For example, we know that fiber is good but that excessive sugar is bad, and we are aware that balanced nutrition is a good hedge against major nutritional missteps.

What a consumer should not do is to blindly buy whatever tastes good—we know that we are almost guaranteed to end up with too much junk food—and we also know that we should not be too naïve about advertising. A critical habit of informed consumers therefore is reading—of product labels and also of the occasional article on the subject.

By reading this book, you are obviously already on a golden path towards becoming an informed user of algorithms. Specifically, I would like to urge you to adopt three habits:

First, ask, ask, ask. Pepper your data scientists with curious questions. Curious questions are questions that purely help your understanding. Don’t assume that your data scientist is evil or hiding facts from you—that only will poison the relationship and tempt your data scientist to become defensive—but to the contrary, let your natural curiosity drive the conversation. Ask how exactly to read the output of the algorithm, and honestly marvel at the fact that the new credit score can pinpoint a group of ultra-safe companies that only have a 0.03% probability of not repaying their debt. Ask how the credit score can achieve this without reading your carefully crafted credit memos, and pick your data scientist’s brain about what might happen if a company forges its financials. Most importantly, ask matter-of-factly what biases the algorithms might have, in which situations you should be particularly careful with the outputs, and what it may take to derail the algorithm. What you want to achieve through this is to help your data scientist to better understand the challenges the algorithm might encounter in the real world and what might be the greatest weaknesses of the algorithm. This allows the data scientist not only to think of improvements for the next version of the score but also to better monitor the algorithm and to protect business outcomes by suggesting specific constraints on the use of the score. The more your data scientist sees you as a partner rather than a critic or enemy, the more productive this relationship will be. And don’t pretend to be a data scientist yourself or be scared off by not being one—your value add is exactly that you approach problems from a totally different, non-technical perspective that is therefore complementary to the data scientist’s statistical perspective.¹

Second, understand for which cases the algorithm has an insufficient basis for a useful prediction, and ask the data scientist to assign a “don’t know” label to them instead of an estimate. It is actually a potentially fatal design issue that many algorithms will always provide you their best estimate (even if it is just the population mean or a random number) rather than honestly stating “I don’t know.”² Labeling certain classes of cases as “don’t know” is a double smart move. On the one hand, it enables superior hybrid decision processes—for example, these cases lend themselves to some human intervention or a conservative decision rule. On the other hand, it also prevents human users to be biased (anchored) by an unreliable estimate—as soon as an algorithm provides an estimate, there is a risk that this number develops a life of its own and will subconsciously bias any overriding human judgment even if it is perfectly clear that this number is purely random. Often that effect is amplified by formatting—an output of 2.47% looks very precise even if it is simply the population mean. Algorithms don’t even round their wildest guesses! The only output that can appropriately caution human users is “I don’t know.”

Third, ask to regularly see meaningful monitoring reports, and if you believe that additional metrics would help you in assessing whether everything is working, try to add them to the report. Just as thoughtful nutrition should be complemented by regular health checks, and high cholesterol results trigger a review of your eating habits, the factual nature of monitoring is invaluable in detecting algorithmic bias (as well as many other problems of algorithms). In fact, monitoring is so important that in the next chapter, we will review in detail how to monitor algorithms. Just remember that these monitoring reports must be meaningful. This means, first of all, that all metrics actually need to mean something to you (otherwise they just waste your time); secondly, that taken together they should be rather comprehensive; and finally that the report should draw your attention to the important insights—it doesn’t help if the one metric screaming “bias!” is buried amongst a thousand other metrics that all look good. Good reports therefore are designed to draw your attention in particular to two types of situations: metrics that are outside of the value range considered “safe” or “OK,” and metrics that show big changes. This is actually also how our brain constantly monitors for dangers—it looks for unusual things (you already encountered the bizarreness effect) and it looks out for big changes. The experience of one of my banking clients illustrates the power of monitoring: one very meaningful metric for home equity loan performance, the scaling factor of so-called vintage curves, showed big changes already in 2005—a full two years before the global financial crisis broke out. This metric was the equivalent of an early warning signal for tsunamis (and will be revisited in the next chapter when we discuss calibration analysis).

Taken together, these three practices not only render users of algorithms informed about risks from algorithmic bias without requiring any particular technical knowledge but also drive to specific solutions.

Understanding the specific risks and limitations of algorithmic biases through conversations with data scientists (e.g., data weaknesses) can inform precautions taken on the business side (such as manual review of certain types of cases and other limitations on the use of the algorithm) as well as the modeling side (such as specific corrections to the data, removal of specific variables from the equation, or targeted monitoring of particular metrics). And effective monitoring helps users detect new biases arising or existing biases growing.

One final note: Among the three habits recommended, the second one (getting the algorithm to admit if it really does not know) is the least common practice. This is rather ironic given that Socrates remarked that he knew more than his fellow Athenians because he knew what he did not know (“Έν οἶδα ὅτι οὐδὲν οἶδα,” “I know that I know nothing”)—as opposed to others not even knowing what they don’t know. Hopefully this book will change this!