One tricky example of how variables contribute to an outcome is the case of masking effect variables. Let's create a toy dataset to exemplify this phenomena. Basically, we are creating two independent variables ( and ). They are positively correlated to each other and they are correlated to , but in opposite directions; is positively correlated and  negatively correlated:

np.random.seed(42)
N = 126
r = 0.8
x_1 = np.random.normal(size=N)
x_2 = np.random.normal(x_1, scale=(1 - r ** 2) ** 0.5)
y = np.random.normal(x_1 - x_2)
X = np.vstack((x_1, x_2)).T
scatter_plot(X, y)

As we did before, we are going to build three related models. The first one, m_x1x2, is a linear regression model with two independent variables,  and  (stacked together in the variable X). The second model, m_x1, is a simple linear regression for  and the third one, m_x2, is a simple linear regression for . After sampling from these models, take a look at the  parameters using a forest plot to compare them in a single plot:

az.plot_forest([trace_x1x2, trace_x1, trace_x2],
               model_names=['m_x1x2', 'm_x1', 'm_x2'],
               var_names=['β1', 'β2'],
               combined=True, colors='cycle', figsize=(8, 3))

According to the posterior, the values of  for m_x1x2 are close to 1 and -1 (as expected, according to the way we generate the data). For the simple linear regression model, that is when we study each variable on its own, we can see that the values for  are instead closer to zero, indicating a weaker effect. 

Notice  is correlated to . In fact, when  increases,  also increases. Also notice that when  increases,  also increases, but  decreases. As a result of this particular arrangement, we get a partial cancellation of effects unless we include both variables in the same linear regression. The linear regression model is able to untable these effects because the model is learning for each data point what is the contribution of  to  is given a value of _, and the other way around for .