2.1.1 The overall wage gap and the structural form

Our identification results for the aggregate decomposition are very general, and hold for any distributional statistic.⁷ Accordingly, we focus on general distributional measures in this subsection of the chapter.

Consider the case where the distributional statistic of interest is , where is a real-valued functional, and where is a class of distribution functions such that if , . The distribution function  represents the distribution of the (potential) outcome for workers in group . is an observed distribution when , and a counterfactual distribution when .

The overall -difference in wages between the two groups measured in terms of the distributional statistic is

     (2)

The more common distributional statistics used to study wage differentials are the mean and the median. The wage inequality literature has focused on the variance of log wages, the Gini and Theil coefficients, and the differentials between the 90th and 10th percentiles, the 90th and 50th percentiles, and the 50th and 10th percentiles. These latter measures provide a simply way of distinguishing what happens at the top and bottom end of the wage distribution. Which statistic is most appropriate depends on the problem at hand.

A typical aim of decomposition methods is to divide , the -overall wage gap between the two groups, into a component attributable to differences in the observed characteristics of workers, and a component attributable to differences in wage structures. In our setting, the wage structure is what links observed characteristics, as well as some unobserved characteristics, to wages.

The decomposition of the overall difference into these two components depends on the construction of a meaningful counterfactual wage distribution. For example, counterfactual states of the world can be constructed to simulate what the distribution of wages would look like if workers had different returns to observed characteristics. We may want to ask, for instance, what would happen if group workers were paid like group workers, or if women were paid like men? When the two groups represent different time periods, we may want to know what would happen if workers in year 2000 had the same characteristics as workers in 1980, but were still paid as in 2000. A more specific counterfactual could keep the return to education at its 1980 level, but set all the other components of the wage structure at their 2000 levels.

As these examples illustrate, counterfactuals used in decompositions often consist of manipulating structural wage setting functions (i.e. the wage structure) linking the observed and unobserved characteristics of workers to their wages for each group. We formalize the role of the wage structure using the following assumption:

Assumption 2 ( Structural Form) A worker belonging to either group or is paid according to the wage structure, and , which are functions of the worker’s observable () and unobservable () characteristics:

     (3)

where has a conditional distribution given , and .

While the wage setting functions are very general at this point, the assumption implies that there are only three reasons why the wage distribution can differ between group  and . The three potential sources of differences are (i) differences between the wage setting functions and , (ii) differences in the distribution of observable () characteristics, and (iii) differences in the distribution of unobservable () characteristics. The aim of the aggregate decomposition is to separate the contribution of the first factor (differences between and ) from the two others.

When the counterfactuals are based on the alternative wage structure (i.e. using the observed wage structure of group as a counterfactual for group ), decompositions can easily be linked to the treatment effects literature. However, other counterfactuals may be based on hypothetical states of the world, that may involve general equilibrium effects. For example, we may want to ask what would be the distribution of wages if group workers were paid according to the pay structure that would prevail if there were no workers, for example if there were no union workers. Alternatively, we may want to ask what would happen if women were paid according to some non-discriminatory wage structure (which differs from what is observed for either men or women)?

We use the following assumption to restrict the analysis to the first type of counterfactuals.

Assumption 3 ( Simple Counterfactual Treatment) A counterfactual wage structure, , is said to correspond to a simple counterfactual treatment when it can be assumed that for workers in group B, or for workers in group A.

It is helpful to represent the assumption using the potential outcomes framework introduced earlier. Consider ,where indicates the potential outcome, while indicates group membership. For group , the observed wage is , while represents the counterfactual wage. For group , is the observed wage while the counterfactual wage is . Note that we add the superscript C to highlight counterfactual wages. For instance, consider the case where workers in group are unionized, while workers in group are not unionized. The dichotomous variable  indicates the union status of workers. For a worker in the union sector (), the observed wage under the “union” treatment is , while the counterfactual wage that would prevail if the worker was not unionized is , . An alternative counterfactual could ask what would be the wage of a non-union worker if this worker was unionized , . We note that the choice of which counterfactual to choose is analogous to the choice of reference group in standard OB decomposition. ⁸

What Assumption 3 rules out is the existence of another counterfactual wage structure such as that represents how workers would be paid if there were no unions in the labor market. Unless there are no general equilibrium effects, we would expect that , and, thus, Assumption 3 to be violated.

2.1.2 Four decomposition terms

With this setup in mind, we can now decompose the overall difference into the four following components of interest:

D.1 Differences associated with the return to observable characteristics under the structural functions. For example, one may have the following counterfactual in mind: What if everything but the return to  was the same for the two groups?

D.2 Differences associated with the return to unobservable characteristics under the structural functions. For example, one may have the following counterfactual in mind: What if everything but the return to was the same for the two groups?

D.3 Differences in the distribution of observable characteristics. We have here the following counterfactual in mind: What if everything but the distribution of  was the same for the two groups?

D.4 Differences in the distribution of unobservable characteristics. We have the following counterfactual in mind: What if everything but the distribution of  was the same for the two groups?

Obviously, because unobservable components are involved, we can only decompose into the four decomposition terms after imposing some assumptions on the joint distribution of observable and unobservable characteristics. Also, unless we make additional separability assumptions on the structural forms represented by the functions, it is virtually impossible to separate out the contribution of returns to observables from that of unobservables. The same problem prevails when one tries to perform a detailed decomposition in returns, that is, provide the contribution of the return to each covariate separately.

2.1.3 Imposing identification restrictions: overlapping support

The first assumption we make to simplify the discussion is to impose a common support assumption on the observables and unobservables. Further, this assumption ensures that no single value of or can serve to identify membership into one of the groups.

Assumption 4 ( Overlapping Support) Let the support of all wage setting factors be . For all in , .

Note that the overlapping support assumption rules out cases where inputs may be different across the two wage setting functions. The case of the wage gap between immigrant and native workers is an important example where the vector may be different for two groups of workers. For instance, the wage of immigrants may depend on their country of origin and their age at arrival, two variables that are not defined for natives. Consider also the case of changes in the wage distribution over time. If group consists of workers in 1980, and group of workers in 2000, the difference in wages over time should take into account the fact that many occupations of 2000, especially those linked to information technologies, did not even exist in 1980. Thus, taking those differences explicitly into account could be important for understanding the evolution of the wage distribution over time.

The case with different inputs can be formalized as follows. Assume that for group , there is a vector of observable and unobservable characteristics that may include components not included in the vector of characteristics for group , where and denote the length of the and vectors, respectively. Define the intersection of these characteristics by the vector , which represent characteristics common to both groups. The respective complements, which are group-specific characteristics, are denoted by tilde as and , such that and .

In that context, the overlapping support assumption could be restated by letting the support of all wage setting factors be . The overlapping support assumption would then guarantee that, for all in , . The assumption rules out the existence of the vectors and .

In the decomposition of gender wage differentials, it is not uncommon to have explanatory variables for which this condition does not hold. Black et al. (2008) and Ñopo (2008) have proposed alternative decompositions based on matching methods to address cases where they are severe gaps in the common support assumption (for observables). For example, Ñopo (2008) divides the gap into four additive terms. The first two are analogous to the above composition and wage structure effects, but they are computed only over the common support of the distributions of observable characteristics, while the other two account for differences in support.

2.1.4 Imposing identification restrictions: ignorability

We cannot separate out the decomposition terms (D.1) and (D.2) unless we impose some separability assumptions on the functional forms of and . For highly complex nonlinear functions of observables and unobservables , there is no clear definition of what would be the component of the functions associated with either or . For instance, if and  represent years of schooling and unobserved ability, respectively, we may expect the return to schooling to be higher for high ability workers. As a result, there is an interaction term between or in the wage equation , which makes it hard to separate the contribution of these two variables to the wage gap.

Thus, consider the decomposition term D.1* that combines (D.1) and (D.2):

D.1* Differences associated with the return to observable and unobservable characteristics in the structural functions.

This decomposition term solely reflects differences in the functions. We call this decomposition term , or the “-wage structure effect” on the “-overall difference”, . The key question here is how to identify the three decomposition terms (D.1*), (D.3) and (D.4) which, under Assumption 4, fully describe ?

We denote the decomposition terms (D.3) and (D.4) as and , respectively. They capture the impact of differences in the distributions of and between groups and on the overall difference, . We can now write

Without further assumptions we still cannot identify these three terms. There are two problems. First, we have not imposed any assumption for the identification of the functions, which could help in our identification quest. Second, we have not imposed any assumption on the distribution of unobservables. Thus, even if we fix the distribution of covariates to be the same for the two groups, we cannot clearly separate all three components because we do not observe what would happen to the unobservables under this scenario.

Therefore, we need to introduce an assumption to make sure that the effect of manipulations of the distribution of observables will not be confounded by changes in the distribution of . As we now show formally, the assumption required to rule out these confounding effects is the well-known ignorability, or unconfoundedness, assumption.

Consider a few additional concepts before stating our main assumption. For each member of the two groups , an outcome variable and some individual characteristics are observed. and have a conditional joint distribution, , and is the support of .

The distribution of is defined using the law of iterated probabilities, that is, after we integrate over the observed characteristics we obtain

     (4)

We can construct a counterfactual marginal wage distribution that mixes the conditional distribution of given and using the distribution of . We denote that counterfactual distribution as , which is the distribution of wages that would prevail for group workers if they were paid like group workers. This counterfactual distribution is obtained by replacing with  (or with ) in Eq. (4):

     (5)

These types of manipulations play a very important role in the implementation of decomposition methods. Counterfactual decomposition methods can either rely on manipulations of , as in DiNardo et al. (1996), or of , as in Albrecht et al. (2003) and Chernozhukov et al. (2009).⁹

Back to our union example, represents the conditional distribution of wages observed in the union sector, while represents the conditional distribution of wages observed in the non-union sector. In the case where , Eq. (4) yields, by definition, the wage distribution in the union sector where we integrate the conditional distribution of wages given over the marginal distribution of in the union sector, . The counterfactual wage distribution is obtained by integrating over the conditional distribution of wages in the non-union sector instead (Eq. (5)). It represents the distribution of wages that would prevail if union workers were paid like non-union workers.

The connection between these conditional distributions and the wage structure is easier to see when we rewrite the distribution of wages for each group in terms of the corresponding structural forms,

Conditional on , the distribution of wages only depends, therefore, on the conditional distribution of , and the wage structure .¹⁰ When we replace the conditional distribution in the union sector, , with the conditional distribution in the non-union sector, , we are replacing both the wage structure and the conditional distribution of . Unless we impose some further assumptions on the conditional distribution of , this type of counterfactual exercise will not yield interpretable results as it will mix differences in the wage structure and in the distribution of .

To see this formally, note that unless has the same conditional distribution across groups, the difference

     (6)

will mix differences in functions and differences in the conditional distributions of given .

We are ultimately interested in a functional (i.e. a distributional statistic) of the wage distribution. The above result means that, in general, . The question is under what additional assumptions will the difference between a statistic from the original distribution of wages and the counterfactual distribution, , solely depend on differences in the wage structure? The answer is that under a conditional independence assumption, also known as ignorability of the treatment in the treatment effects literature, we can identify and the remaining terms and .

Assumption 5 ( Conditional Independence/Ignorability) For , let have a joint distribution. For all in : is independent of given or, equivalently, .

In the case of the simple counterfactual treatment, the identification restrictions from the treatment effects literature may allow the researcher to give a causal interpretation to the results of the decomposition methodology as discussed in Section 2.3. The ignorability assumption has become popular in empirical research following a series of papers by Rubin and coauthors and by Heckman and coauthors.¹¹ In the program evaluation literature, this assumption is sometimes called unconfoundedness or selection on observables, and allows identification of the treatment effect parameter.

2.1.5 Identification of the aggregate decomposition

We can now state our main result regarding the identification of the aggregate decomposition

Proposition 1 ( Identification of the Aggregate Decomposition) UnderAssumption 3(simple counterfactual),4(overlapping support), and5(ignorability), the overall -
gap, can be written as

where

(i) the wage structure term solely reflects the difference between the structural functions and

(ii) the composition effect term solely reflects the effect of differences in the distribution of characteristics (  and ) between the two groups.

This important result means that, under the ignorability and overlapping assumptions, we can give a structural interpretation to the aggregate decomposition that is formally linked to the underlying wage setting models, and . Note also that the wage structure () and composition effect () terms represent algebraically what we have informally defined by terms D.1* and D.3.

As can be seen from Eq. (6), the only source of difference between and  is the difference between the structural functions and . Now note that under Assumptions 4 and 5, we have that , where

Thus, reflects only changes or differences in the distribution of observed covariates. As a result, under Assumptions 4 and 5, we identify by and set . This normalization makes sense as a result of the conditional independence assumption: no difference in wages will be systematically attributed to differences in distributions of once we fix these distributions to be the same given . Thus, all remaining differences beyond are due to differences in the distribution of covariates captured by .

Combining these two results, we get

     (7)

which is the main result in Proposition 1.

When the Assumption 3 (simple counterfactual) and 5 (ignorability) are satisfied, the conditional distribution of  given remains invariant under manipulations of the marginal distribution of . It follows that Eq. (5) represents a valid counterfactual for the distribution of  that would prevail if workers in group were paid according to the wage structure . The intuition for this result is simple. Since , manipulations of the distribution of can only affect the conditional distribution of given if they either (i) change the wage setting function , or (ii) change the distribution of given . The first change is ruled out by the assumption of a simple counterfactual treatment (i.e. no general equilibrium effects), while the second effect is ruled out by the ignorability assumption.

In the inequality literature, the invariance of the conditional distribution is often introduced as the key assumption required for to represent a valid counterfactual (e.g. DiNardo et al., 1996; Chernozhukov et al., 2009).

Assumption 6 ( Invariance of Conditional Distributions) The construction of the counterfactual wage distribution for workers of group that would have prevailed if they were paid like group workers (described in Eq. (5)), assumes that the conditional wage distribution applies or can be extrapolated for , that is, it remains valid when the marginal distribution replaces .

One useful contribution of this chapter is to show the economics underneath this assumption, i.e. that the invariance assumption holds provided that there are no general equilibrium effects (ruled out by Assumption 3) and no selection based on unobservables (ruled out by Assumption 5).

Assumption 6 is also invoked by Chernozhukov et al. (2009) to perform the aggregate decomposition using the following alternative counterfactual that uses group as the reference group. Let be the distribution of wages that would prevail for group workers under the conditional distribution of wages of group workers. In our union example, this would represent the distribution of wages of non-union workers that would prevail if they were paid like union workers. Relative to Eq. (7), the terms of the decomposition equation are now inverted:

Now the first term is the composition effect and the second term the wage structure effect.

Whether the assumption of the invariance of the conditional distribution is likely to be satisfied in practice depends on the economic context. If group were workers in 2005 and group were workers in 2007, perhaps Assumption 6 would be more likely to hold than if group were workers in 2007 and group were workers in 2009 in the presence of the 2009 recession. Thus it is important to provide an economic rationale to justify Assumption 6 in the same way the choice of instruments has to be justified in terms of the economic context when using an instrumental variable strategy.

2.1.6 Why ignorability may not hold, and what to do about it

The conditional independence assumption is a somewhat strong assumption. We discuss three important cases under which it may not hold:

1. Differential selection into labor market. This is the selection problem that Heckman (1979) is concerned with in describing the wage offers for women. In the case of the gender pay gap analysis, it is quite plausible that the decisions to participate in the labor market are quite different for men and women. Therefore, the conditional distribution of may be different from the distribution of . In that case, both the observed and unobserved components may be different, reflecting the fact that men participating in the labor market may be different in observable and unobservable ways from women who also participate. The ignorability assumption does not necessarily rule out the possibility that these distributions are different, but it constrains their relationship. Ignorability implies that the joint densities of observables and unobservables for groups and (men and women) have to be similar up to a ratio of conditional probabilities:

2. Self-selection into groups A and B based on unobservables. In the gender gap example there is no selection into groups, although the consequences of differential selection into the labor market are indeed the same. An example where self-selection based on unobservables may occur is in the analysis of the union wage gap. The conditional independence or ignorability assumption rules out selection into groups based on unobservable components beyond . However, the ignorability assumption does not impose that , so the groups may have different marginal distributions of . But if selection into groups is based on unobservables, then the ratio of conditional joint densities will in general depend on the value of being evaluated, and not only on , as ignorability requires:

3. Choice of  and . In the previous case, the values of and are not determined by group choice, although they will be correlated and may even explain the choice of the group. In the first example of the gender pay gap, values of and such as occupation choice and unobserved effort may also be functions of gender ‘discrimination’. Thus, the conditional independence assumption will not be valid if is a function of , even holding constant. The interpretation of ignorability here is that given the choice of , the choice of will be randomly determined across groups. Pursuing the gender pay gap example, fixing (for example education), men and women would exert the same level of effort. The only impact of anticipated discrimination is that they may invest differently in education.

In Section 6, we discuss several solutions to these problems that have been proposed in the decomposition literature. Those include the use of panel data methods or standard selection models. In case 2 above, one could also use instrumental variable methods to deal with the fact that the choice of group is endogenous. One identification issue we briefly address here is that IV methods would indeed yield a valid decomposition, but only for the subpopulation of compliers.

To see this, consider the case where we have a binary instrumental variable , which is independent of conditional on , where is a categorical variable which indicates ‘type’. There are four possible types: ,, and as described below:

Assumption 7 ( LATE) For , let have a joint distribution in . We define , a random variable that may take on four values , and that can be constructed using and according to the following rule: if and , then ; if and , then ; if and , then ; if and , then .

(i) For all in : is independent of .

(ii) .

These are the LATE assumptions from Imbens and Angrist (1994), which allow us to identify the counterfactual distribution of . We are then able to decompose the -wage gap under that less restrictive assumption, but only for the population of compliers:

2.2 Case 2: The detailed decomposition

One convenient feature of the aggregate decomposition is that it can be performed without any assumption on the structural functional forms, , while constraining the distribution of unobserved () characteristics.¹² Under the assumptions of Proposition 1, the composition effect component reflects differences in the distribution of , while the wage structure component  reflects differences in the returns to either or .

To perform a detailed decomposition, we need to separate the respective contributions of or in both and , in addition to separating the individual contribution of each element of the vector of covariates . Thus, generally speaking, the identification of an interpretable detailed decomposition involves stronger assumptions such as functional form restrictions and/or further restrictions on the distribution of , like independence with respect to and .

Since these restrictions tend to be problem specific, it is not possible to present a general identification theory as in the case of the aggregate decomposition. We discuss instead how to identify the elements of the detailed decomposition in a number of specific cases. Before discussing these issues in detail, it is useful to state what we seek to recover with a detailed decomposition.

Property 1 ( Detailed Decomposition) A procedure is said to provide a detailed decomposition when it can apportion the composition effect, , or the wage structure effect, , into components attributable to each explanatory variable:

1. The contribution of each covariate to the composition effect, , is the portion of  that is only due to differences between the distribution of in groups and . When , the detailed decomposition of the composition effect is said to add up.

2. The contribution of each covariate to the wage structure effect, , is the portion of that is only due to differences in the parameters associated with in group and , i.e. to differences in the parameters of and linked to . Similarly, the contribution of unobservables to the wage structure effect, , is the portion of that is only due to differences in the parameters associated with in and .

Note that unobservables do not make any contribution to the composition effect because of the ignorability assumption we maintain throughout most of the chapter. As we mentioned earlier, it is also far from clear how to divide the parameters of the functions and into those linked to a given covariate or to unobservables. For instance, in a model with a rich set of interactions between observables and unobservables, it is not obvious which parameters should be associated with a given covariate. As a result, computing the elements of the detailed decomposition for the wage structure involves arbitrary choices to be made depending on the economic question of interest.

The adding-up property is automatically satisfied in linear settings like the standard OB decomposition, or the -regression procedure introduced in Section 5.2. However, it is unlikely to hold in non-linear settings when the distribution of each individual covariate  is changed while keeping the distribution of the other covariates unchanged (e.g. in the case discussed in Section 5.3). In such a procedure “with replacement” we would, for instance, first replace the distribution of for group with the distribution of for group , then switch back to the distribution of for group and replace the distribution of instead, etc.

By contrast, adding up would generally be satisfied in a sequential (e.g. “without replacement”) procedure where we first replace the distribution of for group with the distribution of for group , and then do the same for each covariate until the whole distribution of has been replaced. The problem with this procedure is that it would introduce some path dependence in the decomposition since the “effect” of changing the distribution of one covariate generally depends on distribution of the other covariates.

For example, the effect of changes in the unionization rate on inequality may depend on the industrial structure of the economy. If unions have a particularly large effect in the manufacturing sector, the estimated effect of the decline in unionization between, say, 1980 and 2000 will be larger under the distribution of industrial affiliation observed in 1980 than under the distribution observed in 2000. In other words, the order of the decomposition matters when we use a sequential (without replacement) procedure, which means that the property of path independence is violated. As we will show later in the chapter, the lack of path independence in many existing detailed decomposition procedures based on a sequential approach is an important shortcoming of these approaches.

Property 2 ( Path Independence) A decomposition procedure is said to be path independent when the order in which the different elements of the detailed decomposition are computed does not affect the results of the decomposition.

A possible solution to the problem of path dependence suggested by Shorrocks (1999) consists of computing the marginal impact of each of the factors as they are eliminated in succession, and then averaging these marginal effects over all the possible elimination sequences. He calls the methodology the Shapley decomposition, because the resulting formula is formally identical to the Shapley value in cooperative game theory. We return to these issues later in the chapter.

2.3 Decomposition terms and their relation to causality and the treatment effects literature

We end this section by discussing more explicitly the connection between decompositions and various concepts introduced in the treatment effects literature. As it turns out, when the counterfactuals are based on hypothetical alternative wage structures, they can be easily linked to the treatment effects literature. For example: What if group workers were paid according to the wage structure of group ? What if all workers were paid according to the wage structure of group ?

Define the overall average treatment effect () as the difference between average wages if everybody were paid according to the wage structure of group and average wages if everybody were paid according to the wage structure of group . That is:

where switching a worker of from “type ” to “type ” is thought to be the “treatment”.

We also define the average treatment effect on the treated () as the difference between actual average wages of group workers and average wages if group workers were paid according to the pay structure of group . That is:

These treatment effects can be generalized to other functionals or statistics of the wage distribution. For example, define -, the -treatment effect, as

and its version applied to the subpopulation of “treated”, - as

The distributions , and are not observed from data on .¹⁸ Following the treatment effects literature, we could in principle identify these parameters if “treatment” was randomly assigned. This is hardly the case, at least for our examples, and one needs extra identifying restrictions. In fact, we note that ignorability and common support assumptions (which together are termed strong ignorability after (Rosenbaum and Rubin, 1983)) are sufficient to guarantee identification of the previous parameters. For example under strong ignorability, for

Under ignorability, it follows that . Then - and . Reweighting methods, as discussed by DiNardo et al. (1996), Hirano et al. (2003) and Firpo (2007, 2010) have implicitly or explicitly assumed strong ignorability to identify specific -treatment effects.

It is interesting to see how the choice of the reference or base group is related to the treatment effects literature. Consider the treatment effect parameter for the non-treated, -:

Under strong ignorability, we have . Thus, in this case, - and .

We could also consider other decompositions, such as:

where includes the actual wages of group workers and the counterfactual wages of group workers if they were are paid like group workers, and conversely for . In this case, the wage structure effect is -, while the composition effect is the sum .¹⁹

The above discussion reveals that the reference group choice problem is just a matter of choosing a meaningful counterfactual. There will be no right answer. In fact, we see that analogously to the treatment effects literature, where treatment effect parameters are different from each other because they are defined over distinct subpopulations, the many possible ways of performing decompositions will reflect the reference group that we want to emphasize.

We conclude this section by discussing briefly the relationship between causality, structural parameters and decomposition terms. In this section, we show that the decomposition terms do not necessarily rely on the identification of structural forms. Whenever we can identify those structural functions linking observable and unobservable characteristics to wages, we benefit from being able to perform counterfactual analysis that we may not be able to do otherwise. However, that comes at the cost of having to impose either strong independence assumptions, as in the case of nonparametric identification, or restrictive functional form assumptions plus some milder independence assumption (mean independence, for instance) between observables and unobservables within each group of workers.

If we are, however, interested in the aggregate decomposition terms  and , we saw that a less restrictive assumption is sufficient to guarantee identification of these terms. Ignorability is the key assumption here as it allows fixing the conditional distribution of unobservables to be the same across groups. The drawback is that we cannot separate out the wage structure effects associated with particular observable and unobservable characteristics.

The treatment effects literature is mainly concerned with causality. Under what conditions can we claim that although identifiable under ignorability, may have a causal interpretation? The conditions under which we could say that is a causal parameter are very stringent and unlikely to be satisfied in general cases. There are two main reasons for that, in our view.

First, in many cases, “treatment” is not a choice or a manipulable action. When decomposing gender or race gaps in particular, we cannot conceive workers choosing which group to belong to. ²⁰ They may have different labor market participation behavior, which is one case where ignorability may not hold, as discussed in Section 2.1.6. However, workers cannot choose treatment. Thus, if we follow, for example, Holland (1986)’s discussion of causality, we cannot claim that is a causal parameter.

A second reason for failing to assign causality to the pay structure effect is that most of the observable variables considered as our (or unobservables ) are not necessarily pre-treatment variables. ²¹ In fact, may assume different values as a consequence of the treatment. In the treatment effects literature, a confounding variable may have different distributions across treatment groups. But that is not a direct action of the treatment. It should only be a selection problem: People who choose to be in a group may have a different distribution of relative to people who choose to be in the other group. When is affected by treatment, we cannot say that controlling for we will obtain a causal parameter. In fact, what we will obtain is a partial effect parameter, netted from the indirect effect through changes in .

3.1 Basics

Despite its apparent simplicity, there are many important issues of estimation and interpretation in the classic OB decomposition. The goal of the method is to decompose differences in mean wages, , across two groups. The wage setting model is assumed to be linear and separable in observable and unobservable characteristics (Assumption 10):

     (13)

where (Assumption 11). Letting be an indicator of group membership, and taking the expectations over , the overall mean wage gap can be written as

where . Adding and subtracting the average counterfactual wage that group workers would have earned under the wage structure of group , , the expression becomes

Replacing the expected value of the covariates , for , by the sample averages , the decomposition is estimated as

     (14)

     (15)

     (16)

The first term in Eq. (15) is the wage structure effect, , while the second term is the composition effect, . Note that in cases where group membership is linked to some immutable characteristics of the workers, such as race or gender, the wage structure effect has also been called the “unexplained” part of the wage differentials or the part due to “discrimination”.

The OB decomposition is very easy to use in practice. It is computed by plugging in the sample means and the OLS estimates  in the above formula. Various good implementations of the procedure are available in existing software packages.²²Table 2 displays the various underlying elements of the decomposition in the case of the gender wage gap featured in O’Neill and O’Neill (2006) using data from the NLSY79. The composition effect is computed as the difference between the male and female means reported in column (1) multiplied by the male coefficients reported in column (2).²³ The corresponding wage structure effect is computed from the difference between the male and female coefficients reported in columns (2) and (3). The results are reported in column (1) of Table 3. The composition effect accounts for 0.197 (0.018) log points out of the 0.233 (0.015) average log wage gap between men and women in 2000. When the male wage structure is used as reference, only an insignificant 0.036 (0.019) part of the gap (the wage structure effect) is left unexplained.

Table 2 Means and OLS regression coefficients of selected variables from NLSY log wage regressions for workers ages 35-43 in 2000.

Table 3 Gender wage gap: Oaxaca-Blinder decomposition results (NLSY, 2000).

Because of the additive linearity assumption, it is easy to compute the various elements of the detailed decomposition. The wage structure and composition effects can be written in terms of sums over the explanatory variables

     (17)

     (18)

where represents the omitted group effect, and where and represent the th element of and , respectively.  and  are the respective contributions of the th covariate to the composition and wage structure effect. Each element of the sum can be interpreted as the contribution of the difference in the returns to the th covariate to the total wage structure effect, evaluated at the mean value of . Whether or not this decomposition term is economically meaningful depends on the choice of the omitted group, an issue we discuss in detail in Section 3.2 below.²⁴

Similar to O’Neill and O’Neill (2006), Table 3 reports the contribution of single variables and groups of variables to composition (upper panel) and wage structure effects (lower panel). Life-time work experience ‘priced’ at the male returns to experience stands out as the factor with the most explanatory power (0.137 out of 0.197, or 69%) for composition effects. The wage structure effects are not significant in this example, except for the case of industrial sectors which we discuss below.

Because regression coefficients are based on partial correlations, an OB decomposition that includes all explanatory variables of interest satisfies the property of path independence (Property 2). Note, though, that a sequence of Oaxaca-Blinder decompositions, each including a subset of the variables, would suffer from path dependence, as pointed out by Gelbach (2009). Despite these attractive properties, there are some important limitations to the standard OB decomposition that we now address in more detail.

3.2 Issues with detailed decompositions: choice of the omitted group

There are many relevant economic questions that can be answered with the detailed decomposition of the composition effect in Eq. (18). For example, what has been the contribution of the gender convergence in college enrollment to the gender convergence in average pay? There are also some important questions that are based on the detailed decomposition of the wage structure effect . For example, consider the related “swimming upstream” query of Blau and Kahn (1997). To what extent have the increases in the returns to college slowed down the gender convergence in average pay? Or, to what extent has the decline in manufacturing and differences in industry wage premia contributed to that convergence?

Some difficulties of interpretation arise when the explanatory variables of interest are categorical (with more than two categories, or more generally, in the case of scalable variables, such as test scores) and do not have an absolute interpretation. In OB decompositions, categorical variables generate two problems. The first problem is that categorical or scalable variables do not have a natural zero, thus the reference point has to be chosen arbitrarily. The conventional practice is to omit one category which becomes the reference point for the other groups. This generates some interpretation issues even in the detailed decomposition of the composition effect.

Returning to our NLSY example, assume that the industry effects can captured by four dummy variables, to , for the broad sectors: (i) primary, construction, transportation & utilities, (ii) manufacturing, (iii) education and health services & public administration, and (iv) other services. Consider the case where is the omitted category, , and denote by the coefficients from the wage regression, as in column (2) of Table 2. Denote by the coefficients of a wage regression where is the omitted category, , as in column (4) of Table 2, so that, for example, . In our example, given the large difference in the coefficients of manufacturing between columns (2) and (4) of Table 2, this could mistakenly lead one to conclude that the effect of the underrepresentation of women in the manufacturing sector has an effect three times as large in one case (education and health omitted) as in the other case (primary omitted). In the first case, the underrepresentation of women in the manufacturing sector is ‘priced’ at the relative returns in the manufacturing versus the education and health sector, while in the other it is ‘priced’ at the relative returns in the manufacturing versus the primary sector.²⁵

Note, however, that the overall effect of 0.017 (0.006) of gender differences in industrial sectors on the gender wage gap, is the same in columns (1) and (2) of Table 3. To simplify the exposition, consider the special case where industrial sectors are the only explanatory factors in the wage regression. It follows that the composition effect,

     (19)

is unaffected by the choice of omitted category.²⁶

The second problem with the conventional practice of omitting one category to identify the coefficients of the remaining categories is that in the unexplained part of the decomposition one cannot distinguish the part attributed to the group membership (true “unexplained” captured by the difference in intercepts) from the part attributed to differences in the coefficient of the omitted or base category.²⁷ These difficulties with the detailed decomposition of the unexplained part component were initially pointed by Jones (1983) who argued that “this latter decomposition is in most applications arbitrary and uninterpretable” (p. 126). Pursuing the example above, the effect of industry wage differentials on the gender wage gap is given by the right-hand side sums in the following expressions

     (20)

     (21)

where and , . The overall wage structure effect is the same irrespective of the omitted category , as shown in the last row of column (1) and (2) of Table 3. However, the overall effect of differences in the returns to industrial sectors, given by the right hand side sums with either choice of omitted group, −0.092 (0.033) in column (1) and 0.014 (0.028) in column (2), are different because different parts of the effect are hidden in the intercepts [0.128 (0.213) in column (1) and 0.022 (0.212) in column (2)].²⁸

This invariance issue has been discussed by Oaxaca and Ransom (1999), Gardeazabal and Ugidos (2004), and Yun (2005, 2008), who have proposed tentative solutions to it. These solutions impose some normalizations on the coefficients to purge the intercept from the effect of the omitted category, either by transforming the dummy variables before the estimation, or by implementing the restriction, , , via restricted least squares.²⁹ Yun (2005) imposes the constraint that the coefficient on the first category equals the unweighted average of the coefficients on the other categories, along with . While these restrictions may appear to solve the problem of the omitted group, as pointed out by Yun (2008) “some degree of arbitrariness in deriving a normalized equation is unavoidable” (p. 31). For example, an alternative restriction on the coefficients, that goes back to Kennedy (1986), could be a weighted sum, , where the weights reflect the relative frequencies of the categories in the pooled sample. The coefficients would then reflect deviations from the overall sample mean.

The pitfall here is that the normalizations proposed by Gardeazabal and Ugidos (2004) and Yun (2005) may actually leave the estimation and decomposition without a simple meaningful interpretation. Moreover, these normalizations will likely be sample specific and preclude comparisons across studies. By contrast, in the case of educational categories, the common practice of using high school graduates as the omitted category allows the comparison of detailed decomposition results when this omitted category is comparable across studies.

Invariance of the detailed decomposition with respect to the choice of omitted category may appear to be a desirable property, but it is actually elusive and should not come at the expense of interpretability. There is no quick fix to the difficult choice of the appropriate omitted category or base group, which is actually exacerbated in procedures that go beyond the mean. To mimic the case of continuous variables, one may argue that an education category such as less than high school that yields the smallest wage effect should be the omitted one, but this category may vary more across studies than the high school category. Issues of internal logic have to be balanced with comparability across studies.

Another way of reporting the results of counterfactual experiments, proposed in the context of the gender wage gap by industry, is to report the wage structure effects for each category by setting and for in the expression (20) for the total wage structure effect

     (22)

in a case where there are other explanatory variables, , . ³⁰ Initially, such expressions included only the first two terms, the intercept and the effect of the category (Fields and Wolff, 1995). Later, Horrace and Oaxaca (2001) added the wage structure effect associated with the other variables. This allows one to compare the effect of wage structure on gender wage differentials by category while controlling for other explanatory variables , in a way that is invariant to the choice of omitted category. ³¹In columns (1) and (2) of Table 3, the wage structure effect associated with variables other than industrial sectors is essentially zero, and the can be computed as the difference between the male and female coefficients in columns (2) and (3) of Table 2 plus the 0.128 difference in the constant, yielding values of 0.128, 0.022, 0.004, and 0.048 for industries 1 through 4, respectively. Horrace and Oaxaca (2001) also proposed to ex-post normalize the effects of each category with respect to the maximum categorical effect.

One disadvantage of decomposition terms like relative to the usual components of the detailed decomposition is that they do not sum up to the overall wage structure effect. As a result, just looking at the magnitude of the terms gives little indication of their quantitative importance in the decomposition. We propose a normalization to help assess the proportion of the total wage structure effect which can be attributed to a category given that a proportion of group workers belongs to that category, and that is also invariant to the choice of omitted category. The normalization uses the fact that the weighted sum of the , (that is, including the omitted category), is equal to the total wage structure effect, so that the proportional effect of category in the total wage structure can be computed as³²

     (23)

In our empirical example, with group as the reference group, this expression is computed using female averages, thus will tell us the proportion of the total wage structure effect that can be attributed to industrial category given the proportion of women in each category. The numbers are 0.308 for primary, 0.074 for manufacturing, 0.040 for education and health, and 0.578 for other services. Despite being underrepresented in the manufacturing sector, because women’s returns to manufacturing jobs are relatively high, the share of the unexplained gap attributable to that factor turns out not to be that large.

3.3 Alternative choices of counterfactual

On the one hand, the choice of a simple counterfactual treatment is attractive because it allows us to use the identification results from the treatment effects literature. On the other hand, these simple counterfactuals may not always be appropriate for answering the economic question of interest. For instance, the male wage structure may not represent the appropriate counterfactual for the way women would be paid in the absence of labor market discrimination. If the simple counterfactual does not represent the appropriate treatment, it may be more appropriate to posit a new wage structure. For example, in the case of the gender pay gap, typically propositions (Reimers, 1983; Cotton, 1998; Neumark, 1988; Oaxaca and Ransom, 1994) have used a weighted average expression , where corresponds to , corresponds to , and where could reflect a weighting corresponding to the share of the two groups in the population. Another popular choice is the matrix , which captures the sample variation in the characteristics of group and workers. ³³ The decomposition is then based on the triple differences:

Table 3 shows that in the NLSY example, the gender gap decomposition is substantially different when either the female wage structure (column 3) or the weighted sum of the male and female wage structure (column 4) is used as the reference wage structure. Typically (as in Bertrand and Hallock (2001) for example), with the female wage structure as reference, the explained part of the decomposition (composition effect) is smaller than with the male wage structure as reference. Indeed, evaluated at either female ‘prices’ or average of male and female ‘prices’, the total unexplained (wage structure) effect becomes statistically significant.

An alternative measure of “unexplained” differences (see Cain, 1986) in mean wages between group and group workers is given by the coefficient of the group membership indicator variable in the wage regression on the pooled sample, where the coefficients of the observed wage determination characteristics are constrained to be the same for both groups:

     (24)

where the vector of observed characteristics excludes the constant. It follows that,

where . As noted by Fortin (2008), this “regression-compatible” approach is preferable to the one based on a pooled regression that omits the group membership variable (as in Neumark (1988) and Oaxaca and Ransom (1994)), because in the latter case the estimated coefficients are biased (omitted variable bias). Note, however, that this counterfactual corresponds to the case where the group membership dummy is thought to be sufficient to purge the reference wage structure from any group membership effect, an assumption that is maintained in the common practice of using the group membership dummy in a simple regression to assess its effect. The detailed decomposition is obtained using the above triple differences decomposition. ³⁴

The results of this decomposition, reported in column (5) of Table 3, are found to be closest to the one using the female coefficients in column (3), but this is not necessarily always the case. Notice that the magnitude of the total unexplained wage log wage gap 0.092 (0.014) log points corresponds to the coefficient of the female dummy in column (5) of Table 2.

3.4 Reweighted-regression decompositions

A limitation of OB decompositions, discussed by Barsky et al. (2002), is that they may not provide consistent estimates of the wage structure and composition effect when the conditional mean function is non linear. Barsky et al. (2002) look at the role of earnings and other factors in the racial wealth gap. They argue that a standard OB decomposition is inadequate because the wealth-earnings relationship is non linear, and propose a more flexible approach instead.

Under the linearity assumption, the average counterfactual wage that group workers would have earned under the wage structure of group is equal to , and is estimated as the product , a term that appears in both the wage structure and composition effect in Eq. (15). However, when linearity does not hold, the counterfactual mean wage will not be equal to this term.

One possible solution to the problem is to estimate the conditional expectation using non-parametric methods. Another solution proposed by Barsky et al. (2002) is to use a (non-parametric) reweighting approach as in DiNardo et al. (1996) to perform the decomposition. One drawback of this decomposition method, discussed later in the chapter, is that it does not provide, in general, a simple way of performing a detailed decomposition. In the case of the mean, however, this drawback can be readily addressed by estimating a regression in the reweighted sample.

To see this, let be the reweighting function, discussed in Section 4.5, that makes the characteristics of group workers similar to those of group workers. The counterfactual coefficients and the counterfactual mean , are then estimated as:³⁵

where .³⁶ If the conditional expectation of  given was linear, both the weighted and unweighted regressions would yield the same consistent estimate of , i.e. we would have . When the conditional expectation is not linear, however, the weighted and unweighted estimates of  generally differ since OLS minimizes specification errors over different samples. ³⁷

Consider the “reweighted-regression” decomposition of the overall wage gap , where

The composition effect can be divided into a pure composition effect using the wage structure of group , and a component linked to the specification error in the linear model, :

The wage structure effect can be written as

and reduces to the first term as the reweighting error goes to zero in large samples ().

The reweighted-regression decomposition is similar to the usual OB decomposition except for two small differences. The first difference is that the wage structure effect is based on a comparison between  and the weighted estimate  instead of the usual unweighted estimate . As discussed in Firpo et al. (2007), this ensures that the difference  reflects true underlying differences in the wage structure for group and , as opposed to a misspecification error linked to the fact that the underlying conditional expectation is non-linear. Note that is also useful to check whether the reweighting error  is equal to zero (or close to zero), as it should be when the reweighting factor is consistently estimated.

The other difference relative to the OB decomposition is that the composition effects consists of a standard term plus the specification error . If the model was truly linear, the specification error term would be equal to zero. Computing the specification error is important, therefore, for checking whether the linear model is well specified, and adjusting the composition effect in the case where the linear specification is found to be inaccurate.

In the case where the conditional expectation is estimated non-parametrically, a whole different procedure would have to be used to separate the wage structure into the contribution of each covariate. For instance, average derivative methods could be used to estimate an effect akin to the coefficients used in standard decompositions. Unfortunately, these methods are difficult to use in practice, and would not be helpful in dividing up the composition effect into the contribution of each individual covariate.

On a related note, Kline (2009) points out that the standard OB decomposition can be interpreted as a reweighting estimator where the weights have been linearized as a function of the covariates. This suggests that the procedure may actually be more robust to departures from linearity than what has been suggested in the existing literature. Since the procedure is robust to these departures and remains the method of choice when linearity holds, Kline (2009) points out that it is “doubly robust” in the sense of Robins et al. (1994) and Egel et al. (2009).

3.5 Extensions to limited dependent variable models

OB decompositions have been extended to cases where the outcome variable is not a continuous variable. To mention a few examples, Gomulka and Stern (1990) study the changes over time in labor force participation of women in the United Kingdom using a probit model. Even and Macpherson (1990) decomposes the male-female difference in the average probability of unionization, while Doiron and Riddell (1994) propose a decomposition of the gender gap in unionization rate based on a first order Taylor series approximation of the probability of unionization. Fitzenberger et al. (forthcoming) use a probit model to decompose changes over time in the rate of unionization in West and East Germany. Fairlie (1999, 2005) discuss the cases of the racial gaps in self-employment and computer ownership. Bauer and Sinning (2008) discuss the more complicated cases of a count data model, for example where the dependent variable is the number of cigarettes smoked by men and women (Bauer et al., 2007), and of the truncated dependent variable, where for example the outcome of interest is hours of work.

In the case of a limited dependent variable , the conditional expectation of is typically modeled as a non-linear function in , . For example, if is a dichotomous outcome variable () and is a latent variable which is linear in , it follows that  where is the CDF of . When  follows a standard normal distribution, we have a standard probit model and . More generally, under various assumptions regarding the functional form and/or the distribution of the error terms , the models are estimated by maximum likelihood.

Because , the decomposition cannot simply be computed by plugging in the estimated ’s and the mean values of ’s, as in the standard OB decomposition. Counterfactual conditional expectations have to be computed instead, and averaged across observations. For example, if group is thought to be the reference group, will be the counterfactual conditional expectation of that would prevail if the coefficients of the determinants of self-employment (for example) for group were the same as for group . This involves computing predicted (i.e. expected) values based on the estimated model for group , , over all observations in group , and averaging over these predicted values.

The mean gap between group and group is then decomposed as follows

into a component that attributes differences in the mean outcome variable to differences in the characteristics of the individuals, and a component that attributes these differences to differences in the coefficients.

The same difficult issues in the appropriate choice of counterfactuals persist for more general non-linear models. In addition, extra care has to be taken to verify that the sample counterfactual conditional expectation lies within the bounds of the limited dependent variable. For example, Fairlie (1999) checks that average self-employment for Blacks predicted from the White coefficients is not negative.

The non-linear decomposition may perform better than the linear alternative (linear probability model, LPM) when the gap is located in the tails of the distribution or when there are very large differences in the explanatory variables, whose effects would remain unbounded in a LPM. On the other hand, there are many challenges in the computation of detailed decompositions for non-linear models. Because of non-linearity, the detailed decomposition of the two components into the contribution of each variable, even if the decomposition was linearized using marginal effects, would not add up to the total. Gomulka and Stern (1990) and Fairlie (2005) have proposed alternative methodologies based on a series of counterfactuals, where the coefficient of each variable is switched to reference group values in sequence. In the latter cases, the decomposition will be sensitive to the order of the decomposition, that is will be path dependent. We discuss these issues further in the context of the decompositions of entire distributions in Section 5.

3.6 Statistical inference

OB decompositions have long been presented without standard errors. More recently, Oaxaca and Ransom (1998), followed by Greene (2003, p. 53–54), have proposed approximate standard errors based the delta method, under the assumption that the explanatory variables were fixed.³⁸ A more modern approach where, as above, are stochastic was suggested and implemented by Jann (2005). In cases where the counterfactuals are not a simple treatment, or where a non-linear estimator is used, bootstrapping the entire procedure may prove to be the practical alternative.