Chapter 7

Studying Globalization: Methodological Issues

SALVATORE BABONES

INTRODUCTION

Globalization means many things to many people, so many things that it hardly seems worth offering yet one more definition of the term. It would be difficult enough to pick among existing definitions, or even to list them: reviewing trends in 26 indicators of globalization, Guillen (2001) notes that not one has risen as quickly over the past 20 years as the number of academic publications on the topic. Happily for future writers of review articles, a search of the Sociological Abstracts database suggests that this trend has reversed: after peaking at 705 in 2002, the number of peer-reviewed publications returned from a keyword search on globalization declined to 601 in 2003 and 553 in 2004. Perhaps Chase-Dunn and Babones (2006) are right to caution that ‘waves of globalization have been followed by waves of deglobalization in the past, and this is also an entirely plausible scenario for the future’. If academic interest is a leading indicator, this future may come sooner than any of us expect.

Whatever the trend, the sheer number of academic treatments of globalization eliminates the possibility of agreeing on a common definition for the term; a common definition of globalization may not even be desirable from the standpoint of advancing our theorizing on the subject (Smith 2001). Lack of consensus among theorists and practitioners, however, does create difficulties for the methodologist. Ideally, operational definitions of concepts should follow from theoretical definitions, and measurement choices should reflect operational definitions. Without agreement on theory, research tends to be driven by empiricism: concepts are defined to conform to the data that are conveniently available, or simply are not defined at all. The result is the existence of a wide variety of globalization indicators in the empirical literature, none of which can be judged theoretically superior to any other.

Lacking a commonly agreed theoretical definition, individual researchers can (and do) operationalize globalization as they see fit. Many operationalize globalization using country-level economic indicators, such as foreign trade as a proportion of total economic activity. Some take a qualitative case-study approach, operationalizing globalization at a much more local level. A few operationalize globalization temporally, as a sort of all-pervasive force in the world-economy that has existed since 1980 or so. Still others – one suspects the plurality or even the majority of the hundreds of articles mentioned above – use globalization more as a rhetorical backdrop than as a variable to be operationalized and measured.

While the formulation of a general theoretical framework for the study of globalization may be impractical, one broad guideline can be identified: empirical research on globalization should be conceived at a global level of analysis. Level of analysis refers to the scope at which research questions are posed. Even when the evidence adduced to demonstrate an effect of globalization is highly local, research questions must be asked at a global level if the study is to shed any light on globalization as such (Babones 2006). For example, when Macleod (1999) investigates the impact of individual people’s integration into global business networks on gender roles in a small port city in the Canary Islands, the research question is clearly global in scope, despite the extremely local nature of the study.

Important as qualitative case studies are for advancing our understanding of how processes of globalization play out at the level of agency and action, it is difficult to generalize methodological principles about them. In scale they range from localized ethnographic studies like Macleod’s to wide-ranging analyses of transnational social movements (Moghadam 1999). Units of analysis range from the individual person all the way up to the world-system; the methodological tools they employ are equally varied. Quantitative cross-national studies, on the other hand, are more uniform, and thus more tractable from a methodologist’s perspective. They employ a relatively uniform bag of statistical tools to analyse data drawn from a small number of standard sources on similar panels of cases. Thus, with no prejudice to the qualitative case study approach, in what follows I focus on the methodology of quantitative cross-national studies of globalization, though where possible I also highlight opportunities for globally representative qualitative research.

The remainder of this chapter is organized into three sections. In the first section, I describe some of the most important standard data sources used in research on globalization, including sources for the study of economic globalization, cultural globalization and political globalization. In the second section, I discuss the use and misuse of common statistical tools for analysing cross-national panel data, focusing in particular on unintended consequences in various kinds of regression models. In the third section, I consider some more general questions about the nature of countries as cases. I conclude with a few straightforward guidelines for improving future research on globalization.

STANDARD DATA SOURCES

For strictly practical purposes, most cross-national research on globalization relies on published compilations of existing data: it is generally unrealistic for individual researchers to contemplate collecting data that is representative, or even broadly comprehensive, in covering the 200 or so countries of the world. As a result, a relatively small number of data sources have become standard data sources for research dealing with globalization. ‘Generic’ globalization can be subdivided into at least three narrower (but interlaced) processes of globalization: economic globalization, cultural globalization and political globalization (Evans 2005, cited in Lechner 2005). In this section, I examine in turn the standard data sources for studying each of these.

Economic globalization

Three variables have typically been used to operationalize globalization in broadly cross-national panel studies of economic globalization: foreign trade, foreign direct investment and foreign portfolio investment. Foreign trade as a proportion of GDP is the ratio of imports plus exports to total economic activity within a country; when a country is more closely tied into global trade networks, it is more exposed to pressures emanating from the world outside its borders. Foreign direct investment as a proportion of GDP (FDI) is the ratio of active foreign investment (investment that implies some level of management involvement, typically defined as at least 10 per cent of a company) to total economic activity; it tends to vary widely year by year. Foreign portfolio investment as a proportion of GDP (FPI) is the ratio of passive foreign investment (investment that is motivated by speculative gain, with no implications for management control) to total economic activity; it is even more variable than FDI.

In general, foreign trade is more an indicator of exposure to global economic forces, while foreign investment is more an indicator of economic sovereignty. Most empirical studies of globalization as such have operationalized levels of globalization using trade. For example, in their benchmark study tracing the trajectory of world trade over the past two centuries, Chase-Dunn et al. (2000) identify two peaks in the globalization of the world-economy (1880s, 1920s) that predate the current upsurge in world trade. The authors simply equate higher levels of foreign trade with greater globalization. Kim and Shin (2002) focus on the most recent wave of globalization, defining globalization in terms of increasing numbers of partners in foreign trade. Kaplinsky (2001), in a wide review of the costs and benefits of globalization, defines globalization exclusively in terms of the ratio of foreign trade to GDP. Occasional studies, however, also use FDI (Chase-Dunn 1999) or, less often, FPI (Reuveny and Li 2003) alongside foreign trade as indicators of globalization.

That such qualitatively different and only moderately correlated indicators as trade and investment are used interchangeably to operationalize globalization underscores the theoretical ambiguity of the term. In general, any variable that rose over the period 1980–2000 can be used – and probably has been used – as an indicator of globalization. Trade and investment are convenient indicators because they have good face validity (both represent connections between a country and the outside world) and the necessary data are widely available. Both, however, should be used with caution, as explained below.

FOREIGN TRADE

The most common indicator of a country’s level of globalization used in the empirical literature is its ratio of foreign trade to GDP. Foreign trade is the sum of a country’s imports and exports; GDP is discussed in detail below. Foreign trade as a proportion of GDP is most conveniently accessible from the World Bank’s World Development Indicators (WDI) database, which is published annually in April. The WDI reports trade data from 1960 to the year 2 years preceding publication (i.e. WDI 2005 reports data for 1960–2003). Since these data are compiled from figures supplied by national statistical offices, there can be a lag in reporting of as long as 5 years. On the other hand, every year more countries report as statistics improve over time. The net result is that the maximum panel of countries for which data are available is usually for the year 5 or 6 years or so before the last reporting year. In the 2005 WDI, the maximum number of reporting countries is reached in 1997 (199 countries versus 151 for 2003, the most recent year), though the panel is close to this maximum as late as 2002 (183 countries).

As a general rule, one should use the most recent version of the WDI for data from any year 1960 to current, since data in the WDI are always subject to update. There is, however, one major exception to this rule: countries are dropped from WDI when they cease to exist, even for historical data. Thus, for trade figures for East Germany, West Germany, the USSR, united Yugoslavia etc., one must turn to older editions of the WDI; similarly, users should be aware that territorial discontinuities caused by war or separatism may be reflected in WDI data. For pre-1960 data, ad hoc sources, such as Mitchell (1992, 1993, 1995) must be used.

Foreign trade expressed as a proportion of GDP presents some odd qualities. First, though the net trade balance (exports minus imports) is properly a component of GDP, gross foreign trade is not. It is entirely possible for imports plus exports to total more than a country’s total GDP; this is in fact the case for more than 50 countries today. Second, trade as a proportion of GDP has a moderate positive skew, as infinitely positive outliers are theoretically possible (trans-shipment ports such as Hong Kong and Malta score particularly high) but negative outliers are bounded by zero (trade is always positive). Third, small countries engage more (proportionally) in foreign trade than do large countries, simply because they are smaller units (much of what is counted as foreign trade for small countries is equivalent to inter-regional domestic trade in large countries). These complications suggest that foreign trade as a proportion of GDP should not be used to operationalize globalization without careful consideration.

The key to dealing with operationalization issues like these is to theorize what is meant by globalization. If globalization is theorized as the degree to which jobs in a country are exposed to the vagaries of supply and demand on world markets, then exports as a proportion of GDP might be a more appropriate operationalization of the concept than total trade. On the other hand, if globalization is the degree to which consumers are exposed to products from throughout the world, imports might be more appropriate. Imports and exports as proportions of GDP are highly correlated (r ≈ 0.78 between countries, 2000), but it is not clear that they are always manifestations of the same latent concept. In some cases they probably are: if globalization is the degree to which members of a population have contact with the world outside their country, then trade as a proportion of GDP is probably an appropriate operationalization.

The skew in trade as a proportion of GDP can easily be handled by logging the data. For example, using 2000 WDI data, trade as a proportion of GDP has a positive skew of 1.5. The logged series shows a negative skew of less than 0.1. Few researchers bother to log trade data, but both the empirical and the methodological cases for logging are strong. The only question is whether it makes theoretical sense; if globalization is mainly conceived at an ordinal level, one should log the data to remove the skew. If, on the other hand, globalization is conceived at a ratio level, incorporating the implicit claim that each unit of trade as a proportion of GDP produces an equivalent effect in the dependent variable, then trade should not be logged. It is unusual, however, to find social science theories that are truly couched at a ratio level. In most cases, trade should probably be logged.

The solution to the small country problem is more complicated but no less necessary, again subject to theoretical judgment. In some senses, small countries like Belgium (trade as a proportion of GDP = 168 per cent in 2000) are far more exposed to the global economy than are large countries like France (56 per cent in 2000). On the other hand, much of that excess trade in Belgium is ‘local’ foreign trade with neighbouring countries, rather than ‘global’ overseas trade. Taking out the effects of country size would seem appropriate in most cases; as with logging trade to remove skew, it should probably be done in all cases except where an explicit theoretical argument is made not to do it. This can be done by regressing logged foreign trade as a proportion of GDP on logged population size, then using the residual to operationalize trade globalization.

Not only is foreign trade as a proportion of GDP correlated with population size, but that correlation changes over time. Using logged series for both foreign trade and population, the correlation has declined from r= –0.73 in 1980 to r= –0.55 in 2000 for a constant panel of 128 countries; thus, separate regression models must be estimated for each year in the study panel. This observed decline in the correlation of foreign trade with country size is presumably itself a product of globalization: as the world globalized between 1980 and 2000, the ‘global’ portion of total foreign trade was presumably expanding. This would tend to attenuate the correlation between foreign trade and country size, which originates in the ‘local’ portion of total trade.

For some study designs, aggregate trade data are insufficient, and trade flows must be differentiated by partner country. Country to country trade flows are reported by the International Monetary Fund (IMF) in its Direction of Trade Statistics (DOTS) database. The DOTS database reports monthly, quarterly and annual country to country trade flows for 186 countries for the period since 1980. Only the 60 or so largest countries report continuous monthly series, but annual series are available for all countries for most years. Limited pre-1980 data are reported separately in a DOTS historical compendium covering the years 1948–80.

The DOTS database is keyed by country and contains both import and export data for trade between the keyed country and every other country of the world. Since the underlying raw data originate in the individual countries’ statistical offices, the exports recorded from any one country do not necessarily match the imports recorded by the other. Reported imports and exports between country pairs can also be misaligned because the IMF DOTS data have been converted to US dollars, while the underlying raw data are in national currency units. Since the DOTS data are reported in US dollars evaluated at market and official exchange rates, they must be paired with GDP per capita of the same character in order to compute ratios of imports and exports to GDP. These data can be found in the WDI, as described below.

An even more detailed source of trade data between pairs of countries is the United Nations (UN) COMTRADE database, which records each reporting country’s imports and exports by industry classification, beginning in 1962. Over 120 countries, including most major trading countries, currently report trade data to COMTRADE. Industry-specific data are available at the five-digit Standard International Trade Classification (SITC) level, though not all countries report in such detail for all periods. As with the IMF DOTS database, data are keyed by reporting country, so figures reported by pairs of countries may not correspond. Also, as with the DOTS data, COMTRADE statistics should be paired with GDP evaluated at market and official exchange rates to compute ratios.

FOREIGN INVESTMENT

Like foreign trade, foreign investment (FDI or FPI) as a proportion of GDP is most conveniently accessible from the World Bank’s World Development Indicators (WDI) database. As with trade, the WDI reports investment data with a nominal 2 year lag. In general, FDI data are not as well reported as trade data; only about 75 per cent as many countries report FDI figures as report trade figures. In general, the FDI reporting countries are a subset of the trade reporting countries; thus, adding trade to an analysis that already includes FDI results in little or no loss of cases, but adding FDI to a trade analysis can result in substantial loss of cases. This is surely one reason why FDI is less common in the empirical literature on globalization than is trade.

Though foreign direct investment as a proportion of GDP is sometimes used alongside trade as an indicator of globalization, it suffers from a similar lack of theorization. Certainly the aggregate of all global FDI flows is an indicator of the globalization of the world-economy; FDI (and FPI) flows have grown over the past 25 years in step with all other global financial markets. This does not imply, however, that an individual country’s level of FDI as a proportion of GDP is a good general indicator of that country’s level of globalization. It is probably appropriate to use FDI (or FPI) as an indicator only where the theoretical model is explicitly concerned with issues of economic control.

Like trade, FDI as a proportion of GDP is skewed to the right, though it is not highly correlated with population size. Correcting for the right skew is problematic, since FDI is technically a net measure and can take minor negative values (which cannot be logged). One strategy is to use the log of 1 plus FDI, throwing out as outliers any cases in which FDI net inflows are more than 1 per cent of GDP in the red. This gives a result for 2000 that is reasonably well behaved (skew = –0.33).

FDI also presents difficulties due to its high annual volatility. I analysed FDI volatility in a panel of 51 countries for which continuous FDI data are reported in the WDI database for the period 1980–2000. Since these are the countries with the best reporting records, any estimate of FDI volatility derived from them is likely to be conservative. I first found the residual of a linear regression of each country’s FDI series on time to detrend the data. I then computed a coefficient of variation for each country by dividing the standard deviation of the detrended data by the mean of the raw data. The average coefficient of variation across 51 countries was 0.74. By comparison, repeating the exercise using trade data gave an average coefficient of variation of 0.12 (n= 123). Clearly, the high annual volatility in FDI flows calls into question the use of point estimates for magnitudes of FDI.

The obvious solution is to average FDI over periods of several years. This solution, however, creates its own problems. Averaging FDI over a period of Y years creates a variable that is arithmetically a multiple of 1/Y times the sum of FDI over Y years. This accumulation of foreign investment over a period of years is recognized as a distinct variable in the development literature, foreign capital penetration or PEN. A common operationalization of PEN is the ratio of the total accumulated stock of foreign investment in a country divided by that country’s total GDP. Though PEN can be measured directly as foreign ownership interest in a country’s economy at a given point in time, it is not analytically distinct from the sum of past years’ flows of FDI. In fact, the correlation between foreign capital penetration in 2000 reported by the United Nations Conference on Trade and Development (UNCTAD) and the sum of FDI flows over the 10 years 1991–2000 reported in the WDI database is 0.75 for a panel of 99 countries for which full data are available. Considering the fact that the two figures are computed using completely distinct methods, the correspondence is remarkable.

In short, FDI as a proportion of GDP is probably best avoided as a measure of globalization. It is volatile, difficult to transform and, in most cases, poorly theorized. Foreign capital penetration, on the other hand, is a valuable indicator, though not properly an indicator of globalization. Further difficulties arising from the use of FDI in statistical models of globalization are discussed below in the section on statistical tools.

NATIONAL INCOME

National income is the total final value of all goods and services produced by an economy. The two most common measures of national income are gross domestic product (GDP) and gross national product (GNP), though other similar measures do exist. In broad terms, GDP is the total final value of goods and services produced within a country’s borders, while GNP is the total final value of goods and services produced by the citizens of a country. The difference between the two is the net of factor payments (wages, profits, interest) made to resident foreigners and factor payments made by foreigners to citizens. The difference between the two is usually minor, but can exceed 10 per cent in extreme cases (such as countries with large overseas holdings or guest worker programmes).

GDP and GNP are often divided by population to create per capita measures of national income, which are used as proxies for countries’ overall levels of development. These series exhibit a strong positive skew, so they are customarily logged before being used in statistical models. Aggregate GDP and GNP figures are also sometimes used as indicators of country size, though population is more common for this purpose. Again, when used as aggregates GDP and GNP should be logged to correct for extreme positive skew.

Whether expressed as GDP or GNP, national incomes must be converted to a common currency unit (usually US dollars) to facilitate comparisons across countries. A fierce methodological debate has raged in recent years over just how to accomplish this currency conversion, driven by the fact that our interpretation of inequality trends in the world-economy hinges on the currency conversion method chosen (see Babones and Turner 2004 for a summary of the arguments). Two basic options exist. Currencies can be converted to dollars using the combination of market and official exchange rates reported by multilateral agencies such as the IMF and the World Bank (F/X method), or currencies can be converted to dollars at the purchasing power parity level that would equalize the prices of goods across countries (PPP method). Each method has both advocates and opponents, but each also has appropriate uses, which I describe below.

The F/X method is most closely associated with the world-systems school in sociology (Korzeniewicz and Moran 2000), while the PPP method is most closely associated with social demographers (Firebaugh 2000). Dollar denominated national income figures resulting from F/X conversion factors better represent command over goods and services traded on world markets, while those resulting from PPP conversion factors better represent the physical standards of living obtaining within a country. In recent extended treatments of the topic of national income measurement, Korzeniewicz et al. (2004) repeatedly contrast the ‘successes’ of F/X methods with the ‘failures’ of PPP methods, while Firebaugh (2003) labels the F/X estimates ‘implausible (p. 36) and ‘dubious’ (p. 38).

Perhaps the most balanced appraisal of the relative merits of F/X versus PPP based national income figures comes, ironically, from those very scholars who are most responsible for the creation of today’s PPP figures. Summers and Heston (1991) emphasize that their PPP estimates are intended as a ‘companion’ to previously existing F/X methods, ‘not at all a replacement’ for them (p. 355). They explicitly conclude that ‘a country’s international transactions … are best compared with … other countries’ transactions via exchange rates rather than PPPs’ (p. 360). Following Summers and Heston’s guidance, a reasonable solution to the F/X versus PPP controversy is to operationalize national income using F/X series where structural position in the world-economy is the main concern, and to operationalize national income using PPP series where the relative standard of living between countries is the main concern.

In general, the use of GDP per capita tends to be associated with PPP based currency conversions, while the use of GNP per capita tends to be associated with F/X based currency conversions. This makes some methodological sense, since PPP currency conversion factors are based on prices within a country, and thus cannot appropriately be applied to that portion of GNP that represents transfers from abroad. Both dollar denominated series (GDP-PPP and GNP-F/X) are reported in the World Bank’s World Development Indicators. A cleaner but less up-to-date source of GDP-PPP data is the University of Pennsylvania Center for International Comparisons’ Penn World Table.

Ultimately, the heated debate over the validity and reliability of GDP-PPP versus GNP-F/X national income figures is not very relevant to regression-based panel studies of globalization. While the choice of data series is of critical importance in describing the trajectory global inequality, it has very little effect on inferential statistics in regression models. This is because the correlation between logged GDP-PPP per capita and logged GNP-F/X per capita in any given year is in the order of r= 0.97 (based on series reported in the WDI).

Finally, it should be noted that the operationalization of GDP or GNP growth requires no currency conversion at all. Growth figures should be derived from national income reported in local currency units, adjusted for inflation. Using dollar-based figures to compute economic growth inappropriately incorporates currency effects into the resulting growth statistics. Unless this outcome is explicitly desired, currency conversions (and their associated debates) are best avoided.

Cultural globalization

Research on cultural globalization has a long pedigree, dating at least to the controversial ‘modernization’ literature of 40 years ago (e.g. Lerner 1958; Inkeles and Smith 1974). It was not until the 1980s, though, with the first wave of the World Values Survey (WVS), that cross-nationally comparable cultural data based on national probability samples became available. The WVS is a collaborative effort involving researchers in over 80 countries asking parallel questions on values and beliefs (plus basic demographics) in four waves spanning the two decades 1981–2001. Although precautions must be taken to establish the lexicon, contextual and conceptual equivalence of questions across so many cultures, the WVS is an invaluable one-of-a-kind resource. The latest wave, Wave 4 (1999–2001), contains data for over 160,000 individuals from 69 distinct countries and regions.

The key strength of the WVS is national probability sampling. Other surveys that might otherwise provide useful data on cultural globalization, from the early modernization literature to the latest National Institute on Aging National Character Survey (Terracciano et al. 2005), are limited by their samples of convenience. Their results can be considered no better than indicative.

Political globalization

A standard source of data used in studies of political globalization is the Europa World Year Book. The Europa yearbook reports annual relational data on the diplomatic representation of every country to and from every other country, as well as memberships in major multilateral institutions, signatories of major international treaties etc. Military data from the International Institute of Strategic Studies’ (IISS) annual volume The Military Balance is often used to complement political data from the Europa yearbook. The Military Balance reports data on troop levels, military exchanges and the like. The IISS also maintains an Armed Conflict Database identifying and providing background statistics on substantially all armed conflicts ongoing throughout the world. For data on international terrorism, the US State Department’s annual Patterns of Global Terrorism report is the standard source for historical data, but beginning with the 2005 report (on 2004 activity) the detailed data section of the report is classified. Hopefully these reports will once again be made public in the future on completion or after a reasonable time lag.

STATISTICAL TOOLS

In this section I review some common but sometimes quite subtle errors in interpreting the results of statistical analyses of country level panel data. Multiple linear regression, in which a single dependent variable is regressed on a vector of several independent variables, is a workhorse tool of globalization research. When the independent variables in a multiple linear regression model are uncorrelated with each other and only moderately correlated with the dependent variable, their coefficients can be interpreted relatively straightforwardly as the effects of the independent variables on the dependent variable. Unfortunately, in globalization research this benign scenario almost never comes to pass. Independent variables are often highly correlated with each other, and lagged versions of dependent variables are often themselves used as independent variables.

Making matters more confusing, multiple ratio variables in a regression equation often share the same numerator or denominator. For example, it would not be uncommon to find GDP, GDP/population, population, trade/GDP and investment/GDP together in a single regression equation. Of course, all of these variables trend over time, though at different rates in different countries. Underlying all of this might be a country level fixed or even variable effect. Throw in a few interaction terms, and it is clear that models like this must be interpreted with care, if they can be interpreted at all. Models like this are, however, quite common.

It has become commonplace to observe that the ease of use of statistical software has allowed the ability to run statistical models to far outstrip the ability to understand them. In my experience, the problem is usually not a lack of technical understanding of the mathematics that underlie the tools, but a lack of careful thought being put into understanding the meaning of coefficients, especially partial coefficients (‘controlling for’ other variables). Many times the risk is as simple as being seduced by the name of a variable; what could be more obvious than that the effect of the variable ‘GDP per capita’ is the effect of GDP per capita? As I demonstrate below, in many very common research designs it is not. In this section, I address several such situations in which the difficulty lies not in understanding the mathematics of the statistical models estimated but in understanding the substantive meaning of the results.

Difficulties interpreting multiple regression coefficients

Despite the fact that multiple linear regression has long been a basic tool of statistical analysis in the social sciences, and despite the fact that the mathematics underlying multiple linear regression are relatively accessible even to non-mathematicians, the behaviour of variables in multiple regression equations is still often the subject of serious controversy. Consider, for example, the now famous (or infamous) ‘denominator effect’ debate on the effects of foreign capital penetration on growth (Firebaugh 1992, 1996; Dixon and Boswell 1996a, 1996b). A total of 68 pages of one of sociology’s top journals were devoted, fundamentally, to the question of how to interpret the coefficient for foreign capital penetration in a model of economic growth. Throughout this debate (which still ripples through the dependency literature; see Kentor and Boswell 2003), the mathematics and numerical results were never at issue; the debate was over what words should be attached to the agreed numbers.

Whatever the actual effect of foreign capital on economic growth, several methodological lessons can be learned from the debate, beyond the mere fact that ratio variables are tricky (Firebaugh and Gibbs 1985). First and foremost, one should take great care in interpreting models when the same variable appears in multiple places in a single regression equation. This caution applies equally when two different variables are so highly correlated as to be effectively indistinguishable. Second, as a corollary, the use of lagged dependent variables in regression models should be avoided. It is simpler – and more direct – to study change in the dependent variable instead. Third, another corollary, one should take great care in interpreting interaction effects. Interactions generally involve variables that appear elsewhere as main effects; main and interaction effects cannot be interpreted separately from one another.

The use of lagged dependent variables is particularly problematic. Do their coefficients represent ‘stability effects’, underscoring the fact that earlier values of dependent variables are often the best guides to predicting later values? Or do they simply represent change scores on the sly: a lagged coefficient of 1, when moved to the dependent side of the equation, becomes a change score. The answer is likely a combination of the two. For example, Reuveny and Li (2003) report a lagged dependent variable coefficient of 0.7 in a study of within-country income inequality. Moving a ‘phantom’ coefficient of 1.0 to the dependent variable side of the equation changes the dependent variable from inequality at time t to inequality growth over the period from t – 1 to t. The remaining coefficient for income inequality at time t – 1, –0.3, is consistent with a model of regression to the mean.

The use of multilevel models in globalization research

Since we are now in the midst of an ‘age of globalization’, every year that passes generates one more year of data for studying globalization. As the time period for which broadly cross-national data are available has lengthened, researchers have naturally sought to expand their panels longitudinally. Where researchers once studied outcomes for a single panel of countries over the full period for which data were available, they are now able to study multiple unbalanced panels over shorter periods. There has also been an explosion in multiple time point cross-sectional research, in which countries appear for each year or period for which data are available. Studies in which individual countries each appear multiple times almost always adopt a multilevel modelling approach, whether or not such an approach is called for.

There are two basic varieties of multilevel models: fixed effects models (FEMs) and random effects models (REMs). The fixed effects model as used in globalization research is a hybrid between a one-way analysis of variance (ANOVA) model (in which the country is the fixed factor) and a multiple linear regression model (in which all other independent variables are covariates); it is thus the equivalent of estimating a multiple linear regression with indicator variables representing countries. Each country’s effect on the dependent variable is ‘fixed’ across all observations and the country mean is taken out of the analysis; in effect, the problem is reduced to estimating the relationships between the independent variables (‘covariates’) and the within-country variation of the dependent variable. Between-country variation in the dependent variable is ignored in estimating the effects of other independent variables in FEMs.

The random effects model is similar to the fixed effects model, but as the name implies the country effects are not fixed at a single value but instead are modelled as random variables with their own means and variances. The REM assumes that the array of unobserved factors unique to each country has the same expected impact on the dependent variable across all time points, but that there is some element of randomness in the actual impact recorded for any particular realization of the dependent variable. Because they allow some variability in the realized magnitude of country effects at multiple time periods, REMs allow independent variables to affect realizations of the dependent variable both within and between countries. The relative influence of between-country variation in estimating the effects of independent variables in REMs increases as the number of time points over which each country is observed declines.

Multilevel models were developed for use in experimental settings where the choice between FEMs and REMs is clear: when values of the treatment variable (analogous to the country in cross-national applications) are fixed by the experimenter, use FEMs; when values of the treatment variable are not under the control of the experimenter (and thus might be associated with other unobserved variables), use REMs. In experimental research, REMs are the more conservative choice of the two, since the main object of interest is the effectiveness of the treatment; REMs allow for the fact that part of the apparent effect of the treatment may be attributable to other unobserved variables. As the number of distinct observations for each treatment increases, treatment means become better defined, and the REM converges to a FEM.

In experimental research, the effects of covariates (other independent variables) are a distraction; in most statistical packages, they are not even included in the default output for multilevel models. In globalization research, of course, priorities are reversed, and the effects of the independent variables are of primary interest; the actual country effects are typically not even reported. The result of this flip in priorities is to make FEMs the more conservative approach; it is often difficult to ‘achieve’ statistically significant results with FEMs, especially when the number of time points observed per country is small. The temptation is strong to use REMs in these cases, but this often flies in the face of the rationale for using multilevel models in the first place. The motivation for using multilevel models in cross-national research is to control for time-invariant unmeasured variables at the country level (Firebaugh and Beck 1994). Where statistical power is more important than controlling for unmeasured variables, other strategies are more appropriate. Multilevel models should not be employed simply because multiple observations exist for each country in a study; this situation can be handled by allowing for the correlation of errors within countries.

Another situation where multilevel models are inappropriate occurs when the researcher wants to estimate the effects of time-invariant variables. For example, Alderson and Nielsen (1999) used multilevel models to study the effects of foreign capital penetration on within-country inequality. They chose an REM because they also wanted to include two time-invariant controls in their models; in an FEM, the effects of time-invariant variables cannot be estimated, since in FEMs all time-invariant covariates are incorporated into the country level fixed effects. Tellingly, the results of their REM were ‘substantively identical’ (p. 616) to those given by an FEM for their main variables of interest. Since the time-invariant variables were only used as controls, they were unnecessary to the model, and their absence in the FEM did not affect the performance of other variables. Where time-invariant variables are not merely controls but are in fact the main variables of interest, regression models incorporating correlations of errors by country are the better choice.

Where the object is to control for country level unmeasured variables, the experimental analogy provides a guide for deciding whether FEMs or REMs are more appropriate. In experimental research, REMs are used when unmeasured variables partially or wholly determine the values of the treatments for which the dependent variable is to be evaluated. For example, educational sociologists regularly use REMs to control for school effects in studying student performance: what school a child attends is the result of many unmeasurable factors beyond the researchers’ control, factors that likely also effect student performance. It is difficult to imagine a scenario in cross-national research in which the assignment of cases to be subjected to country effects is due to unmeasurable factors that are systematically related to the dependent variable. Thus, FEMs are probably the methodologically appropriate choice in almost all cases. They are certainly the more conservative choice.

Multilevel models often strain statistical intuition to the breaking point, and thus there are many pitfalls to avoid in interpreting the coefficients of FEMs, never mind REMs. A particularly insidious pitfall for globalization researchers results from the fact that most variables associated with globalization exhibit strong secular trends. One example is GDP per capita; it consistently rises over time in almost all countries, though at very different rates. GDP per capita is near ubiquitous as a control variable; it is rarely the variable of interest, but it is present in almost every model of the effects of globalization. When used in country level FEMs, however, the statistical power of GDP per capita has nothing to do with level of development, since each country’s mean level of GDP per capita is accounted for by its fixed effect; all that remains is each country’s trend in GDP per capita over time. For the 24 historical members of the OECD over the period 1975–2000, these trends are correlated on average r= 0.98 with time. Practically speaking, GDP per capita operationalizes in fixed effects models as time, though time that ticks at a different rate in each country.

COUNTRIES AS CASES

Pragmatically, the unit of analysis in almost all globalization research is the country, since in today’s world the country is the primary political unit (and thus the primary data collection unit). When countries representing more than 90 per cent of the world’s population are included in a study, most scholars accept it as globally representative. It should be noted that this is not a sampling issue: in all cases involving country level data, the sampling ratio is 100 per cent. There is, however, an unavoidable gap between the target population (cases covering all people in all countries) and the sampling frame (those countries for which data are available). Unfortunately, data availability is not distributed at random with respect to variables of interest: small, poor and conflict-ridden countries are those most likely to be missing. Meeting a 90 per cent (or higher) world population threshold ensures that the study coverage is reasonably broad.

The political independence of a country, however, does not necessarily imply its statistical independence as a case that is free to vary independently of other countries. The classic formulation of the interdependence of societies is Galton’s problem of cultural diffusion. Nineteenth-century geneticist Frances Galton questioned whether similar marriage customs arose independently in multiple societies or were adopted in each society through cultural diffusion from a single source. A modern variant of Galton’s problem is at the heart of the debate between Wallerstein’s (2004) world-systems and Meyer et al.’s (1997) world society approaches to studying globalization: is globalization at the country level best modelled as a similar response to common structural forces affecting all countries simultaneously (world-systems approach) or as a process of cultural diffusion in which practices that are associated with successful countries are emulated by countries that aspire to be successful (world society approach)?

Solutions to these modern variants of Galton’s problem are not easy to come by. Naroll (1968) suggests that researchers compare cases for which diffusion would have been impossible, but this is difficult to apply in practice. Chase-Dunn (1989: 311) suggests diffusion be endogenized and itself studied as a system property. This is, however, virtually impossible to operationalize in a cross-national panel context. The world-systems–world society debate is probably more amenable to qualitative comparative case studies rather than to quantitative statistical models. When it comes to quantitative modelling, the truth is that we probably have fewer independent cases – a smaller N, so to speak – than we think we have.

The larger problem of the interdependence of cases is a hidden plague on all research that uses the country as the unit of analysis. The simple fact is that countries are not independent cases when it comes to the study of globalization. A basic assumption of regression analysis is that errors are uncorrelated across cases, but this assumption probably does not hold for neighbouring countries or those occupying similar structural positions in the world-economy. Of course, the use of country level fixed effects solves this problem, but creates others (as discussed above); also, fixed effects models are infeasible when data are available for only one or two time periods.

Environmental scientists have recently begun addressing problems of spatial autocorrelation using Bayesian statistical methods based on Markov random fields (see Rue 2005). In the Markov random field approach an underlying error structure is specified in which each country’s error terms are assumed to be influenced by neighbouring countries’ error terms. A graph is constructed in which each country is represented by a vertex which is connected to each neighbouring country by an edge; this graph forms an (irregular) lattice across which errors are allowed to ‘flow’: countries’ errors are associated most strongly with their immediate neighbours’ errors, but are also associated to their neighbours’ neighbours’ errors, and so on across the entire lattice.

Models using Markov random fields have begun to be adopted in the epidemiological literature, but have not yet appeared in the mainstream of political economics research; a seminal application in economic geography is Dezzani (2004). One factor holding back their use is the lack of well-developed tools for statistical inference: research to date has focused on using Markov random fields for estimation and prediction. Currently, only rather crude Monte Carlo techniques are available for computing confidence intervals for coefficients estimated using Markov random field models. This gap in inferential statistics is an area of active research in mathematical statistics, and is likely to be filled with time.

A complementary problem to the lack of independence of countries as units of analysis is the lack of independence between observations of the same country at different points in time, or temporal autocorrelation. A simple and direct fix for temporal autocorrelation in the dependent variable, which often leads to temporal autocorrelation of errors, is to take the first difference in the dependent variable, subtracting its value at time t – 1 from its value at time t. Differencing transforms the dependent variable from being the level of a phenomenon to being the degree of change in a phenomenon; special care must be taken in determining whether or not to difference independent variables as well (Firebaugh and Beck 1994). Thought should also be put into determining the appropriate period over which to take the difference, what Chase-Dunn (1989: 321–2) calls the ‘width of a time point’. Too often in the economics literature annual first differences are applied mechanically to dependent variables without appropriate consideration of the implications this holds for the interpretation of the model or the roles of other variables in it.

An alternative to differencing is to explicitly model regression errors as autoregressive processes. Usually a first-order or AR(1) model, in which each country’s error at time t is conditioned on its error at time t – 1, is sufficient, though more complicated error structures can be modelled. As with the treatment of spatial autocorrelation, so too with temporal autocorrelation the Bayesian Markov random field approach may ultimately become a standard tool. In addition to linking countries geographically to their neighbours, Markovian lattices can be used to link countries to themselves at previous and future time points. While these emerging techniques present intriguing possibilities for improving our understanding of globalization, they have not yet been tested in practice.

CONCLUSION

The foregoing discussion outlines many subtle and sometimes quite technical methodological issues to keep in mind when studying the causes or consequences of globalization. In the end, they are nothing more than special cases of much more general principles of sound methodology. Empirical research should be grounded in theory. Theoretical models should guide the formulation of statistical models. Operational definitions of variables should match their theoretical definitions. Partial regression coefficients should not be interpreted as simple regression coefficients. Occam’s razor should be applied at all times.

What is distinct about broadly cross-national research on globalization is that for the most part we are all using the same data. Thus, methodological considerations come to the fore as they do in few other areas of social scientific research. Often, as in the international inequality debate, only the operationalization of a key variable stands between wildly opposing substantive conclusions; as a result, there is now an entire literature evaluating the appropriateness of different measures of national income for studying international inequality. Where research is less controversial – and the level of scrutiny is lower – methodological oversights can more easily go unnoticed.

Hopefully, future creators and consumers of globalization research will find the pointers given here useful in improving their own understandings of the complex methodological issues raised in the study of globalization. The literature on economic globalization is methodologically quite advanced; in many cases, the level of methodological sophistication may be said to have outstripped the ability of the data to support it. The research literatures on cultural and political globalization are not as extensive as that on economic globalization, but given that theory is well in advance of research in these areas the research literatures will likely catch up quickly. As all three research literatures become ever more sophisticated methodologically, it is well to remember that parsimony should be valued at a premium.

References

Alderson, A.S. and Nielsen, F. 1999. ‘Income inequality, development, and dependence: A reconsideration’, American Sociological Review, 64, 606–31.

Babones, S.J. 2006. ‘Conducting global social research’. In C.K. Chase-Dunn and S.J. Babones (eds), Global Social Change, ch. 2. Baltimore, MD: The Johns Hopkins University Press.

Babones, S.J. and Turner, J.H. 2004. ‘Global inequality’. In G. Ritzer (ed.), Handbook of Social Problems, 101–20. Thousand Oaks, CA: Sage.

Bornschier, V. and Chase-Dunn, C.K. 1985. Transnational Corporations and Underdevelopment. New York: Praeger Press.

Bornschier, V., Chase-Dunn, C.K. and Rubinson, R. 1978. ‘Cross-national evidence of the effects of foreign investment and aid on economic growth and inequality: A survey of findings and a reanalysis’, American Journal of Sociology, 84, 651–83.

Chase-Dunn, C.K. 1989. Global Formation: Structures of the World-Economy. Cambridge, MA: Basil Blackwell.

Chase-Dunn, C.K. 1999. ‘Globalization: A world-systems perspective’, Journal of World-Systems Research, 5, 187–215.

Chase-Dunn, C.K. and Babones, S.J. 2006. ‘Global social change’. In C.K. Chase-Dunn and S.J. Babones (eds), Global Social Change, ch. 1. Baltimore, MD: The Johns Hopkins University Press.

Chase-Dunn, C., Kawano, Y. and Brewer, B.D. 2000. ‘Trade globalization since 1795: Waves of integration in the world-system’, American Sociological Review, 65, 77–95.

Dezzani, R. 2004. ‘Spatial probability modeling of regional convergence and development’. Paper presented at the 2004 Annual Meeting of the Association of American Geographers, Philadelphia, PA.

Dixon, W.J. and Boswell, T. 1996a. ‘Dependency, disarticulation, and denominator effects: Another look at foreign capital penetration’, American Journal of Sociology, 102, 543–62.

Dixon, W.J. and Boswell, T. 1996b. ‘Differential productivity, negative externalities, and foreign capital dependency: Reply to Firebaugh’, American Journal of Sociology, 102, 576–84.

Evans, P. 2005. ‘Counterhegemonic globalization: Transnational social movements in contemporary political economy’. In T. Janoski et al. (eds), Handbook of Political Sociology: States, Civil Societies, and Globalization, 655–70. New York: Cambridge University Press. Firebaugh, G. 1992. ‘Growth effects of foreign and domestic investment’, American Journal of Sociology, 98, 105–30.

Firebaugh, G. 1996. ‘Does foreign capital harm poor nations? New estimates based on Dixon and Boswell’s measures of capital penetration’, American Journal of Sociology, 102, 563–75.

Firebaugh, G. 2000. ‘Observed trends in between-nation income inequality and two conjectures’, American Journal of Sociology, 106, 215–21.

Firebaugh, G. 2003. The New Geography of Global Income Inequality. Cambridge, MA: Harvard University Press.

Firebaugh, G. and Beck, F.D. 1994. ‘Does economic growth benefit the masses? Growth, dependence, and welfare in the Third World’, American Sociological Review, 59, 631–53.

Firebaugh, G. and Gibbs, J.P. 1985. ‘User’s guide to ratio variables’, American Sociological Review, 50, 713–22.

Guillen, M.F. 2001. ‘Is globalization civilizing, destructive or feeble? A critique of five key debates in the social science literature’, Annual Review of Sociology, 27, 235–60.

Inkeles, A. and Smith, D.H. 1974. Becoming Modern: Individual Change in Six Developing Countries. Cambridge, MA: Harvard University Press.

Kaplinsky, R. 2001. ‘Is globalization all it is cracked up to be?’ Review of International Political Economy, 8, 45–65.

Kentor, J. and Boswell, T. 2003. ‘Foreign capital dependence and development: A new direction’, American Sociological Review, 68, 301–13.

Kim, S. and Shin, E.H. 2002. ‘A longitudinal analysis of globalization and regionalization in international trade: A social network approach’, Social Forces, 81, 445–71.

Kohn, M.L. 1987. ‘Cross-national research as an analytic strategy’, American Sociological Review, 52, 713–31.

Korzeniewicz, R.P. and Moran, T.P. 2000. ‘Measuring world income inequalities’, American Journal of Sociology, 106, 209–14.

Korzeniewicz, R.P., Stach, A., Patil, V. and Moran, T.P. 2004. ‘Measuring national income: A critical assessment’, Comparative Studies in Sociology and History, 46, 535–86.

Lechner, F.J. 2005. ‘Globalization and inequality: The “great reversal” and its implications’. Paper presented at the 100th Annual Meeting of the American Sociological Association, Philadelphia.

Lerner, D. 1958. The Passing of Traditional Society: Modernizing the Middle East. Glencoe, IL: Free Press.

Macleod, D.V.L. 1999. ‘Tourism and the globalization of a Canary Island’, Journal of the Royal Anthropological Institute, 5, 443–56.

Meyer, J.W., Boli, J., Thomas, G.M. and Ramirez, F. 1997. ‘World society and the nation-state’, American Journal of Sociology, 103, 144–81.

Mitchell, B.R. 1992. International Historical Statistics: Europe 1750–1988, 3rd edn. New York: Stockton Press.

Mitchell, B.R. 1993. International Historical Statistics: The Americas 1750–1988, 2nd edn. New York: Stockton Press.

Mitchell, B.R. 1995. International Historical Statistics: Africa, Asia, and Oceania 1750–1988, 2nd edn. New York: Stockton Press.

Moghadam, V.M. 1999. ‘Gender and globalization: Female labor and women’s mobilization’, Journal of World-Systems Research, 5, 367–88.

Naroll, R. 1968. ‘Some thoughts on comparative method in cultural anthropology’. In H.M. Blalock Jr and A.B. Blalock (eds), Methodology in Social Research, 236–77. New York: McGraw-Hill.

Reuveny, R. and Li, Q. 2003. ‘Economic openness, democracy, and income inequality: An empirical analysis’, Comparative Political Studies, 36, 575–601.

Rue, H. 2005. Gaussian Markov Random Fields: Theory and Applications. Boca Raton, FL: Chapman & Hall.

Smith, D.A. 2001. ‘Editor’s introduction – Globalization and social problems’, Social Problems, 48, 429–34.

Summers, R. and Heston, A. 1991. ‘The Penn World Table (Mark 5): An expanded set of international comparisons, 1950–1988’, Quarterly Journal of Economics, 106, 327–68.

Terracciano, A., Abdel-Khalek, A.M., Ádám, N. et al. 2005. ‘National character does not reflect mean personality trait levels in 49 cultures’, Science, 310 (5745), 96–100.

Wallerstein, I. 2004. World-Systems Analysis: An Introduction. Durham, NC: Duke University Press.