Chapter 9

Nonlinear Regression Models

Chapter 9 Preview

When you have completed reading this chapter you will be able to:

• Address nonlinearities with linear least squares regression.

• Estimate a nonlinear regression (four different nonlinear regressions).

• Know when it may be reasonable to use a nonlinear regression model.

• Identify common uses of nonlinear regressions.

• Interpret the results from an estimate of a nonlinear model.

Introduction

All of the regression models you have looked at thus far have been linear. Some have had one independent variable, while others have had several independent variables, but all have been linear. Linear functions appear to work very well in many situations. However, cases do arise in which nonlinear models are called for. The real world does not always present you with linear (i.e., straight-line) relationships.

In some situations, the economic/business theory underlying a relationship may lead you to expect a nonlinear relationship. Examples of such real world cases include the U-shaped average cost functions and S-shaped production functions, both of which are caused by the existence of the law of variable proportions or diminishing marginal returns. Profit and revenue functions often increase at a decreasing rate and then decline, shaped somewhat like an inverted-U().

Earlier, you learned that plotting data in a scattergram before doing a regression analysis can be helpful. Among other things, this may help us decide on the appropriate form for the model. Perhaps some nonlinear form would appear more consistent with the data than would a linear function.

Quadratic Functions

Look at the two graphs in Figure 9.1. The data for the scattergrams depicted in each graph are listed to the right of the graphs. In the upper graph, the 10 observations appear to form a U-shaped function such as what you might expect of an average variable cost curve. The lower scattergram suggests a function that increases at a decreasing rate and then eventually declines (like an upside-down bowl placed on a table). Clearly, a linear function would not fit either set of data very well. This lack of a linear fit is illustrated by the inclusion of the OLS linear regression line in each graph. In neither case does the (linear) regression line offer a close fit for the data.

These scattergrams are indicative of quadratic functions, that is, functions in which X 2 appears as an independent variable. But most regression programs are designed for linear problems. Is it possible to use such programs to estimate this type of function? The answer is “yes” and the technique for doing so is surprisingly simple.

You need only define a new variable to be the square of X. Suppose that you use Z. Then Z = X 2. You can now estimate a multiple linear regression model of Y as a function of X and Z, which may be expressed as:

This is a function that is linear in the variables X and Z.

Estimating a function such as this for the data in the upper graph of Figure 9.1, you get:

But remember that Z = X². Rewriting the function in terms of Y, X, and X², you have:

This is a quadratic function in X and has the U-shape you would expect to fit the scattergram (see Figure 9.2).

The data in the lower graph in Figure 9.1 also appear to be quadratic, but this time the function at first goes up and then down (rather than vice versa, as in the previous case). Even though the curvature of this data is an upside-down U-shape, you can use the same technique. Again let Z = X 2 and estimate:

This time you expect the sign of bl to be positive and the sign of b2 to be negative (The opposite was true in the previous case). The regression results are:

Substituting X2 for Z, you have:

This function would have the inverted U-shape that the scattergram of the data would suggest (see Figure 9.3).

Cubic Functions

How would you handle situations in which the perceived curve exhibited by a set of data has more than a single curvature? Is OLS regression able to model such data? Yes, you can model data with a double curvature but the technique will require the use of a cubed term. Now take a look at the scattergrams and data shown in Figure 9.4. You can see that a linear function would not fit either of these data sets very well. The data in the upper graph might represent a cost function, while that in the lower graph may be a production function. Both are polynomials, but they are cubic functions, rather than quadratic functions.

To estimate these cubic functions using a multiple linear regression model, you can let Z = X 2 and W = X 3. The linear function is then:

Using the data in the lower graph in Figure 9.4, you get the following regression equation:

Substituting X2 for Z and X3 for W, you get:

You can see this result in Figure 9.5.1

The t-ratios are fairly low even though the function fits the data pretty well which might be puzzling. The main reason is related to the small number of data points relative to the number of variables. You would rarely estimate a regression analysis with just 10 observations, especially when using 3 independent variables. However, using just ten observations is helpful in learning the concepts.

You see that the upper scattergram in Figure 9.4 also appears to have two turning points: one near X = 6 and the other in the vicinity of X = 11. This indicates that a cubic function might again be appropriate. You again let Z = X2 and W = X3. The linear function is then:

Using the data in the upper graph in Figure 9.4, you get the following regression equation:

Substituting X2 for Z and X 3 for W, you get:

You can see this result in Figure 9.6.

Reciprocal Functions

Now look at the scattergrams and data in Figure 9.7. In the upper graph, you see that the Y values fall rapidly as X increases at first, but then as X continues to increase, the Y value seems to level off. The lower graph is similar except that, as X increases, at first Y increases rapidly but seems to reach a plateau. Now, consider using a reciprocal transformation to estimate each of these sets of data; that is, use 1/X rather than X as the independent variable.

You can let U = 1/X and proceed to estimate the function: Y = a + bU. Using the data from the upper graph in Figure 9.7, the regression ­equation is:

Substituting 1/X for U, you have:

For this function, as X increases 1/X decreases. As a result, this function approaches the value of the constant term of the regression (a = 5.91) from above as X gets large as you see in Figure 9.8.

Using the same approach for the data in the lower graph in Figure 9.7, the regression results yield:

For this function, as X increases, this function also approaches the value of the constant term of the regression (a = 12.38), but from below rather than above. You can see this graphically in Figure 9.9.

Multiplicative Functions2

The final type of nonlinear function included here involves a double-log transformation. This allows you to use a multiple linear regression program to estimate the following functional form:

Functions of this type are called power functions or multiplicative functions. The most common of these functions is the Cobb-Douglas production function. The general form for such a function is:

where, Q is output, L is labor input, and K is capital. This example uses just L and K but each of these could have subcomponents so there could be many Ls and many Ks.

These functions are clearly nonlinear, so you must use some transformation to get them into a linear form. If you take the logarithms of both sides of the equation, you have:

where ln represents the natural logarithm. Now, you can let W = InY, a = lnA, U = lnX1, and V = lnX2. The function is now linear in W, U, and V, as shown below:

A function such as this can be estimated using a standard multiple regression program. For example, consider the data in Table 9.1. Y is the dependent variable, X1 and X2 are independent variables, and you would like to estimate the following function:

Using Excel, you can easily find the natural logarithms of all three variables (W = lnY, U = lnXl, and V = lnX 2). These values are also shown in Table 9.1.3

Using the natural logarithms as the variables, you estimate the following multiple linear regression:

Making the appropriate substitutions for W, U, and V, you have:

Taking the antilogarithm of the equation, you obtain:

where e represents the Naperian number (2.7183), which is the base for natural logarithms. Thus,

Converting 2.71832.402 to 11.04 is easily done using the EXP function in Excel. You see that this fairly complex-looking function can be estimated with relative ease using the power of multiple regression and the logarithmic transformation. Power, or multiplicative functions, such as this are not only commonly found as production functions but are also sometimes used for demand functions.

Even though the basic regression model is designed to estimate linear relationships, nonlinear models can be estimated by using various transformations of the variables. Polynomial functions can be estimated by using second, third, and/or higher powers of one or more of the ­independent variables. Logarithmic and reciprocal transformations are also commonly used.

What You Have Learned in Chapter 9

• How nonlinearities can be addressed with linear least squares regression.

• To estimate a nonlinear regression (four different nonlinear regressions).

• To recognize when it may be reasonable to use a nonlinear regression model.

• To be familiar with common uses of nonlinear regressions.

• How to interpret the results of an estimate of a nonlinear model.

NOTES