3.1. Introduction

Regression analysis primarily deals with the issues related to estimating or predicting the expected value of the dependent or response variable using the known values of one or more independent or predictor variables. Usually a model is postulated relating the response variable to the predictor variables with certain unknown coefficients. A model that is linear in these coefficients is often referred to as a linear regression model or simply a linear model.

The multivariate linear regression model is a natural generalization of a (univariate) linear regression model. That is, two or more possibly correlated dependent variables are simultaneously modeled as the linear functions of the same set of predictor variables. For reasons of mathematical convenience in developing an appropriate theory, it is required that the particular model for each response variable be in exactly the same functional form. For example, if y₁ and y₂ are the response variables and x₁ and x₂ are the predictors, then the univariate models y₁ = a₀ + a₁x₁ + a₂x₂ + ϵ₁ and y₂ + b₀ + b₁x₁ + b₂x₂ + ϵ₂ have the same functional forms, whereas the models y₁ = a₀+ a₁x₁ + a₂x₂ + ϵ₁ and y₂ = b₀ + b₁x₁ + ϵ₂ do not. It is possible to argue that the last model (for y₂) has the same functional form as the model for y₁ with the choice b₂ = 0. However, this suggests that b₂ is completely known and hence its estimation is irrelevant. But multivariate regression theory assumes that all the coefficients in the model are unknown and are to be estimated.

In the univariate regression models, we assume that there are n observations available on a response variable y as well as on predictors x₁,...,x_k. Suppose these n data values on y are stored in an n by 1 column vector y and values on x_i, i = 1,...,k are stacked, in the same order, in an n by 1 vector x_i. Then the complete linear model for the data can be expressed as the linear relation between these column vectors

y = β₀1_n + β₁x₁ +... +, β_kx_k + ϵ.

In multivariate situations, that is, when there are two or more response variables, the functional form of the linear model for each of these response variables is assumed to be the same as above. However, each model will have a different set of unknown coefficients β₀,...,β_k and a different error vector.