Goodness of fit

Goodness-of-fit measures assess how well a model explains the variation in the outcome. They help to assess the quality of model specification, for instance, to select among different model designs. They differ in how they evaluate the fit. The measures discussed here provide in-sample information; we will use out-of-sample testing and cross-validation when we focus on predictive models in the next section.

Prominent goodness-of-fit measures include the (adjusted) R2 that should be maximized and is based on the least-squares estimate:

 

  • R2 measures the share of the variation in the outcome data explained by the model and is computed as , where TSS is the sum of squared deviations of the outcome from its mean. It also corresponds to the squared correlation coefficient between the actual outcome values and those estimated (fitted) by the model. The goals is to maximize R2 but it never decreases as the model adds more variables and, hence, encourages overfitting.
  • The adjusted R2 penalizes R2 for adding more variables; each additional variable needs to reduce RSS significantly to produce better goodness of fit.

Alternatively, the Akaike (AIC) and the Bayesian Information Criterion (BIC) are to be minimized and are based on the maximum-likelihood estimate:

  • , where  is the value of the maximized likelihood function, k is the number of parameters
  •  where N is the sample size

Both metrics penalize for complexity, with BIC imposing a higher penalty so that it might underfit whereas AIC might overfit in relative terms. Conceptually, AIC aims at finding the model that best describes an unknown data-generating process, whereas BIC tries to find the best model among the set of candidates. In practice, both criteria can be used jointly to guide model selection when the goal is in-sample fit; otherwise, cross-validation and selection based on estimates of generalization error are preferable.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset
54.234.136.147