Glossary

2^k factorial experiment. A full factorial experiment with k factors and all factors tested at only two levels (settings) each.

2^k–p factorial experiment. A fractional factorial experiment with k factors tested in a 2^–p fraction with all factors tested at only two levels (settings) each.

Acceptance region. In hypothesis testing, a region in the sample space of the test statistic such that if the test statistic falls within it, the null hypothesis cannot be rejected. This terminology is used because rejection of H₀ is always a strong conclusion and acceptance of H₀ is generally a weak conclusion.

Addition rule. A formula used to determine the probability of the union of two (or more) events from the probabilities of the events and their intersection(s).

Additivity property of χ². If two independent random variables X₁ and X₂ are distributed as chi-square with v₁ and v₂ degrees of freedom, respectively, Y = X₁ + X₂ is a chi-square random variable with u = v₁+ v₂ degrees of freedom. This generalizes to any number of independent chi-square random variables.

Adjusted R². A variation of the R² statistic that compensates for the number of parameters in a regression model. Essentially, the adjustment is a penalty for increasing the number of parameters in the model.

Alias. In a fractional factorial experiment when certain factor effects cannot be estimated uniquely, they are said to be aliased.

All possible (subsets) regressions. A method of variable selection in regression that examines all possible subsets of the candidate regressor variables. Efficient computer algorithms have been developed for implementing all possible regressions.

Alternative hypothesis. In statistical hypothesis testing, this is a hypothesis other than the one that is being tested. The alternative hypothesis contains feasible conditions, whereas the null hypothesis specifies conditions that are under test.

Analysis of variance (ANOVA). A method of decomposing the total variability in a set of observations, as measured by the sum of the squares of these observations from their average, into component sums of squares that are associated with specific defined sources of variation.

Analytic study. A study in which a sample from a population is used to make inference to a future population. Stability needs to be assumed. See Enumerative study.

Arithmetic mean. The arithmetic mean of a set of numbers x₁, x₂,..., x_n is their sum divided by the number of observations, or (1/n). The arithmetic mean is usually denoted by , and is often called the average.

Assignable cause. The portion of the variability in a set of observations that can be traced to specific causes, such as operators, materials, or equipment. Also called a special cause.

Asymptotic relative efficiency (ARE). Used to compare hypothesis tests. The ARE of one test relative to another is the limiting ratio of the sample sizes necessary to obtain identical error probabilities for the two procedures.

Attribute. A qualitative characteristic of an item or unit, usually arising in quality control. For example, classifying production units as defective or nondefective results in attributes data.

Attribute control chart. Any control chart for a discrete random variable. See Variables control chart.

Average. See Arithmetic mean.

Average run length, or ARL. The average number of samples taken in a process monitoring or inspection scheme until the scheme signals that the process is operating at a level different from the level in which it began.

Axioms of probability. A set of rules that probabilities defined on a sample space must follow. See Probability.

Backward elimination. A method of variable selection in regression that begins with all of the candidate regressor variables in the model and eliminates the insignificant regressors one at a time until only significant regressors remain.

Baye's estimator. An estimator for a parameter obtained from a Bayesian method that uses a prior distribution for the parameter along with the conditional distribution of the data given the parameter to obtain the posterior distribution of the parameter. The estimator is obtained from the posterior distribution.

Baye's theorem. An equation for a conditional probability such as P(A|B) in terms of the reverse conditional probability P(B|A).

Bernoulli trials. Sequences of independent trials with only two outcomes, generally called “success” and “failure”, in which the probability of success remains constant.

Bias. An effect that systematically distorts a statistical result or estimate, preventing it from representing the true quantity of interest.

Biased estimator. See Unbiased estimator.

Bimodal distribution. A distribution with two modes.

Binomial random variable. A discrete random variable that equals the number of successes in a fixed number of Bernoulli trials.

Bivariate distribution. The joint probability distribution of two random variables.

Bivariate normal distribution. The joint distribution of two normal random variables.

Block. In experimental design, a group of experimental units or material that is relatively homogeneous. The purpose of dividing experimental units into blocks is to produce an experimental design wherein variability within blocks is smaller than variability between blocks. This allows the factors of interest to be compared in an environment that has less variability than in an unblocked experiment.

Box plot (or box and whisker plot). A graphical display of data in which the box contains the middle 50% of the data (the interquartile range) with the median dividing it, and the whiskers extend to the smallest and largest values (or some defined lower and upper limits).

C chart. An attribute control chart that plots the total number of defects per unit in a subgroup. Similar to a defects-per-unit or U chart.

Categorical data. Data consisting of counts or observations that can be classified into categories. The categories may be descriptive.

Causal variable. When y = f(x) and y is considered to be caused by x, x is sometimes called a causal variable.

Cause-and-effect diagram. A chart used to organize the various potential causes of a problem. Also called a fishbone diagram.

Center line. A horizontal line on a control chart at the value that estimates the mean of the statistic plotted on the chart. See Control chart.

Central composite design (CCD). A second-order response surface design in k variables consisting of a two-level factorial, 2k axial runs, and one or more center points. The two-level factorial portion of a CCD can be a fractional factorial design when k is large. The CCD is the most widely used design for fitting a second-order model.

Central limit theorem. The simplest form of the central limit theorem states that the sum of n independently distributed random variables will tend to be normally distributed as n becomes large. It is a necessary and sufficient condition that none of the variances of the individual random variables are large in comparison to their sum. There are more general forms of the central theorem that allow infinite variances and correlated random variables, and there is a multivariate version of the theorem.

Central tendency. The tendency of data to cluster around some value. Central tendency is usually expressed by a measure of location such as the mean, median, or mode.

Chance cause. The portion of the variability in a set of observations that is due to only random forces and which cannot be traced to specific sources, such as operators, materials, or equipment. Also called a common cause.

Chi-square (or chi-squared) random variable. A continuous random variable that results from the sum of squares of independent standard normal random variables. It is a special case of a gamma random variable.

Chi-square test. Any test of significance based on the chi-square distribution. The most common chi-square tests are (1) testing hypotheses about the variance or standard deviation of a normal distribution and (2) testing goodness of fit of a theoretical distribution to sample data.

Coefficient of determination. See R².

Combination. A subset selected without replacement from a set used to determine the number of outcomes in events and sample spaces.

Comparative experiment. An experiment in which the treatments (experimental conditions) that are to be studied are included in the experiment. The data from the experiment are used to evaluate the treatments.

Completely randomized design (or experiment). A type of experimental design in which the treatments or design factors are assigned to the experimental units in a random manner. In designed experiments, a completely randomized design results from running all of the treatment combinations in random order.

Components of variance. The individual components of the total variance that are attributable to specific sources. This usually refers to the individual variance components arising from a random or mixed model analysis of variance.

Conditional mean. The mean of the conditional probability distribution of a random variable.

Conditional probability. The probability of an event given that the random experiment produces an outcome in another event.

Conditional probability density function. The probability density function of the conditional probability distribution of a continuous random variable.

Conditional probability distribution. The distribution of a random variable given that the random experiment produces an outcome in an event. The given event might specify values for one or more other random variables.

Conditional probability mass function. The probability mass function of the conditional probability distribution of a discrete random variable.

Conditional variance. The variance of the conditional probability distribution of a random variable.

Confidence coefficient. The probability 1 – α associated with a confidence interval expressing the probability that the stated interval will contain the true parameter value.

Confidence interval. If it is possible to write a probability statement of the form

where L and U are functions of only the sample data and θ is a parameter, then the interval between L and U is called a confidence interval (or a 100(1 – α)% confidence interval). The interpretation is that a statement that the parameter θ lies in this interval will be true 100(1 – α)% of the times that such a statement is made.

Confidence level. Another term for the confidence coefficient.

Confounding. When a factorial experiment is run in blocks and the blocks are too small to contain a complete replicate of the experiment, one can run a fraction of the replicate in each block, but this results in losing information on some effects. These effects are linked with or confounded with the blocks. In general, when two factors are varied such that their individual effects cannot be determined separately, their effects are said to be confounded.

Consistent estimator. An estimator that converges in probability to the true value of the estimated parameter as the sample size increases.

Contingency table. A tabular arrangement expressing the assignment of members of a data set according to two or more categories or classification criteria.

Continuity correction. A correction factor used to improve the approximation to binomial probabilities from a normal distribution.

Continuous distribution. A probability distribution for a continuous random variable.

Continuous random variable. A random variable with an interval (either finite or infinite) of real numbers for its range.

Continuous uniform random variable. A continuous random variable with range of a finite interval and a constant probability density function.

Contour plot. A two-dimensional graphic used for a bivariate probability density function that displays curves for which the probability density function is constant.

Contrast. A linear function of treatment means with coefficients that total zero. A contrast is a summary of treatment means that is of interest in an experiment.

Control chart. A graphical display used to monitor a process. It usually consists of a horizontal center line corresponding to the in-control value of the parameter that is being monitored and lower and upper control limits. The control limits are determined by statistical criteria and are not arbitrary, nor are they related to specification limits. If sample points fall within the control limits, the process is said to be in-control, or free from assignable causes. Points beyond the control limits indicate an out-of-control process; that is, assignable causes are likely present. This signals the need to find and remove the assignable causes.

Control limits. See Control chart.

Convolution. A method to derive the probability density function of the sum of two independent random variables from an integral (or sum) of probability density (or mass) functions.

Cook's distance. In regression, Cook's distance is a measure of the influence of each individual observation on the estimates of the regression model parameters. It expresses the distance that the vector of model parameter estimates with the ith observation removed lies from the vector of model parameter estimates based on all observations. Large values of Cook's distance indicate that the observation is influential.

Correction factor. A term used for the quantity (1/n)² that is subtracted from to give the corrected sum of squares defined as(1/n)x_i(x_i – )². The correction factor can also be written as n².

Correlation. In the most general usage, a measure of the interdependence among data. The concept may include more than two variables. The term is most commonly used in a narrow sense to express the relationship between quantitative variables or ranks.

Correlation coefficient. A dimensionless measure of the linear association between two variables, usually lying in the interval from −1 to +1, with zero indicating the absence of correlation (but not necessarily the independence of the two variables).

Correlation matrix. A square matrix that contains the correlations among a set of random variables, say, X₁, X₂,..., X_k. The main diagonal elements of the matrix are unity and the off-diagonal elements r_ij are the correlations between X_i and X_j.

Counting techniques. Formulas used to determine the number of elements in sample spaces and events.

Covariance. A measure of association between two random variables obtained as the expected value of the product of the two random variables around their means; that is, Cov(X, Y) = E[(X − μ_X)(Y − μ_Y)].

Covariance matrix. A square matrix that contains the variances and covariances among a set of random variables, say, X₁, X₂,..., X_k. The main diagonal elements of the matrix are the variances of the random variables and the off-diagonal elements are the covariances between X_i and X_j. Also called the variance-covariance matrix. When the random variables are standardized to have unit variances, the covariance matrix becomes the correlation matrix.

Critical region. In hypothesis testing, this is the portion of the sample space of a test statistic that will lead to rejection of the null hypothesis.

Critical value(s). The value of a statistic corresponding to a stated significance level as determined from the sampling distribution. For example, if P(Z ≥ z_0.0025) = P(Z ≥ 1.96) = 0.025, then Z_0.025 = 1.96 is the critical value of z at the 0.025 level of significance.

Crossed factors. Another name for factors that are arranged in a factorial experiment.

Cumulative distribution function. For a random variable X, the function of X defined as P(X ≤ x) that is used to specify the probability distribution.

Cumulative normal distribution function. The cumulative distribution of the standard normal distribution, often denoted as Φ(x) and tabulated in Appendix Table II.

Cumulative sum control chart (CUSUM). A control chart in which the point plotted at time t is the sum of the measured deviations from target for all statistics up to time t.

Curvilinear regression. An expression sometimes used for nonlinear regression models or polynomial regression models.

Decision interval. A parameter in a tabular CUSUM algorithm that is determined from a trade-off between false alarms and the detection of assignable causes.

Defect. Used in statistical quality control, a defect is a particular type of nonconformance to specifications or requirements. Sometimes defects are classified into types, such as appearance defects and functional defects.

Defect concentration diagram. A quality tool that graphically shows the location of defects on a part or in a process.

Defects-per-unit control chart. See U chart.

Defining relation. A subset of effects in a fractional factorial design that define the aliases in the design.

Degrees of freedom. The number of independent comparisons that can be made among the elements of a sample. The term is analogous to the number of degrees of freedom for an object in a dynamic system, which is the number of independent coordinates required to determine the motion of the object.

Deming. W. Edwards Deming (1900–1993) was a leader in the use of statistical quality control.

Deming's 14 points. A management philosophy promoted by W. Edwards Deming that emphasizes the importance of change and quality.

Density function. Another name for a probability density function.

Dependent variable. The response variable in regression or a designed experiment.

Design matrix. A matrix that provides the tests that are to be conducted in an experiment.

Designed experiment. An experiment in which the tests are planned in advance and the plans usually incorporate statistical models. See Experiment.

Discrete distribution. A probability distribution for a discrete random variable.

Discrete random variable. A random variable with a finite (or countably infinite) range.

Discrete uniform random variable. A discrete random variable with a finite range and constant probability mass function.

Dispersion. The amount of variability exhibited by data.

Distribution free method(s). Any method of inference (hypothesis testing or confidence interval construction) that does not depend on the form of the underlying distribution of the observations. Sometimes called non-parametric method(s).

Distribution function. Another name for a cumulative distribution function.

Efficiency. A concept in parameter estimation that uses the variances of different estimators; essentially, an estimator is more efficient than another estimator if it has smaller variance. When estimators are biased, the concept requires modification.

Empirical model. A model to relate a response to one or more regressors or factors that is developed from data obtained from the system.

Enumerative study. A study in which a sample from a population is used to make inference to the population. See Analytic study.

Erlang random variable. A continuous random variable that is the sum of a fixed number of independent, exponential random variables.

β-error (or β-risk). In hypothesis testing, an error incurred by failing to reject a null hypothesis when it is actually false (also called a type II error).

α-error (or α-risk). In hypothesis testing, an error incurred by rejecting a null hypothesis when it is actually true (also called a type I error).

Error mean square. The error sum of squares divided by its number of degrees of freedom.

Error of estimation. The difference between an estimated value and the true value.

Error propagation. An analysis of how the variance of the random variable that represents that output of a system depends on the variances of the inputs. A formula exists when the output is a linear function of the inputs and the formula is simplified if the inputs are assumed to be independent.

Error sum of squares. In analysis of variance, this is the portion of total variability that is due to the random component in the data. It is usually based on replication of observations at certain treatment combinations in the experiment. It is sometimes called the residual sum of squares, although this is really a better term to use only when the sum of squares is based on the remnants of a model-fitting process and not on replication.

Error variance. The variance of an error term or component in a model.

Estimate (or point estimate). The numerical value of a point estimator.

Estimator (or point estimator). A procedure for producing an estimate of a parameter of interest. An estimator is usually a function of only sample data values, and when these data values are available, it results in an estimate of the parameter of interest.

Event. A subset of a sample space.

Exhaustive. A property of a collection of events that indicates that their union equals the sample space.

Expected value. The expected value of a random variable X is its long-term average or mean value. In the continuous case, the expected value of X is E(X) = xf(x)dx where f(x) is the density function of the random variable X.

Experiment. A series of tests in which changes are made to the system under study.

Exponential random variable. A continuous random variable that is the time between events in a Poisson process.

Extra sum of squares method. A method used in regression analysis to conduct a hypothesis test for the additional contribution of one or more variables to a model.

Factorial experiment. A type of experimental design in which every level of one factor is tested in combination with every level of another factor. In general, in a factorial experiment, all possible combinations of factor levels are tested.

False alarm. A signal from a control chart when no assignable causes are present.

F distribution. The distribution of the random variable defined as the ratio of two independent chi-square random variables, each divided by its number of degrees of freedom.

Finite population correction factor. A term in the formula for the variance of a hypergeometric random variable.

First-order model. A model that contains only first-order terms. For example, the first-order response surface model in two variables is y = β₀ + β₁x₁ + β₂x₂ + ε. A first-order model is also called a main effects model.

Fisher's least significant difference (LSD) method. A series of pair-wise hypothesis tests of treatment means in an experiment to determine which means differ.

Fixed factor (or fixed effect). In analysis of variance, a factor or effect is considered fixed if all the levels of interest for that factor are included in the experiment. Conclusions are then valid about this set of levels only, although when the factor is quantitative, it is customary to fit a model to the data for interpolating between these levels.

Forward selection. A method of variable selection in regression, where variables are inserted one at a time into the model until no other variables that contribute significantly to the model can be found.

Fraction defective control chart. See P chart.

Fraction defective. In statistical quality control, that portion of a number of units or the output of a process that is defective.

Fractional factorial experiment. A type of factorial experiment in which not all possible treatment combinations are run. This is usually done to reduce the size of an experiment with several factors.

Frequency distribution. An arrangement of the frequencies of observations in a sample or population according to the values that the observations take on.

F-test. Any test of significance involving the F distribution. The most common F-tests are (1) testing hypotheses about the variances or standard deviations of two independent normal distributions, (2) testing hypotheses about treatment means or variance components in the analysis of variance, and (3) testing significance of regression or tests on subsets of parameters in a regression model.

Gamma function. A function used in the probability density function of a gamma random variable that can be considered to extend factorials.

Gamma random variable. A random variable that generalizes an Erlang random variable to noninteger values of the parameter r.

Gaussian distribution. Another name for the normal distribution, based on the strong connection of Karl F. Gauss to the normal distribution; often used in physics and electrical engineering applications.

Generating function. A function that is used to determine properties of the probability distribution of a random variable. See Moment-generating function.

Generator. Effects in a fractional factorial experiment that are used to construct the experimental tests used in the experiment. The generators also define the aliases.

Geometric mean. The geometric mean of a set of n positive data values is the nth root of the product of the data values; that is, = ()^1/n.

Geometric random variable. A discrete random variable that is the number of Bernoulli trials until a success occurs.

Goodness of fit. In general, the agreement of a set of observed values and a set of theoretical values that depend on some hypothesis. The term is often used in fitting a theoretical distribution to a set of observations.

Harmonic mean. The harmonic mean of a set of data values is the reciprocal of the arithmetic mean of the reciprocals of the data values; that is, = .

Hat matrix. In multiple regression, the matrix H = X(X′X)⁻¹X′. This a projection matrix that maps the vector of observed response values into a vector of fitted values by = X(X′X)⁻¹ X′y = Hy.

Hidden extrapolation. An extrapolation is a prediction in a regression analysis that is made at point (x₁, x₂,..., x_k) that is remote from the data used to generate the model. Hidden extrapolation occurs when it is not obvious that the point is remote. This can occur when multicollinearity is present in the data used to construct the model.

Histogram. A univariate data display that uses rectangles proportional in area to class frequencies to visually exhibit features of data such as location, variability, and shape.

Homogeneity test. In a two-way (r by c) contingency table, this tests if the proportions in the c categories are the same for all r populations.

Hypergeometric random variable. A discrete random variable that is the number of success obtained from a sample drawn without replacement from a finite populations.

Hypothesis (as in statistical hypothesis). A statement about the parameters of a probability distribution or a model, or a statement about the form of a probability distribution.

Hypothesis testing. Any procedure used to test a statistical hypothesis.

Independence. A property of a probability model and two (or more) events that allows the probability of the intersection to be calculated as the product of the probabilities.

Independence test. In a two-way (r by c) contingency table, this tests if the row and column categories are independent.

Independent random variables. Random variables for which P(X ∈ A, Y ∈ B) = P(X ∈ A)P(Y ∈ B) for any sets A and B in the range of X and Y, respectively. There are several equivalent descriptions of independent random variables.

Independent variable. The predictor or regressor variables in a regression model.

Inference. Conclusion from a statistical analysis. It usually refers to the conclusion from a hypothesis test or an interval estimate.

Indicator variable(s). Variables that are assigned numerical values to identify the levels of a qualitative or categorical response. For example, a response with two categorical levels (yes and no) could be represented with an indicator variable taking on the values 0 and 1.

Individuals control chart. A Shewhart control chart in which each plotted point is an individual measurement, rather than a summary statistic. See Control chart, Shewhart control chart.

Influential observation. An observation in a regression analysis that has a large effect on estimated parameters in the model. Influence is measured by the change in parameters when the influential observation is included and excluded in the analysis.

Interaction. In factorial experiments, two factors are said to interact if the effect of one variable is different at different levels of the other variables. In general, when variables operate independently of each other, they do not exhibit interaction.

Intercept. The constant term in a regression model.

Interquartile range. The difference between the third and first quartiles in a sample of data. The interquartile range is less sensitive to extreme data values than the usual sample range.

Interval estimation. The estimation of a parameter by a range of values between lower and upper limits, in contrast to point estimation, where the parameter is estimated by a single numerical value. A confidence interval is a typical interval estimation procedure.

Intrinsically linear model. In regression analysis, a nonlinear function that can be expressed as a linear function after a suitable transformation is called intrinsically linear.

Jacobian. A matrix of partial derivatives that is used to determine the distribution of transformed random variables.

Joint probability density function. A function used to calculate probabilities for two or more continuous random variables.

Joint probability distribution. The probability distribution for two or more random variables in a random experiment. See Joint probability mass function, Joint probability density function.

Joint probability mass function. A function used to calculate probabilities for two or more discrete random variables.

Kurtosis. A measure of the degree to which a unimodal distribution is peaked.

Lack of memory property. A property of a Poisson process. The probability of a count in an interval depends only on the length of the interval (and not on the starting point of the interval). A similar property holds for a series of Bernoulli trials. The probability of a success in a specified number of trials depends only on the number of trials (and not on the starting trial).

Least significance difference test (or Fisher's LSD test). An application of the t-test to compare pairs of means following rejection of the null hypothesis in an analysis of variance. The error rate is difficult to calculate exactly because the comparisons are not all independent.

Least squares (method of). A method of parameter estimation in which the parameters of a system are estimated by minimizing the sum of the squares of the differences between the observed values and the fitted or predicted values from the system.

Least squares estimator. Any estimator obtained by the method of least squares.

Level of significance. If Z is the test statistic for a hypothesis, and the distribution of Z when the hypothesis is true are known, then we can find the probabilities P(Z ≤ z_L) and P(Z ≥ z_U). Rejection of the hypothesis is usually expressed in terms of the observed value of Z falling outside the interval from z_L to z_U. The probabilities P(Z ≤ z_L) and P(Z ≥ z_U) are usually chosen to have small values, such as 0.01, 0.025, 0.05, or 0.10, and are called levels of significance. The actual levels chosen are somewhat arbitrary and are often expressed in percentages, such as a 5% level of significance.

Levels of a factor. The settings (or conditions) used for a factor in an experiment.

Likelihood function. Suppose that the random variables X₁, X₂,..., X_n have a joint distribution given by f(x₁, x₂,..., x_n; θ₁, θ₂,..., θ_p) where the θs are unknown parameters. This joint distribution, considered as a function of the θs for fixed x's, is called the likelihood function.

Likelihood principle. This principle states that the information about a model given by a set of data is completely contained in the likelihood.

Likelihood ratio. Let x₁, x₂,..., x_n be a random sample from the population f(x;θ). The likelihood function for this sample is L = f(x_i; θ) We wish to test the hypothesis H₀ : θ ∈ ω, where ω is a subset of the possible values Ω for θ. Let the maximum value of L with respect to θ over the entire set of values that the parameter can take on be denoted by L(), and let the maximum value of L with θ restricted to the set of values given by ω be L(). The null hypothesis is tested by using the likelihood ratio λ = L() / L(), or a simple function of it. Large values of the likelihood ratio are consistent with the null hypothesis.

Likelihood ratio test. A test of a null hypothesis versus an alternative hypothesis using a test statistic derived from a likelihood ratio.

Linear function of random variables. A random variable that is defined as a linear function of several random variables.

Linear model. A model in which the observations are expressed as a linear function of the unknown parameters. For example, y = β₀ + β₁x + ε and y = β₀ + β₁x + β₂x² + ε are linear models.

Location parameter. A parameter that defines a central value in a sample or a probability distribution. The mean and the median are location parameters.

Logistic regression. A regression model that is used to model a categorical response. For a binary (0, 1) response, the model assumes that the logarithm of the ratio of probabilities (for zero and one) is linearly related to the regressor variables.

Lognormal random variable. A continuous random variable with probability distribution equal to that of exp(W) for a normal random variable W.

Main effect. An estimate of the effect of a factor (or variable) that independently expresses the change in response due to a change in that factor, regardless of other factors that may be present in the system.

Marginal probability density function. The probability density function of a continuous random variable obtained from the joint probability distribution of two or more random variables.

Marginal probability distribution. The probability distribution of a random variable obtained from the joint probability distribution of two or more random variables.

Marginal probability mass function. The probability mass function of a discrete random variable obtained from the joint probability distribution of two or more random variables.

Maximum likelihood estimation. A method of parameter estimation that maximizes the likelihood function of a sample.

Mean. The mean usually refers either to the expected value of a random variable or to the arithmetic average of a set of data.

Mean square. In general, a mean square is determined by dividing a sum of squares by the number of degrees of freedom associated with the sum of squares.

Mean square(d) error. The expected squared deviation of an estimator from the true value of the parameter it estimates. The mean square error can be decomposed into the variance of the estimator plus the square of the bias; that is, MSE() = E( – θ)² = V() + [E() – θ]² .

Mechanistic model. A model developed from theoretical knowledge or experience in contrast to a model developed from data. See Empirical model.

Median. The median of a set of data is that value that divides the data into two equal halves. When the number of observations is even, say 2n, it is customary to define the median as the average of the nth and (n + 1)st rank-ordered values. The median can also be defined for a random variable. For example, in the case of a continuous random variable X, the median M can be defined as f(x)dx = f(x)dx = 1/2.

Method of steepest ascent. A technique that allows an experimenter to move efficiently toward a set of optimal operating conditions by following the gradient direction. The method of steepest ascent is usually employed in conjunction with fitting a first-order response surface and deciding that the current region of operation is inappropriate.

Mixed model. In an analysis of variance context, a mixed model contains both random and fixed factors.

Mode. The mode of a sample is that observed value that occurs most frequently. In a probability distribution f(x) with continuous first derivative, the mode is a value of x for which df(x) / dx = 0 and d²f(x) / dx² < 0. There may be more than one mode of either a sample or a distribution.

Moment (or population moment). The expected value of a function of a random variable such as E(X – c)^r for constants c and r. When c = 0, it is said that the moment is about the origin. See Moment-generating function.

Moment estimator. A method of estimating parameters by equating sample moments to population moments. Since the population moments will be functions of the unknown parameters, this results in equations that may be solved for estimates of the parameters.

Moment-generating function. A function that is used to determine properties (such as moments) of the probability distribution of a random variable. It is the expected value of exp(tX). See Generating function, Moment.

Moving range. The absolute value of the difference between successive observations in time-ordered data. Used to estimate chance variation in an individual control chart.

Multicollinearity. A condition occurring in multiple regression where some of the predictor or regressor variables are nearly linearly dependent. This condition can lead to instability in the estimates of the regression model parameters.

Multinomial distribution. The joint probability distribution of the random variables that count the number of results in each of k classes in a random experiment with a series of independent trials with constant probability of each class on each trial. It generalizes a binomial distribution.

Multiplication rule. For probability, a formula used to determine the probability of the intersection of two (or more) events. For counting techniques, a formula used to determine the number of ways to complete an operation from the number of ways to complete successive steps.

Mutually exclusive events. A collection of events whose intersections are empty.

Natural tolerance limits. A set of symmetric limits that are three times the process standard deviation from the process mean.

Negative binomial random variable. A discrete random variable that is the number of trials until a specified number of successes occur in Bernoulli trials.

Nonlinear regression model. A regression model that is nonlinear in the parameters. It is sometimes applied to regression models that are nonlinear in the regressors or predictors, but this is an incorrect usage.

Nonparametric statistical method(s). See Distribution free method(s).

Normal approximation. A method to approximate probabilities for binomial and Poisson random variables.

Normal equations. The set of simultaneous linear equations arrived at in parameter estimation using the method of least squares.

Normal probability plot. A specially constructed plot for a variable x (usually on the abscissa) in which y (usually on the ordinate) is scaled so that the graph of the normal cumulative distribution is a straight line.

Normal random variable. A continuous random variable that is the most important one in statistics because it results from the central limit theorem. See Central limit theorem.

NP chart. An attribute control chart that plots the total of defective units in a subgroup. Similar to a fraction-defective chart or P chart.

Nuisance factor. A factor that probably influences the response variable, but which is of no interest in the current study. When the levels of the nuisance factor can be controlled, blocking is the design technique that is customarily used to remove its effect.

Null distribution. In a hypothesis test, the distribution of the test statistic when the null hypothesized is assumed to be true.

Null hypothesis. This term generally relates to a particular hypothesis that is under test, as distinct from the alternative hypothesis (which defines other conditions that are feasible but not being tested). The null hypothesis determines the probability of type I error for the test procedure.

Observational study. A system is observed and data might be collected, but changes are not made to the system. See Experiment.

Odds ratio. The odds equals the ratio of two probabilities. In logistic regression, the logarithm of the odds is modeled as a linear function of the regressors. Given values for the regressors at a point, the odds can be calculated. The odds ratio is the odds at one point divided by the odds at another.

One-way model. In an analysis of variance context, this involves a single variable or factor with a different levels.

Operating characteristic curves (OC curves). A plot of the probability of type II error versus some measure of the extent to which the null hypothesis is false. Typically, one OC curve is used to represent each sample size of interest.

Optimization experiment. A experiment conducted to improve (or optimize) a system or process. It is assumed that the important factors are known.

Orthogonal. There are several related meanings, including the mathematical sense of perpendicular, two variables being said to be orthogonal if they are statistically independent, or in experimental design where a design is orthogonal if it admits statistically independent estimates of effects.

Orthogonal design. See Orthogonal.

Outcome. An element of a sample space.

Outlier(s). One or more observations in a sample that are so far from the main body of data that they give rise to the question that they may be from another population.

Overcontrol. Unnecessary adjustments made to processes that increase the deviations from target.

Overfitting. Adding more parameters to a model than is necessary.

P chart. An attribute control chart that plots the proportion of defective units in a subgroup. Also called a fraction-defective control chart. Similar to an NP chart.

Parameter estimation. The process of estimating the parameters of a population or probability distribution. Parameter estimation, along with hypothesis testing, is one of the two major techniques of statistical inference.

Parameter. An unknown quantity that may vary over a set of values. Parameters occur in probability distributions and in statistical models, such as regression models.

Pareto chart. A bar chart used to rank the causes of a problem.

PCR. A process capability ratio with numerator equal to the difference between the product specification limits and denominator equal to six times the process standard deviation. Said to measure the potential capability of the process because the process mean is not considered. See Process capability, Process capability ratio, Process capability study, and PCR_k. Sometimes denoted as C_p in other references.

PCR_k. A process capability ratio with numerator equal to the difference between the product target and the nearest specification limit and denominator equal to three times the process standard deviation. Said to measure the actual capability of the process because the process mean is considered. See Process capability, Process capability ratio, Process capability study, and PCR. Sometimes denoted as C_pk in other references.

Percentage point. A particular value of a random variable determined from a probability (expressed as a percentage). For example, the upper 5 percentage point of the standard normal random variable is z_0.05 = 1.645.

Percentile. The set of values that divide the sample into 100 equal parts.

Permutation. An ordered sequence of the elements in a set used to determine the number of outcomes in events and sample spaces.

Point estimator. See Estimator.

Poisson process. A random experiment with events that occur in an interval and satisfy the following assumptions. The interval can be partitioned into subintervals such that the probability of more than one event in a subinterval is zero, the probability of an event in a subinterval is proportional to the length of the subinterval, and the event in each subinterval is independent of other subintervals.

Poisson random variable. A discrete random variable that is the number of events that occur in a Poisson process.

Pooled t-test. A hypothesis to compare the means of two populations with the variances assumed to be equal.

Pooling. When several sets of data can be thought of as having been generated from the same model, it is possible to combine them, usually for purposes of estimating one or more parameters. Combining the samples for this purpose is usually called pooling and it is commonly used to estimate a variance.

Population standard deviation. See Standard deviation.

Population variance. See Variance.

Population. Any finite or infinite collection of individual units or objects.

Posterior distribution. The probability distribution for a parameter in a Bayesian analysis calculated from the prior distribution and the conditional distribution of the data given the parameter.

Power. The power of a statistical test is the probability that the test rejects the null hypothesis when the null hypothesis is indeed false. Thus, the power is equal to one minus the probability of type II error.

Prediction. The process of determining the value of one or more statistical quantities at some future point in time. In a regression model, predicting the response y for some specified set of regressors or predictor variables also leads to a predicted value, although there may be no temporal element to the problem.

Prediction interval. The interval between a set of upper and lower limits associated with a predicted value designed to show on a probability basis the range of error associated with the prediction.

Predictor variable(s). The independent or regressor variable(s) in a regression model.

PRESS statistic. In regression analysis, the predicted residual sum of squares. Delete each point and estimate the parameters of the model from the data that remain. Estimate the deleted point from this model. Restore the point and then delete the next point. Each point is estimated once and the sum of squares of these errors is calculated.

Prior distribution. The initial probability distribution assumed for a parameter in a Bayesian analysis.

Probability. A numerical measure between 0 and 1 assigned to events in a sample space. Higher numbers indicate the event is more likely to occur. See Axioms of probability.

Probability density function. A function used to calculate probabilities and to specify the probability distribution of a continuous random variable.

Probability distribution. For a sample space, a description of the set of possible outcomes along with a method to determine probabilities. For a random variable, a probability distribution is a description of the range along with a method to determine probabilities.

Probability mass function. A function that provides probabilities for the values in the range of a discrete random variable.

Probability plot. A scatter plot used to judge if data can reasonably be assumed to follow a particular probability distribution. A normal probability plot is often used to evaluate the normality assumption of data or residuals.

Process capability. The capability of a process to produce product within specification limits. See Process capability ratio, Process capability study, PCR, and PCR_k.

Process capability ratio. A ratio that relates the width of the product specification limits to measures of process performance. Used to quantify the capability of the process to produce product within specifications. See Process capability, Process capability study, PCR, and PCR_k.

Process capability study. A study that collects data to estimate process capability. See Process capability, Process capability ratio, PCR, and PCR_k.

P-Value. The exact significance level of a statistical test; that is, the probability of obtaining a value of the test statistic that is at least as extreme as that observed when the null hypothesis is true.

Qualitative (data). Data derived from nonnumeric attributes, such as sex, ethnic origin or nationality, or other classification variables.

Quality control. Systems and procedures used by an organization to assure that the outputs from processes satisfy customers.

Quantiles. The set of n − 1 values of a variable that partition it into a number n of equal proportions. For example, n − 1 = 3 values partition data into four quantiles, with the central value usually called the median and the lower and upper values usually called the lower and upper quartiles, respectively.

Quantitative (data). Data in the form of numerical measurements or counts.

Quartiles. The three values of a variable that partition it into four equal parts. The central value is usually called the median and the lower and upper values are usually called the lower and upper quartiles, respectively. See Quantiles.

R². A quantity used in regression models to measure the proportion of total variability in the response accounted for by the model. Computationally, R² = SS_Regression/SS_Total, and large values of R² (near unity) are considered good. However, it is possible to have large values of R² and find that the model is unsatisfactory. R² is also called the coefficient of determination (or the coefficient of multiple determination in multiple regression).

R chart. A control chart that plots the range of the measurements in a subgroup that is used to monitor the variance of the process.

Random. Nondeterministic, occurring purely by chance, or independent of the occurrence of other events.

Random effects model. In an analysis of variance context, this refers to a model that involves only random factors.

Random error. An error (usually a term in a statistical model) that behaves as if it were drawn at random from a particular probability distribution.

Random experiment. An experiment that can result in different outcomes, even though it is repeated in the same manner each time.

Random factor. In analysis of variance, a factor whose levels are chosen at random from some population of factor levels.

Random order. A sequence or order for a set of objects that is carried out in such a way that every possible ordering is equally likely. In experimental design, the runs of the experiment are typically arranged and carried out in random order.

Random sample. A sample is said to be random if it is selected in such a way that every possible sample has the same probability of being selected.

Random variable. A function that assigns a real number to each outcome in the sample space of a random experiment.

Randomization. Randomly assign treatments to experimental units or conditions in an experiment. This is done to reduce the opportunity for a treatment to be favored or disfavored (biased) by test conditions.

Randomized complete block design. A type of experimental design in which treatment or factor levels are assigned to blocks in a random manner.

Range. The largest minus the smallest of a set of data values. The range is a simple measure of variability and is widely used in quality control.

Range (control) chart. A control chart used to monitor the variability (dispersion) in a process. See Control chart.

Rank. In the context of data, the rank of a single observation is its ordinal number when all data values are ordered according to some criterion, such as their magnitude.

Rational subgroup. A sample of data selected in a manner to include chance sources of variation and to exclude assignable sources of variation to the extent possible.

Reference distribution. The distribution of a test statistic when the null hypothesis is true. Sometimes a reference distribution is called the null distribution of the test statistic.

Reference value. A parameter set in a tabular CUSUM algorithm that is determined from the magnitude of the process shift that should be detected.

Regression. The statistical methods used to investigate the relationship between a dependent or response variable y and one or more independent variables x. The independent variables are usually called regressor variables or predictor variables.

Regression coefficient(s). The parameter(s) in a regression model.

Regression diagnostics. Techniques for examining a fitted regression model to investigate the adequacy of the fit and to determine if any of the underlying assumptions have been violated.

Regression line (or curve). A graphical display of a regression model, usually with the response y on the ordinate and the regressor x on the abscissa.

Regression sum of squares. The portion of the total sum of squares attributable to the model that has been fit to the data.

Regressor variable. The independent or predictor variable in a regression model.

Rejection region. In hypothesis testing, this is the region in the sample space of the test statistic that leads to rejection of the null hypothesis when the test statistic falls in this region.

Relative frequency. The relative frequency of an event is the proportion of times the event occurred in a series of trials of a random experiment.

Reliability. The probability that a specified mission will be completed. It usually refers to the probability that a lifetime of a continuous random variable exceeds a specified time limit.

Replicates. One of the independent repetitions of one or more treatment combinations in an experiment.

Replication. The independent execution of an experiment more than once.

Reproductive property of the normal distribution. A linear combination of independent, normal random variables is a normal random variable.

Residual. Generally this is the difference between the observed and the predicted value of some variable. For example, in regression a residual is the difference between the observed value of the response and the corresponding predicted value obtained from the regression model.

Residual analysis (and plots). Any technique that uses the residuals, usually to investigate the adequacy of the model that was used to generate the residuals.

Residual sum of squares. See Error sum of squares.

Resolution. A measure of severity of aliasing in a fractional factorial design. We commonly consider resolution III, IV, and V designs.

Response (variable). The dependent variable in a regression model or the observed output variable in a designed experiment.

Response surface. When a response y depends on a function of k quantitative variables x₁ x₂,..., x_k, the values of the response may be viewed as a surface in k + 1 dimensions. This surface is called a response surface. Response surface methodology is a subset of experimental design concerned with approximating this surface with a model and using the resulting model to optimize the system or process.

Response surface designs. Experimental designs that have been developed to work well in fitting response surfaces. These are usually designs for fitting a first- or second-order model. The central composite design is a widely used second-order response surface design.

Ridge regression. A method for fitting a regression model that is intended to overcome the problems associated with using standard (or ordinary) least squares when there is a problem with multicollinearity in the data.

Rotatable design. In a rotatable design, the variance of the predicted response is the same at all points that are the same distance from the center of the design.

Run rules. A set of rules applied to the points plotted on a Shewhart control chart that are used to make the chart more sensitized to assignable causes. See Control chart, Shewhart control chart.

Runs test. A nonparametric test to compare two distributions or check for independence of measurements.

S chart. A control chart that plots the standard deviation of the measurements in a subgroup that is used to monitor the variance of the process.

Sample. Any subset of the elements of a population.

Sample mean. The arithmetic average or mean of the observations in a sample. If the observations are x₁, x₂,..., x_n, then the sample mean is (1/n). The sample mean is usually denoted by .

Sample moment. The quantity (1/n) is called the kth sample moment.

Sample range. See Range.

Sample size. The number of observations in a sample.

Sample space. The set of all possible outcomes of a random experiment.

Sample standard deviation. The positive square root of the sample variance. The sample standard deviation is the most widely used measure of variability of sample data.

Sample variance. A measure of variability of sample data, defined as s² = [1 / (n − 1)](x_i − )², where is the sample mean.

Sampling distribution. The probability distribution of a statistic. For example, the sampling distribution of the sample mean is the normal distribution.

Scatter diagram. A diagram displaying observations on two variables, x and y. Each observation is represented by a point showing its x-y coordinates. The scatter diagram can be very effective in revealing the joint variability of x and y or the nature of the relationship between them.

Screening experiment. An experiment designed and conducted for the purpose of screening out or isolating a promising set of factors for future experimentation. Many screening experiments are fractional factorials, such as two-level fractional factorial designs.

Second-order model. A model that contains second-order terms. For example, the second-order response surface model in two variables is y = β₀ + β₁x₁ + β₂ x₂ + β₁₂x₁x₂ + + ε. The second order terms in this model are β₁₂x₁x₂, , and .

Shewhart control chart. A specific type of control chart developed by Walter A. Shewhart. Typically, each plotted point is a summary statistic calculated from the data in a rational subgroup. See Control chart.

Sign test. A statistical test based on the signs of certain functions of the observations and not their magnitudes.

Signed-rank test. A statistical test based on the differences within a set of paired observations. Each difference has a sign and a rank, and the test uses the sum of the differences with regard to sign.

Significance. In hypothesis testing, an effect is said to be significant if the value of the test statistic lies in the critical region.

Significance level. See Level of significance.

Six-sigma process. Originally used to describe a process with the mean at least six standard deviations from the nearest specification limits. It has now been used to describe any process with a defect rate of 3.4 parts per million.

Skewness. A term for asymmetry usually employed with respect to a histogram of data or a probability distribution.

Specification limits. Numbers that define the region of measurement for acceptable product. Usually there is an upper and lower limit, but one-sided limits can also be used.

Standard deviation. The positive square root of the variance. The standard deviation is the most widely used measure of variability.

Standard error. The standard deviation of the estimator of a parameter. The standard error is also the standard deviation of the sampling distribution of the estimator of a parameter.

Standard normal random variable. A normal random variable with mean zero and variance one that has its cumulative distribution function tabulated in Appendix Table II.

Standardize. The transformation of a normal random variable that subtracts its mean and divides by its standard deviation to generate a standard normal random variable.

Standardized residual. In regression, the standardized residual is computed by dividing the ordinary residual by the square root of the residual mean square. This produces scaled residuals that have, approximately, a unit variance.

Statistic. A summary value calculated from a sample of observations. Usually, a statistic is an estimator of some population parameter.

Statistical inference. See Inference.

Statistical Process Control (SPC). A set of problem-solving tools based on data that are used to improve a process.

Statistical quality control. Statistical and engineering methods used to measure, monitor, control, and improve quality.

Statistics. The science of collecting, analyzing, interpreting, and drawing conclusions from data.

Steepest ascent (or descent). A strategy for a series of tests to optimize a response used along with response surface models.

Stem-and-leaf diagram. A method of displaying data in which the stem corresponds to a range of data values and the leaf represents the next digit. It is an alternative to the histogram but displays the individual observations rather than sorting them into bins.

Stepwise regression. A method of selecting variables for inclusion in a regression model. It operates by introducing the candidate variables one at a time (as in forward selection) and then attempting to remove variables following each forward step.

Studentized range. The range of a sample divided by the sample standard deviation.

Studentized residual. In regression, the studentized residual is calculated by dividing the ordinary residual by its exact standard deviation, producing a set of scaled residuals that have, exactly, unit standard deviation.

Sufficient statistic. An estimator is said to be a sufficient statistic for an unknown parameter if the distribution of the sample given the statistic does not depend on the unknown parameter. This means that the distribution of the estimator contains all of the useful information about the unknown parameter.

Tabular CUSUM. A numerical algorithm used to detect assignable causes on a cumulative sum control chart. See V mask.

Tampering. Another name for overcontrol.

t Distribution. The distribution of the random variable defined as the ratio of two independent random variables. The numerator is a standard normal random variable and the denominator is the square root of a chi-square random variable divided by its number of degrees of freedom.

Test statistic. A function of a sample of observations that provides the basis for testing a statistical hypothesis.

Time series. A set of ordered observations taken at points in time.

Tolerance interval. An interval that contains a specified proportion of a population with a stated level of confidence.

Tolerance limits. A set of limits between which some stated proportion of the values of a population must fall with a specified level of confidence.

Total probability rule. Given a collection of mutually exclusive events whose union is the sample space, the probability of an event can be written as the sum of the probabilities of the intersections of the event with the members of this collection.

Treatment. In experimental design, a treatment is a specific level of a factor of interest. Thus, if the factor is temperature, the treatments are the specific temperature levels used in the experiment.

Treatment effect. The mean change to the response due to the presence of the treatment.

Treatment sum of squares. In analysis of variance, this is the sum of squares that accounts for the variability in the response variable due to the different treatments that have been applied.

t-test. Any test of significance based on the t distribution. The most common t-tests are (1) testing hypotheses about the mean of a normal distribution with unknown variance, (2) testing hypotheses about the means of two normal distributions, and (3) testing hypotheses about individual regression coefficients.

Two-level factorial experiment. A full or fractional factorial experiment with all factors tested at only two levels (settings) each. See 2^k factorial experiment.

Type I error. In hypothesis testing, an error incurred by rejecting a null hypothesis when it is actually true (also called an α-error).

Type II error. In hypothesis testing, an error incurred by failing to reject a null hypothesis when it is actually false (also called a β-error).

U chart. An attribute control chart that plots the average number of defects per unit in a subgroup. Also called a defects-per-unit control chart. Similar to a C chart.

Unbiased estimator. An estimator that has its expected value equal to the parameter that is being estimated is said to be unbiased.

Uniform random variable. Refers to either a discrete or continuous uniform random variable.

Uniqueness property of moment-generating function. Refers to the fact that random variables with the same moment-generating function have the same distribution.

Universe. Another name for population.

V mask. A geometrical figure used to detect assignable causes on a cumulative sum control chart. With appropriate values for parameters, identical conclusions can be made from a V mask and a tabular CUSUM.

Variable selection. The problem of selecting a subset of variables for a model from a candidate list that contains all or most of the useful information about the response in the data.

Variables control chart. Any control chart for a continuous random variable. See Attribute control chart.

Variance. A measure of variability defined as the expected value of the square of the random variable around its mean.

Variance component. In analysis of variance models involving random effects, one of the objectives is to determine how much variability can be associated with each of the potential sources of variability defined by the experimenters. It is customary to define a variance associated with each of these sources. These variances in some sense sum to the total variance of the response, and are usually called variance components.

Variance inflation factors. Quantities used in multiple regression to assess the extent of multicollinearity (or near linear dependence) in the regressors. The variance inflation factor for the ith regressor VIF_i can be defined as VIF_i = [1/(1 − )], where is the coefficient of determination obtained when x_i is regressed on the other regressor variables. Thus, when x_i is nearly linearly dependent on a subset of the other regressors, will be close to unity and the value of the corresponding variance inflation factor will be large. Values of the variance inflation factors that exceed 10 are usually taken as a signal that multicollinearity is present.

Warning limits. Horizontal lines added to a control chart (in addition to the control limits) that are used to make the chart more sensitive to assignable causes.

Weibull random variable. A continuous random variable that is often used to model the time until failure of a physical system. The parameters of the distribution are flexible enough that the probability density function can assume many different shapes.

Western Electric rules. A specific set of run rules that were developed at Western Electric Corporation. See Run rules.

Wilcoxon rank-sum test. A nonparametric test for the equality of means in two populations. This is sometimes called the Mann-Whitney test.

Wilcoxon signed-rank test. A distribution-free test of the equality of the location parameters of two otherwise identical distributions. It is an alternative to the two-sample t-test for nonnormal populations.

With replacement. A method to select samples in which items are replaced between successive selections.

Without replacement. A method to select samples in which items are not replaced between successive selections.

chart. A control chart that plots the average of the measurements in a subgroup that is used to monitor the process mean.