2.1.1. Time plots of trends

2.1.2. Paired t-test

Let ${\overline{Y}}_{\text{pre}}$ and ${\overline{Y}}_{\text{post}}$ be the mean scores of the response before and after a medical treatment for a sample of N patients. If the researcher wants to test whether the two mean scores are different, the null and the alternative hypotheses should be written as ${\text{H}}_0:{\overline{Y}}_{\text{post}}={\overline{Y}}_{\text{pre}}$ and ${\text{H}}_1:{\overline{Y}}_{\text{post}}\ne {\overline{Y}}_{\text{pre}}$, respectively, for a two-tail test. Suppose that the variances of the pre- and posttest mean scores are equal with the same sample size. The equation of the paired t-test is

$t=\frac{{\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}}{s_D/\sqrt{N}}\text{,}$

(2.1)

where s_D is the sample standard deviation of the differences between all pre- and posttest pairs, and the denominator on the right of the equation is the corresponding standard error. The above statistic asymptotically follows a Student’s t distribution. Consequently, the corresponding p-value can be readily obtained from the distribution, generating results for testing the H₀ hypothesis. If the t score is associated with a p-value smaller than α, the null hypothesis about the difference between ${\overline{Y}}_{\text{pre}}$ and ${\overline{Y}}_{\text{post}}$ should be rejected; otherwise, the null hypothesis is accepted.

$t=\frac{{\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}}{\sqrt{\frac{s_{\text{pre}}^2}{N_{\text{pre}}}+\frac{s_{\text{post}}^2}{N_{\text{post}}}}}\text{,}$

(2.2)

where $s_{\text{pre}}^2$ and $s_{\text{post}}^2$ are the sample variances of the pre- and posttest scores, respectively, and N_pre, N_post are the sample sizes at the two time points. Equation (2.2) permits the researcher to use summary statistics for analyzing paired samples with data missing in one or the other samples. This convenient estimator using summary statistics, however, is based on the assumption that Y_pre and Y_post for each subject are independent.

2.1.3. Effect size between two means and its confidence interval

2.1.3.1. General specification of effect size

As a statistical index displaying the degree of departures from the null hypothesis, the effect size statistic is viewed as a complement to the significance test statistics such as the t-test. Although the significance test provides information on the probability of obtaining the given value based on alpha, the effect size index provides information on the practical significance (in biomedical studies, it is referred to as clinical significance). In other words, an effect size enables the researcher to interpret the meaning of the effect from the practical standpoint based on defined thresholds. Furthermore, effect sizes are standardized thereby permitting direct comparison between/among effect estimates of different metric units like the use of Z-scores. Given their practical focus and capacity to compare results across studies, the computation and presentation of effect sizes is popular in medical and psychological research. There are estimators for various types of data. In this text, effect size is portrayed as a descriptive, complementary means of describing longitudinal data. Therefore, I focus on the computation of an effect size on the distance of a measurement between two time points.

I start with Cohen’s, (Cohen, 1988, 1992) mathematical notation of effect size. Let ${\overline{Y}}_{\text{pre}}$ and ${\overline{Y}}_{\text{post}}$ be the mean scores of a measurement before and after a medical treatment for a sample of N patients. Cohen defines d as the difference between the two means divided by a standard deviation statistic, given by

$d_{\text{C}}=\frac{{\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}}{\sigma }\text{,}$

(2.3)

where d_C is Cohen’s d, and σ is the standard deviation for either population, assuming the statistic to be equal for the two populations (Cohen, 1988). Here, Cohen’s d measures the standardized mean difference between two population groups.

Because the population statistic σ is usually unknown, especially in observational studies, it may be more practical to replace σ with the sample standard deviation s for expressing an effect size (Glass, 1976; Hedges, 1981). Additionally, the sample standard deviations of the means for samples at two time points are often unequal due to missing observations in follow-ups, and therefore, it is important to specify how the sample standard deviation s for the mean difference is estimated. Glass (1976) proposes the use of the standard deviation for the second mean to estimate an effect size. In the case of a pre–post clinical trial, the Glass’s effect size is given by

$d_{\text{G}}=\frac{{\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}}{s_{\text{post}}}\text{,}$

(2.4)

where d_G is Glass’s d, and s_post is the sample standard deviation at the second time point. His argument depends on the fact that in clinical trials, the sample standard deviations of different groups generally differ, and the use of the post statistics is more appropriate.

Hedges (1981) suggests the use of a pooled standard deviation to calculate an effect size, given by

$d_{\text{H}}=\frac{{\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}}{s_{\text{pooled}}}\text{,}$

(2.5)

where d_H is Hedges’s d, and the pooled standard deviation s_pooled can be estimated by

$s_{\text{pooled}}=\sqrt{\frac{(N_{\text{pre}}-1)s_{\text{pre}}^2+(N_{\text{post}}-1)s_{\text{post}}^2}{N_{\text{pre}}+N_{\text{post}}-2}}\text{.}$

(2.6)

The terms $s_{\text{pre}}^2$, $s_{\text{post}}^2$ are sample variances for ${\overline{Y}}_{\text{pre}}$ and ${\overline{Y}}_{\text{post}}$, respectively, given by

$s_{\text{pre}}^2=\frac{1}{N_{\text{pre}}-1}\sum _{i=1}^{n_{\text{pre}}}{(Y_{\text{pre},i}-{\overline{Y}}_{\text{pre}})}^2\text{,}$

(2.7)

$s_{\text{post}}^2=\frac{1}{N_{\text{post}}-1}\sum _{i^{\prime }=1}^{n_{\text{post}}}{(Y_{\text{post},i^{\prime }}-{\overline{Y}}_{\text{post}})}^2.$

(2.8)

Hedges (1981) and Hedges and Olkin (1985) consider d_H to be a better estimator than d_G because the bias and variance of d_H are both smaller than the bias and variance of the corresponding d_G. Hedges and Olkin (1985) also developed some more advanced estimators for calculating effect sizes in cases of categorical variables, regression coefficients, odds ratios, as well as on stochastic processes in the distance between two means. Given the focus of this book, those additional techniques are not further described. The interested reader is referred to Cohen (1988), Glass (1976), Hedges (1981), and Hedges and Olkin (1985) for more comprehensive discussions on effect sizes.

The above equations of effect size display a major difference between the effect size and the t-test statistics, although the two appear close in form. The denominator in the equation of the t-test statistic has the component $\sqrt{N}$, suggesting that the significance level is enhanced with an increase in the sample size. By contrast, effect size estimators typically are not affected by the sample size, thus eliminating the scale difference. In the longitudinal setting, the effect size index is a statistic displaying the degree of the absolute deviation between scores measured at two time points. A higher effect size estimate indicates a greater standardized “effect” while a lower value indicates a lesser one. For example, an effect size of 0.5 indicates that the effect is a half standard deviation greater, while an effect size of 1.0 indicates that the effect is a full standard deviation greater. As originally developed, the effect size index has no attached probability, and therefore, it cannot be statistically significant or insignificant (Robey, 2004). Cohen (1988) categorized small, medium, and large effect sizes, with threshold values of the three categories set as d = 0.20, d = 0.50, and d = 0.80, respectively. Notably, in some special situations or in some other fields, the same value of an effect size statistic can imply a different degree of deviation between two means. Therefore, considerable caution must be applied in classifying effect sizes into several arbitrary levels, since they can mean different strengths of association in various disciplines (Robey, 2004), particularly in psychosocial studies.

2.1.3.2. Meta-analysis on an estimated effect size

The computation of an effect size posits a random sample as the hypothetical population so that the effect size estimate is an absolute value without being linked to a specific probability distribution. This specification has triggered some skepticism about its justification. As obtained from a single population sample, the calculated effect size does not produce a precise estimate because the value of its estimate can differ markedly using another sample. Therefore, when deriving an effect size, it is more informative to examine a series of studies on the same measurement to estimate the true effect size more accurately. A simple, convenient method in this regard is to combine results from different studies on a common measurement, referred to as meta-analysis.

Meta-analysis, originally coined by Glass (1976), refers to the quantitative methods for combining evidence across a series of studies on a common measurement. This approach usually relies on summary statistics obtained from primary analyses of a series of carefully selected studies. Essentially, a meta-analysis is an analysis of the results from several analyses (Glass, 1976; Hedges and Olkin, 1985). Using this approach allows the estimate of an effect size to be derived as a weighted average of the common measurement, with weights related to the sample sizes of individual studies. If the series of studies are well selected, meta-analysis can derive a more precise effect size estimate than a single dataset.

In clinical experimental studies, the researcher can conduct a meta-analysis by using several clinical trials of a common medical treatment, and then combining the results from these trials to estimate an effect size. The individual-based data can be used to derive the effect size statistic. However, the more popular meta-analytic approach is to identify studies using a common outcome measurement and then utilizing the aggregate results data to generate a summary measure (a weighted average of a series of reported effect sizes). This approach requires a thorough and systematic search of the literature on a specific topic, and the selection of specific studies based on preidentified quality criteria.

For example, to conduct a meta-analysis on the impact of a medical treatment on PTSD, the researcher first needs to identify all related studies on the subject, then examine whether each study meets the necessary criteria for inclusion. This procedure requires randomization, blindness, and representativeness of patients. Based on these quality checks, studies can be selected that satisfy the criteria and the investigator can obtain the pre–post effect size of the treatment (in practice, Hedges’s d is recommended) for each selected study. Finally, the summary effect size statistic can be computed as a weighted average, given by

${\overline{d}}_{\text{H}}=\sum _{g=1}^G{w}_g{d}_{\text{H},g}\text{,}$

(2.9)

where ${\overline{d}}_{\text{H}}$ is the summary measure of the effect size statistic, d_H,g is Hedges’s d for study g (g = 1, …, G), and w_g is the weight for study g that can be obtained from the inverse variance of the effect size estimator for each study. In the pre–post setting, w_g can be calculated by

$w_g=\frac{\frac{1}{s_g^2}}{\sum _{l=1}^G\left(\frac{1}{s_l^2}\right)}.$

(2.10)

In the above equation, studies with larger sample sizes have greater weights than those with smaller sample sizes in derivation of the summary effect size statistic.

In practice, meta-analysis can be used as an alternative to significance testing, thereby providing supplementary information concerning a pre–post effect for guiding further data collection and analysis. This approach has several obvious advantages. First, the researcher can widen the generality of an effect size from a sample estimate to a larger population parameter. Provided that the studies are correctly selected, including more data points improves the precision of the effect size estimate. Second, the effect size estimates from a number of single studies are combined within an integrated framework, and consequently, sampling errors and the resulting inconsistencies can be evaluated. Third, meta-analyses provide information about the dispersion of a summarized effect size, thereby facilitating hypothesis testing on the summary statistic. More recently, some scientists have advanced meta-analysis by aggregating published prediction models from different studies (Debray et al., 2012, 2013).

Regardless of the aforementioned strengths, meta-analysis is a rough statistical approach to estimate effect sizes and is subject to several serious limitations. Most notably, it is difficult to find studies that have exactly the same design, sampling strategy, and representativeness of the sample. A weighted average from samples with different designs and levels of representativeness can result in an erroneous summary estimate. Another important problem is that of publication bias. Much of the meta-analysis literature relies on published studies and available analytic results, which can result in strong selection bias in a weighted average. In the fields of medicine and psychology, study results displaying negative results or insignificant findings are much less likely to be accepted by scientific journals for publication. As many study results failing to show significant results presumably still lie in researchers’ file drawers, this publication bias is vividly referred to as the “file drawer” problem (Rosenthal, 1979). To date, there has been no sufficiently satisfactory solution for correcting this type of bias. If the studies included are nonrandomly selected based on publication bias, the distribution of effect sizes will be skewed, resulting in a biased weighted average of effect sizes.

2.1.3.3. Computation of confidence intervals for effect size from a single study

Recently, there are compelling calls in biomedical fields for reporting confidence intervals built around a point estimate of effect size (Bird, 2002; Robey, 2004; Wilkinson and APA Task Force on Statistical Inference, 1999). It is contended that although the use of the effect size statistic is meant to estimate the magnitude of an effect for demonstrating the strength of association, it is not sufficient to just report the point estimate for a complete interpretation of that effect. The report of the American Psychological Association (APA) Task Force on Statistical Inference recommended that confidence intervals should be given for any effect size involving principal outcomes (Wilkinson and APA Task Force on Statistical Inference, 1999, p. 599). From the statistical standpoint, confidence intervals can be computed from a single study that corresponds to the tradition for assessing the dispersion of a point estimate.

Computation of a confidence interval for an effect size estimate is based on the selection of the distribution containing the necessary critical value (Cumming and Finch, 2001). For example, an effect size relating to a t-test has a distribution of t; an effect size relating to an F-test follows the distribution of F. As effect sizes are used to examine departures from the null hypothesis, the required distribution for a valid confidence interval usually relies on the noncentral distribution of the underlying test statistic. For within-study effect sizes, however, an optimal confidence interval around a point effect estimate can be approximated by using the central distribution for large samples (Hedges and Olkin, 1985; Robey, 2004).

In this section, a simplified approach is described for constructing a confidence interval around a point estimate of the pre–post effect size obtained from a single study, given an equal sample size for the pre–post pair (Robey, 2004). Specifically, this simple construction requires nine steps, as summarized by Robey (2004).

Step 1. Specify a priori criterion for Type I error tolerance (the probability of rejecting a true null hypothesis), referred to as α. In empirical analyses, α is regularly set at 0.05 for each contrast. The bandwidth of the corresponding confidence interval is then 1 − α/2 or 0.975 in the case of α = 0.05.

Step 2. For an effect size statistic obtained from a single study, find the critical value in the central distribution of t for α and the number of error degrees of freedom, referred to as CC (Bird, 2002). The value of CC can be obtained from the SAS software given the value of α and the number of the degrees of freedom.

Step 3. Compute the standard error for the pre–post contrast in terms of the effect size. The calculation can be easily accomplished by a hand-calculation given the standard formula, given by

$\text{s}{\text{e}}_d=\sqrt{\frac{s_d^2}{0.5(N_{\text{pre}}+N_{\text{post}})}}\text{,}$

(2.11)

where $s_d^2$ is the variance for the pre–post contrast in the effect size, which can be obtained from the pair-wise differences given an equal sample size.

Step 4. Compute the value of the pooled standard deviation, s_pooled, either by using the pooled standard deviation (see Subsection 2.1.3.1) or by using the following simplified equation:

$s_{\text{pooled}}=\sqrt{\frac{s_{\text{pre}}^2+s_{\text{post}}^2}{2}}\text{.}$

(2.12)

Step 5. Compute the upper limit for the confidence interval (RUL) of the difference in the raw pre–post scores, using the CC value from a t distribution, given by

$\text{RUL}={\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}+\text{CC}(\text{s}{\text{e}}_d)\text{.}$

(2.13)

Step 6. Likewise, the lower limit for the confidence interval (RUL) of the difference in the raw pre–post scores is

$\text{RLL}={\overline{Y}}_{\text{post}}-{\overline{Y}}_{\text{pre}}-\text{CC}(\text{s}{\text{e}}_d)\text{.}$

(2.14)

Step 7. Compute the point estimate of effect size by using one of the estimators described in Subsection 2.1.3.1 (Cohen’s, Glass’s, or Hedges’s d).

Step 8. Compute the upper limit for the confidence interval about the effect size estimate (SUL), written as

$\text{SUL}=\frac{\text{RUL}}{s_{\text{pooled}}}\text{.}$

(2.15)

Step 9. Likewise, the lower limit for the confidence interval about the effect size d (SLL) is

$\text{SLL}=\frac{\text{RLL}}{s_{\text{pooled}}}\text{.}$

(2.16)

As indicated earlier, these steps describe a simplified procedure given an equal size for the pre and the post samples. The computational procedure for unequal sample sizes is not covered in this book given the focus on paired differences. For the between-studies effect size estimate, obtained from using the weighted average approach, the confidence interval needs to be computed through complex iterative processes. Given the focus of this text, those more advanced techniques are not elaborated.

2.1.4. Empirical illustration: descriptive analysis on the effectiveness of acupuncture treatment in reduction of PTSD symptom severity

2.2.1. Specifications of one-factor ANOVA

$\begin{array}{l}{\text{H}}_0:{\mu }_1={\mu }_2,=\cdots ={\mu }_K\\ {\text{H}}_1:\text{not}\text{all}\mu \text{'}\text{s}\text{are}\text{equal.}\end{array}$

$\text{SS}(\text{factor})=N_1{({\overline{Y}}_1-\overline{Y})}^2+N_2{({\overline{Y}}_2-\overline{Y})}^2+\cdots +N_K{({\overline{Y}}_K-\overline{Y})}^2\text{,}$

(2.17)

where N₁, N₂, ···, N_K are sample sizes for each of the K levels for the factor, respectively, ${\overline{Y}}_1,{\overline{Y}}_2,\cdots ,{\overline{Y}}_K$ are the corresponding sample means, and $\overline{Y}$ is the grand mean of the entire sample.

$\text{SS}(\text{error})=\sum {(Y_1-{\overline{Y}}_1)}^2+\sum {(Y_2-{\overline{Y}}_2)}^2+\cdots +\sum {(Y_K-{\overline{Y}}_K)}^2\text{,}$

(2.18)

where $Y_1,Y_2,\cdots ,Y_K$ are subject-specific measures within each of the K groups.

$\text{SS}(\text{total})=\sum {(Y-\overline{Y})}^2\text{.}$

(2.19)

$\text{MS}(\text{factor})=\frac{\text{SS}(\text{factor})}{K-1}\text{,}$

(2.20)

$\text{MS}(\text{error})=\frac{\text{SS}(\text{error})}{N-K}\text{,}$

(2.21)

The test statistic for testing ${\text{H}}_0:{\mu }_1={\mu }_2,=\cdots ={\mu }_K$ versus the alternative hypothesis is the ratio of MS(factor) over MS(error), which follows an F-distribution, given by

$F=\frac{\text{SS}(\text{factor})}{\text{MS}(\text{error})}\text{.}$

(2.22)

2.2.2. One-factor repeated measures ANOVA

$\text{SS}(\text{total})=\text{SS}(\text{time}\text{factor})+\text{SS}(\text{blocks})+\text{SS}(\text{error})\text{,}$

(2.23)

$\text{SS}(\text{time})=N\left[{({\overline{Y}}_1-\overline{Y})}^2+{({\overline{Y}}_2-\overline{Y})}^2+\cdots +{({\overline{Y}}_n-\overline{Y})}^2\right]\text{,}$

(2.24)

$\text{SS}(\text{subjects})=n\left[{({\overline{Y}}_1-\overline{Y})}^2+{({\overline{Y}}_2-\overline{Y})}^2+\cdots +{({\overline{Y}}_N-\overline{Y})}^2\right]\text{.}$

(2.25)

$\text{MS}(\text{time})=\frac{\text{SS}(\text{time})}{n-1}\text{,}$

(2.26)

$\text{MS}(\text{subjects})=\frac{\text{SS}(\text{subjects})}{N-1}\text{,}$

(2.27)

$\text{MS}(\text{error})=\frac{\text{SS}(\text{error})}{\left[(n-1)\right](N-1)}\text{.}$

(2.28)

Like the classical ANOVA, in repeated measures ANOVA, testing of the null hypothesis ${\text{H}}_0:{\mu }_1={\mu }_2,=\cdots ={\mu }_n$ versus the alternative hypothesis is based on the ratio of MS(time) over MS(error), which follows an F-distribution and is specified by Equation (2.22). Given the F-value, the p-value can be obtained from the distribution with two degree of freedom terms: n − 1 for time and (n − 1)(N − 1) for error. The null and the alternative hypotheses can subsequently be statistically tested. The amount of MS(error) is usually smaller than the results derived from the classical ANOVA because SS(subjects) absorbs some variations in the observed measurements thereby reducing MS(error) given a fixed MS(total) value. As a result, the F-value is increased, elevating the likelihood that the null hypothesis is to be rejected.

The ANOVA assumes a common variance σ² across all levels, or all time points in the context of longitudinal data. Consequently, MS(error) can be viewed as the estimate of σ², denoted by s², with the square root of MS(error) representing the sample standard error s. Therefore, using the randomized block design, the confidence interval for the difference between two sample means, ${\overline{Y}}_j$ and ${\overline{Y}}_{j^{\prime }}$ where $j\ne j^{\prime }$, can be computed from MS(error). Mathematically, the confidence interval for $\left({\overline{Y}}_j-{\overline{Y}}_{j^{\prime }}\right)$ with coverage probability (1 − α) satisfies the following condition:

$\mathrm{Pr}\left\{\left[({\overline{Y}}_j-{\overline{Y}}_{j^{\prime }})-\tilde{w}\right]\le ({\overline{Y}}_j-{\overline{Y}}_{j^{\prime }})\le \left[({\overline{Y}}_j-{\overline{Y}}_{j^{\prime }})+\tilde{w}\right]\right\}=1-\alpha \text{,}$

(2.29)

where $\tilde{w}$ is the confidence width determined by α, which can be approximated by

$\tilde{w}=t_{\alpha /2,\text{df}}\times \sqrt{\text{MS}(\text{error})\left(\frac{2}{N}\right)}\text{,}$

In this one-factor repeated measures ANOVA, the F-test is applied to test the effect due to subjects given the null and the alternative hypotheses that ${\text{H}}_0^{\text{subjects}}:{\mu }_1={\mu }_2,=\cdots ={\mu }_N$ and ${\text{H}}_1^{\text{subjects}}$: not all subject means are equal. The F-value is calculated by

$F_{\text{subjects}}=\frac{\text{MS}(\text{subjects})}{\text{MS}(\text{error})}\text{,}$

(2.30)

$Y_{ij}=\mu +b_i+{\tau }_j+{\varepsilon }_{ij}\text{,}$

(2.31)

where Y_ij is the measurement for subject i at time point j, μ is the grand mean indicated earlier, b_i is the subject i’s deviation from μ assumed to be constant over time, τ_j is the time point’s deviation from μ also assumed to be the same across all subjects, and ɛ_ij is the random error for subject i at time point j. By definition, the sum of τ_j across all time points equals zero. The term b_i is defined as the between-subjects random effect assumed to be distributed as $\text{N}\left(\text{0,}{\sigma }_b^2\right)$. Likewise, ɛ_ij is the within-subject random error distributed as $\text{N}\left(\text{0,}{\sigma }_{\varepsilon }^2\right)$. It follows that the one-factor repeated-measures ANOVA specifies two components of effects – the fixed effect of time and the random effect of subjects, thereby engendering the original form of a mixed-effects model. In fact, various mixed-effects linear models, which will be described in the succeeding chapters, can be regarded as extensions of Equation (2.31).

2.2.3. Specifications of two-factor repeated measures ANOVA

$\text{SS}(\text{total})=\text{SS}(\text{A})+\text{SS}(\text{B})+\text{SS}(\text{AB})+\text{SS}(\text{subjects})+\text{SS}(\text{error})\text{,}$

(2.32)

$\text{SS}(\text{A})=N\left[{({\overline{Y}}_{1.}-\overline{Y})}^2+{({\overline{Y}}_{2.}-\overline{Y})}^2+\cdots +{({\overline{Y}}_{n.}-\overline{Y})}^2\right]\text{,}$

(2.33)

where, given two factors, ${\overline{Y}}_{1.},{\overline{Y}}_{2.},\cdots ,{\overline{Y}}_{n.}$ are marginal means of Y at each of the n time points across all K levels, and the subscript “.” indicates margins across all levels of another factor (given this specification, the grand mean $\overline{Y}$ can be expressed as $\overline{Y}\mathrm{.}\text{.}$).

$\text{SS}(\text{B})=n\left[{({\overline{Y}}_{.1}-\overline{Y})}^2+{({\overline{Y}}_{.2}-\overline{Y})}^2+\cdots +{({\overline{Y}}_{.K}-\overline{Y})}^2\right]\text{,}$

(2.34)

where, ${\overline{Y}}_{.1},{\overline{Y}}_{.2},\cdots ,{\overline{Y}}_{.K}$ are marginal means of Y at each of the K levels across all n time points.

$\text{SS}(\text{AB})=\sum _{j=1}^n\sum _{k=1}^K{N}_j{({\overline{Y}}_{jk}-{\overline{Y}}_{j.}-{\overline{Y}}_{.k}+\overline{Y})}^2\text{.}$

(2.35)

$\text{SS}(\text{subjects})=n\sum _{k=1}^K\sum _{i=1}^{N_k}{({\overline{Y}}_{.k}-\overline{Y})}^2\text{.}$

(2.36)

$\text{MS}(\text{A})=\frac{\text{SS}(\text{A})}{n-1}\text{,}$

(2.37)

$\text{MS}(\text{B})=\frac{\text{SS}(\text{B})}{K-1}\text{,}$

(2.38)

$\text{MS}(\text{AB})=\frac{\text{SS}(\text{AB})}{(n-1)(K-1)}\text{,}$

(2.39)

$\text{MS}(\text{subjects})=\frac{\text{SS}(\text{subjects})}{N-K}\text{,}$

(2.40)

$\text{MS}(\text{error})=\frac{\text{SS}(\text{error})}{(N-K)(n-1)}\text{.}$

(2.41)

$F_{\text{factor}\text{B}}=\frac{\text{MS}(\text{B})}{\text{MS}(\text{error})}\text{.}$

(2.42)

$Y_{ij}=\mu +{\tau }_j+{\lambda }_k+{(\tau \lambda )}_{jk}+b_{i(k)}+{\varepsilon }_{ijk}\text{,}$

(2.43)

where λ_k is the effect of group k, (τλ)_jk measures the effect of the interaction between time point j and group k, b_i(k) indicates the subject’s random effect nested in group k, and ɛ_ijk is the random error for subject i at time point j. As deviations of group means from the grand mean, μ, the sums of τ_j’s, l_k’s, and (τl)_jk’s are all zero, respectively. Again, b_i is defined as the between-subjects random effect assumed to be distributed as $\text{N}(0,{\sigma }_b^2)$, and ɛ_ij is the within-subjects random disturbances distributed as $\text{N}(\text{0,}{\sigma }_{\varepsilon }^2)$. The two-factor repeated-measures ANOVA specifies two effect components, the fixed effects of the two factors and the random effect of subjects; accordingly, the variance of the response measurement consists of two components – between-subjects variance ${\sigma }_b^2$ and within-subject variance ${\sigma }_{\varepsilon }^2$.

2.2.4. Empirical illustration: a two-factor repeated measures ANOVA – the effectiveness of acupuncture treatment on PCL revisited

2.3.1. General MANOVA

$S_{\text{total}}=S_{\text{bet}}+S_{\text{within}}\text{,}$

(2.44)

$\begin{array}{c}{S}_{\text{bet}}=N_{jk}\sum _{j-1}^n\sum _{k=1}^K{\left({\overline{Y}}_{jk}-\overline{Y}\right)}^2=N_j\sum _{j-1}^n{\left({\overline{Y}}_j-\overline{Y}\right)}^2+\sum _{k=1}^K{\left({\overline{Y}}_k-\overline{Y}\right)}^2\\ +\left[N_{jk}\sum _{j-1}^n\sum _{k=1}^K{\left({\overline{Y}}_{jk}-\overline{Y}\right)}^2-N_j\sum _{j-1}^n{\left({\overline{Y}}_j-\overline{Y}\right)}^2-\sum _{k=1}^K{\left({\overline{Y}}_k-\overline{Y}\right)}^2\right]\text{.}\end{array}$

(2.45)

$S_{\text{bet}}=S_{X_1}+S_{X_2}+S_{X_1{X}_2}\text{.}$

(2.46)

$S_{\text{within}}=\sum _i\sum _j\sum _k\left(Y_{ijk}-{\overline{Y}}_{jk}\right){\left(Y_{ijk}-{\overline{Y}}_{jk}\right)}^{\prime }\text{.}$

(2.47)

$\begin{array}{c}{S}_{\text{total}}=\sum _i\sum _j\sum _k\left(Y_{ijk}-\overline{Y}\right){\left(Y_{ijk}-\overline{Y}\right)}^{\prime }=N_j\sum _j\left({\overline{Y}}_j-\overline{Y}\right){\left({\overline{Y}}_j-\overline{Y}\right)}^{\prime }\\ +\sum _{k=1}^K\left({\overline{Y}}_k-\overline{Y}\right){\left({\overline{Y}}_k-\overline{Y}\right)}^{\prime }+\left[N_{jk}\sum _{j-1}^n\sum _{k=1}^K\left({\overline{Y}}_{jk}-\overline{Y}\right){\left({\overline{Y}}_{jk}-\overline{Y}\right)}^{\prime }\right.\\ \left.-N_j\sum _{j-1}^n\left({\overline{Y}}_j-\overline{Y}\right){\left({\overline{Y}}_j-\overline{Y}\right)}^{\prime }-\sum _{k=1}^K\left({\overline{Y}}_k-\overline{Y}\right){\left({\overline{Y}}_k-\overline{Y}\right)}^{\prime }\right]\\ +\sum _i\sum _j\sum _k\left(Y_{ijk}-{\overline{Y}}_{jk}\right){\left(Y_{ijk}-{\overline{Y}}_{jk}\right)}^{\prime },\end{array}$

(2.48)

$S_{\text{total}}=S_{X_1}+S_{X_2}+S_{X_1{X}_2}+\hspace{0.17em}{S}_{\text{within}}\text{.}$

(2.49)

2.3.2. Hypothesis testing on effects in MANOVA

$\Lambda =\frac{\left|S_{\text{within}}\right|}{\left|S_{\text{effect}}+\hspace{0.17em}{S}_{\text{within}}\right|}\text{,}$

(2.50)

$\text{Approximate}F\left(\text{d}{\text{f}}_1,\text{d}{\text{f}}_2\right)=\left(\frac{1-\tilde{y}}{\tilde{y}}\right)\left(\frac{\text{d}{\text{f}}_2}{\text{d}{\text{f}}_2}\right)\text{,}$

(2.51)

$\tilde{y}=\sqrt[\tilde{s}]{\Lambda }\text{,}$

(2.52)

$\tilde{s}=\sqrt{\frac{M^2{\left(\text{d}{\text{f}}_{\text{effect}}\right)}^2-4}{M^2+{\left(\text{d}{\text{f}}_{\text{effect}}\right)}^2-5}}\text{.}$

(2.53)

$\text{d}{\text{f}}_1=M\left(\text{d}{\text{f}}_{\text{effect}}\right)\text{,}$

$\text{d}{\text{f}}_2=\tilde{s}\left[\left(\text{d}{\text{f}}_{\text{error}}\right)-\left(\frac{M-\text{d}{\text{f}}_{\text{error}}+1}{2}\right)\right]-\left[\frac{M\left(\text{d}{\text{f}}_{\text{error}}\right)-2}{2}\right]\text{.}$

${\tilde{\eta }}^2=1-\Lambda \text{,}$

(2.54)

where ${\tilde{\eta }}^2$ indicates the variance accounted for by the best linear combination of dependent variables because Λ is the variance not accounted for by the combined measurements. Empirically, however, the value of ${\tilde{\eta }}^2$ can be greater than one in MANOVA given its multivariate nature. Therefore, a recommended alternative, when $\tilde{s}>1$, is

$\text{partial}{\tilde{\eta }}^2=1-\sqrt[\tilde{s}]{\Lambda }\text{.}$

(2.55)

In addition to Wilks’ Lambda, there are some other multivariate test statistics for MANOVA, such as Hotelling’s trace criterion, Pillai’s criterion, and Roy’s greatest root criterion. The Hotelling’s trace test statistic, denoted by $\tilde{T}$, is a pooled ratio of effect variance over error variance, given by

$\tilde{T}=\text{trace}\left(\frac{S_{\text{effect}}}{S_{\text{within}}}\right)\text{,}$

(2.56)

Similarly, Pillai’s trace statistic, denoted $\tilde{P}$, pools effect variances:

$\tilde{P}=\text{trace}\left(\frac{S_{\text{effect}}}{S_{\text{effect}}+S_{\text{within}}}\right)\text{.}$

(2.57)

2.3.3. Repeated measures MANOVA

$Y_{ik}=\tau +{\lambda }_k+{\varepsilon }_{ik}\text{,}$

(2.58)

$\begin{array}{c}P{Y}_{ik}=P\tau +P{\lambda }_k+P{\varepsilon }_{ik}\\ ={\tau }^{*}+{\lambda }_k^{*}+{\varepsilon }_{ik}^{*},\end{array}$

(2.59)

where ${\varepsilon }_{ik}^{*}\sim \text{N}\left(0\text{,}{P\Sigma P}^{\prime }\right)=\text{N}\left(0\text{,}{\Sigma }^{*}\right)$.

After the orthogonal transformation, the total sums of squares in the repeated measures MANOVA, denoted by $S_{\text{total}}^{*}$, is written as

$S_{\text{total}}^{*}=P{S}_{\text{total}}{P}^{\prime }=P\left[\sum _i\sum _j\sum _k\left(Y_{ijk}-\overline{Y}\right){\left(Y_{ijk}-\overline{Y}\right)}^{\prime }\right]P^{\prime }\text{.}$

(2.60)

Let S_time be the S matrix for the effect of time and S_group be the S matrix for the effect of group, and $S_{\text{time}}^{*}$, $S_{\text{group}}^{*}$ are the corresponding sums of squares after orthogonal transformation. The two S matrix components in repeated measures MANOVA are then given by

$S_{\text{time}}^{*}=P{S}_{\text{time}}{P}^{\prime }=NP\left[{}_j\sum _j\left({\overline{Y}}_j-\overline{Y}\right){\left({\overline{Y}}_j-\overline{Y}\right)}^{\prime }\right]P^{\prime }\text{,}$

(2.61)

$S_{\text{group}}^{*}=P{S}_{\text{group}}{P}^{\prime }=P\left[\sum _{k=1}^K\left({\overline{Y}}_k-\overline{Y}\right){\left({\overline{Y}}_k-\overline{Y}\right)}^{\prime }\right]P^{\prime }\text{.}$

(2.62)

Likewise, the S matrix for error after orthogonal transformation, denoted $S_{\text{within}}^{*}$, is

$S_{\text{within}}^{*}=P{S}_{\text{within}}{P}^{\prime }=P\left(S_{\text{total}}^{*}-S_{t\text{ime}}^{*}-S_{\text{group}}^{*}\right)P^{\prime }\text{.}$

(2.63)

The above S matrices, all symmetric, provide multivariate information about between-subjects variability. The multivariate hypothesis tests on various S components can be performed by using the four multivariate test statistics described in Section 2.3.2 – Wilks’ Lambda, Hotelling’s trace criterion, Pillai’s criterion, and Roy’s greatest root criterion. The availability of these matrices is also sufficient to derive the univariate repeated measures ANOVA results by extracting the diagonal elements from the respective S matrices. Specifically, SS(Time), the univariate total sum of squares due to time, can be obtained from the sum of the diagonal elements in $S_{\text{time}}^{*}$ except the first one; SS(Group), the total sum of squares due to group, is simply the first element in $S_{\text{group}}^{*}$; and SS(Time × Group) is the sum of the rest of the diagonal elements in $S_{\text{group}}^{*}$. Similarly, SS(subjects), the univariate sum of squares due to subjects, is the first element in the matrix $S_{\text{within}}^{*}$, and the rest of the diagonal elements in this S^* matrix for error are then summed up to make SS(error). Given these ANOVA-type statistics, the degree of freedom for each SS component is identical to those specified for the two-factor repeated measures ANOVA. As a result, the univariate MS and F statistics can be obtained from using the ANOVA procedure described in Section 2.2.3.

2.3.4. Empirical illustration: a two-factor repeated measures MANOVA on the effectiveness of acupuncture treatment on two psychiatric disorders