Decomposing the Variance and Its Meaning

DECOMPOSING THE VARIANCE AND ITS MEANING

Cochran’s theorem is the basis for the sums of squares having independent chisquare distributions when Equation 13.1 holds [see Rao (1997), page 4]. It can be deduced from Cochran’s theorem in the case of the one-way ANOVA that Σ(X_ij – )² = Σ(X_ij – X_i)² + Σ(X_i – )², where the following holds:

X_ij is normally distributed with mean μ_i

The variance is σ²

X_ij is the jth observation from the ith group

X_i. is the average of all observations in the ith group

X is the average over all the observations in all groups

Let Q, Q₁, and Q₂ refer to total sum of squares, within-groups sum of squares, and between-groups sum of squares, respectively. We have that Q = Q₁ + Q₂ Q = S(X_ij – )² normalized has a chi-square distribution with n_w + n_b – 1 degrees of freedom; Q₁ = Σ(X_ij – X_i.)² has a chi-square distribution with n_w degrees of free-dom; and Q₂ = Σ(X_i. – )² has a chi-square distribution with n_b – 1 degrees of freedom. Q₂ is independent of Σ(X_ij – X_i.)². The symbol n_b is the number of groups and n_w is the number of degrees of freedom for error. The total sample size equals n – n_b. For Q₂ to have a chi-square distribution when appropriately nor-malized, we need the null hypothesis that all μ_i are equal to be true. The F distri bution is obtained by taking [Q₂/(n_b – 1)]/[Q₁/n_w]. When the alternative holds, the normalized Q₂ has what is called a noncentral chi-square distribution, and the ra-tio tends to be centered above 1. The distribution of [Q₂/(n_b – 1)]/[Q₁/n_w] is then called a noncentral F distribution.

The mathematical relationship between this F statistic and the sample multiple correlation coefficient R² (discussed in Chapter 12) is as follows: R² = (n_b – 1)F/ {(n_b – 1)F + n_w} or F = {R²/(n_b – 1)}/{(1 – R²)/n_w}