MATH 376 -- Probability and Statistics II
A Comment on WMS, page 525.
March 29, 2010
I want to point out that there is a discussion in our text, Wackerly, Mendenhall,
and Schaeffer, Mathematical Statistics with Applications, 7th edition, that is somewhat
dated and that does not completely square with the current understanding in
statistical science. The statement in question is from the discussion of
small sample tests for means on page 525 -- ``Such investigations [of
the empirical distributions of the t-statistic] have shown that moderate
departures from normality in the distribution of the population have
little effect on the distribution of the test statistic'' -- a property
called robustness. This was certainly the consensus for a long time (say
through the 1970's.) However, more recent research has shown that
this is not always the case (or to be fair perhaps, that the precise meaning
of ``moderate departures from normality'' needs to be more carefully stated!)
How actually robust is the t-test in the case where the normality assumptions
are violated?
Let us study some small samples from a lognormal distribution. (See the pdf
plotted below.) Note this is a skewed distribution with a relatively long right
tail. Many quantities observed in scientific experiments have distributions
like this, so this is not just a theoretical example.
(1) |
We generate 1000 random samples of size 8 from this distribution, and plot the means and
variances of the samples in a scatter plot:
Recall that in the definition of t-distributions, we said that when sampling from a
normal population,
T =
had a t-distribution with n - 1 degrees of freedom, because it could be
rearranged to put it into the standard form , where Z is standard
normal, U has a distribution with n - 1 degrees of freedom, and
Z and U are independent. This is the same as saying Ybar and S are
independent. Does that match what we are seeing with these
samples??
Conclusion: For these samples, it is not valid to assume that the sample mean and
the sample variance (or standard deviation) are independent(!) It is clearly true
that the larger the sample mean is, the larger the sample variance is too.
Does this make a difference??
(2) |
Now, let us compare the empirical distribution of the t-statistics from these samples
with what we expect from the theoretical t - density function:
(3) |
(4) |
Note that the empirical distribution of the t-statistics is
quite a bit different from the theoretical t-PDF for
8 - 1 = 7 degrees of freedom. The red empirical distribution
has significantly more probability mass in the lower tail than
the t-curve, and correspondingly less in the center and upper
tail. This means that confidence intervals or hypothesis
tests using these samples will lead to incorrect conclusions
more often than we expect. (That is, for instance, the α and β
for hypothesis tests will not achieve the desired values.)
This kind of thing is more common than people thought until relatively
recently. Hence a whole new branch of the subject devoted to developing
robust statistics (that is, statistics that are not overly affected by the
failure of an assumption such as normality) has developed in the last 20 or
30 years. Our book, being an introduction to mathematical statistics,
unfortunately does not discuss these methods.