Mathematics 376 – Mathematical Statistics Problem Set 8 due: Friday, April 20, 2012

A) Assume we have a linear statistical model

Y = β_{0} + β_{1}x + ϵ

and ϵ is normally distributed with mean μ = 0, and variance σ^{2}.

1) Given observations (x_{1}, y_{1}), …, (x_{n}, y_{n}) for values x = x_{i} (known exactly), explain why the likelihood of these observations (as a function of β_{0} and β_{1}) is given by

L(β_{0}, β_{1}) = (\frac{1}{\sqrt{2πσ^{2}}} )^{n} exp ((− ∑_{i=1}^{n}(y_{i} − β_{0} − β_{1}x_{i})^{2}) \frac{1}{2σ^{2}} )

2) Find the maximum likelihood estimators for β_{0} and β_{1} by computing the appropriate partial derivatives of ln(L), setting equal to zero, and solving for β_{0}, β_{1}. (You should get the same formulas as we obtained for the least squares estimators.)

3) Show that your answer in part 2 is really a maximum of L or ln(L) using the Second Derivative Test for functions of two variables.

B) In class, by solving the normal equations using Cramer’s Rule, we obtained the following formulas (all summations extend from i = 1 to i = n, so we omit limits of summation for simplicity):

(1)  β̂_{0} = \frac{(Σy_{i}) (Σx_{i}^{2}) − (Σx_{i}) (Σx_{i}y_{i})}{n (Σx_{i}^{2}) − (Σx_{i})^{2}}

and

(2)  β̂_{1} = \frac{n (Σx_{i}y_{i}) − (Σx_{i}) (Σy_{i})}{n (Σx_{i}^{2}) − (Σx_{i})^{2}}

Recall that in class we also introduced the quantities

S_{xx} = ∑(x_{i} − \overline{x})^{2} S_{xy} = ∑(x_{i} − \overline{x})(y_{i} − \overline{y})

Show that (2) is equivalent to

β̂_{1} = \frac{S_{xy}}{S_{xx}}

and then that (1) is equivalent to

β̂_{0} = \overline{y} − β̂_{1}\overline{x}

From the text: Chapter 11/4 (show all work for calculations by hand, check with R),9 (show all work for calculations by hand, check with R), 14 (can be done entirely in R), 69 (can be done entirely in R).