Mathematics 376 – Probability and Statistics II Problem Set 8 due: Thursday, April 15, 2010

A) Assume we have a linear statistical model

Y = β_{0} + β_{1}x + ϵ

and ϵ is normally distributed with mean μ = 0, and variance σ^{2}.

1) Given observations (x_{1}, y_{1}), …, (x_{n}, y_{n}) of Y explain why the likelihood of these observations (as a function of β_{0} and β_{1}) is given by

L(β_{0}, β_{1}) = (\frac{1}{\sqrt{2πσ^{2}}} )^{n} exp ((− ∑_{i=1}^{n}(y_{i} − β_{0} − β_{1}x_{i})^{2}) \frac{1}{2σ^{2}} )

2) Find the maximum likelihood estimators for β_{0} and β_{1} by computing the appropriate partial derivatives of ln(L), setting equal to zero, and solving for β_{0}, β_{1}. (You should get the same formulas as we obtained for the least squares estimators.)

3) Show that your answer in part 2 is really a maximum of L or ln(L) using the Second Derivative Test for functions of two variables.

B) In class, by solving the normal equations using Cramer’s Rule, we obtained the following formulas (all summations extend from i = 1 to i = n, so we omit limits of summation for simplicity):

(1)  β̂_{0} = \frac{(Σy_{i}) (Σx_{i}^{2}) − (Σx_{i}) (Σx_{i}y_{i})}{n (Σx_{i}^{2}) − (Σx_{i})^{2}}

and

(2)  β̂_{1} = \frac{n (Σx_{i}y_{i}) − (Σx_{i}) (Σy_{i})}{n (Σx_{i}^{2}) − (Σx_{i})^{2}}

Recall that in class we also introduced the quantities

S_{xx} = ∑(x_{i} − \overline{x})^{2} S_{xy} = ∑(x_{i} − \overline{x})(y_{i} − \overline{y})

Show that (2) is equivalent to

β̂_{1} = \frac{S_{xy}}{S_{xx}}

and then that (1) is equivalent to

β̂_{0} = \overline{y} − β̂_{1}\overline{x}

From the text: Chapter 11/8,12,13,69. (Note: parts of problems that call for graphing of a scatter plot and the associated regression line can be done conveniently using Maple – see handouts from course homepage).