MATH 376 -- Probability and Statistics 2 

April 12, 2010 

Multiple Regression and Least Squares -- Matrix Formulation. 

Example 1. 

Normal equations for fitting the linear model

 

to the data points [0,0,2], [1,0,1], [0,1,4], [1,1,5] 

>

 

>	X:=Matrix([[1.0,0,0],[1,1,0],[1,0,1],[1,1,1]]);

 

(1)

 

>	Z:=Matrix([[2],[1],[4],[5]]);

 

(2)

 

>	XtX:=Multiply(Transpose(X),X);

 

(3)

 

>	XtZ:=Multiply(Transpose(X),Z);

 

(4)

 

>	beta:=LinearSolve(XtX,XtZ);

 

(5)

 

>	PlP:=plot3d(beta[1,1]+beta[2,1]*x+beta[3,1]*y,x=-0.5..1.5,y=-0.5..1.5,axes=boxed):

 

>	SP1:=plot3d([0.1*cos(theta)*sin(phi),0.1*sin(theta)*sin(phi),2+0.1*cos(phi)],theta=0..2*Pi,phi=0..Pi,color=blue):

 

>	SP2:=plot3d([1+0.1*cos(theta)*sin(phi),0.1*sin(theta)*sin(phi),1+0.1*cos(phi)],theta=0..2*Pi,phi=0..Pi,color=blue):

 

>	SP3:=plot3d([0.1*cos(theta)*sin(phi),1+0.1*sin(theta)*sin(phi),4+0.1*cos(phi)],theta=0..2*Pi,phi=0..Pi,color=blue):

 

>	SP4:=plot3d([1+0.1*cos(theta)*sin(phi),1+0.1*sin(theta)*sin(phi),5+0.1*cos(phi)],theta=0..2*Pi,phi=0..Pi,color=blue):

 

>	display3d(SP1,SP2,SP3,SP4,PlP,scaling=constrained);

 

We see the plane in  that comes closest to containing the 4 data points. 

Example 2. 

Normal equations for fitting the linear model  

 Y =  

to the data points  [0,1], [1,4], [2,7], [3,8]. 

We let   and   

The normal equations are  

>	X:=Matrix([[1.0,0,0],[1,1,1],[1,2,4],[1,3,9]]);

 

(6)

 

>	Y:=Matrix([[1],[4],[7],[8]]);

 

(7)

 

>	XtX:=Multiply(Transpose(X),X);

 

(8)

 

>	XtY:=Multiply(Transpose(X),Y);

 

(9)

 

>	beta:=LinearSolve(XtX,XtY);

 

(10)

 

>	Pts:=[[0,1],[1,4],[2,7],[3,8]];

 

(11)

 

>	PP:=plot(Pts,style=point,symbol=circle,color=blue):

 

>	RP:=plot(beta[1,1] + beta[2,1]*x + beta[3,1]*x^2,x=-1..4):

 

>	display(PP,RP);

 

>