MATH 372 -- Numerical Linear Algebra
Least Squares via QR -- the full rank case
March 16, 2007

Consider the overdetermined system Ax = b
for

>> A = [1 3 -2 ; 2 -2 5; 1 0 1; -2 3 2]

A =

     1     3    -2
     2    -2     5
     1     0     1
    -2     3     2

>> b = [3; 5; 6; -8]

b =

     3
     5
     6
    -8

This is a "full-rank" example, since 
A has 3 columns and rank(A) = dim R(A) 
= dim Col(A) = 3:

>> rank(A)

ans =

     3

Our solution method in this case is to find the 
QR factorization of A:

>> [Q,R]=qr(A)

Q =

   -0.3162    0.8948    0.2030    0.2413
   -0.6325   -0.1451   -0.7001    0.2981
   -0.3162    0.1693   -0.1422   -0.9226
    0.6325    0.3869   -0.6697   -0.0426


R =

   -3.1623    2.2136   -1.5811
         0    4.1352   -1.5719
         0         0   -5.3879
         0         0         0

Then from A = QR we have Ax = b <=> Q^t A x = Q^t b 
<=> Rx = Q^t b.  Since A has full rank, R has rank 
3 also, and the upper 3 x 3 block in R is upper-triangular
and invertible.  We solve R x = Q^t b to get the least-squares
solution of Ax = b:

>> R \ Q'*b

ans =

    3.5496
   -0.1430
   -0.2994

Finally, we check this against MATLAB's builtin least-squares 
solver:

>> A \ b

ans =

    3.5496
   -0.1430
   -0.2994