Mathematics 172 – Numerical Analysis II Discussion 3 – “How Big Is This Vector? How Big Is This Matrix?” March 18, 1999

Background

To understand the “bad” behavior of some of the systems Ax = b we have seen when floating point arithmetic is used with Gaussian Elimination, and to have tools at our disposal for analyzing other methods for solving systems, we will consider next the question: How we can measure how close two matrices are to each other, or measure the “size” of a matrix. In this context, by “size” we will not mean the number of rows and columns; instead given an n, we will want to be able to compare any two n × n matrices A and B, determine which is “bigger,” determine how close A is to B and so on.

For vectors in R^{n} you should recall that similar questions can be addressed by using the usual “vector norm:”

(1)  ∥x⃗∥ = ∥(x_{1}, x_{2}, …, x_{n})∥ = \sqrt{x_{1}^{2} + x_{2}^{2} + ⋯ + x_{n}^{2}}

The size of the number ∥x⃗∥ gives us a measure of the “size” of x⃗ (its geometric length), and the distance between x⃗ and y⃗ can be determined by finding ∥x⃗ − y⃗∥. From now on, we’ll drop the “arrow” notation for vectors.

Discussion Questions

Let’s start with vectors and consider (1) in more detail.

A) What are the important properties of (1)? In particular, 1) What kind of numbers are norms of vectors? What vectors satisfy ∥x∥ = 0?

2) What happens to the norm if you multiply x by a scalar λ? How are ∥λx∥ and ∥x∥ related?

3) How does (1) interact with linear combinations of vectors? (For instance, what can you say about ∥x + y∥?)

For our purposes, it will be best to think of (1) as defining a function n : R^{n} → R, where n(x) = ∥x∥ = \sqrt{∑_{i=1}^{n} x_{i}^{2}}.

B) Which (if any) of the following functions from R^{n} to R satisfy the same properties as (1)? Explain.

1) n_{1}(x) = ^{3} \sqrt{∑_{i=1}^{n} x_{i}^{3}}.

2) n_{2}(x) = |x_{1}| · |x_{2}| · · · · · |x_{n}|.

3) n_{3}(x) = |x_{1}| + |x_{2}| + ⋯ + |x_{n}|.

4) n_{4}(x) = max{|x_{i}|, i = 1, …, n} (the largest component of the vector in absolute value).

C) For each of the functions n_{j} above that does have properties similar to the usual vector norm, draw a picture of the set of vectors x ∈ R^{2} satisfying n_{j}(x) = 1 (analogues of the unit circle for (1)!)

One possible way to measure the “size” of a matrix A (thinking of it as a mapping) is to measure how it affects the sizes of vectors, when you multiply A times them, for instance by considering the ratios

r_{A}(x) = \frac{∥Ax∥}{∥x∥}

for x ≠ 0. If A magnifies the lengths of vectors a lot, we will think of it as “big”. If A shrinks the lengths of vectors a lot, we will think of it as “small”. For this purpose, any vector norm on R^{n} can be used (that is any function satisfying the same properties as the usual vector norm, as in question A).

D) How are r_{A}(x) and r_{A}(λx) related? What does this tell you? (Note, for any non-zero vector x, there is a vector of norm 1 (a “unit vector”) in the same direction:

u = \frac{1}{∥x∥}x

Is it necessary to consider all vectors to study the possible values of r_{A}(x)?)

E) What is the biggest possible value you can get for r_{A}(x) for

A = ( \begin{matrix}7 & 1 \\ −1 & 3 \end{matrix} )

1) using the vector ℓ_{1} norm on R^{2}: ∥(x, y)∥_{1} = |x| + |y|?

2) using the vector ℓ_{∞} norm on R^{2}: ∥(x, y)∥_{∞} = max{|x|, |y|}?

3) Do you notice any patterns here? Can you determine what the biggest r_{A}(x) will be for a general 2 × 2 matrix A, using the norms from 1 and 2.

4) Recall that an eigenvalue of a matrix A is a number λ such that there exists a non-zero vector x satisfying an equation Ax = λx. How do the eigenvalues of A relate to the maximum of the quotients r_{A}(x)? Is there an equality you can state? An inequality?