MATH 241 -- Multivariable Calculus 

Constrained Optimization and Lagrange Multipliers 

November 2, 2010 

 

Consider the problem of optimizing  f(x, y) = `+`(`*`(`^`(x, 2)), `-`(`*`(`^`(y, 2))))at the  

points of the ``constraint curve'' 

  

                        `and`(g(x, y) = `+`(`*`(`^`(`+`(x, `-`(1)), 2)), `*`(`/`(1, 4), `*`(`^`(`+`(y, `-`(1)), 2))), `-`(1)), `+`(`*`(`^`(`+`(x, `-`(1)), 2)), `*`(`/`(1, 4), `*`(`^`(`+`(y, `-`(1)), 2))), `-`(1)) = 0.)

Here is a picture of the constraint curve superimposed on the
 

contour plot of the function  f
 

> with(plots); -1
 

> `assign`(fP, contourplot(`+`(`*`(`^`(x, 2)), `-`(`*`(`^`(y, 2)))), x = -4 .. 4, y = -4 .. 4, contours = [-8, -7, -6, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4])); -1
 

> `assign`(gP, implicitplot(`+`(`*`(`^`(`+`(x, `-`(1)), 2)), `*`(`/`(1, 4), `*`(`^`(`+`(y, `-`(1)), 2))), `-`(1)), x = -4 .. 4, y = -4 .. 4, color = blue, grid = [40, 40])); -1
 

> display(fP, gP); 1
 

Plot_2d
 


Think of the contour plot here as the map of a landscape.  The constraint
 

curve might then describe a hike we are taking through the landscape. 

The question we want to answer is:

  
At which point(s) did we reach a maximum elevation on the hike,
  and where did we reach a minimum elevation?



In this case we can parametrize the constraint curve easily since it is an ellipse
 

with center at  (1,1)  semimajor axis  2  parallel to the  y-axis,  and semiminor 

axis  1  parallel to the  x-axis: 

 

   alpha(t) = (`+`(1, cos(t)), `+`(1, `*`(2, `*`(sin(t))))), `and`(`<=`(0, t), `<=`(t, `+`(`*`(2, `*`(Pi))))) 

If we substitute the coordinate functions from this into  f (x, y),  then we can 

see explicitly where  f(x(t), y(t))achieves its greatest and smallest values:

 

> plot(`+`(`*`(`^`(`+`(1, cos(t)), 2)), `-`(`*`(`^`(`+`(1, `*`(2, `*`(sin(t)))), 2)))), t = 0 .. `+`(`*`(2, `*`(Pi)))); 1
 

Plot_2d
 

Note how this plot relates to what we see in the first plot of the constraint curve
and the contours.   There is an interesting geometric relation between the constraint
curve and the contour whenever we ``
hit''  a critical point of the function  f(x, y)restricted to the contour.   Can you see what it is? 

The example, continued 

 

One way to say the relation is that the contour of  f(x, y)and the constraint
curve
are tangent to each other at each of those points.   (That is, the two
curves have
the same tangent line!)

This observation holds
in general for the following reason.  Suppose that we
can parametrize a portion of the constraint curve near a point  x[0], y[0]where
the restriction of f(x, y)has a critical point as a curve  alpha(t).  (This is
always
possible when  `<>`(Typesetting:-delayGradient(g(x[0], y[0]), cartesian), 0, 0)by a more advanced result called the  Implicit
Function Theorem.)   Say  If the restricted function has a
critical point at  t = t[0], then
 

 

   `/`(`*`(d, `*`(f(alpha(t)))), `*`(dt)) = 0  at   However, by the multivariable Chain Rule we discussed earlier, this implies 

 

   In other words, the gradient vector of  f  at  x[0], y[0]is orthogonal to the tangent 

line to the constraint curve at that point.  Since the gradient vector of  g  at  

that point is also orthogonal to the tangent line to the constraint curve,  

we have that  the two vectors  Typesetting:-delayGradient(f(x[0], y[0]), cartesian)and  `*`(Typesetting:-delayDotProduct(`*`(Typesetting:-delayGradient(g(x[0], y[0]), cartesian), `*`(must, `*`(be, `*`(scalar, `*`(multiples, `*`(of, `*`(one, `*`(another)))))))), In), `*`(other, `*`(words)...
`*`(Typesetting:-delayDotProduct(`*`(Typesetting:-delayGradient(g(x[0], y[0]), cartesian), `*`(must, `*`(be, `*`(scalar, `*`(multiples, `*`(of, `*`(one, `*`(another)))))))), In), `*`(other, `*`(words)...
 

(called a Lagrange multiplier )  such that  

 

             Typesetting:-delayGradient(f(x[0], y[0]), cartesian) = `*`(lambda, `*`(Typesetting:-delayGradient(g(x[0], y[0]), cartesian))) 


Hence we get the ``method of Lagrange multipliers" for solving the constrained
 

optimization problem: 

 

   1.  Set up the Lagrange equations :

        

        
 

   2.  Solve them simultaneously for  x, y,  (and sometimes λ)  

   3.  By comparing the values of  
f  at all of the points 

        found in step 2,  we can determine the
        maximum and minimum values of  
f  on the constraint curve.

        (Note: it is often the case that the values of  λ  in the
 

        solutions of the Lagrange equations are not relevant,  

        and we will not use them for anything.  Still, there are 

        times when it might be convenient to solve for  λ  
        and use those values to get  
x  and  y.

Here is how this ``program'' works in our example above:

   1.  The Lagrange equations are

          The
hardest step is almost always actually solving 

        the Lagrange equations.  A good general strategy is 

        to try to use the equations to eliminate variables and 

        reduce down to an equation in one variable (if possible).
        
For some equations, though, this may require a good deal 

        of cleverness to carry through.  This is an example where 

        that is the case.  

        Here note that the first two equations allow us to
 

        eliminate  λ:  

            `and`(`/`(`*`(x), `*`(`+`(x, `-`(1)))) = lambda, `and`(lambda = `+`(`-`(`/`(`*`(4, `*`(y)), `*`(`+`(y, `-`(1)))))), `+`(`-`(`/`(`*`(4, `*`(y)), `*`(`+`(y, `-`(1)))))) = `+`(`-`(4), `-`(`/`(`*`(4), `*`...
 

 

        So  `and`(`+`(x, `-`(1)) = `/`(1, `*`(`+`(`-`(5), `-`(`/`(`*`(4), `*`(`+`(y, `-`(1)))))))), `/`(1, `*`(`+`(`-`(5), `-`(`/`(`*`(4), `*`(`+`(y, `-`(1)))))))) = `/`(`*`(`+`(y, `-`(1))), `*`(`+`(`-`(`*`(5, `*...Then we can eliminate the  x  in the constraint  

        equation by substituting for the  `+`(x, `-`(1)); -1 

 

           `+`(`/`(`*`(`^`(`+`(y, `-`(1)), 2)), `*`(`^`(`+`(`-`(`*`(5, `*`(y))), 1), 2))), `*`(`/`(1, 4), `*`(`^`(`+`(y, `-`(1)), 2))), `-`(1)) = 0 

        So

             `+`(`*`(`^`(`+`(y, `-`(1)), 2)), `*`(`/`(1, 4), `*`(`^`((`*`(`^`(`+`(y, `-`(1)), 2)))(`+`(`-`(`*`(5, `*`(y))), 1)), 2))), `-`(`*`(`^`(`+`(`-`(`*`(5, `*`(y))), 1), 2)))) = 0
 

> `assign`(poly, expand(`+`(`*`(`^`(`+`(y, `-`(1)), 2)), `*`(`/`(1, 4), `*`(`^`(`+`(y, `-`(1)), 2), `*`(`^`(`+`(`-`(`*`(5, `*`(y))), 1), 2)))), `-`(`*`(`^`(`+`(`-`(`*`(5, `*`(y))), 1), 2)))))); 1
 

`+`(`-`(`*`(`/`(25, 2), `*`(`^`(y, 2)))), `*`(5, `*`(y)), `/`(1, 4), `*`(`/`(25, 4), `*`(`^`(y, 4))), `-`(`*`(15, `*`(`^`(y, 3))))) (1.1)
 

> factor(poly); 1
 

`+`(`-`(`*`(`/`(25, 2), `*`(`^`(y, 2)))), `*`(5, `*`(y)), `/`(1, 4), `*`(`/`(25, 4), `*`(`^`(y, 4))), `-`(`*`(15, `*`(`^`(y, 3))))) (1.2)
 


So this is a ``hard'' example where there are no rational roots and
 

where using a numerical root-finding method is probably needed.
The Maple command for this is the  
fsolve  command.  To set up  

for the fact that there are several different  y - values.  We collect 

them in a list: 

 

> `assign`(yvals, [fsolve(`+`(`*`(`+`(`*`(25, `*`(`^`(y, 4)))), `/`(1, 4)), `-`(`*`(15, `*`(`^`(y, 3)))), `-`(`*`(`/`(25, 2), `*`(`^`(y, 2)))), `*`(5, `*`(y)), `/`(1, 4)), y)]); 1
 

[-.8742938229, -0.4517900248e-1, .3398653489, 2.979607476] (1.3)
 


    For each of these  
y - values, there is exactly one corresponding  x - value, 

    given by       We can have Maple do all the calculations for us like this, using
    the ``for loop'' structure to build up a corresponding list of  
x-values.
 

> for i to 4 do `assign`(xvals[i], `+`(1, `/`(`*`(`+`(yvals[i], `-`(1))), `*`(`+`(`-`(`*`(5, `*`(yvals[i]))), 1))))) end do; -1
 


    Next, we print out the 4 points and the values of  
f  at each one:
 

> for i to 4 do [xvals[i], yvals[i]]; subs({x = xvals[i], y = yvals[i]}, `+`(`*`(`^`(x, 2)), `-`(`*`(`^`(y, 2))))) end do; 1
 

 

 

 

 

 

 

 

[.6510649539, -.8742938229]
-.3405041146
[.1474155684, -0.4517900248e-1]
0.1969020754e-1
[1.943957394, .3398653489]
3.663461895
[.8575620844, 2.979607476]
-8.142647982 (1.4)
 



So we see that

   the
maximum value of  f  on the constraint curve is  about 3.66,  achieved at the
   third critical point in this list:   [1.943957394, .3398653489] , and

   the
minimum value is about  - 8.14,  achieved at the last one:  [.8575620844, 2.979607476].  

Note how this agrees with our graph of the restriction of  
f  to the constraint set  

before(!)   The two other points give a local minimum and a local maximum that are not 

global extrema on the constraint curve.