Chapter2

CLASSICAL THEORY OF MAXIMA AND MINIMA

 
Locating Local Maxima and Minima (Necessary Conditions) top

    Based on geometric intuition from the previous example, we can understand the famous Weierstrass' theorem (11,12) which guarantees the existence of maxima and minima. It states:

           Every function which is continuous in a closed domain possesses a maximum and minimum Value

            either in the interior or on the boundary of the domain.

     The proof is by contradiction.
          There is another theorem (13) which tells how to locate extreme points in the interior of a region of a continuous function. It states:

          A continuous function of n variables attains a maximum or a minimum in the interior of a region, only at those values of the variables for which the n partial derivatives either vanish simultaneously (stationary points) or at which one or more of these derivatives cease to exist (i.e., are discontinuous).

The proof involves examining the Taylor Series expansion at the points where the partial derivatives either vanish or cease to exist.
    Thus the problem becomes one of locating points where the partial derivatives are zero or where some of them are discontinuous. The stationary points can be located by solving the algebraic equations which result in setting the partial derivatives of the function equal to zero. Also the algebraic equations must be examined for points of discontinuities, and this has to be accomplished by inspection.

Evaluating Local Maxima and Minima (Sufficient Conditions) top

         As we have seen, it is not necessary for all stationary points to be local maxima and minima, since there is a possibility of saddle or inflection points. Now we need to develop procedures to determine if stationary points are maxima or minima. These sufficient conditions will be developed for one independent variable first and then extended for two and n independent variables, using the same concepts. Once the local maxima and minima are located, it is necessary to compare the individual points to locate the global maximum and minimum.

Sufficient Conditions for One Independent Variable top

        To develop criteria establishing whether a stationary point is a local maximum or minimum, we begin by performing a Taylor series expansion about the stationary point xo.

y(x) = y(xo) + y'(xo) (x - xo) + ½ y''(xo) (x - xo)2 + higher order terms

Now, select x sufficiently close to xo so the higher order terms become negligible compared to the second-order terms. Since the first derivative is zero at the stationary point, the above equation becomes

y(x) = y(xo) + ½y" (xo) (x - xo)                                                (2-1)

We can determine if xo is a local maximum or minimum by examining the value of y"(xo), since (x - xo)2 is always positive. If y"(xo) is positive, then the terms ½y"(xo) (x - xo)2 will always add to y(xo) in equation (2-1) for x taking on values that are less than of greater than xo. For this case y(xo) is a local minimum. This is summarized in the following:

______________________________________________________________

If y''(xo) > 0 then y(xo) is a minimum

y''(xo) < 0 y(xo) is a maximum

y''(xo) = 0 no statement can be made
______________________________________________________________

If the second derivative is zero, it is necessary to examine higher order derivatives. In general if y''(xo) = ... = yn-1(xo) = 0, the Taylor series expansion becomes


y(x) = y(xo) +(1/n!) y(n)(xo) (x - xo)n                                 (2-2)

If n is even, then (x - xo)n is always positive, and the result is:

______________________________________________________________

If y(n)(xo) > 0 then y(xo) is a minimum

y(n)(xo) < 0 y(xo) is a maximum
______________________________________________________________


If n is odd then (x - xo)n changes sign as x moves from x < xo to x > xo, and thus there is an inflection point. These results can be summarized in the following theorem (1).


           If at a stationary point the first and possibly some of the higher derivatives vanish, then  

          the point is or is not an extreme point, according as the first non-vanishing derivative is

         of   even or odd order. If it is even, there is a maximum or minimum according as the

         derivative is negative or positive.


The proof of this theorem follows the discussion given above. The following example illustrates the principles discussed.

Example 2-1
    Locate the extreme points of the following two functions:

                              y(x) = x4/4 - x2/2

                           y'(x) = x3 - x = x(x2 - 1) = x(x - 1)(x+1) = 0

       Stationary points are x = 0,1,-1

                           y"(x) = 3x2 - 1

                          y"(0) = -1 maximum

                          y"(1) = 2 minimum

                          y"(-1) = 2 minimum


               y(x) = x5

              y'(x) = 5x4 = 0 stationary point is x = 0

             y"(x) = 20x3 y"(0) = 0

             y"'(x) = 60x2 y"'(0) = 0 no statement can be made

             y(4)(x) = 120x y(4)(0) = 0

            y(5)(x) = 120 y(5)(0) = 120 n is odd, and the stationary
                                                        point is an inflection point.

 

Sufficient Conditions for Two Independent Variables top

      To develop the criteria for a local maximum or minimum for a stationary point xo(x10, x20) of a function of two variables, a Taylor's series expansion is made about this point.

y(x1, x2)  =  y(x10, x20)  +  yx 1 (x1-x10)  +  yx 2 (x2-x20)

+ [yx1 x1 (x1-x10)2   +   2yx1 x2 (x1-x10)(x2-x20)                  (2-3)  

+ yx2 x2 (x2-x20)2]   +    higher order terms

where the subscripts x1 and x2 indicate partial differentiation with respect to those variables and evaluation at the stationary point.

Again we select y(x1, x2) sufficiently close to y(x10, x20) so the higher order terms become negligible compared to the second order terms. Also, the first derivatives are zero at the stationary point. Thus equation (2-3) can be written in matrix form as:

Equ2-4.jpg (9834 bytes)

In matrix-vector notation the above equation can be written as

y(x)  =   y(xo)  +  [(x - xo)T Ho (x - xo)]                                                                    (2-5)

where Ho is the matrix of second partial derivatives evaluated at the stationary point xo and is called the Hessian matrix.

      The term in the bracket of equation (2-5) is called a differential quadratic form, and y(xo) will be a minimum or a maximum accordingly if this term is always positive or always negative. Based on this concept, it can be shown (1) that if the following results apply xo is a maximum or a minimum. If they do not hold, xo could be a saddle point and is not a maximum or a minimum.

Pg15.jpg (15810 bytes)

 

 

 

 

 

An illustration of the above results is given in example 2-2. The term in the bracket of equation(2-5) is an example of a quadratic form. It will be necessary to describe a quadratic form briefly before giving the sufficient conditions for maxima and minima for n-independent variables.

Sign of a Quadratic Form top

       To perform a similar analysis for a function with more than two independent variables, it is necessary to determine what is called the sign of the quadratic form. The general quadratic form (1) is written as:

Equ2-6.jpg (5454 bytes)

where aij are the components of symmetric matrix A, e.g., aij = aji.

It turns out (1) that we can determine if Q is always positive or negative, for all finite values of xi and xj, by evaluating the signs of Di, the determinants of the principal submatrices of A.

Equ2-7.jpg (7030 bytes)

 

The important results that will be used subsequently are:

______________________________________________________________________________

If Di > 0 for i = 1,2,...n then: A is positive definite and Q(A,x) > 0

If Di < 0 for i = 1,3...   and

   Di > 0 for i = 2,4,...   then: A is negative definite and Q(A,x) < 0

If Di is neither of these, then: Q(A,x) and depends on the values of xi and xj.

______________________________________________________________________________

Sufficient Conditions for N Independent Variables top

       The result of the previous two sections can be extended to the case of n independent variables by considering the Taylor series expansion for n independent variables around stationary point xo:

Equ2-8.jpg (12834 bytes)                                                

 

Again select x sufficiently close to xo, so the higher order terms become negligible compared to the second order terms. Also, the first derivatives are zero at the stationary point. Thus, equation (2-8) can be written in matrix-vector notation as:

y(x) = y(xo) + (x - xo)T Ho (x-xo)                                                               (2-9)


where x is the column vector of independent variables, and Ho, the matrix of second partial derivative evaluated at the stationary point xo, in the Hessian matrix. This is the same equation as equation (2-5), which was written for two independent variables.

Equ2-10.jpg (6170 bytes)

The second term on the right hand side of equation (2-9) is called a differential quadratic form as shown below.

Q[ Ho, (x-xo)] = (x-xo)T Ho (x-xo)                                                              (2-11)


Equation (2-11) corresponds to equation (2-6) in the previous section, and the determinants of the principal submatrices of Ho as defined below correspond to equation (2-7).

Equ2-12.jpg (8287 bytes)

      We can now use the same procedure in evaluating the character of the stationary points for n independent variables. For example, if the term containing the Hessian matrix is always positive for perturbations of the independent variables around the stationary point, then the stationary point is a local minimum. For this differential quadratic form to be positive always, the |Ho| > 0 for i = 1,2,...n.  The same reasoning can be applied for a local maximum, and the results for these two cases are summarized below:

___________________________________________________________________

y(xo) is a minimum if |Hoi| > 0 for i = 1,2,...,n

y(xo) is a maximum if |Hoi| < 0 for i = 1,3,5,...

   |Hoi| > 0 i = 2,4,6,...

___________________________________________________________________


     If zeros occur in the place of some of the positive or negative number in the tests above (semi-definite quadratic form), then there is insufficient information to determine the character of the stationary point (1). As discussed in Avriel (10) higher order terms may have to be examined, or local exploration can be performed. If the test is not met (indefinite quadratic form), then the point is neither a maximum or minimum (1). The following theorem from Cooper (7) summarize these results. It states:

If y(x) and its first two partial derivatives are continuous, then a sufficient condition for y(x) to have a relative minimum (maximum) at xo, the when dy(xo)/dxj = 0,  j = 1,2, ...n, is that Hessian matrix be positive definite (negative definite).

The proof of this theorem employs arguments similar to those given above.

    The following example illustrates these methods.

 

Example 2-2

The flow diagram of a simple process is shown in Figure 2-2 (2) where the hydrocarbon feed is mixed with recycle and compressed before being passed into a catalytic reactor. The product and unreacted material are separated by distillation, and the unreacted material is recycled. The pressure, P, in psi and recycle ratio, R, must be selected to minimized the total annual cost for the required production rate of 107 pounds per year. The feed is brought up to pressure at an annual cost of $1000P, mixed with the recycle stream, and fed to the reactor at an annual cost of $4 x 109/PR. The product is removed in a separator at a cost of $105R per year, and the unreacted material is recycled in a recirculating compressor which consumes $1.5 x 105R annually. Determine the optimal operating pressure, recycle ratio, and total annual cost; and show that the cost is a minimum

Solution: The equation giving the total operating cost is:

C ($/yr.) = 1000P + 4 x 109/ P R + 2.5 x 105R

Equating the partial derivatives of C with respect P and R to zero gives two algebraic equations to be solved for P and R.

dC/dP = 1000 - 4 x 109 / R P2 = 0

dC/dR = 2.5 x 105 - 4 x 109 / P R2 = 0


Solving simultaneously gives

P = 1000psi and R = 4

Substituting to determine the corresponding total operating cost gives

C = $ 3 x 106 per year

C (P,R) is a minimum if

Ex2-2a.jpg (8398 bytes)

Performing the appropriate partial differentiation and evaluation at the stationary point ( P = 1000, R = 4) gives

Ex2-2b.jpg (8528 bytes)

Thus, the stationary point is a minimum since both determinants are positive.