Chapter2

CLASSICAL THEORY OF MAXIMA AND MINIMA

• Introduction
• Analytical Methods without Constraints
  • Locating Local Maxima and Minima (Necessary Conditions)
  • Evaluating Local Maxima and Minima (Sufficient Conditions)
  • Sufficient Conditions for One Independent Variables
  • Sufficient Conditions for Two Independent Variables
  • Sign of a Quadratic Form
  • Sufficient Conditions for N Independent Variables
• Analytical Methods Applicable for Constraints
  • Direct Substitution
  • Constrained Variation
  • Lagrange Multipliers
  • Method of Steepest Ascent
  • Economic Interpretation of the Lagrange Multipliers
  • Inequality Constraints
• Necessary and Sufficient Conditions for Constrained Problems
• Closure
• References
• Problems

    Based on geometric intuition from the previous example, we can understand the famous Weierstrass' theorem (11,12) which guarantees the existence of maxima and minima. It states:

           Every function which is continuous in a closed domain possesses a maximum and minimum Value

            either in the interior or on the boundary of the domain.

     The proof is by contradiction.
          There is another theorem (13) which tells how to locate extreme points in the interior of a region of a continuous function. It states:

          A continuous function of n variables attains a maximum or a minimum in the interior of a region, only at those values of the variables for which the n partial derivatives either vanish simultaneously (stationary points) or at which one or more of these derivatives cease to exist (i.e., are discontinuous).

The proof involves examining the Taylor Series expansion at the points where the partial derivatives either vanish or cease to exist.
    Thus the problem becomes one of locating points where the partial derivatives are zero or where some of them are discontinuous. The stationary points can be located by solving the algebraic equations which result in setting the partial derivatives of the function equal to zero. Also the algebraic equations must be examined for points of discontinuities, and this has to be accomplished by inspection.

         As we have seen, it is not necessary for all stationary points to be local maxima and minima, since there is a possibility of saddle or inflection points. Now we need to develop procedures to determine if stationary points are maxima or minima. These sufficient conditions will be developed for one independent variable first and then extended for two and n independent variables, using the same concepts. Once the local maxima and minima are located, it is necessary to compare the individual points to locate the global maximum and minimum.

        To develop criteria establishing whether a stationary point is a local maximum or minimum, we begin by performing a Taylor series expansion about the stationary point x_o.

y(x) = y(x_o) + y'(x_o) (x - x_o) + ½ y''(x_o) (x - x_o)² + higher order terms

Now, select x sufficiently close to x_o so the higher order terms become negligible compared to the second-order terms. Since the first derivative is zero at the stationary point, the above equation becomes

y(x) = y(xo) + ½y" (x_o) (x - x_o)                                                (2-1)

We can determine if x_o is a local maximum or minimum by examining the value of y"(x_o), since (x - x_o)² is always positive. If y"(x_o) is positive, then the terms ½y"(x_o) (x - x_o)² will always add to y(x_o) in equation (2-1) for x taking on values that are less than of greater than x_o. For this case y(xo) is a local minimum. This is summarized in the following:

______________________________________________________________

If y''(x_o) > 0 then y(x_o) is a minimum

y''(x_o) < 0 y(x_o) is a maximum

y''(x_o) = 0 no statement can be made
______________________________________________________________

If the second derivative is zero, it is necessary to examine higher order derivatives. In general if y''(xo) = ... = y^n-1(xo) = 0, the Taylor series expansion becomes

y(x) = y(xo) +(1/n!) y(n)(xo) (x - xo)ⁿ                                 (2-2)

If n is even, then (x - xo)n is always positive, and the result is:

______________________________________________________________

If y⁽ⁿ⁾(xo) > 0 then y(xo) is a minimum

y⁽ⁿ⁾(xo) < 0 y(xo) is a maximum
______________________________________________________________

If n is odd then (x - xo)ⁿ changes sign as x moves from x < xo to x > xo, and thus there is an inflection point. These results can be summarized in the following theorem (1).

           If at a stationary point the first and possibly some of the higher derivatives vanish, then  

          the point is or is not an extreme point, according as the first non-vanishing derivative is

         of   even or odd order. If it is even, there is a maximum or minimum according as the

         derivative is negative or positive.

The proof of this theorem follows the discussion given above. The following example illustrates the principles discussed.

Example 2-1
    Locate the extreme points of the following two functions:

                              y(x) = x⁴/4 - x²/2

                           y'(x) = x³ - x = x(x² - 1) = x(x - 1)(x+1) = 0

       Stationary points are x = 0,1,-1

                           y"(x) = 3x² - 1

                          y"(0) = -1 maximum

                          y"(1) = 2 minimum

                          y"(-1) = 2 minimum

               y(x) = x⁵

              y'(x) = 5x⁴ = 0 stationary point is x = 0

             y"(x) = 20x³ y"(0) = 0

             y"'(x) = 60x² y"'(0) = 0 no statement can be made

             y(4)(x) = 120x y(4)(0) = 0

            y(5)(x) = 120 y(5)(0) = 120 n is odd, and the stationary
                                                        point is an inflection point.

      To develop the criteria for a local maximum or minimum for a stationary point x_o(x₁₀, x₂₀) of a function of two variables, a Taylor's series expansion is made about this point.

y(x₁, x₂)  =  y(x₁₀, x₂₀)  +  y_{x 1} (x₁-x₁₀)  +  y_{x 2} (x₂-x₂₀)

+ ½[y_{x1 x1} (x₁-x₁₀)²   +   2y_{x1 x2} (x₁-x₁₀)(x₂-x₂₀)                  (2-3)  

+ y_{x2 x2} (x₂-x₂₀)²]   +    higher order terms

where the subscripts x₁ and x₂ indicate partial differentiation with respect to those variables and evaluation at the stationary point.

Again we select y(x₁, x₂) sufficiently close to y(x₁₀, x₂₀) so the higher order terms become negligible compared to the second order terms. Also, the first derivatives are zero at the stationary point. Thus equation (2-3) can be written in matrix form as:

In matrix-vector notation the above equation can be written as

y(x)  =   y(x_o)  +  ½[(x - x_o)^T H^o (x - x_o)]                                                                    (2-5)

where H^o is the matrix of second partial derivatives evaluated at the stationary point x_o and is called the Hessian matrix.

      The term in the bracket of equation (2-5) is called a differential quadratic form, and y(x_o) will be a minimum or a maximum accordingly if this term is always positive or always negative. Based on this concept, it can be shown (1) that if the following results apply x_o is a maximum or a minimum. If they do not hold, x_o could be a saddle point and is not a maximum or a minimum.

An illustration of the above results is given in example 2-2. The term in the bracket of equation(2-5) is an example of a quadratic form. It will be necessary to describe a quadratic form briefly before giving the sufficient conditions for maxima and minima for n-independent variables.

       To perform a similar analysis for a function with more than two independent variables, it is necessary to determine what is called the sign of the quadratic form. The general quadratic form (1) is written as:

where a_ij are the components of symmetric matrix A, e.g., a_ij = a_ji.

It turns out (1) that we can determine if Q is always positive or negative, for all finite values of x_i and x_j, by evaluating the signs of D_i, the determinants of the principal submatrices of A.

The important results that will be used subsequently are:

______________________________________________________________________________

If D_i > 0 for i = 1,2,...n then: A is positive definite and Q(A,x) > 0

If D_i < 0 for i = 1,3...   and

   D_i > 0 for i = 2,4,...   then: A is negative definite and Q(A,x) < 0

If D_i is neither of these, then: Q(A,x) and depends on the values of x_i and x_j.

______________________________________________________________________________

       The result of the previous two sections can be extended to the case of n independent variables by considering the Taylor series expansion for n independent variables around stationary point x_o:

Again select x sufficiently close to x_o, so the higher order terms become negligible compared to the second order terms. Also, the first derivatives are zero at the stationary point. Thus, equation (2-8) can be written in matrix-vector notation as:

y(x) = y(x_o) + ½(x - x_o)^T H^o (x-x_o)                                                               (2-9)

where x is the column vector of independent variables, and H_o, the matrix of second partial derivative evaluated at the stationary point x_o, in the Hessian matrix. This is the same equation as equation (2-5), which was written for two independent variables.

The second term on the right hand side of equation (2-9) is called a differential quadratic form as shown below.

Q[ H^o, (x-x_o)] = (x-x_o)^T H^o (x-x^o)                                                              (2-11)

Equation (2-11) corresponds to equation (2-6) in the previous section, and the determinants of the principal submatrices of H^o as defined below correspond to equation (2-7).

      We can now use the same procedure in evaluating the character of the stationary points for n independent variables. For example, if the term containing the Hessian matrix is always positive for perturbations of the independent variables around the stationary point, then the stationary point is a local minimum. For this differential quadratic form to be positive always, the |H^o| > 0 for i = 1,2,...n.  The same reasoning can be applied for a local maximum, and the results for these two cases are summarized below:

___________________________________________________________________

y(x_o) is a minimum if |H^o_i| > 0 for i = 1,2,...,n

y(x_o) is a maximum if |H^o_i| < 0 for i = 1,3,5,...

   |H^o_i| > 0 i = 2,4,6,...

___________________________________________________________________

     If zeros occur in the place of some of the positive or negative number in the tests above (semi-definite quadratic form), then there is insufficient information to determine the character of the stationary point (1). As discussed in Avriel (10) higher order terms may have to be examined, or local exploration can be performed. If the test is not met (indefinite quadratic form), then the point is neither a maximum or minimum (1). The following theorem from Cooper (7) summarize these results. It states:

If y(x) and its first two partial derivatives are continuous, then a sufficient condition for y(x) to have a relative minimum (maximum) at x_o, the when dy(x_o)/dx_j = 0,  j = 1,2, ...n, is that Hessian matrix be positive definite (negative definite).

The proof of this theorem employs arguments similar to those given above.

    The following example illustrates these methods.

Example 2-2

The flow diagram of a simple process is shown in Figure 2-2 (2) where the hydrocarbon feed is mixed with recycle and compressed before being passed into a catalytic reactor. The product and unreacted material are separated by distillation, and the unreacted material is recycled. The pressure, P, in psi and recycle ratio, R, must be selected to minimized the total annual cost for the required production rate of 10⁷ pounds per year. The feed is brought up to pressure at an annual cost of $1000P, mixed with the recycle stream, and fed to the reactor at an annual cost of $4 x 10⁹/PR. The product is removed in a separator at a cost of $10⁵R per year, and the unreacted material is recycled in a recirculating compressor which consumes $1.5 x 10⁵R annually. Determine the optimal operating pressure, recycle ratio, and total annual cost; and show that the cost is a minimum

Solution: The equation giving the total operating cost is:

C ($/yr.) = 1000P + 4 x 10⁹/ P R + 2.5 x 10⁵R

Equating the partial derivatives of C with respect P and R to zero gives two algebraic equations to be solved for P and R.

dC/dP = 1000 - 4 x 10⁹ / R P² = 0

dC/dR = 2.5 x 10⁵ - 4 x 10⁹ / P R² = 0

Solving simultaneously gives

P = 1000psi and R = 4

Substituting to determine the corresponding total operating cost gives

C = $ 3 x 10⁶ per year

C (P,R) is a minimum if

Performing the appropriate partial differentiation and evaluation at the stationary point ( P = 1000, R = 4) gives

Thus, the stationary point is a minimum since both determinants are positive.