EEC254 – Optimization

3 units – Winter Quarter; alternate years

Lecture: 3 hours

Prerequisite: Math 22A; knowledge of FORTRAN or C

Grading: Letter; homework (60%) and final examination (40%).

Catalog Description:

Modeling optimization problems existing in engineering design and other applications; optimality conditions; unconstrained optimization (gradient, Newton, conjugate gradient and quasi-Newton methods); duality and Lagrangian relaxation; constrained optimization (Primal method and an introduction to penalty and augmented Lagrangian methods).

Expanded Course Description:

  1. Modeling Optimization Problems
    1. Evolution of optimization-based engineering design
    2. Modeling optimization problems existing in a variety of engineering design situations
  2. Unconstrained Optimization
    1. First- and second-order optimality conditions
    2. Convergence and rate of convergence
  3. Univariate Optimization
    1. Various methods (including Fibonacci search, golden section, and curve fitting) for one-dimensional minimization
  4. Basic Descent Methods
    1. Steepest descent and Armijo gradient algorithms
    2. Newton’s method and local convergence
  5. Conjugate Gradient Method
    1. Conjugate directions
    2. Conjugate gradient algorithm
    3. Rate of convergence
    4. Partial conjugate gradient method
  6. Quasi-Newton Methods
    1. Variable metric concept
    2. Rank one and rank two updates of the approximate Hessian
  7. Constrained Minimization
    1. Optimality conditions
    2. Local duality and Lagrangian relaxation
  8. Primal Methods
    1. Active set method
    2. Gradient projection method
  9. Other Methods
    1. Penalty and barrier methods
    2. Augmented Lagrangian methods


  1. Linear and Nonlinear Programming, Second Edition, David G. Luenberger, Addison-Wesley, 1984.
  2. Nonlinear Programming, Dimitri P. Bertsekas, Athena Scientific, 1995.

Instructor: Chang


Last revised: February 1997