ENGINEERING OPTIMIZATION

Transcription

1 ENGINEERING OPTIMIZATION Engineering Optimization: Methods and Applications, Second Edition. A. Ravindran, K. M. Ragsdell and G. V. Reklaitis 2006 John Wiley & Sons, Inc. ISBN:

2 ENGINEERING OPTIMIZATION Methods and Applications SECOND EDITION A. Ravindran Industrial and Manufacturing Engineering Pennsylvania State University K. M. Ragsdell Engineering Management and Systems Engineering University of Missouri Rolla G. V. Reklaitis Chemical Engineering Purdue University John Wiley & Sons, Inc.

3 This book is printed on acid-free paper. Copyright 2006 by John Wiley & Sons. All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) , fax (978) , or on the web at Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) , fax (201) , or online at go/ permission. Limit of Liability/ Disclaimer of Warranty: While the publisher and the author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor the author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information about our other products and services, please contact our Customer Care Department within the United States at (800) , outside the United States at (317) or fax (317) Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. For more information about Wiley products, visit our web site at Library of Congress Cataloging-in-Publication Data: Ravindran, A., 1944 Engineering optimization: methods and applications/ A. Ravindran, K. M. Ragsdell, G. V. Reklaitis. 2nd ed. p. cm. Includes bibliographical references and indexes. ISBN (cloth) ISBN (cloth) 1. Engineering Mathematical models. 2. Mathematical optimization. I. Ragsdell, K. M. II. Reklaitis, G. V., 1942 III. Title. TA342.R dc Printed in the United States of America

4 CONTENTS Preface xiii 1 Introduction to Optimization Requirements for the Application of Optimization Methods / Defining the System Boundaries / Performance Criterion / Independent Variables / System Model / Applications of Optimization in Engineering / Design Applications / Operations and Planning Applications / Analysis and Data Reduction Applications / Classical Mechanics Applications / Taguchi System of Quality Engineering / Structure of Optimization Problems / Scope of This Book / 29 References / 30 2 Functions of a Single Variable Properties of Single-Variable Functions / Optimality Criteria / Region Elimination Methods / Bounding Phase / Interval Refinement Phase / Comparison of Region Elimination Methods / Polynomial Approximation or Point Estimation Methods / Quadratic Estimation Methods / 56 v

5 vi CONTENTS Successive Quadratic Estimation Method / Methods Requiring Derivatives / Newton Raphson Method / Bisection Method / Secant Method / Cubic Search Method / Comparison of Methods / Summary / 70 References / 71 Problems / 71 3 Functions of Several Variables Optimality Criteria / Direct-Search Methods / The S 2 (Simplex Search) Method / Hooke Jeeves Pattern Search Method / Powell s Conjugate Direction Method / Gradient-Based Methods / Cauchy s Method / Newton s Method / Modified Newton s Method / Marquardt s Method / Conjugate Gradient Methods / Quasi-Newton Methods / Trust Regions / Gradient-Based Algorithm / Numerical Gradient Approximations / Comparison of Methods and Numerical Results / Summary / 137 References / 137 Problems / Linear Programming Formulation of Linear Programming Models / 149

6 CONTENTS vii 4.2 Graphical Solution of Linear Programs in Two Variables / Linear Program in Standard Form / Handling Inequalities / Handling Unrestricted Variables / Principles of the Simplex Method / Minimization Problems / Unbounded Optimum / Degeneracy and Cycling / Use of Artificial Variables / Two-Phase Simplex Method / Computer Solution of Linear Programs / Computer Codes / Computational Efficiency of the Simplex Method / Sensitivity Analysis in Linear Programming / Applications / Additional Topics in Linear Programming / Duality Theory / Dual Simplex Method / Interior Point Methods / Integer Programming / Goal Programming / Summary / 206 References / 206 Problems / Constrained Optimality Criteria Equality-Constrained Problems / Lagrange Multipliers / Economic Interpretation of Lagrange Multipliers / Kuhn Tucker Conditions / Kuhn Tucker Conditions or Kuhn Tucker Problem / Interpretation of Kuhn Tucker Conditions / Kuhn Tucker Theorems / 229

7 viii CONTENTS 5.6 Saddlepoint Conditions / Second-Order Optimality Conditions / Generalized Lagrange Multiplier Method / Generalization of Convex Functions / Summary / 254 References / 254 Problems / Transformation Methods Penalty Concept / Various Penalty Terms / Choice of Penalty Parameter R / Algorithms, Codes, and Other Contributions / Method of Multipliers / Penalty Function / Multiplier Update Rule / Penalty Function Topology / Termination of the Method / MOM Characteristics / Choice of R-Problem Scale / Variable Bounds / Other MOM-Type Codes / Summary / 293 References / 294 Problems / Constrained Direct Search Problem Preparation / Treatment of Equality Constraints / Generation of Feasible Starting Points / Adaptations of Unconstrained Search Methods / Difficulties in Accommodating Constraints / Complex Method / Discussion / 320

8 CONTENTS ix 7.3 Random-Search Methods / Direct Sampling Procedures / Combined Heuristic Procedures / Discussion / Summary / 330 References / 330 Problems / Linearization Methods for Constrained Problems Direct Use of Successive Linear Programs / Linearly Constrained Case / General Nonlinear Programming Case / Discussion and Applications / Separable Programming / Single-Variable Functions / Multivariable Separable Functions / Linear Programming Solutions of Separable Problems / Discussion and Applications / Summary / 372 References / 373 Problems / Direction Generation Methods Based on Linearization Method of Feasible Directions / Basic Algorithm / Active Constraint Sets and Jamming / Discussion / Simplex Extensions for Linearly Constrained Problems / Convex Simplex Method / Reduced Gradient Method / Convergence Acceleration / Generalized Reduced Gradient Method / Implicit Variable Elimination / Basic GRG Algorithm / 410

9 x CONTENTS Extensions of Basic Method / Computational Considerations / Design Application / Problem Statement / General Formulation / Model Reduction and Solution / Summary / 441 References / 441 Problems / Quadratic Approximation Methods for Constrained Problems Direct Quadratic Approximation / Quadratic Approximation of the Lagrangian Function / Variable Metric Methods for Constrained Optimization / Discussion / Problem Scaling / Constraint Inconsistency / Modification of H (t) / Comparison of GRG with CVM / Summary / 475 References / 476 Problems / Structured Problems and Algorithms Integer Programming / Formulation of Integer Programming Models / Solution of Integer Programming Problems / Guidelines on Problem Formulation and Solution / Quadratic Programming / Applications of Quadratic Programming / Kuhn Tucker Conditions / Complementary Pivot Problems / Goal Programming / Summary / 518

10 CONTENTS xi References / 518 Problems / Comparison of Constrained Optimization Methods Software Availability / A Comparison Philosophy / Brief History of Classical Comparative Experiments / Preliminary and Final Results / Summary / 539 References / Strategies for Optimization Studies Model Formulation / Levels of Modeling / Types of Models / Problem Implementation / Model Assembly / Preparation for Solution / Execution Strategies / Solution Evaluation / Solution Validation / Sensitivity Analysis / Summary / 594 References / 594 Problems / Engineering Case Studies Optimal Location of Coal-Blending Plants by Mixed-Integer Programming / Problem Description / Model Formulation / Results / Optimization of an Ethylene Glycol Ethylene Oxide Process / Problem Description / 610

11 xii CONTENTS Model Formulation / Problem Preparation / Discussion of Optimization Runs / Optimal Design of a Compressed Air Energy Storage System / Problem Description / Model Formulation / Numerical Results / Discussion / Summary / 630 References / 631 Appendix A Review of Linear Algebra 633 A.1 Set Theory / 633 A.2 Vectors / 633 A.3 Matrices / 634 A.3.1 Matrix Operations / 635 A.3.2 Determinant of a Square Matrix / 637 A.3.3 Inverse of a Matrix / 637 A.3.4 Condition of a Matrix / 639 A.3.5 Sparse Matrix / 639 A.4 Quadratic Forms / 640 A.4.1 Principal Minor / 641 A.4.2 Completing the Square / 642 A.5 Convex Sets / 646 Appendix B Convex and Concave Functions 648 Appendix C Gauss Jordan Elimination Scheme 651 Author Index 653 Subject Index 659

12 PREFACE We are deeply gratified by the enthusiastic response given to the first edition of our book by our colleagues and students. We took great care in preparing the second edition. We added some new material, revised the explanation and presentation of some topics, and increased the number of problems. Examples of new material include more application of optimization problems in Chapter 1, discussion of duality theory and interior point methods for solving LP problems in Chapter 4, new sections on the generalized Lagrange multiplier method and generalization of convex functions in Chapter 5, a new section on goal programming for solving multiobjective optimization problems in Chapter 11, and the inclusion of software availability for solving nonlinear programs in Chapter 12. To make room for the new material and to keep the size of the book down, we deleted discussion of cutting plane methods in Chapter 8, gradient projection methods, in Chapter 9 and geometric programming in Chapter 11. This is a text on the practical aspects of optimization methodology, with a major focus on the techniques and stratagems relevant to engineering application arising in design, operations, and analysis. Attention is given primarily to techniques applicable to problems in continuous variables that involve realvalued constraint functions and a single real-valued objective function. In short, we treat the methodology often categorized as nonlinear programming. Within this framework a broad survey is given of all-important families of optimization methods, ranging from those applicable to the minimization of a single-variable function to those most suitable for large-scale nonlinear constrained problems. Included are discussions not only of classical methods, important for historical reasons and for their role in motivating subsequent developments in this field, but also of promising new techniques, such as those of successive quadratic programming and goal programming. Our development is aimed at imparting an understanding of the logic of the methods, of the key assumptions that underlie them, and of the comparative merits of the methods. Proofs and mathematical derivations are given only if they serve to explain key steps or properties of algorithms. Generally, we simply cite the original literature source for proofs and use the pages of this book to motivate and explain the key steps underlying the mathematical constructions. Thus, our aim is to inform the engineer-user of optimization methodology rather than to prepare the software specialist who will develop computer implementations of the algorithms. In keeping with this aim, we xiii

13 xiv PREFACE have given considerable attention to practical issues such as model formulation, implementation, preparation for solution, starting point generation, and the selection of execution strategies. A major chapter (Chapter 13) is devoted to strategies for carrying out optimization studies; another (Chapter 12) reviews the state-of-the-art optimization software and the results of existing comparative studies for solving nonlinear programs; and a third (Chapter 14) discusses three significant engineering case studies. In addition, a considerable fraction of each chapter is allocated to engineering examples drawn from the chemical, industrial, and mechanical engineering backgrounds of the authors. While a number of excellent books are available that deal in detail with the rich theoretical and numerical analysis issues that are relevant to nonlinear programming, this book is unique in the features outlined above: broad treatment of up-to-date method; conceptual, rather than formal, presentation; and focus on issues relevant to engineering studies. The first edition of the book was developed over a period of eight years in which various drafts were used in a one-semester interdisciplinary engineering optimization course team-taught by the authors to senior undergraduate and first-year graduate students at Purdue University. For these students, this course was typically the first systematic exposure to optimization methods. The students mathematical preparation consisted of the calculus and linear algebra coursework typical of BS engineering curricula; hence, that is all that is required of the reader of this book. The organization of the book has also benefited considerably from the authors experience in teaching a television course on engineering optimization broadcast to regional campuses, nonresident MS, and professional engineering audiences. The authors have used the first edition as a text in engineering optimization courses taught at Arizona, Oklahoma, and Purdue universities. The book has also been used in in-plant training courses, with National Technological University courses, and courses taught live over the Internet. We are therefore confident that the book can serve as a text for conventional classroom lectures, for television courses, for on-line courses, and for industrial short courses as well as for self-study. Two different course organizations have been followed in teaching from this text: an all-lecture, one-semester course involving 45 fifty-minute lectures and a lecture recitation format involving 30 lectures and 15 recitation discussion sessions. In the former case, the entire contents of the book, except for Chapter 14, can be covered in numerical chapter sequence. In the latter case, Chapters 1, 13, and 14, as well as additional case studies and examples, are discussed in the recitation sessions, while the methodology chapters (Chapters 2 10) and Chapter 12 are covered in the lectures. In this format, Chapter 11 was omitted because of the limited lecture hours. Homework problems in both formats should include problems and computer exercises given at the end of the chapters. Student computer solutions can be carried out using the software programs referenced in Chapter 12. The development and evolution of this book have benefited substantially from the advice and counsel, both conscious and unintentional, of our col-

14 PREFACE xv leagues and mentors in our respective engineering disciplines, who are too numerous to acknowledge individually. We are indebted for the numerous questions and pleas for clarification raised by our students in engineering optimization, who have persevered through various revisions of the manuscript. Their persistent, often biting, and usually justifiable criticism, overt and sub rosa, has been a key driving force in the revision process. We wish to thank our Wiley editor, Robert Argentieri, for his perseverance and patience. We express our sincere appreciation to Ajay Natarajan, an industrial engineering doctoral student at Penn State University, for his careful preparation of the author and subject indexes for the second edition. Finally we are grateful to the instructors who have adopted our first edition and for their encouragement and helpful suggestions that made the second edition a reality. University Park, Pennsylvania Rolla, Missouri West Lafayette, Indiana A. (RAVI) RAVINDRAN K. M. RAGSDELL G. V. REKLAITIS