Some Optimization Fundamentals

Transcription

1 ISyE 3133B Engineering Optimization Some Optimization Fundamentals Shabbir Ahmed Homepage:

2 Basic Building Blocks min or max s.t. objective as a function of the decision variables constraints on the decision variables The functions defining the objective 1 and constraints also involve parameters or data, which need to be quantified before the model can be solved. Variable-type or Domain constraints specify the type of values the decision variables can take. Main constraints specify limits on and interactions between the decision variables. 1 We will be primarily concerned with optimization problems involving a single objective function.

3 The Generic Optimization Model min or max f(x 1, x 2,..., x n ) s.t. g i (x 1, x 2,..., x n ) = b i i = 1,..., m x j is continuous or discrete j = 1,..., n. The tractability of the above optimization model largely depends on the structure of the functions f, g 1,..., g m that define the objective and the constraints.

4 X = The Generic Optimization Model Let x = (x 1, x 2,..., x n ) be the n 1 vector of decision variables, and X R n be the set of feasible solutions defined by the constraints, i.e., { x R n : g i (x 1, x 2,..., x n ) Then a generic optimization model is = b i i = 1,..., m } x j is continuous or discrete j = 1,..., n. min f(x) s.t. x X. Note: max f(x) min f(x), therefore it is sufficient to study the problem in minimization form.

5 Linear Programs A function f(x 1, x 2,..., x n ) is linear if it is a constant-weighted sum of the variables, i.e., n f(x 1, x 2,..., x n ) = a j x j j=1 for some constant (fixed) weights a 1, a 2,..., a n. If all the functions f, g 1,..., g m defining the objective and constraints of an optimization are linear, and all the decision variables are allowed to take continuous (fractional or whole) values, the optimization problem is a linear program.

6 Classification Functions Variables Type All linear All continuous Linear Program (LP) One or more nonlinear All continuous Non-linear Program (NLP) All linear All discrete Pure Integer Linear Program All linear Discrete & continuous Mixed Integer Linear Program (MILP) One or more nonlinear All discrete Pure Integer Non-linear Program One or more nonlinear Discrete & continuous Mixed Integer Non-linear Program (MINLP)

7 Graphing Optimization Models An example LP:

8 Graphing Optimization Models An example LP:

9 Graphing Optimization Models An example LP:

10 Graphing Optimization Models An example LP:

11 Graphing Optimization Models An example LP:

12 Graphing Optimization Models An example LP:

13 Graphing Optimization Models An example LP:

14 Graphing Optimization Models An example LP:

15 Graphing Optimization Models An example LP:

16 Graphing Optimization Models An example LP:

17 Graphing Optimization Models An example NLP:

18 Graphing Optimization Models An example NLP:

19 Visualizing Nonlinear Objectives Contour Maps

20 Graphing Optimization Models An example NLP:

21 Graphing Optimization Models An example NLP:

22 Graphing Optimization Models An example NLP:

23 Graphing Optimization Models An example IP:

24 Graphing Optimization Models An example IP:

25 Graphing Optimization Models An example IP:

26 Possible Outcomes Infeasible Unbounded

27 Possible Outcomes Unique Optimal solution Multiple Optimal solutions

28 Possible Outcomes No Optimal solution No Optimal solution ( ] [ ]

29 When is an optimal solution guaranteed to exist? We need to discuss some properties of functions and sets. A set is closed if it includes all its boundary points. Closed Not Closed

30 When is an optimal solution guaranteed to exist? A set is bounded if it can be enclosed in a large enough sphere. Bounded Not Bounded

31 When is an optimal solution guaranteed to exist? A function is continuous if it does not have any jumps. Continuous Not continuous Any polynomial function (e.g. linear)

32 Conditions guaranteeing the existence of an optimal solution 1. The set of feasible solutions is non-empty. 2. The set of feasible solutions is bounded. 3. The set of feasible solutions is closed. 4. The objective function is continuous. The above conditions are only sufficient but not necessary, i.e., an optimization problem not satisfying one or more of the above may still have an optimal solution.

33 Local and Global Optimal Solutions A solution x i is a feasible solution of problem P if x i X. A solution x i is a global optimal solution of problem P if x i is feasible and f(x i ) f(x) for all x X. A solution x i is a local optimal solution of problem P if x i is feasible and f(x i ) f(x) for all x X within a small positive distance from x i (neighborhood of x i ). For a given positive number ɛ > 0, the ɛ-neighborhood of x i is N ɛ (x i ) = {x x x i ɛ}.

34 Local and Global Optimal Solutions (contd.) Thus, a solution x i is a local optimal solution of problem P if x i is feasible and if there is an ɛ > 0 such that f(x i ) f(x) for all x X N ɛ (x i ). Any global optimal solution is also a local optimal solution, but not vice versa.

35 Examples [ ] [ ] [ ] = Local Optimal solution = Global Optimal solution

36 Example = Local Optimal solution = Global Optimal solution

37 Convexity The line segment between the two points x 1 and x 2 consists of all points of the form x 1 + λ(x 2 x 1 ) or (1 λ)x 1 + λx 2 with 0 λ 1. A set X is convex if for any x 1, x 2 X we have that (1 λ)x 1 +λx 2 X for 0 λ 1. A function f is convex if for any two points x 1 and x 2, we have f((1 λ)x 1 + λx 2 ) (1 λ)f(x 1 ) + λf(x 2 ) for 0 λ 1. An optimization problem P where f is a convex function and X is a convex set is called a convex program. E.g. A linear program is a convex program.

38 Convex Programs For a convex program, every local optimal solution is also globally optimal. Most often, finding a local optimal solution is easy but finding a global optimal solution is hard. Convex programs are easier than non-convex ones.

39 Bounds Let v = min{f(x) : x X}. Here v = + if the problem is infeasible, and v = if it is unbounded. Often it is easier to obtain a lower bound on v. Such a bound helps provide a quality certificate for a given feasible solution. Let ˆx X be a feasible solution and let LB be a known lower bound on v then LB v f(ˆx) (Note that a feasible solution provides an upper bound on v ) Then the Optimality Gap of the solution ˆx is f(ˆx) v f(ˆx) LB i.e. we can get an estimate of the optimality gap

40 Relaxations Given an optimization problem (P ) : v = min{f(x) : x X} the problem (R) : v R = min{g(x) : x Y } is a Relaxation of (P) if g(x) f(x) x X and/or X Y Clearly v R v, i.e. relaxations give lower bounds Relaxations are often easier to solve Typically relaxations are obtained by dropping constraints