# 7 Branch and Bound, and Dynamic Programming

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 7 Branch and Bound, and Dynamic Programming 7.1 Knapsack An important combinatorial optimization problem is the Knapsack Problem, which can be defined as follows: Given nonnegative integers n, c 1,..., c n, w 1,..., w n and W. Find a subset S {1,..., n} such that i S w i W and i S c i is maximal. In the standard formulation c i and w i are the costs and weight of item i, respectively, and W is the capacity of the knapsack. The problem is to put items into the knapsack such that the total weight does not exceed the capacity, and the total costs are maximal. The above definition leads straightforwardly to the following ILP formulation: Maximize subject to ni=1 c i x i n i=1 w i x i W 0 x i 1 (i = 1,..., n) x i integral (i = 1,..., n) Here x i is a decision variable that gets value 1 if item i is in the knapsack, and 0 otherwise. This ILP formulation of the knapsack problem has the advantage that it is very easy to solve its LP-relaxation. Proposition 1 Let c 1,..., c n, w 1,..., w n and W be nonnegative integers with and let c 1 w 1 c 2 w 2 c n w n, j k := min j {1,..., n} : i=1 w i > W. Then an optimum solution of the LP-relaxation of the ILP model of the knapsack problem is defined by: 1 i = 1,..., k 1, k 1 W x i = i=1 w j w k i = k, 0 i = k + 1,..., n. The value of this LP-relaxation is clearly an upper bound for the optimal value of the knapsack problem. Notice that the solution of the LP-relaxation has at most one variable with a fractional value. If all weights (w i ) are smaller than W, then the two solutions {1,..., k 1} and {k} are both feasible the better of these two solutions achieves at least half the optimum value. To be more precise: Proposition 2 Let for an instance I of the knapsack problem OP T (I) be its optimal value and LP (I) be the value of the LP-relaxation. If all weights are at most W then 1 LP (I) < OP T (I) LP (I). 2 1

2 7.2 Branch and Bound The branch and bound method is based on the idea of intelligently enumerating all the feasible points of a combinatorial optimization problem. In the branch and bound method we search for an optimal solution based on successive partitioning of the solution space. The branch in branch and bound refers to this partitioning process; the bound refers to lower bounds that are used to eliminate parts of the solution space that will not contain an optimum. First we shall develop the method for ILP, and then put things in a more abstract framework. Consider the ILP problem Problem 0: minimize z = c x = c(x) subject to Ax b x 0, x Z. If we solve the LP relaxation, we obtain a solution x 0, which in general is not integer. The cost c(x 0 ) of this solution is, however, a lower bound on the optimal cost c(x ) (where x is the optimal solution to Problem 0), and if x 0 were integer, we would in fact be done. In the cutting plane algorithm, we would now add a constraint to the relaxed problem that does not exclude feasible solutions. Here, however, we are going to split the problem into two subproblems by adding two mutually exclusive and exhaustive constraints. Suppose that component x 0 i of x0 is noninteger, for example. Then the two subproblems are Problem 1: minimize z = c x = c(x) subject to Ax b x 0, x Z x i x 0 i and Problem 2: minimize z = c x = c(x) subject to Ax b x 0, x Z x i x 0 i + 1 The solution to the original problem must lie in the feasible region of one of these two problems, simply because one of the following two statements must be true. x i x 0 i, x i x 0 i + 1. We now choose one of the subproblems, say Problem 1, which is after all an LP, and solve it. The solution x 1 will in general not be integer, and we may split Problem 1 into two subproblems just as we split Problem 0, creating Problems 3 and 4. We can visualize this process as a successive finer and finer subdivision of the feasible region. Each subset in a given partition represents a subproblem i, with relaxed solution x i and lower bound z i = c(x i ) on the cost of any solution in the subset. We can also visualize this process as a tree, as is shown in Figure 1. The root represents the original feasible region and each node represents a subproblem. Splitting the feasible region at a node by the addition of the inequalities is represented by the branching to the node s two children. If the original ILP has a bounded feasible region, this process cannot continue indefinitely, because eventually the inequalities at a node in the branching tree will lead to an integer solution to the corresponding LP, which is an optimal solution to the original ILP. The branching process can fail at a particular node for one of two reasons: (1) the LP solution can be integer; or (2) the LP problem can be infeasible. 2

3 0 x i x 0 i x i x 0 i x j x 1 j x k x 2 k x j x 1 j + 1 x k x 2 k Figure 1: Representation of solution space subdivision by a binary tree x 2 Problem 0 Problem 1 Problem 2 x 2 x 0 x 1 x 2 (a) x 1 (b) x 1 x 2 Problem 4 x 2 Problem 3 x 3 x 5 Problem 5 Problem 6 (c) x 1 (d) x 1 Figure 2: Stages in the solution of an ILP by branch and bound 3

4 A simple ILP is shown in Figure 2(a); the solution is x = (2, 1) and c(x ) = (x 1 +x 2 ) = 3. The initial relaxed problem has the solution x 0 = ( 3 2, 5 2 ) with cost c(x0 ) = 4. Figure 2(b) shows the two subproblems generated by choosing the noninteger component x 0 1 = 3 2 and introducing the constraints x 1 1 and x 1 2. If we continue branching from Problem 2 in Figure 2, we obtain the branching tree shown in Figure 3. Three leaves are reached in the right subtree; these leaves correspond to two feasible LP s and one LP with an integer solution x 5 = (2, 1) with cost z 5 = c(x 5 ) = 3. What we have z 0 = 4 0 x 1 1 x 1 2 z 1 = 2, 5 1 z 2 = 3, 5 2 killed by x 5 x 2 1 x 2 2 z 3 = 3, empty x 1 2 x 1 3 z 5 = integer: x 1 = 2 x 2 = 1 empty Figure 3: The binary tree leading to a solution to the problem described up to this point comprises the branching part of the branch and bound. If we continue the branching process until all the nodes are leaves of the tree and correspond either to integer solutions or infeasible LP s, then the leaf with the smallest cost must be the optimal solution to the original ILP. We come now to an important component of the branch and bound approach: Suppose at some point the best complete integer solution obtained so far has cost z m and that we are contemplating branching from a node at which the lower bound z k = c(x k ) is greater than or equal to z m. This means that any solution x that would be obtained as a descendent of x k would have cost c(x) z k z m and hence we need not proceed with a branching from x k. In such a case, we say that the node x k has been killed or fathomed, and refer to it as dead. The remaining nodes, from which branching is still possibly fruitful, are referred to as alive. For our example (Figures 2, 3), the node corresponding to Problem 1 has associated with it a lower bound of 2 1 2, which is greater than the solution cost of 3 associated with node 5. It is therefore killed by node 5, as shown. Since no live nodes remain, node 5 must represent the optimal solution. There are now still two important details in the algorithm that need to be specified: We must decide how to choose, at each branching step, which node to branch from; and we must decide how to choose which noninteger variable is to determine the added constraint. The first choice is referred to as the search strategy. The two most common strategies are depth first search and frontier search. In case of depth first search we first consider the 4

5 last generated nodes to branch from. It has to be specified in which order these last nodes are considered. This strategy has the advantage that the amount of storage needed is limited and furthermore that in an early stage feasible solutions are found. In case of frontier search we first consider the node with the lowest lower bound because this is the most promising one. This strategy looks more effective, but for larger problems the number of subproblems that have to be stored is too large. For the second choice-the variable to add the constraint- the best strategy is to find that constraint which leads to the largest increase in the lower bound after that constraint is added, and to add either that constraint or its alternative. The motivation is to find the branch from a given node that is most likely to get killed, and in this way keep the search tree shallow. There are no theorems to tell us the best strategy, and computational experience and intuition are the only guides to the design of fast algorithms of this type. If the feasible region is bounded, it is not difficult to see that the algorithm terminates. Indeed, then each variable can only be chosen a finite number of times for splitting. 7.3 Branch and Bound in a General Context The idea of branch and bound is applicable not only to a problem formulated as an ILP (or mixed ILP), but to almost any problem of a combinatorial nature. We next describe the method in a very general context. Two things are needed to develop the tree in the branch and bound algorithm for ILP: 1. Branching A set of solutions, which is represented by a node, can be partitioned into mutually exclusive sets. Each subset in the partition is represented by a child of the original node. 2. Lower Bounding An algorithm is available for calculating a lower bound on the cost of any solution in a given subset. No other properties of ILP were used. We may therefore formulate the method for any optimization problem in which (1) and (2) are available. Specification of the branching, the lower bounding and the search strategy leads to a branch and bound algorithm for this optimization problem. There is always some set of subproblems, or active nodes. This set is initialized by the original problem and the algorithm stops as soon as this set is empty. In each step of the algorithm one subproblem is selected. The choice is specified by the search strategy. There might be three reasons to eliminate this subproblem. 1. The subproblem is infeasible. 2. The lower bound is larger than or equal to the upper bound (cost of the best solution so far). 3. The subproblem can be solved. If the cost are less than the upper bound, the upper bound is replaced by these cost and the solution is stored. If the subproblem can not be eliminated, it is split in a finite number of smaller subproblems that are added to the set of subproblems. There are now many choices in how we implement a branch and bound algorithm for a given problem; we shall discuss some of them. First, there is a choice of the branching itself; there may be many schemes for partitioning the solution space for that matter. 5

6 Next, there is the lower bound calculation. One often has a choice here between bounds that are relatively tight but require relatively large computation time and bounds that are not so tight but can be computed fast. Third, there is the choice at each branching step of which node to branch from. The usual alternatives are least lower bound next (frontier search), last in first out (depth first search) or first in first out. Still another choice must be made at the start of the algorithm. It is often practical to generate an initial solution by some heuristic construction. This gives us an initial upper bound and may be very useful for killing nodes early in the algorithm. As usual, however, we must trade off the time required for the heuristic against possible benefit. It should be clear from now that the branch and bound idea is not one specific algorithm, but rather a very wide class. Its effective use is dependent on the design of a strategy for the particular problem at hand. In this course we discuss two different branch and bound algorithms one for ILP (which we just did), and one for the traveling salesman problem (see Section 10). 7.4 Knapsack with Branch and Bound The branch and bound algorithm for ILP has been illustrated in case of two variables, because then the LP-relaxations can easily be solved by hand. Another case where the LP-relaxation are easy to solve if for the knapsack problem. We illustrate this for the following example. Maximize 60x x x x x 5 + 9x 6 + 3x 7, subject to 3x 1 + 5x 2 + 4x 3 + x 4 + 4x 5 + 3x 6 + x 7 10, x i {0, 1}, i = 1,..., 7. This problem can be solved using the branch and bound algorithm with the following specifications: (A) Branching. Partition a subset F i of the feasible region into two disjunct subsets by adding an extra condition: x j = 0 and x j = 1 respectively. The choice of the variable x j follows from the calculation of the upper bound for the subset F i. This corresponds to solving the LP-relaxation of some smaller knapsack problem. The solution of this relaxation has at most one variable with a fractional value. This variable is chosen as splitting variable. (B) Upper Bound calculation. A subproblem F i of the knapsack problem consists of the original knapsack where some variables are forced to be 0 or 1. This is again the feasible region of a knapsack problem since if some x i = 0 this item can be removed and if x i = 1, the knapsack capacity can be reduced by w i. The upper bound calculation for F i is again solving the LP-relaxation of a knapsack problem. (C) Search Strategy. As search strategy we take the Last-in-First-out -rule. Of the last two generated subsets we first consider the one with x j = 1. 6

7 F 0 BG=140 OG=0 F (1, 1, 2 1, 0, 0, 0, 0) F F (1, 1, 0, 1, 1, 0, 0) 4 F F (1, 5 3, 1, 0, 0, 0, 0) F F 16 (1, 1, 0, 1, 0, 1 3, 0) F (1, 3, 0, 0, 1, 0, 0) 5 F (1, 0, 1, 1, 1, 0, 0) 2 F F ( 1, 1, 1, 0, 0, 0, 0) 3 F 3 0 (1, 1, 0, 1, 0, 0, 1) F 15 (1, 5 4, 0, 0, 0, 1, 0) F 14 (1, 0, 1, 1, 0, 2 3, 0) F 9 ( 2, 0, 1, 0, 1, 0, 0) 3 F 8 (0, 1, 1, 1, 0, 0, 0) no feasible solution (1, 0, 0, 1, 0, 1, 1) ( 3 2, 1, 0, 0, 0, 1, 0) (1, 0, 1, 1, 0, 0, 1) (1, 0, 1, 0, 0, 1, 0) Figure 4: The Branch and Bound tree of the knapsack problem under consideration Using these rules we get the following results. The items are already sorted by nonincreasing ratio c i w i. The boldface 0 s and 1 s denote the variables that are fixed to 0 or 1 during the branching. The corresponding branch and bound tree is shown in Figure 4. Name Solution LP-relaxation U.B. L.B. Next Action F 0 (1, 1, 1 2, 0, 0, 0, 0) branch w.r.t. x 3 F 1 (1, 3 5, 1, 0, 0, 0, 0) branch w.r.t. x 2 F 2 ( 1 3, 1, 1, 0, 0, 0, 0) branch w.r.t. x 1 F 3 no feasible solution 0 F 3 empty, backtrack F 4 (0, 1, 1, 1, 0, 0, 0) integral solution, backtrack F 5 (1, 0, 1, 1, 1 2, 0, 0) branch w.r.t. x 5 F 6 ( 2 3, 0, 1, 0, 1, 0, 0) u.b.<l.b., backtrack F 7 (1, 0, 1, 1, 0, 2 3, 0) branch w.r.t. x 6 F 8 (1, 0, 1, 0, 0, 1, 0) integral solution and u.b.<l.b., backtrack F 9 (1, 0, 1, 1, 0, 0, 1) integral solution, backtrack F 10 (1, 1, 0, 1, 1 4, 0, 0) branch w.r.t. x 5 F 11 (1, 3 5, 0, 0, 1, 0, 0) u.b.<l.b., backtrack F 12 (1, 1, 0, 1, 0, 1 3, 0) branch w.r.t. x 6 F 13 (1, 4 5, 0, 0, 0, 1, 0) branch w.r.t. x 2 F 14 ( 2 3, 1, 0, 0, 0, 1, 0) u.b.<l.b., backtrack F 15 (1, 0, 0, 1, 0, 1, 1) integral solution and u.b.<l.b., backtrack F 16 (1, 1, 0, 1, 0, 0, 1) integral solution, backtrack, finished 7

8 The optimal solution has value 133 and is given by: x 1 = 1, x 2 = 1, x 3 = 0, x 4 = 1, x 5 = 0, x 6 = 0, x 7 = 1 The number of considered subproblems can be 2 n and solving the LP-relaxation of a subproblem takes O(n). So the complexity of the branch and bound algorithm for knapsack is O(n2 n ). 7.5 Dynamic Programming Dynamic programming (DP) is, like branch and bound, an enumerative method to solve combinatorial optimization problems. The method can be characterized by the fact that it uses some recurrence relation to solve the problem. This recurrence relation relates the optimal solution of some problem to optimal solutions of smaller problems of the same type. For these smaller problems the same recurrence relation holds etcetera. Like the tree for the branch and bound method, one can define a directed graph to represent the dynamic programming method. The vertices of the graph correspond to the subproblems and there is an arc from subproblem Π 1 to Π 2 if Π 1 appears in the recurrence relation for Π 2. The original problem is then the endpoint of the graph and some trivial subproblem(s) is(are) the starting point(s) of the graph. We will call this graph the DP-graph. The optimal solution of some subproblem can be determined using the recurrence relation if the optimal solutions of its predecessors are known. Since the graph is acyclic, one can solve all subproblems one-by-one starting with the trivial problems and ending with the original problem. In some cases solving a problem reduces to finding the shortest or longest path from the starting point to the end point in the DP-graph. This is for instance the case if the optimum OP T (Π) of some subproblem Π is equal to the maximum or minimum of {OP T (Π 1 ) + c(π 1, Π),..., OP T (Π k ) + c(π k, Π)} where Π 1,..., Π k are the predecessors of problem Π and c(.,.) are costs that can be computed easily for some problem instance. De complexity of some dynamic programming algorithm depends on the number of subproblems that have to be considered and the amount of work per subproblem. Often the total number of arcs in the DP-graph is a good measure. In this course we have seen examples of dynamic programming algorithms already: The algorithms of Bellman-Ford and Floyd. We will now discuss a dynamic programming algorithm for the knapsack problem. Let 1 m n en 0 W W. Define f m (W ) as the optimal solution of the knapsack problem with items 1, 2,..., m en knapsack capacity W. Then clearly f 0 (W ) = 0 W = 0, 1,..., W { f 1 (W 0 W ) = = 0, 1,..., w 1 1 W = w 1, w 1 + 1,..., W c 1 We have the following recurrence relation for f m (W ): f m (W ) = { fm 1 (W ) W = 0, 1,..., w m 1 max{f m 1 (W ), f m 1 (W w m ) + c m } W = w m, w m + 1,... W f n (W ) is the optimum solution of the original problem. Dynamic programming now consists of calculating the values f m (W ). This is done from m = 0 to m = n and for some m from W = W downto W = 0. This leads to the following algorithm: 8

9 At the start of the m th step of the algorithm the variables have the following values: P [W ] = f m 1 (W ) for W = 0, 1,..., W X[W ] = S {1, 2,..., m 1} for W = 0, 1,..., W where S is the subset of the set of items {1, 2,..., m 1} that leads to the maximum value P [W ] for capacity W. Input: n, W, (c i ), (w i ); Output: z; begin for W = 0 to W do begin P [W ] := 0; X[W ] := ; end for m := 1 to n do for W = W downto w m do if P [W ] < P [W w m ] + c m then begin P [W ] := P [W w m ] + c m ; X[W ] := X[W w m ] {m} end; z := P [W ]; end. This algorithm has time complexity O(nW ) since the number of considered subproblems is nw and the amount of work per subproblem does not grow when the problem becomes larger. One only has to choose the best of two alternatives. Notice that the size of the input is O(n log W + n log C), where C is equal to the largest profit of the items. For our example we find the following values for f m (W ): W m So, again the optimal solution has value 133 and is realized by choosing the items 1, 2, 4 and 7. 9

10 Exercises 1. Solve the following ILP problem by use of Branch and Bound with frontier search and draw the search tree. Minimize 11x + 10y subject to 13x + 10y 65 8x + 17y 68 x 0, y 0 x, y integer. 2. Give an ILP formulation of the knapsack problem given below with knapsack capacity 17. Solve the ILP problem with Branch and Bound with frontier search. Draw the search tree. object value weight Solve the following knapsack problem with knapsack capacity 7, by use of dynamic programming. object value weight Show that if for an instance I of the knapsack problem not all weights are at most the capacity W, then LP (I)/OP T (I) can be arbitrarily large. 5. Consider a knapsack problem with capacity W, items 1,..., n, weights w 1,..., w n, and values (costs) c 1,..., c n. Define C = n j=1 c j. For an integer m n, define g m (C ) as the minimal total weight of a subset S {1, 2,..., m} for which c i = C (take g m (C ) = if such a set doesn t exist). i S a. What is g 0 (C ) for 0 C C? b. How can OP T (I) be derived from the values of g n (C ) (0 C C)? c. Derive a recursion formula for g m+1 (C ) in terms of the g m (C ) and g m (C c m+1 ) (provided C c m+1 ). d. Show that this gives a dynamic programming algorithm with complexity O(nC). e. Use this algorithm to solve the knapsack problem of Exercise 3. 10

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. !-approximation algorithm.

Approximation Algorithms Chapter Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of

### Chapter 13: Binary and Mixed-Integer Programming

Chapter 3: Binary and Mixed-Integer Programming The general branch and bound approach described in the previous chapter can be customized for special situations. This chapter addresses two special situations:

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### Near Optimal Solutions

Near Optimal Solutions Many important optimization problems are lacking efficient solutions. NP-Complete problems unlikely to have polynomial time solutions. Good heuristics important for such problems.

### Scheduling Home Health Care with Separating Benders Cuts in Decision Diagrams

Scheduling Home Health Care with Separating Benders Cuts in Decision Diagrams André Ciré University of Toronto John Hooker Carnegie Mellon University INFORMS 2014 Home Health Care Home health care delivery

### Dynamic programming. Doctoral course Optimization on graphs - Lecture 4.1. Giovanni Righini. January 17 th, 2013

Dynamic programming Doctoral course Optimization on graphs - Lecture.1 Giovanni Righini January 1 th, 201 Implicit enumeration Combinatorial optimization problems are in general NP-hard and we usually

### INTEGER PROGRAMMING. Integer Programming. Prototype example. BIP model. BIP models

Integer Programming INTEGER PROGRAMMING In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is

### Analysis of Algorithms I: Binary Search Trees

Analysis of Algorithms I: Binary Search Trees Xi Chen Columbia University Hash table: A data structure that maintains a subset of keys from a universe set U = {0, 1,..., p 1} and supports all three dictionary

### A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem

A Branch and Bound Algorithm for Solving the Binary Bi-level Linear Programming Problem John Karlof and Peter Hocking Mathematics and Statistics Department University of North Carolina Wilmington Wilmington,

### A Constraint Programming based Column Generation Approach to Nurse Rostering Problems

Abstract A Constraint Programming based Column Generation Approach to Nurse Rostering Problems Fang He and Rong Qu The Automated Scheduling, Optimisation and Planning (ASAP) Group School of Computer Science,

### Scheduling Single Machine Scheduling. Tim Nieberg

Scheduling Single Machine Scheduling Tim Nieberg Single machine models Observation: for non-preemptive problems and regular objectives, a sequence in which the jobs are processed is sufficient to describe

### Discuss the size of the instance for the minimum spanning tree problem.

3.1 Algorithm complexity The algorithms A, B are given. The former has complexity O(n 2 ), the latter O(2 n ), where n is the size of the instance. Let n A 0 be the size of the largest instance that can

### Branch-and-Price Approach to the Vehicle Routing Problem with Time Windows

TECHNISCHE UNIVERSITEIT EINDHOVEN Branch-and-Price Approach to the Vehicle Routing Problem with Time Windows Lloyd A. Fasting May 2014 Supervisors: dr. M. Firat dr.ir. M.A.A. Boon J. van Twist MSc. Contents

### The Problem of Scheduling Technicians and Interventions in a Telecommunications Company

The Problem of Scheduling Technicians and Interventions in a Telecommunications Company Sérgio Garcia Panzo Dongala November 2008 Abstract In 2007 the challenge organized by the French Society of Operational

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### Single machine models: Maximum Lateness -12- Approximation ratio for EDD for problem 1 r j,d j < 0 L max. structure of a schedule Q...

Lecture 4 Scheduling 1 Single machine models: Maximum Lateness -12- Approximation ratio for EDD for problem 1 r j,d j < 0 L max structure of a schedule 0 Q 1100 11 00 11 000 111 0 0 1 1 00 11 00 11 00

### Algorithm Design and Analysis

Algorithm Design and Analysis LECTURE 27 Approximation Algorithms Load Balancing Weighted Vertex Cover Reminder: Fill out SRTEs online Don t forget to click submit Sofya Raskhodnikova 12/6/2011 S. Raskhodnikova;

### 11. APPROXIMATION ALGORITHMS

11. APPROXIMATION ALGORITHMS load balancing center selection pricing method: vertex cover LP rounding: vertex cover generalized load balancing knapsack problem Lecture slides by Kevin Wayne Copyright 2005

### Chapter 15: Dynamic Programming

Chapter 15: Dynamic Programming Dynamic programming is a general approach to making a sequence of interrelated decisions in an optimum way. While we can describe the general characteristics, the details

### Classification - Examples

Lecture 2 Scheduling 1 Classification - Examples 1 r j C max given: n jobs with processing times p 1,...,p n and release dates r 1,...,r n jobs have to be scheduled without preemption on one machine taking

### 1.204 Lecture 10. Greedy algorithms: Job scheduling. Greedy method

1.204 Lecture 10 Greedy algorithms: Knapsack (capital budgeting) Job scheduling Greedy method Local improvement method Does not look at problem globally Takes best immediate step to find a solution Useful

### Analysis of Algorithms I: Optimal Binary Search Trees

Analysis of Algorithms I: Optimal Binary Search Trees Xi Chen Columbia University Given a set of n keys K = {k 1,..., k n } in sorted order: k 1 < k 2 < < k n we wish to build an optimal binary search

### Dynamic programming formulation

1.24 Lecture 14 Dynamic programming: Job scheduling Dynamic programming formulation To formulate a problem as a dynamic program: Sort by a criterion that will allow infeasible combinations to be eli minated

### IE 680 Special Topics in Production Systems: Networks, Routing and Logistics*

IE 680 Special Topics in Production Systems: Networks, Routing and Logistics* Rakesh Nagi Department of Industrial Engineering University at Buffalo (SUNY) *Lecture notes from Network Flows by Ahuja, Magnanti

### Routing in Line Planning for Public Transport

Konrad-Zuse-Zentrum für Informationstechnik Berlin Takustraße 7 D-14195 Berlin-Dahlem Germany MARC E. PFETSCH RALF BORNDÖRFER Routing in Line Planning for Public Transport Supported by the DFG Research

### Computer Algorithms. NP-Complete Problems. CISC 4080 Yanjun Li

Computer Algorithms NP-Complete Problems NP-completeness The quest for efficient algorithms is about finding clever ways to bypass the process of exhaustive search, using clues from the input in order

### Modeling and Solving the Capacitated Vehicle Routing Problem on Trees

in The Vehicle Routing Problem: Latest Advances and New Challenges Modeling and Solving the Capacitated Vehicle Routing Problem on Trees Bala Chandran 1 and S. Raghavan 2 1 Department of Industrial Engineering

### THE SCHEDULING OF MAINTENANCE SERVICE

THE SCHEDULING OF MAINTENANCE SERVICE Shoshana Anily Celia A. Glass Refael Hassin Abstract We study a discrete problem of scheduling activities of several types under the constraint that at most a single

Load Balancing Backtracking, branch & bound and alpha-beta pruning: how to assign work to idle processes without much communication? Additionally for alpha-beta pruning: implementing the young-brothers-wait

### Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal

Approximating Minimum Bounded Degree Spanning Trees to within One of Optimal ABSTACT Mohit Singh Tepper School of Business Carnegie Mellon University Pittsburgh, PA USA mohits@andrew.cmu.edu In the MINIMUM

### Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design

Bicolored Shortest Paths in Graphs with Applications to Network Overlay Design Hongsik Choi and Hyeong-Ah Choi Department of Electrical Engineering and Computer Science George Washington University Washington,

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Applied Algorithm Design Lecture 5

Applied Algorithm Design Lecture 5 Pietro Michiardi Eurecom Pietro Michiardi (Eurecom) Applied Algorithm Design Lecture 5 1 / 86 Approximation Algorithms Pietro Michiardi (Eurecom) Applied Algorithm Design

### A single minimal complement for the c.e. degrees

A single minimal complement for the c.e. degrees Andrew Lewis Leeds University, April 2002 Abstract We show that there exists a single minimal (Turing) degree b < 0 s.t. for all c.e. degrees 0 < a < 0,

### Classification/Decision Trees (II)

Classification/Decision Trees (II) Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Right Sized Trees Let the expected misclassification rate of a tree T be R (T ).

### Dantzig-Wolfe bound and Dantzig-Wolfe cookbook

Dantzig-Wolfe bound and Dantzig-Wolfe cookbook thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline LP strength of the Dantzig-Wolfe The exercise from last week... The Dantzig-Wolfe

### Discrete Optimization

Discrete Optimization [Chen, Batson, Dang: Applied integer Programming] Chapter 3 and 4.1-4.3 by Johan Högdahl and Victoria Svedberg Seminar 2, 2015-03-31 Todays presentation Chapter 3 Transforms using

### Integrating Benders decomposition within Constraint Programming

Integrating Benders decomposition within Constraint Programming Hadrien Cambazard, Narendra Jussien email: {hcambaza,jussien}@emn.fr École des Mines de Nantes, LINA CNRS FRE 2729 4 rue Alfred Kastler BP

### Modeling and Performance Evaluation of Computer Systems Security Operation 1

Modeling and Performance Evaluation of Computer Systems Security Operation 1 D. Guster 2 St.Cloud State University 3 N.K. Krivulin 4 St.Petersburg State University 5 Abstract A model of computer system

### How to speed-up hard problem resolution using GLPK?

How to speed-up hard problem resolution using GLPK? Onfroy B. & Cohen N. September 27, 2010 Contents 1 Introduction 2 1.1 What is GLPK?.......................................... 2 1.2 GLPK, the best one?.......................................

### Fairness in Routing and Load Balancing

Fairness in Routing and Load Balancing Jon Kleinberg Yuval Rabani Éva Tardos Abstract We consider the issue of network routing subject to explicit fairness conditions. The optimization of fairness criteria

Logic Cuts Generation in a Branch and Cut Framework for Location Problems Mara A. Osorio Lama School of Computer Science Autonomous University of Puebla, Puebla 72560 Mexico Rosalba Mujica Garca Abstract

### A Column-Generation and Branch-and-Cut Approach to the Bandwidth-Packing Problem

[J. Res. Natl. Inst. Stand. Technol. 111, 161-185 (2006)] A Column-Generation and Branch-and-Cut Approach to the Bandwidth-Packing Problem Volume 111 Number 2 March-April 2006 Christine Villa and Karla

### IEEE/ACM TRANSACTIONS ON NETWORKING, VOL. 19, NO. 3, JUNE 2011 709

IEEE/ACM TRANSACTIONS ON NETWORKING, VOL 19, NO 3, JUNE 2011 709 Improved Bounds on the Throughput Efficiency of Greedy Maximal Scheduling in Wireless Networks Mathieu Leconte, Jian Ni, Member, IEEE, R

### Outline. NP-completeness. When is a problem easy? When is a problem hard? Today. Euler Circuits

Outline NP-completeness Examples of Easy vs. Hard problems Euler circuit vs. Hamiltonian circuit Shortest Path vs. Longest Path 2-pairs sum vs. general Subset Sum Reducing one problem to another Clique

### Nan Kong, Andrew J. Schaefer. Department of Industrial Engineering, Univeristy of Pittsburgh, PA 15261, USA

A Factor 1 2 Approximation Algorithm for Two-Stage Stochastic Matching Problems Nan Kong, Andrew J. Schaefer Department of Industrial Engineering, Univeristy of Pittsburgh, PA 15261, USA Abstract We introduce

### ASSIGNMENT 4 PREDICTIVE MODELING AND GAINS CHARTS

DATABASE MARKETING Fall 2015, max 24 credits Dead line 15.10. ASSIGNMENT 4 PREDICTIVE MODELING AND GAINS CHARTS PART A Gains chart with excel Prepare a gains chart from the data in \\work\courses\e\27\e20100\ass4b.xls.

### Treemaps with bounded aspect ratio

technische universiteit eindhoven Department of Mathematics and Computer Science Master s Thesis Treemaps with bounded aspect ratio by Vincent van der Weele Supervisor dr. B. Speckmann Eindhoven, July

### A Linear Programming Based Method for Job Shop Scheduling

A Linear Programming Based Method for Job Shop Scheduling Kerem Bülbül Sabancı University, Manufacturing Systems and Industrial Engineering, Orhanlı-Tuzla, 34956 Istanbul, Turkey bulbul@sabanciuniv.edu

### Transportation Polytopes: a Twenty year Update

Transportation Polytopes: a Twenty year Update Jesús Antonio De Loera University of California, Davis Based on various papers joint with R. Hemmecke, E.Kim, F. Liu, U. Rothblum, F. Santos, S. Onn, R. Yoshida,

### Scheduling of Mixed Batch-Continuous Production Lines

Université Catholique de Louvain Faculté des Sciences Appliquées Scheduling of Mixed Batch-Continuous Production Lines Thèse présentée en vue de l obtention du grade de Docteur en Sciences Appliquées par

### Complexity Theory. IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar

Complexity Theory IE 661: Scheduling Theory Fall 2003 Satyaki Ghosh Dastidar Outline Goals Computation of Problems Concepts and Definitions Complexity Classes and Problems Polynomial Time Reductions Examples

### Chapter 3 INTEGER PROGRAMMING 3.1 INTRODUCTION. Robert Bosch. Michael Trick

Chapter 3 INTEGER PROGRAMMING Robert Bosch Oberlin College Oberlin OH, USA Michael Trick Carnegie Mellon University Pittsburgh PA, USA 3.1 INTRODUCTION Over the last 20 years, the combination of faster

### Scheduling a sequence of tasks with general completion costs

Scheduling a sequence of tasks with general completion costs Francis Sourd CNRS-LIP6 4, place Jussieu 75252 Paris Cedex 05, France Francis.Sourd@lip6.fr Abstract Scheduling a sequence of tasks in the acceptation

### CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma

CMSC 858T: Randomized Algorithms Spring 2003 Handout 8: The Local Lemma Please Note: The references at the end are given for extra reading if you are interested in exploring these ideas further. You are

### Single-Link Failure Detection in All-Optical Networks Using Monitoring Cycles and Paths

Single-Link Failure Detection in All-Optical Networks Using Monitoring Cycles and Paths Satyajeet S. Ahuja, Srinivasan Ramasubramanian, and Marwan Krunz Department of ECE, University of Arizona, Tucson,

### Chapter 10: Network Flow Programming

Chapter 10: Network Flow Programming Linear programming, that amazingly useful technique, is about to resurface: many network problems are actually just special forms of linear programs! This includes,

### Cloud Branching. Timo Berthold. joint work with Domenico Salvagnin (Università degli Studi di Padova)

Cloud Branching Timo Berthold Zuse Institute Berlin joint work with Domenico Salvagnin (Università degli Studi di Padova) DFG Research Center MATHEON Mathematics for key technologies 21/May/13, CPAIOR

### JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS. Received December May 12, 2003; revised February 5, 2004

Scientiae Mathematicae Japonicae Online, Vol. 10, (2004), 431 437 431 JUST-IN-TIME SCHEDULING WITH PERIODIC TIME SLOTS Ondřej Čepeka and Shao Chin Sung b Received December May 12, 2003; revised February

### Branch, Cut, and Price: Sequential and Parallel

Branch, Cut, and Price: Sequential and Parallel T.K. Ralphs 1, L. Ladányi 2, and L.E. Trotter, Jr. 3 1 Department of Industrial and Manufacturing Systems Engineering, Lehigh University, Bethlehem, PA 18017,

### Integer Programming. subject to: (i = 1, 2,..., m),

Integer Programming 9 The linear-programming models that have been discussed thus far all have been continuous, in the sense that decision variables are allowed to be fractional. Often this is a realistic

### THE PROBLEM WORMS (1) WORMS (2) THE PROBLEM OF WORM PROPAGATION/PREVENTION THE MINIMUM VERTEX COVER PROBLEM

1 THE PROBLEM OF WORM PROPAGATION/PREVENTION I.E. THE MINIMUM VERTEX COVER PROBLEM Prof. Tiziana Calamoneri Network Algorithms A.y. 2014/15 2 THE PROBLEM WORMS (1)! A computer worm is a standalone malware

### Math 181 Handout 16. Rich Schwartz. March 9, 2010

Math 8 Handout 6 Rich Schwartz March 9, 200 The purpose of this handout is to describe continued fractions and their connection to hyperbolic geometry. The Gauss Map Given any x (0, ) we define γ(x) =

### Reconnect 04 Solving Integer Programs with Branch and Bound (and Branch and Cut)

Sandia is a ultiprogra laboratory operated by Sandia Corporation, a Lockheed Martin Copany, Reconnect 04 Solving Integer Progras with Branch and Bound (and Branch and Cut) Cynthia Phillips (Sandia National

### Solutions to Homework 6

Solutions to Homework 6 Debasish Das EECS Department, Northwestern University ddas@northwestern.edu 1 Problem 5.24 We want to find light spanning trees with certain special properties. Given is one example

### Noncommercial Software for Mixed-Integer Linear Programming

Noncommercial Software for Mixed-Integer Linear Programming J. T. Linderoth T. K. Ralphs December, 2004. Revised: January, 2005. Abstract We present an overview of noncommercial software tools for the

### Multi-layer MPLS Network Design: the Impact of Statistical Multiplexing

Multi-layer MPLS Network Design: the Impact of Statistical Multiplexing Pietro Belotti, Antonio Capone, Giuliana Carello, Federico Malucelli Tepper School of Business, Carnegie Mellon University, Pittsburgh

### IBM Research Report. A Pricing Problem under Monge Property

RC23823 (W0512-010) December 2, 2005 Mathematics IBM Research Report A Pricing Problem under Monge Property Oktay Günlük IBM Research Division Thomas J. Watson Research Center P.O. Box 218 Yorktown Heights,

### Offline sorting buffers on Line

Offline sorting buffers on Line Rohit Khandekar 1 and Vinayaka Pandit 2 1 University of Waterloo, ON, Canada. email: rkhandekar@gmail.com 2 IBM India Research Lab, New Delhi. email: pvinayak@in.ibm.com

### 3.3. Solving Polynomial Equations. Introduction. Prerequisites. Learning Outcomes

Solving Polynomial Equations 3.3 Introduction Linear and quadratic equations, dealt within Sections 3.1 and 3.2, are members of a class of equations, called polynomial equations. These have the general

### The Goldberg Rao Algorithm for the Maximum Flow Problem

The Goldberg Rao Algorithm for the Maximum Flow Problem COS 528 class notes October 18, 2006 Scribe: Dávid Papp Main idea: use of the blocking flow paradigm to achieve essentially O(min{m 2/3, n 1/2 }

### Graphs without proper subgraphs of minimum degree 3 and short cycles

Graphs without proper subgraphs of minimum degree 3 and short cycles Lothar Narins, Alexey Pokrovskiy, Tibor Szabó Department of Mathematics, Freie Universität, Berlin, Germany. August 22, 2014 Abstract

### Why? A central concept in Computer Science. Algorithms are ubiquitous.

Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

### Diversity Coloring for Distributed Data Storage in Networks 1

Diversity Coloring for Distributed Data Storage in Networks 1 Anxiao (Andrew) Jiang and Jehoshua Bruck California Institute of Technology Pasadena, CA 9115, U.S.A. {jax, bruck}@paradise.caltech.edu Abstract

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### Single machine parallel batch scheduling with unbounded capacity

Workshop on Combinatorics and Graph Theory 21th, April, 2006 Nankai University Single machine parallel batch scheduling with unbounded capacity Yuan Jinjiang Department of mathematics, Zhengzhou University

### Satisfiability Checking

Satisfiability Checking SAT-Solving Prof. Dr. Erika Ábrahám Theory of Hybrid Systems Informatik 2 WS 10/11 Prof. Dr. Erika Ábrahám - Satisfiability Checking 1 / 40 A basic SAT algorithm Assume the CNF

### A binary search tree is a binary tree with a special property called the BST-property, which is given as follows:

Chapter 12: Binary Search Trees A binary search tree is a binary tree with a special property called the BST-property, which is given as follows: For all nodes x and y, if y belongs to the left subtree

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### Mining Social Network Graphs

Mining Social Network Graphs Debapriyo Majumdar Data Mining Fall 2014 Indian Statistical Institute Kolkata November 13, 17, 2014 Social Network No introduc+on required Really? We s7ll need to understand

### CSC148 Lecture 8. Algorithm Analysis Binary Search Sorting

CSC148 Lecture 8 Algorithm Analysis Binary Search Sorting Algorithm Analysis Recall definition of Big Oh: We say a function f(n) is O(g(n)) if there exists positive constants c,b such that f(n)

### npsolver A SAT Based Solver for Optimization Problems

npsolver A SAT Based Solver for Optimization Problems Norbert Manthey and Peter Steinke Knowledge Representation and Reasoning Group Technische Universität Dresden, 01062 Dresden, Germany peter@janeway.inf.tu-dresden.de

### Key words. multi-objective optimization, approximate Pareto set, bi-objective shortest path

SMALL APPROXIMATE PARETO SETS FOR BI OBJECTIVE SHORTEST PATHS AND OTHER PROBLEMS ILIAS DIAKONIKOLAS AND MIHALIS YANNAKAKIS Abstract. We investigate the problem of computing a minimum set of solutions that

### Lecture 4: BK inequality 27th August and 6th September, 2007

CSL866: Percolation and Random Graphs IIT Delhi Amitabha Bagchi Scribe: Arindam Pal Lecture 4: BK inequality 27th August and 6th September, 2007 4. Preliminaries The FKG inequality allows us to lower bound

### 5. Factoring by the QF method

5. Factoring by the QF method 5.0 Preliminaries 5.1 The QF view of factorability 5.2 Illustration of the QF view of factorability 5.3 The QF approach to factorization 5.4 Alternative factorization by the

### Bounded Treewidth in Knowledge Representation and Reasoning 1

Bounded Treewidth in Knowledge Representation and Reasoning 1 Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien Luminy, October 2010 1 Joint work with G.

### Determination of the normalization level of database schemas through equivalence classes of attributes

Computer Science Journal of Moldova, vol.17, no.2(50), 2009 Determination of the normalization level of database schemas through equivalence classes of attributes Cotelea Vitalie Abstract In this paper,

### Network File Storage with Graceful Performance Degradation

Network File Storage with Graceful Performance Degradation ANXIAO (ANDREW) JIANG California Institute of Technology and JEHOSHUA BRUCK California Institute of Technology A file storage scheme is proposed

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### SMT 2014 Algebra Test Solutions February 15, 2014

1. Alice and Bob are painting a house. If Alice and Bob do not take any breaks, they will finish painting the house in 20 hours. If, however, Bob stops painting once the house is half-finished, then the

### Completion Time Scheduling and the WSRPT Algorithm

Completion Time Scheduling and the WSRPT Algorithm Bo Xiong, Christine Chung Department of Computer Science, Connecticut College, New London, CT {bxiong,cchung}@conncoll.edu Abstract. We consider the online

### Het inplannen van besteld ambulancevervoer (Engelse titel: Scheduling elected ambulance transportation)

Technische Universiteit Delft Faculteit Elektrotechniek, Wiskunde en Informatica Delft Institute of Applied Mathematics Het inplannen van besteld ambulancevervoer (Engelse titel: Scheduling elected ambulance

### The positive minimum degree game on sparse graphs

The positive minimum degree game on sparse graphs József Balogh Department of Mathematical Sciences University of Illinois, USA jobal@math.uiuc.edu András Pluhár Department of Computer Science University

### root node level: internal node edge leaf node CS@VT Data Structures & Algorithms 2000-2009 McQuain

inary Trees 1 A binary tree is either empty, or it consists of a node called the root together with two binary trees called the left subtree and the right subtree of the root, which are disjoint from each

### A Branch-Cut-and-Price Approach to the Bus Evacuation Problem with Integrated Collection Point and Shelter Decisions

A Branch-Cut-and-Price Approach to the Bus Evacuation Problem with Integrated Collection Point and Shelter Decisions Marc Goerigk, Bob Grün, and Philipp Heßler Fachbereich Mathematik, Technische Universität