# Linear Programming is the branch of applied mathematics that deals with solving

Save this PDF as:

Size: px
Start display at page:

Download "Linear Programming is the branch of applied mathematics that deals with solving"

## Transcription

1 Chapter 2 LINEAR PROGRAMMING PROBLEMS 2.1 Introduction Linear Programming is the branch of applied mathematics that deals with solving optimization problems of a particular functional form. A linear programming problem consists of a linear objective function (of decision variables) which is to be minimized or maximized, subject to a certain set of linear constraints on decision variables. The constraints are usually in the form of linear inequalities of decision variables used in the objective function. Linear programming is a relatively recent mathematical discipline. The Russian mathematician Leonid V. Kantorovich formulated the first problem in linear programming in This was what is now known as the transportation problem. The English economist George Stigler (1945) described a problem of determining an optimal diet as a linear programming problem. The simplex algorithm was developed by the American mathematician George B. Dantzig in 1947, and the theory of duality was developed by John von Neumann the same year i.e. in It was also Dantzig who coined the name Linear Programming to this class of problems. In 1972, Klee and Minty showed that the simplex method has exponential computational complex- 15

2 ity. In 1979, Khachiyan showed that linear programming problems can be solved by polynomial time algorithms. Karmarkar, in 1984, described his interior-point algorithm and claimed that it is a polynomial-time algorithm and works better than Khachiyan s algorithm. What is common among all the algorithms mentioned above is that they are all iterative and therefore sequential. That is, every subsequent solution depends on a preceding solution. On one side, this introduces uniqueness in each of these algorithms. On the other hand, however, the same feature makes these algorithms rather restrictive. This chapter aims at developing a method that is neither sequential not restricted to the surface of the polytope representing the set of feasible solutions of the linear programming problem. This development is motivated by the need for a speedy algorithm, since each of the classical techniques often requires a vast amount of calculations [15]. 16

3 2.2 Linear Programming Problems in Canonical Form Linear programming problems can be expressed in the canonical form. The canonical form of a linear programming problem is Maximize C X (2.1) subject to AX b, (2.2) and X 0. (2.3) Where C and X R n, b R m and A R m n. Here (2.1) is called the objective function, (2.2) is a system of linear inequation constraints and (2.3) is a non-negativity constraint Features of linear programming problems In this section we state some standard definitions and some of the important characteristics of a solution to a linear programming problem formulated in the canonical form. Feasible solution set. A set of values of the decision variables x 1, x 2,, x n which satisfy all the constraints and also the non-negativity condition is called the feasible solution set of the linear programming problem. Feasible solution. Each element of the feasible solution set is called a feasible solution. A feasible solution is a solution which satisfies all the constraints and also the non-negativity conditions of the linear programming problem. 17

4 Optimal solution. An optimal solution X is a feasible solution subject to C X = max {C X : AX b, X 0}. (2.4) A (LP ) is feasible if there exists at least one feasible solution. Otherwise, it is said to be infeasible. Convex set. A set S is said to be convex set if, given any two points in the set, the line joining those two points is completely contained in the set. Mathematically, S is a convex set if for any two vectors X (1) and X (2) in S, the vector X = λx (1) + (1 λ)x (2) is also in S for any real number λ [0, 1] [55]. Figure 1 and 2 represent convex set, whereas Figure 3 is not a convex set. Figure 1 Figure 2 Figure 3 Extreme point. An extreme point of the convex set is a feasible point that can not lie on a line segment joining any two distinct feasible points in the set. Actually, extreme points are the same as corner points [61]. The set of all feasible solutions of LP P is a convex set. The optimal solution of LP P can always be associated with a feasible extreme point of feasible solution set. 18

5 Linearity of the objective function results in convexity of the objective function. Linear Programming Problems are solved by some methods like the graphical method, the systematic trial and error method (enumeration method) and the simplex method. The graphical method is not applicable in more than two dimensional space. The enumerate method is a native approach to solve a linear programming (which has an optimal solution) would be to generate all possible extreme points and determine which one of them gives the best objective function value. But, the simplex method, Which is a well-known and widely used method for solving linear programming problem, does this in a more efficient manner by examining only a fraction of the total number of extreme points of feasible solution set Simplex method The general steps of the simplex method are as follows. 1. Start with an initial extreme point of the feasible solution set. 2. Improve the initial solution if possible by finding another extreme point of feasible solution set with a better objective function value. At this step the simplex method implicitly eliminates from consideration all those extreme points of the feasible solution set whose objective function values are worse than the present one. This makes the procedure more efficient than the enumeration method. 3. Continue to find better extreme point of feasible solution set, improving the objective function value at every step. 19

6 4. When a particular extreme point of feasible solution set cannot be improved further, it becomes an optimal solution and the simplex method terminates Simplex method in tableau form The various steps of the simplex method can be carried out in a more compact manner by using a tableau form to represent the constraints and the objective function. In addition by developing some simple formulas, the various calculations can be made mechanical [55]. In this section the simplex method is presented by illustrating the steps and using a tableau form for an example in canonical form. We consider the following linear programming problem. maximize z = 6x 1 + 8x 2 subject to 5x x 2 60, 4x 1 + 4x 2 40, x 1, x

7 The standard form of the above LP problem is shown below. maximize z = 6x 1 + 8x 2 + 0s 1 + 0s 2 subject to 5x x 2 + s 1 60, 4x 1 + 4x 2 + s 2 40, x 1, x 2, s 1, s 2 0, where s 1 and s 2 are slack variables, which are introduced to balance the constraints. Basic variable. A variable is said to be basic variable if it has unit coefficient in one of the constraints and zero coefficient in the remaining constraints [53]. If all the constraints are, then the standard form is to be treated as the canonical form. The canonical form is generally used to prepare the initial simplex tableau [53]. The initial simplex table of the above problem is shown in Table 2.1. Table 2.1: Initial Simplex tableau c j CB i Basic x 1 x 2 s 1 s 2 Solution Ratio Variable 0 s /10 = 6 0 s /4=10 z j c j z j key column key row Here, c j is the coefficient of the j th term of the objective function and CB i is the coefficient of the i th basic variable. The value at the intersection of the key column and the key row is called key element. The value of z j is computed using the following formula. z j = 2 (CB i )(a ij ) i=1 21

8 Where a ij is the coefficient for the i th row and j th column of the table. c j z j is the relative contribution (profit). In this term, c j is the objective function coefficient for the j th variable. The value of z j against the solution column is the value of the objective function and in this iteration, it is zero. Optimality condition: For maximization problem, if all c j z j are less than or equal to zero, then optimality is reached; otherwise select the variable with the maximum c j z j value as the entering variable (For minimization problem, if all c j z j are greater than or equal to zero, the optimality is reached; otherwise select the value with the most negative value as the entering variable). In Table 2.1, all the values for c j z j are either equal to or greater than zero. Hence, the solution can be improved further. c j z j is the maximum for the variable x 2. So, x 2 enter the basis. this is known as intering variable, and the corresponding column is called key column. Feasibility condition: To maintain the feasibility of the solution in each iteration, the following steps need to be followed: 1. In each row, find the ratio between the solution column value and the value in the key column. 2. Then, select the variable from the present set of basic variables with respect to the minimum ratio (break tie randomly). Such variable is the leaving variable and the corresponding row is called the key row. The value at the intersection of the key row and the key column is called key element or pivot element. 22

9 In Table 2.1, the leaving variable is s 1 and the row 1 is the key row. Key element is 10. The next iteration is shown in Table 2.2. In this table, the basic variable s 1 of the previous table is replaced by x 2. The formula to compute the new values of table 2.2 is as shown below: Here Table 2.2: Iteration 1 c j CB i Basic x 1 x 2 s 1 s 2 Solution Ratio Variable 8 x 2 1/2 1 1/ /(1/2)=12 0 s / /2=8 z j 4 8 4/ c j z j 2 0-4/5 0 New value = Old value Key column value Key row value Key value As a sample calculation, the computation of the new value of row and column x 1 is shown below. New value = = = 4 2 = 2 Computation of te cell values of different tables using this formula is a boring process. So, a different procedure can be used as explained below. 1. Key row: New Key row value = Current Key row value Key element value 2. All other rows: New row = Current row (Its Key column value) (New Key row) 23

10 These computations are applied to the Table 2.1. in the following manner: 1. New Key row (x 2 row) = Current Key row (s 1 row) Key element(10) 2. New s 2 row = Current s 2 row - (4) New key row (x 2 row) The results are shown in Table 2.2. The solution in the Table 2.2 is not optimal. The criterion row value for the variable x 1 is the maximum positive value. Hence, the variable x 1 is selected as entering variable and after computing the ratio, s 2 is selected as leaving variable. The next iteration is shown in Table 2.3. Table 2.3: Iteration 2 c j CB i Basic x 1 x 2 s 1 s 2 Solution Ratio Variable 8 x /5-1/4 2 6 x /5 1/2 8 z j 6 8 2/ c j z j 0 0-2/5-1 In Table 2.3, all the values for c j z j are either 0 (corresponding to basic variables) or negative (corresponding to nonbasic variables). Hence, the optimality is reached. The corresponding optimal solution is as follows: x 1 = 8 and x 2 = 2 and the optimal objective function value, z is

11 2.2.4 Revised simplex method The simplex method discussed in section 2.2 performs calculation on the entire tableau during each iteration. However, updating all the elements in tableau during a basis change is not really necessary for using the simplex method. The only information needed in moving from one tableau (basic feasible solution) to another tableau is as follows: 1. The relative profit coefficients (c j z j row). 2. The column corresponding to the entering variable (Key column). 3. The current basic variables and their values (Solution column). The information contained in other columns of the tableau plays no role in the simplex process. Hence the solution of large linear programming problems on a digital computer will become very inefficient and costly if the simplex method were to be used in its full tableau form. However, revised simplex method is an improvement over simplex method. The revised simplex method is computationally more efficient and accurate. Its benefit is clearly comprehended in case of large LP problems. The revised simplex method uses exactly the same steps as those in simplex method but at each iteration the entire tableau is never calculated. The relevant information it needs to move from one basic feasible solution to another is directly generated from the original equations. The steps of the revised simplex method (using matrix manipulations) can be summarized as follows: Step 1. Obtain the initial feasible basis B. Determine the corresponding feasible solution x B = B 1 b. 25

12 Step 2. Obtain the corresponding simplex multipliers π = c B B 1. Check the optimality of the current basic feasible solution (BFS). If the current basis is optimal, then STOP. Step 3. If the current BFS is not optimal, identify the entering variable x j (that is, c j z j = c j m i=1 π ja ij = c j πp j < 0). Step 4. Obtain the column P j = B 1 P j and perform the minimum ratio test to determine the leaving variable x l. Step 5. Update the basis B (or B 1 ) and go to Step 2. For more information and details, refer to [55]. 26

13 2.3 Duality Theory and Its Applications From both the theoretical and practical point of view, the theory of duality is one of the most important and interesting concepts in linear programming. The basic idea behind the duality theory is that every linear programming problem has an associated linear programming called its dual such that a solution to the original linear program also gives a solution to its dual. Thus, whenever a linear program is solved by the simplex method, we are actually getting solutions for two linear programming problems. The Dual can be considered as the inverse of the Primal in every respect. The column coefficients in the Primal constrains become the row co-efficients in the Dual constraints. The coefficients in the Primal objective function become the right hand side constraints in the Dual constraints. The column of constants on the right hand side of the Primal constraints becomes the row of coefficients of the dual objective function. The direction of the inequalities are reversed. If the primal objective function is a Maximization function then the dual objective function is a Minimization function and vice-versa. The concept of duality is very much in useful to obtain additional information about the variation in the optimal solution when certain changes are effected in the constraint coefficient, resource availabilities and objective function co-efficients. This is termed as post optimality or sensitivity analysis. 27

14 The concept of a dual is introduced by the following programme. Example 2.1 Maximze Z = x 1 + 2x 2-3x 3 + 4x 4 subject to x 1 + 2x 2 + 2x 3-3x 4 25, 2x 1 + x 2-3x 3 + 2x 4 15, x 1, x 2, x 3, x 4 0. The above linear programme has two constraints and four variables. The dual of this problem is written as Minimize W = 25y y 2 subject to y 1 + 2y 2 1, 2y 1 + y 2 2, 2y 1-3y 2-3, 3y 1 + 2y2 4, y 1, y 2 0. y 1 and y 2 are called the dual variables. The original problem is called the primal problem. Comparing the primal and the dual problems, we observe the following relationships: 1. The objective function coefficients of the primal objective function have become the right-hand-side constants of dual. Similarly, the right-hand-side constants of primal problem have become the objective function coefficients of the dual. 2. The inequalities have been reversed in the constraints. 3. The objective is changed from maximization in primal to minimization in dual. 28

15 4. Each column in the primal corresponds to a constraint (row) in the dual. Thus, the number of dual constraints is equal to the number of primal variables. 5. Each constraint(row) in the primal corresponds to a column in the dual. Hence, there is one dual variable for every primal constraint. 6. The dual of the dual is the primal problem. In both of the primal and the dual problems, the variables are nonnegative and the constraints are inequalities. Such problems are called symmetric dual linear programmes. Definition. A linear programme is said to be in symmetric form, if all the variables are restricted to be nonnegative, and all the constraints are inequalities (in a maximization problem the inequalities must be in less than or equal to form and in a minimization problem they must be in greater than or equal to ). In matrix notation the symmetric dual linear programmes are: Primal: Dual: Maximize Z = C X subject to AX b, X 0. Minimize W = b Y subject to A Y C, Y 0. 29

16 Where A is an m n matrix, b is an m 1 column vector, C is a n 1 column vector, X is an n 1 column vector and Y is an m 1 column vector. Te following table describes more general relations between the primal and dual that can be easily derived from the symmetric form definition. They relate the sense of constraint i in the primal with the sign restriction for y i in the dual, and sign restriction of x j in the primal with the sense of constraint j in the dual. Note that when these alternative definitions are allowed there are many ways to write the primal and dual problems; however, they are all equivalent. Modifications in the primal-dual formulations Primal model Dual model M aximization M inimization Constraint i is y i 0 Constraint i is = y i is unrestricted Constraint i is y i 0 x j 0 Constraint j is x j is unrestricted Constraint j is = x j 0 Constraint j is Economic interpretation of the dual problem Primal: Consider a manufacturer who makes n products out of m resources. To make one unit of product j it takes a ij units of resource i. The manufacturer has obtained b i units of resources i in hand, and the unit price of product j is c j. Therefore, the primal problem leads the manufacturer to find an optimal production plan that maximizes the sales with available resources. 30

17 Dual: Lets assume the manufacturer gets the resources from a supplier. The manufacturer wants to negotiate the unit purchasing price y i for resource i with the supplier. Therefore, the manufacturers objective is to minimize the total purchasing price b Y in obtaining the resources b i. Since the market price c j and the product-resource conversion ratio a ij are open information on market, the manufacturer knows that, at least, a smart supplier would like to charge him as much as possible, so that a 1j y 1 + a 2j y a mj y m c j. In this way, the dual linear program leads the manufacturer to come up with a least-cost plan in which the purchasing prices are acceptable to the smart supplier. An example is giving to get more understanding on above economic interpretation. Let n = 3 (products are Desk, Table and Chair) and m = 3 (resources are Lumber, Finishing hours(needed time) and Carpentry hours). The amount of each resource which is needed to make one unit of certain type of furniture is as follows: Product Resource Desk Table Chair Lumber 4 lft 6 lft 1 lft Finishing hours 4 hrs 2 hrs 1.5 hrs Carpentry hours 2 hrs 1.5 hrs 0.5 hrs 48 lft of lumber, 20 hours of finishing hours and 4 carpentry hours are available. A desk sells for \$60, a table for \$30 and a chair for \$20. How many of each type furniture should manufacturer produce? Let x 1, x 2, and x 3 respectively denote the number of desks, tables and chairs 31

18 produced. This problem is solved with the following LP: Maximize Z = 60x x x 3 subject to 8x 1 + 6x 2 + x 3 48, 4x 1 + 2x x 3 20, 2x x x 3 8, x 1, x 2, x 3 0. Suppose that an investor wants to buy the resources. What are the fair prices, y 1, y 2, y 3, that the investor should pay for a lft of lumber, one hour of finishing and one hour of carpentry? The investor wants to minimize buying cost. Then, his objective can be written as min W = 48y y 2 + 8y 3 In exchange for the resources that could make one desk, the investor is offering (8y 1 + 4y 2 + 3y 3 ) dollars. This amount should be larger than what manufacturer could make out of manufacturing one desk (\$60). Therefore, 8y 1 + 4y 2 + 2y 3 60 Similarly, by considering the fair amounts that the investor should pay for the combination of resources that are required to make one table and one chair, we conclude 6y 1 + 2y y 3 30 y y y

19 Consequently, the investor should pay the prices y 1, y 2, y 3, solution to the following LP: Minimize W = 48y y 2 + 8y 3 subject to 8y 1 + 4y 2 + 2y 3 60, 6y 1 + 2y y 3 30, y y y 3 20, y 1, y 2, y 3 0. The above LP is the dual to the manufacturers LP. The dual variables are often referred to as shadow prices, the fair market prices, or the dual prices. Now, we shall turn our attention to some of the duality theorems that gives important relationships between the primal and the dual solution. Theorem 2.1: Weak Duality Theorem Consider the symmetric primal-dual linear programmes, Max Z = C X, s. t. AX b, X 0, and Min W = b Y, s. t. A X C, Y 0. The value of the objective function of the minimum problem (dual) for any feasible solution is always greater than or equal to that of the maximum problem (primal). From the weak duality theorem we can infer the following important results: Corollary 2.1 The value of the objective function of the maximum (primal) problem for any (primal) feasible solution is a lower bound to the minimum value of the dual objective. Corollary 2.2 Similarly the objective function value of the minimum problem (dual) for any (dual) feasible solution is an upper bound to the maximum value of the primal objective. 33

20 Corollary 2.3 If the primal problem is feasible and its objective is unbounded (i.e., max Z + ), then the dual problem is infeasible. Corollary 2.4 Similarly, if the dual problem is feasible, and is unbounded (i.e.,min W ), then the primal problem is infeasible. Theorem 2.2: Optimality Criterion Theorem If there exist feasible solutions X 0 and Y 0 for the symmetric dual linear pragrammes such that the corresponding values of their objective functions are equal, then these feasible solution are in fact optimal solutions to their respective problems. Theorem 2.3: Main Duality Theorem If both the primal and dual problems are feasible, then they both have optimal solutions such their optimal values of the objective functions are equal. We will use the properties of duality theory in proposed search algorithm, see section 5. 34

21 2.4 Computational Problems in Simplex Method There are a number of computational problems that may arise during the actual application of the simplex method for solving a linear programming problem. In this section we discuss some complication that may occur. 1. Ties in the selection of the nonbasic variable: In a maximization problem, the nonbasic variable with the largest positive value in the c j z j row is chosen. In case there exists more than one variable with the same largest positive value in the c j z j row, then we have a tie for selecting the nonbasic variable. 2. Ties in the minimum ratio rule and degeneracy: While applying the minimum ratio rule it is possible for two or more constraints to give the same least ratio value. This result in a tie for selecting which basic variable should leave the basis. This complication causes degeneracy in basic feasible solution. Degenerate basic feasible solution: A basic feasible solution is said to be degenerate basic feasible solution if at least one basic variable in the solution equals zero. 3. Cycling: At a degenerate basic solution, there is a risk of getting trapped in a cycle. When there exist degeneracy in feasible solution we can t say that simplex method will be terminated in a finite number of iterations, because this phenomenon causes that we obtain a new basic feasible solution in the next iteration which has no effect on the objective function value. It is then the- 35

22 oretically possible to select a sequence of admissible bases that is repeatedly selected without over satisfying the optimality criteria and, hence, never reaches a optimal solution. A number of anticycling procedure have been developed [17, 14, 66, 49, 10]. These anticycling method are not used in practice partly because they would slow the computer program down and partly because of the difficulty of recognizing a true zero numerically [7] Complexity of the simplex method The simplex method is not a good computational algorithm. It may lead to an exponential number of iteration as the example which is provided by Klee and Minty (1972). They showed that the simplex method as formulated by Dantzig visit all 2 n vertices before arriving at the optimal vertex, this shows that the worst case complexity of the simplex algorithm is exponential time. Also, the number of arithmetic operations in the algorithm grows exponentially with the number of variables. The first polynomial time (interior point) algorithms, Ellipsoid method, for linear programming were given by Khachian (1979), then Karmarkar (1984) suggested the second polynomial time algorithm. Unlike the ellipsoid method Karmarkar s method solves the very larg linear programming problems faster than does the simplex method [58]. Both the ellipsoid method and the karmarkar s method are mathematically iterative and they are not better than simplex algorithm certainly for small linear programming problems and possibly for reasonably large linear programming problems [58]. According this restrictions of simplex algorithm and interior point methods provided by Khachian (1979) and Karmarkar (1984) and their variants, we need a new algorithm. We are going to give a stochastic search algorithm to obtain a near-optimal solution. Stochastic search algorithm are useful for applications 36

23 where stable and acceptable (i. e. near-optimal) answers are desired quickly [27]. In general, Stochastic search algorithms do not require knowledge of a derivative (as in gradient search methods) and perform best with highly nonlinear or highly combinatorial problems [12]. 37

24 2.5 Proposed Stochastic Search Algorithm Consider the linear programming problem in the canonical form: P: max z = c 1 x 1 + c 2 x c n x n subject to a 11 x 1 + a 12 x a 1n x n b 1, a 21 x 1 + a 22 x a 2n x n b 2,..... a m1 x 1 + a m2 x a mn x n b m, x 1, x 2,..., x n 0. Consider the decision variable x j for some j = 1, 2,..., n. If all other decision variables are set to zero, then x j must satisfy a ij x j b i, i = 1, 2,..., m. To satisfy all of this condition for all i, we must have { } bi 0 x j min, 1 j n. {i: a ij >0} a ij { { b The bounding box B = x 0 x j min i {i: aij >0} a ij } }, 1 j n, B E n, contains the simplex set of feasible solutions of P. All feasible solutions of P are elements of B, though the converse may not hold. In this case all dual decision variables have zero lower bound, in fact, the positive quadrant contains the simplex representing the set of feasible solution of dual problem. We call it DB. Now, generate a random element each from B and DB using a specified distribution, p r, on B and DB. If this element is primal feasible, then compute the primal objective function at this solution. Ignore all infeasible elements. Compute the largest value of the primal objective functions for these primal feasible solutions. The same process is applied to the dual problem by computing the smallest value of dual objective functions for the generated feasible dual solutions. In this process, a feasible primal solution is declared inadmissible if the value of the primal objective function does not exceed the highest value obtained till then. A feasible 38

25 dual solution is declared inadmissible if the value of the dual objective function is not smaller than the smallest value obtained till then. We keep track of the number of generated elements, the number of feasible solutions among these and the number of admissible points among the latter, in both problems. The procedure can be terminated after generating a specified number of 1. random elements 2. feasible solutions, or 3. admissible solutions. The best solution at termination of the procedure is declared as the nearoptimal solution. A second possible termination rule is obtained by considering the gap between near optimal primal and near optimal dual solutions. The procedure is terminated when this gap (or interval) is sufficiently small Multiple starts The simulation approach obtains a near-optimal solution for every starting value. Near-optimality of the final solution does not depend on the initial value and hence the rate of convergence can not be determined. Therefore, it may be convenient to begin with more than one initial value, generating an independent sequence of solutions for every initial value. This will give us several end-points and the best of these will be better than the solution obtained from any one of them. The maximum of the primal objective function and the minimum of the dual objective function obtained through the above procedure are both near-optimal. This method is not iterative in the sense that consecutive solutions may not improve the objective function. Also, it is simple from the mathematical point of view and there are no complicated computations. 39

26 Algorithmically, the method is described as follows. Initialization in primal problem (PP): Set i = 1, z n o = 0, x n o = 0, i o = 100 (number of feasible points). 1. Generation in PP: Step 1.1. Generate x i B using uniform distribution on the bounding box B. Step Test feasibility of x i. Step 1.3. If x i is feasible, go to step 1.4. Otherwise, go to step Computation in PP: Step 1.4. Compute z i = c x i, i = 1, 2,, i o. Step 1.5. z n o = max i (z i ), i = 1, 2,, i o and x n o = associated x i with z n o. 3. Termination in PP: Step 1.6. Keep the values of x n o, z n o and stop. Initialization in dual problem (DP): Set j = 1, g n o is infinity, y n o is infinity, j o = 100 (number of feasible points). 1. Generation in DP: Step 2.1. Generate y j DB using uniform distribution on the bounding box DB. Step 2.2. Test feasibility of y j. Step 2.3. If y j is feasible, go to step 2.4. Otherwise, go to step

27 2. Computation in DP: Step 2.4. Compute g j = b y j, j = 1, 2,, j o. Step 2.5. g n o = min j (g j ), j = 1, 2,, j o and g n o = associated y j with g n o. 3. Termination in DP: Step 2.6. Keep the values of y n o, g n o and stop. Termination in restarting: if (g n o z n o ) ɛ, small positive value, or the number of restarting get equal to its initial value, then output the values of z n o, x n o, g n o, y n o and stop restart Choice of probability distribution (p r ) For the probability distribution, we have two choices. 1. Uniform distribution. In this case, every element is generated from B and DB with equal probability of selection. A significant limitation of uniform distribution is the progressively increasing rejection rate (rejection can occur for one of the two reasons: infeasibility and inadmissibility). This happens because points are generated with equal probability and rejection depends on the position of the current solution. 2. Non-Uniform distributions. Due to the limitation of Uniform distribution, we propose to use a nonuniform distribution. As a pilot study, we use the normal distribution with mean µ j = upper bound of x j, j = 1,, n in P P and ν i = lower bound of 41

28 y i, i = 1,, m in DP and unit variances. The results of pilot study show that the normal distribution is more suitable than uniform distribution. A major limitation of the normal distribution is that half of the generated points are expected to exceed the mean. This implies a rejection rate exceeding 1/2. This can be overcome by modifying the mean as follows. We take a fraction f (0, 1) and use the mean f µ j, 1 j n or f ν i, 1 i m. We tried some fractions to identify a good value for the fraction. Another feature of the normal distribution is its unbounded support. Since the set of feasible primal solutions is bounded, we considered a distribution on a bounded support. In particular, we tried the multivariate Beta distribution, [ { }] with the marginal distribution of x j on 0, min {i: aij >0} : 1 j n with parameters α, β > 0 so that α > β or α = k β with k > 1. The gamma distribution is reasonable for the dual problem because its feasible solution set is unbounded. For a problem having integer optimal solution, a discrete probability distribution is found to be very efficient. This is more so if the coordinates of bounding points are small multiples of optimal x j or y i. b i a ij 42

29 2.6 Results of Simulation Studies We consider some examples and implement the proposed simulation algorithm under some non-uniform distributions, namely normal, beta, Weibull and gamma, with different parameters and two discrete distributions, namely geometric and Poisson. The following symbols are used in the tables. Z n o and g n o are the primal and dual near-optimal objective functions,respectively. gap is the length of interval between primal and dual near-optimal values (gap). CPU time/sec. is the average of running time of stochastic algorithm in terms of second per restart. 1. Beale s Example The following example is called Beale s example and falls into cycling when the simplex algorithm is used. The optimum value of objective function is max z = 0.75x 1 150x x 3 6x 4 s.t. 0.25x 1 60x x 3 + 9x 4 0, 0.5x 1 90x x 3 + 3x 4 0, x 3 1, x 1, x 2, x 3, x

30 In this example based on algorithm steps the following steps are carried out in both problems, Primal and Dual: (a) The points are generated randomly from different distribution with different parameters. (b) Test the feasibility of generated points. (c) Evaluate the primal and dual objective function values,z i and g j for feasible points. (d) Compute z n o = max i (z i ) and g n o = min j (g j ) (e) if (g n o z n o ) ɛ, small positive value, or the number of restarting get equal to its initial value, then the algorithm is stopped and outputs the values of z n o, x n o, g n o, y n o and stop restart. The summary of results are shown in the following table. Table 2.4: Near-Optimal values of Primal and Dual objective functions in LPP by using stochastic search algorithm with different distributions in primal and dual problem. Distribution z n o g n o # infeasible points gap CPU time/ Primal Dual Total sec. Uniform(0, µ j ) N(µ j, µ 2 j) N(1/2µ j, µ 2 j) N(0.75µ j, µ 2 j) Beta(0.5, 0.75) Beta(0.75, 0.5) Beta(0.05, 0.05)

31 As we mentioned before, the performance of non-uniform statistical distributions is better than uniform distribution. This can be understood from running time of proposed algorithm and gap (distance between primal and dual near-optimal objective function values). When normal distribution with the mean at the half of upper (lower) bound for primal (dual) decision variable is considered the length of gap is smaller than other cases of mean, this means the primal and dual near-optimal value are nearer to each other than other cases of mean. The results which have obtained from Beta distribution with parameters 0 < α, β < 1 are better than other cases of α, β. 2. Kuhn s Example This example faces to cycling by using simplex algorithm. The optimum value of objective function is 2. max z = 2x 1 + 3x 2 x 3 12x 4 s.t. 2x 1 9x 2 + x 3 + 9x 4 0, 1 3 x 1 + x x 3 2x 4 0, 2x 1 + 3x 2 x 3 12x 4 2, x 1, x 2, x 3, x 4 0. In this example based on algorithm steps the following steps are carried out in both problems, Primal and Dual: (a) The points are generated randomly from different distribution with different parameters. 45

32 (b) Test the feasibility of generated points. (c) Evaluate the primal and dual objective function values,z i and g j for feasible points. (d) Compute z n o = max i (z i ) and g n o = min j (g j ) (e) if (g n o z n o ) ɛ, small positive value, or the number of restarting get equal to its initial value, then the algorithm is stopped and outputs the values of z n o, x n o, g n o, y n o and stop restart. The summary of results are shown in the following table. Table 2.5: Near-Optimal values of Primal and Dual objective functions in LPP by using stochastic search algorithm with different distributions in primal and dual problem. Distribution z n o g n o # infeasible points gap CPU time/ Primal Dual Total sec. Uniform(0, µ j ) N(µ j, 1) N(µ j, µ 2 j) N(1/2µ j, µ 2 j) Beta(0.5, 0.75) Beta(0.5, 0.5) Gamma(1, 1) Gamma(1, 2) Geo(0.5) For this example uniform and non-uniform distributions with considering a discrete distribution (Geometric(p = 0.5)) for dual problem have good and almost same performance. The problem has an integer optimal solution then, a discrete distribution (Geometric(p = 0.5))without considering the upper bounds of decision 46

33 variables in primal problem was applied and the exact optimal solution for both problems obtained. 3. Klee and Minty s Example: In this example, simplex method as formulated by Dantzig, visits all 2 n vertices before arriving at the optimal vertex. The optimal objective function value is 625. max z = 8x 1 + 4x 2 + 2x 3 + x 4 s.t. 16x 1 + 8x 2 + 4x 3 + x 4 625, 8x 1 + 4x 2 + x 3 125, 4x 1 + x 2 25, x 1 5, x 1, x 2, x 3, x 4 0. In this example based on algorithm steps the following steps are carried out in both problems, Primal and Dual: (a) The points are generated randomly from different distribution with different parameters. (b) Test the feasibility of generated points. (c) Evaluate the primal and dual objective function values,z i and g j for feasible points. (d) Compute z n o = max i (z i ) and g n o = min j (g j ) 47

34 (e) if (g n o z n o ) ɛ, small positive value, or the number of restarting get equal to its initial value, then the algorithm is stopped and outputs the values of z n o, x n o, g n o, y n o and stop restart. The summary of results are shown in the following table. Table 2.6: Near-Optimal values of Primal and Dual objective functions in LPP by using stochastic search algorithm with different distributions in primal and dual problem. Distribution z n o g n o # infeasible points gap time/ Primal Dual Total sec. Uniform(0, µ j ) N(µ j, µ 2 j) na na na na na na na N(1/2µ j, µ 2 j) Beta(0.5, 0.75) Beta(0.5, 1) Beta(0.25, 1) Beta(0.25, 0.75) Beta(0.05, 0.05) Geo(0.5) Also in this example, the efficiency of most of non-uniform statistical distributions is better than uniform distribution. This can be understood from running time of proposed algorithm and gap (distance between primal and dual near-optimal objective function values). The normal distribution with the mean at upper (lower) bound for primal (dual) decision variable is not applicable. But normal distribution with mean at the half of the upper (lower) bound for primal (dual) decision variable has better performance than uniform distribution. The results which are obtained from Beta distribution with parameters 0 < α < 1 and β = 1 are better than other cases of α, β. 48

35 The coordinates of bounding points of this example are multiples of optimal solution x opt, y opt then a discreet distribution is very efficient and here the Geometric distribution with parameter p = 0.5 reaches to the exact optimal solution which is integer. 4. Further example This example has integer optimal solution in both problems and the optimal objective function value is max z = 45x x 2 s.t. 5x x 2 400, 10x x 2 450, x 1, x 2 0. In this example based on algorithm steps the following steps are carried out in both problems, Primal and Dual: (a) The points are generated randomly from different distribution with different parameters. (b) Test the feasibility of generated points. (c) Evaluate the primal and dual objective function values,z i and g j for feasible points. (d) Compute z n o = max i (z i ) and g n o = min j (g j ) (e) if (g n o z n o ) ɛ, small positive value, or the number of restarting get equal to its initial value, then the algorithm is stopped and outputs the values of z n o, x n o, g n o, y n o and stop restart. 49

36 The summary of results are shown in the following table. Table 2.7: Near-Optimal values of Primal and Dual objective functions in LPP by using stochastic search algorithm with different distributions in primal and dual problem. Distribution z n o g n o # infeasible points gap CPU time/ Primal Dual Total sec. N(µ j, µ 2 j) N(1/2µ j, µ 2 j) W eibull(µ j, 3.5) P oisson(µ j ) P oisson(1/2µ j ) The Uniform distribution has no efficiency for this example. Then, we applied Non-uniform distribution for both primal and dual problems. Normal and Weibull distributions for primal and Gamma distribution for dual problem. As it can be seen when a fraction of mean parameter of Normal distribution is considered the efficiency of algorithm is increased. It is clear that the Weibull distribution for scale parameters bigger than 3.4 leads to Normal distribution and we see the behavior of this distribution. Because of existence an integer solution for the problem, discrete distributions are best for this problem. All this work has been published in the Pakistan Journal of Statistics, Vol. 27, No. 1, pp

37 2.7 Conclusions 1. Since stochastic search technique uses pseudo-random numbers, there is no learning in the simulation based algorithm. As a result, more work does not necessarily imply improvement in the solution. On the other hand, the result is not necessarily bad if we have not worked much. Hence, an early solution can be a near-optimal solution and may not significantly improve with more work. 2. Generating feasible solutions is the major work in this algorithm. The bounding box is either a hyper-cube or cuboid and then there is a high rejection rate, or the complete Euclidean space with the same consequence. Attempts are being made to improve the performance of random solution generator. 3. Stochastic search algorithm is not an iterative procedure. 4. Stochastic search algorithm is simple from mathematical point of view and there are no complicated computations. 5. Stopping rule can be based on number feasible points in generation, number restarts the algorithm, length of interval of optimal objective function value (the length of gap between primal near-optimal and dual near-optimal values). 6. Discrete distributions like Geometric and Poisson are very efficient in certain special cases. 51

### LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS. 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method

LECTURE 5: DUALITY AND SENSITIVITY ANALYSIS 1. Dual linear program 2. Duality theory 3. Sensitivity analysis 4. Dual simplex method Introduction to dual linear program Given a constraint matrix A, right

### 4.6 Linear Programming duality

4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP. Different spaces and objective functions but in general same optimal

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### Practical Guide to the Simplex Method of Linear Programming

Practical Guide to the Simplex Method of Linear Programming Marcel Oliver Revised: April, 0 The basic steps of the simplex algorithm Step : Write the linear programming problem in standard form Linear

### Linear Programming for Optimization. Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc.

1. Introduction Linear Programming for Optimization Mark A. Schulze, Ph.D. Perceptive Scientific Instruments, Inc. 1.1 Definition Linear programming is the name of a branch of applied mathematics that

### Definition of a Linear Program

Definition of a Linear Program Definition: A function f(x 1, x,..., x n ) of x 1, x,..., x n is a linear function if and only if for some set of constants c 1, c,..., c n, f(x 1, x,..., x n ) = c 1 x 1

### 3 Does the Simplex Algorithm Work?

Does the Simplex Algorithm Work? In this section we carefully examine the simplex algorithm introduced in the previous chapter. Our goal is to either prove that it works, or to determine those circumstances

### Special Situations in the Simplex Algorithm

Special Situations in the Simplex Algorithm Degeneracy Consider the linear program: Maximize 2x 1 +x 2 Subject to: 4x 1 +3x 2 12 (1) 4x 1 +x 2 8 (2) 4x 1 +2x 2 8 (3) x 1, x 2 0. We will first apply the

### Linear Programming. Widget Factory Example. Linear Programming: Standard Form. Widget Factory Example: Continued.

Linear Programming Widget Factory Example Learning Goals. Introduce Linear Programming Problems. Widget Example, Graphical Solution. Basic Theory:, Vertices, Existence of Solutions. Equivalent formulations.

### Duality in Linear Programming

Duality in Linear Programming 4 In the preceding chapter on sensitivity analysis, we saw that the shadow-price interpretation of the optimal simplex multipliers is a very useful concept. First, these shadow

### Simplex method summary

Simplex method summary Problem: optimize a linear objective, subject to linear constraints 1. Step 1: Convert to standard form: variables on right-hand side, positive constant on left slack variables for

### Linear Programming. March 14, 2014

Linear Programming March 1, 01 Parts of this introduction to linear programming were adapted from Chapter 9 of Introduction to Algorithms, Second Edition, by Cormen, Leiserson, Rivest and Stein [1]. 1

### Solving Linear Programs

Solving Linear Programs 2 In this chapter, we present a systematic procedure for solving linear programs. This procedure, called the simplex method, proceeds by moving from one feasible solution to another,

### Operation Research. Module 1. Module 2. Unit 1. Unit 2. Unit 3. Unit 1

Operation Research Module 1 Unit 1 1.1 Origin of Operations Research 1.2 Concept and Definition of OR 1.3 Characteristics of OR 1.4 Applications of OR 1.5 Phases of OR Unit 2 2.1 Introduction to Linear

### Standard Form of a Linear Programming Problem

494 CHAPTER 9 LINEAR PROGRAMMING 9. THE SIMPLEX METHOD: MAXIMIZATION For linear programming problems involving two variables, the graphical solution method introduced in Section 9. is convenient. However,

### Linear Programming I

Linear Programming I November 30, 2003 1 Introduction In the VCR/guns/nuclear bombs/napkins/star wars/professors/butter/mice problem, the benevolent dictator, Bigus Piguinus, of south Antarctica penguins

### 1 Introduction. Linear Programming. Questions. A general optimization problem is of the form: choose x to. max f(x) subject to x S. where.

Introduction Linear Programming Neil Laws TT 00 A general optimization problem is of the form: choose x to maximise f(x) subject to x S where x = (x,..., x n ) T, f : R n R is the objective function, S

### Linear Programming Notes VII Sensitivity Analysis

Linear Programming Notes VII Sensitivity Analysis 1 Introduction When you use a mathematical model to describe reality you must make approximations. The world is more complicated than the kinds of optimization

### Linear Programming in Matrix Form

Linear Programming in Matrix Form Appendix B We first introduce matrix concepts in linear programming by developing a variation of the simplex method called the revised simplex method. This algorithm,

### Sensitivity Analysis 3.1 AN EXAMPLE FOR ANALYSIS

Sensitivity Analysis 3 We have already been introduced to sensitivity analysis in Chapter via the geometry of a simple example. We saw that the values of the decision variables and those of the slack and

### Linear Programming Notes V Problem Transformations

Linear Programming Notes V Problem Transformations 1 Introduction Any linear programming problem can be rewritten in either of two standard forms. In the first form, the objective is to maximize, the material

### What is Linear Programming?

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

### 3. Evaluate the objective function at each vertex. Put the vertices into a table: Vertex P=3x+2y (0, 0) 0 min (0, 5) 10 (15, 0) 45 (12, 2) 40 Max

SOLUTION OF LINEAR PROGRAMMING PROBLEMS THEOREM 1 If a linear programming problem has a solution, then it must occur at a vertex, or corner point, of the feasible set, S, associated with the problem. Furthermore,

### Lecture 3. Linear Programming. 3B1B Optimization Michaelmas 2015 A. Zisserman. Extreme solutions. Simplex method. Interior point method

Lecture 3 3B1B Optimization Michaelmas 2015 A. Zisserman Linear Programming Extreme solutions Simplex method Interior point method Integer programming and relaxation The Optimization Tree Linear Programming

### This exposition of linear programming

Linear Programming and the Simplex Method David Gale This exposition of linear programming and the simplex method is intended as a companion piece to the article in this issue on the life and work of George

### Lecture 3: Linear Programming Relaxations and Rounding

Lecture 3: Linear Programming Relaxations and Rounding 1 Approximation Algorithms and Linear Relaxations For the time being, suppose we have a minimization problem. Many times, the problem at hand can

### Chapter 6. Linear Programming: The Simplex Method. Introduction to the Big M Method. Section 4 Maximization and Minimization with Problem Constraints

Chapter 6 Linear Programming: The Simplex Method Introduction to the Big M Method In this section, we will present a generalized version of the simplex method that t will solve both maximization i and

### Chapter 2 Solving Linear Programs

Chapter 2 Solving Linear Programs Companion slides of Applied Mathematical Programming by Bradley, Hax, and Magnanti (Addison-Wesley, 1977) prepared by José Fernando Oliveira Maria Antónia Carravilla A

### Chapter 6: Sensitivity Analysis

Chapter 6: Sensitivity Analysis Suppose that you have just completed a linear programming solution which will have a major impact on your company, such as determining how much to increase the overall production

### 7.4 Linear Programming: The Simplex Method

7.4 Linear Programming: The Simplex Method For linear programming problems with more than two variables, the graphical method is usually impossible, so the simplex method is used. Because the simplex method

### Interior Point Methods and Linear Programming

Interior Point Methods and Linear Programming Robert Robere University of Toronto December 13, 2012 Abstract The linear programming problem is usually solved through the use of one of two algorithms: either

### Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood

PERFORMANCE EXCELLENCE IN THE WOOD PRODUCTS INDUSTRY EM 8720-E October 1998 \$3.00 Using the Simplex Method to Solve Linear Programming Maximization Problems J. Reeb and S. Leavengood A key problem faced

### Linear Programming Problems

Linear Programming Problems Linear programming problems come up in many applications. In a linear programming problem, we have a function, called the objective function, which depends linearly on a number

### Nonlinear Programming Methods.S2 Quadratic Programming

Nonlinear Programming Methods.S2 Quadratic Programming Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard A linearly constrained optimization problem with a quadratic objective

### 56:171. Operations Research -- Sample Homework Assignments Fall 1992 Dennis Bricker Dept. of Industrial Engineering University of Iowa.

56:171 Operations Research -- Sample Homework Assignments Fall 1992 Dennis Bricker Dept. of Industrial Engineering University of Iowa Homework #1 (1.) Linear Programming Model Formulation. SunCo processes

### 3. Linear Programming and Polyhedral Combinatorics

Massachusetts Institute of Technology Handout 6 18.433: Combinatorial Optimization February 20th, 2009 Michel X. Goemans 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the

### Can linear programs solve NP-hard problems?

Can linear programs solve NP-hard problems? p. 1/9 Can linear programs solve NP-hard problems? Ronald de Wolf Linear programs Can linear programs solve NP-hard problems? p. 2/9 Can linear programs solve

### Lecture 2: August 29. Linear Programming (part I)

10-725: Convex Optimization Fall 2013 Lecture 2: August 29 Lecturer: Barnabás Póczos Scribes: Samrachana Adhikari, Mattia Ciollaro, Fabrizio Lecci Note: LaTeX template courtesy of UC Berkeley EECS dept.

### 1. (a) Multiply by negative one to make the problem into a min: 170A 170B 172A 172B 172C Antonovics Foster Groves 80 88

Econ 172A, W2001: Final Examination, Possible Answers There were 480 possible points. The first question was worth 80 points (40,30,10); the second question was worth 60 points (10 points for the first

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### The application of linear programming to management accounting

The application of linear programming to management accounting Solutions to Chapter 26 questions Question 26.16 (a) M F Contribution per unit 96 110 Litres of material P required 8 10 Contribution per

### Mathematical finance and linear programming (optimization)

Mathematical finance and linear programming (optimization) Geir Dahl September 15, 2009 1 Introduction The purpose of this short note is to explain how linear programming (LP) (=linear optimization) may

### Linear Programming. Solving LP Models Using MS Excel, 18

SUPPLEMENT TO CHAPTER SIX Linear Programming SUPPLEMENT OUTLINE Introduction, 2 Linear Programming Models, 2 Model Formulation, 4 Graphical Linear Programming, 5 Outline of Graphical Procedure, 5 Plotting

### Matrix Norms. Tom Lyche. September 28, Centre of Mathematics for Applications, Department of Informatics, University of Oslo

Matrix Norms Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo September 28, 2009 Matrix Norms We consider matrix norms on (C m,n, C). All results holds for

### 9.4 THE SIMPLEX METHOD: MINIMIZATION

SECTION 9 THE SIMPLEX METHOD: MINIMIZATION 59 The accounting firm in Exercise raises its charge for an audit to \$5 What number of audits and tax returns will bring in a maximum revenue? In the simplex

### Degeneracy in Linear Programming

Degeneracy in Linear Programming I heard that today s tutorial is all about Ellen DeGeneres Sorry, Stan. But the topic is just as interesting. It s about degeneracy in Linear Programming. Degeneracy? Students

### FE670 Algorithmic Trading Strategies. Stevens Institute of Technology

FE670 Algorithmic Trading Strategies Lecture 6. Portfolio Optimization: Basic Theory and Practice Steve Yang Stevens Institute of Technology 10/03/2013 Outline 1 Mean-Variance Analysis: Overview 2 Classical

### IEOR 4404 Homework #2 Intro OR: Deterministic Models February 14, 2011 Prof. Jay Sethuraman Page 1 of 5. Homework #2

IEOR 4404 Homework # Intro OR: Deterministic Models February 14, 011 Prof. Jay Sethuraman Page 1 of 5 Homework #.1 (a) What is the optimal solution of this problem? Let us consider that x 1, x and x 3

### 24. The Branch and Bound Method

24. The Branch and Bound Method It has serious practical consequences if it is known that a combinatorial problem is NP-complete. Then one can conclude according to the present state of science that no

### OPRE 6201 : 2. Simplex Method

OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2

### 4.1 Learning algorithms for neural networks

4 Perceptron Learning 4.1 Learning algorithms for neural networks In the two preceding chapters we discussed two closely related models, McCulloch Pitts units and perceptrons, but the question of how to

### 7.1 Modelling the transportation problem

Chapter Transportation Problems.1 Modelling the transportation problem The transportation problem is concerned with finding the minimum cost of transporting a single commodity from a given number of sources

### Linear Programming. April 12, 2005

Linear Programming April 1, 005 Parts of this were adapted from Chapter 9 of i Introduction to Algorithms (Second Edition) /i by Cormen, Leiserson, Rivest and Stein. 1 What is linear programming? The first

### Probability and Statistics

CHAPTER 2: RANDOM VARIABLES AND ASSOCIATED FUNCTIONS 2b - 0 Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be

### Convex Optimization SVM s and Kernel Machines

Convex Optimization SVM s and Kernel Machines S.V.N. Vishy Vishwanathan vishy@axiom.anu.edu.au National ICT of Australia and Australian National University Thanks to Alex Smola and Stéphane Canu S.V.N.

### princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Sanjeev Arora Scribe: One of the running themes in this course is the notion of

### Linear Programming: Theory and Applications

Linear Programming: Theory and Applications Catherine Lewis May 11, 2008 1 Contents 1 Introduction to Linear Programming 3 1.1 What is a linear program?...................... 3 1.2 Assumptions.............................

### Zeros of a Polynomial Function

Zeros of a Polynomial Function An important consequence of the Factor Theorem is that finding the zeros of a polynomial is really the same thing as factoring it into linear factors. In this section we

### 2.3 Scheduling jobs on identical parallel machines

2.3 Scheduling jobs on identical parallel machines There are jobs to be processed, and there are identical machines (running in parallel) to which each job may be assigned Each job = 1,,, must be processed

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### 56:171 Operations Research Midterm Exam Solutions Fall 2001

56:171 Operations Research Midterm Exam Solutions Fall 2001 True/False: Indicate by "+" or "o" whether each statement is "true" or "false", respectively: o_ 1. If a primal LP constraint is slack at the

### Algebra 2 Chapter 1 Vocabulary. identity - A statement that equates two equivalent expressions.

Chapter 1 Vocabulary identity - A statement that equates two equivalent expressions. verbal model- A word equation that represents a real-life problem. algebraic expression - An expression with variables.

### Optimization Modeling for Mining Engineers

Optimization Modeling for Mining Engineers Alexandra M. Newman Division of Economics and Business Slide 1 Colorado School of Mines Seminar Outline Linear Programming Integer Linear Programming Slide 2

### The Steepest Descent Algorithm for Unconstrained Optimization and a Bisection Line-search Method

The Steepest Descent Algorithm for Unconstrained Optimization and a Bisection Line-search Method Robert M. Freund February, 004 004 Massachusetts Institute of Technology. 1 1 The Algorithm The problem

### Nonlinear Optimization: Algorithms 3: Interior-point methods

Nonlinear Optimization: Algorithms 3: Interior-point methods INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2006 Jean-Philippe Vert,

### 26 Linear Programming

The greatest flood has the soonest ebb; the sorest tempest the most sudden calm; the hottest love the coldest end; and from the deepest desire oftentimes ensues the deadliest hate. Th extremes of glory

Approximation Algorithms Chapter Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should I do? A. Theory says you're unlikely to find a poly-time algorithm. Must sacrifice one

### 0.1 Linear Programming

0.1 Linear Programming 0.1.1 Objectives By the end of this unit you will be able to: formulate simple linear programming problems in terms of an objective function to be maximized or minimized subject

### ! Solve problem to optimality. ! Solve problem in poly-time. ! Solve arbitrary instances of the problem. #-approximation algorithm.

Approximation Algorithms 11 Approximation Algorithms Q Suppose I need to solve an NP-hard problem What should I do? A Theory says you're unlikely to find a poly-time algorithm Must sacrifice one of three

### Duality of linear conic problems

Duality of linear conic problems Alexander Shapiro and Arkadi Nemirovski Abstract It is well known that the optimal values of a linear programming problem and its dual are equal to each other if at least

### 4 UNIT FOUR: Transportation and Assignment problems

4 UNIT FOUR: Transportation and Assignment problems 4.1 Objectives By the end of this unit you will be able to: formulate special linear programming problems using the transportation model. define a balanced

### International Doctoral School Algorithmic Decision Theory: MCDA and MOO

International Doctoral School Algorithmic Decision Theory: MCDA and MOO Lecture 2: Multiobjective Linear Programming Department of Engineering Science, The University of Auckland, New Zealand Laboratoire

### 1 if 1 x 0 1 if 0 x 1

Chapter 3 Continuity In this chapter we begin by defining the fundamental notion of continuity for real valued functions of a single real variable. When trying to decide whether a given function is or

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### Review of Fundamental Mathematics

Review of Fundamental Mathematics As explained in the Preface and in Chapter 1 of your textbook, managerial economics applies microeconomic theory to business decision making. The decision-making tools

### 1. LINEAR EQUATIONS. A linear equation in n unknowns x 1, x 2,, x n is an equation of the form

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

### Simulating the Multiple Time-Period Arrival in Yield Management

Simulating the Multiple Time-Period Arrival in Yield Management P.K.Suri #1, Rakesh Kumar #2, Pardeep Kumar Mittal #3 #1 Dean(R&D), Chairman & Professor(CSE/IT/MCA), H.C.T.M., Kaithal(Haryana), India #2

### Duality in General Programs. Ryan Tibshirani Convex Optimization 10-725/36-725

Duality in General Programs Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: duality in linear programs Given c R n, A R m n, b R m, G R r n, h R r : min x R n c T x max u R m, v R r b T

### Module1. x 1000. y 800.

Module1 1 Welcome to the first module of the course. It is indeed an exciting event to share with you the subject that has lot to offer both from theoretical side and practical aspects. To begin with,

### Proximal mapping via network optimization

L. Vandenberghe EE236C (Spring 23-4) Proximal mapping via network optimization minimum cut and maximum flow problems parametric minimum cut problem application to proximal mapping Introduction this lecture:

### Regression Using Support Vector Machines: Basic Foundations

Regression Using Support Vector Machines: Basic Foundations Technical Report December 2004 Aly Farag and Refaat M Mohamed Computer Vision and Image Processing Laboratory Electrical and Computer Engineering

### Solving Linear Diophantine Matrix Equations Using the Smith Normal Form (More or Less)

Solving Linear Diophantine Matrix Equations Using the Smith Normal Form (More or Less) Raymond N. Greenwell 1 and Stanley Kertzner 2 1 Department of Mathematics, Hofstra University, Hempstead, NY 11549

### Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi

Project and Production Management Prof. Arun Kanda Department of Mechanical Engineering Indian Institute of Technology, Delhi Lecture - 15 Limited Resource Allocation Today we are going to be talking about

### CHAPTER 3 Numbers and Numeral Systems

CHAPTER 3 Numbers and Numeral Systems Numbers play an important role in almost all areas of mathematics, not least in calculus. Virtually all calculus books contain a thorough description of the natural,

### Linear Programming: Chapter 11 Game Theory

Linear Programming: Chapter 11 Game Theory Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Rock-Paper-Scissors

### Tetris: Experiments with the LP Approach to Approximate DP

Tetris: Experiments with the LP Approach to Approximate DP Vivek F. Farias Electrical Engineering Stanford University Stanford, CA 94403 vivekf@stanford.edu Benjamin Van Roy Management Science and Engineering

### Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

### a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

### FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL

FEGYVERNEKI SÁNDOR, PROBABILITY THEORY AND MATHEmATICAL STATIsTICs 4 IV. RANDOm VECTORs 1. JOINTLY DIsTRIBUTED RANDOm VARIABLEs If are two rom variables defined on the same sample space we define the joint

### Chapter 6. Cuboids. and. vol(conv(p ))

Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both

### The Equivalence of Linear Programs and Zero-Sum Games

The Equivalence of Linear Programs and Zero-Sum Games Ilan Adler, IEOR Dep, UC Berkeley adler@ieor.berkeley.edu Abstract In 1951, Dantzig showed the equivalence of linear programming problems and two-person

### SENSITIVITY ANALYSIS AS A MANAGERIAL DECISION

SENSITIVITY ANALYSIS AS A MANAGERIAL DECISION MAKING TOOL SENSITIVITY ANALYSIS AS A MANAGERIAL DECISION MAKING TOOL SUMMARY Martina Briš, B.Sc.(Econ) Faculty of Economics in Osijek 87 Decision making is

### CHAPTER 9. Integer Programming

CHAPTER 9 Integer Programming An integer linear program (ILP) is, by definition, a linear program with the additional constraint that all variables take integer values: (9.1) max c T x s t Ax b and x integral

### 5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1

5 INTEGER LINEAR PROGRAMMING (ILP) E. Amaldi Fondamenti di R.O. Politecnico di Milano 1 General Integer Linear Program: (ILP) min c T x Ax b x 0 integer Assumption: A, b integer The integrality condition

### Chapter 4. Duality. 4.1 A Graphical Example

Chapter 4 Duality Given any linear program, there is another related linear program called the dual. In this chapter, we will develop an understanding of the dual linear program. This understanding translates

### Several Views of Support Vector Machines

Several Views of Support Vector Machines Ryan M. Rifkin Honda Research Institute USA, Inc. Human Intention Understanding Group 2007 Tikhonov Regularization We are considering algorithms of the form min

### Approximation Algorithms

Approximation Algorithms or: How I Learned to Stop Worrying and Deal with NP-Completeness Ong Jit Sheng, Jonathan (A0073924B) March, 2012 Overview Key Results (I) General techniques: Greedy algorithms