CSCE 310J Data Structures & Algorithms. Dynamic programming 01 Knapsack problem. Dynamic programming. Dynamic Programming. Knapsack problem (Review)


 Blaise Montgomery
 1 years ago
 Views:
Transcription
1 CSCE J Data Structures & Algorithms Dynamic programming  Knapsac problem Dr. Steve Goddard CSCE J Data Structures & Algorithms Giving credit where credit is due:» Most of slides for this lecture are based on slides created by Dr. David Luebe, University of Virginia.» Some slides are based on lecture notes created by Dr. Chuc Cusac, UNL.» I have modified them and added new slides. Dynamic Programming Another strategy for designing algorithms is dynamic programming» A metatechnique, not an algorithm (lie divide & conquer)» The word programming is historical and predates computer programming Use when problem breas down into recurring small subproblems Dynamic programming It is used when the solution can be recursively described in terms of solutions to subproblems (optimal substructure). Algorithm finds solutions to subproblems and stores them in memory for later use. More efficient than bruteforce methods, which solve the same subproblems over and over again. Summarizing the Concept of Dynamic Programming Basic idea:» Optimal substructure: optimal solution to problem consists of optimal solutions to subproblems» Overlapping subproblems: few subproblems in total, many recurring instances of each» Solve bottomup, building a table of solved subproblems that are used to solve larger ones Variations:» Table could be dimensional, triangular, a tree, etc. Knapsac problem (Review) Given some items, pac the napsac to get the maximum total value. Each item has some weight and some value. Total weight that we can carry is no more than some fixed number W. So we must consider weights of items as well as their value. Item # Weight Value 8 6 6
2 Knapsac problem There are two versions of the problem:.  napsac problem and. Fractional napsac problem. Items are indivisible; you either tae an item or not. Solved with dynamic programming. Items are divisible: you can tae any fraction of an item. Solved with a greedy algorithm. We have already seen this version  Knapsac problem Given a napsac with maximum capacity W, and a set S consisting of n items Each item i has some weight w i and benefit value b i (all w i, b i and W are integer values) Problem: How to pac the napsac to achieve maximum total value of paced items? 8  Knapsac problem: a picture  Knapsac problem This is a napsac Max weight: W = W = Items Weight w i 9 Benefit value b i 8 Problem, in other words, is to find max i T b subject to i i T w W The problem is called a  problem, because each item must be entirely accepted or rejected. In the Fractional Knapsac Problem, we can tae fractions of items. i 9  Knapsac problem: bruteforce approach  Knapsac problem: bruteforce approach Let s first solve this problem with a straightforward algorithm Since there are n items, there are n possible combinations of items. We go through all combinations and find the one with maximum value and with total weight less or equal to W Running time will be O( n ) Can we do better? Yes, with an algorithm based on dynamic programming We need to carefully identify the subproblems Let s try this: If items are labeled..n, then a subproblem would be to find an optimal solution for S = {items labeled,,.. }
3 Defining a Subproblem If items are labeled..n, then a subproblem would be to find an optimal solution for S = {items labeled,,.. } This is a reasonable subproblem definition. The question is: can we describe the final solution (S n ) in terms of subproblems (S )? Unfortunately, we can t do that. Defining a Subproblem w = b = w = b = w = b =8 w = Max weight: W = For S : Total weight: ; Maximum benefit: w = b = w = b = w = b =8 Weight Benefit Item # b = w i b i? w =9 b = For S : Total weight: Maximum benefit: 6 S S 9 8 Solution for S is not part of the solution for S!!! Defining a Subproblem (continued) Recursive Formula for subproblems As we have seen, the solution for S is not part of the solution for S So our definition of a subproblem is flawed and we need another one! Let s add another parameter: w, which will represent the exact weight for each subset of items The subproblem then will be to compute,w] Recursive formula for subproblems:, w] =, w] if w > w max{, w],, w w ] + b } It means, that the best subset of S that has total weight w is: ) the best subset of S  that has total weight w, or ) the best subset of S  that has total weight ww plus the item 6 Recursive Formula, w] =, w] if w > w max{, w],, w w ] + b } The best subset of S that has the total weight w, either contains item or not. First case: w >w. Item can t be part of the solution, since if it was, the total weight would be > w, which is unacceptable. Second case: w w. Then the item can be in the solution, and we choose the case with greater value.  Knapsac Algorithm for w = to W,w] = for i = to n i,] = for i = to n for w = to W if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w 8
4 Running time for w = to W,w] = for i = to n i,] = for i = to n for w = to W < the rest of the code > O(W) Repeat n times O(W) What is the running time of this algorithm? O(n*W) Remember that the bruteforce algorithm taes O( n ) 9 Example Let s run our algorithm on the following data: n = (# of elements) W = (max weight) Elements (weight, benefit): (,), (,), (,), (,6) Example () Example () i\w i\w for w = to W,w] = for i = to n i,] = Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i =
5 Example (6) i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = 6 Example (8) i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = Example (9) i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = 8 Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = 9
6 Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= ww i = Example () i\w : (,) : (,) : (,) i= : (,6) b i = w i = w=.. Example () i\w : (,) : (,) : (,) i= : (,6) b i = w i = w= w w i = if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w Example (6) i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i = w i = w= w w i = Example () i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i =6 w i = w=.. 6 6
7 Example (8) i\w if b i + i,ww i ] > i,w] i,w] = b i + i,w w i ] i,w] = i,w] i,w] = i,w] // w i > w : (,) : (,) : (,) i= : (,6) b i =6 w i = w= w w i = Comments This algorithm only finds the max possible value that can be carried in the napsac» I.e., the value in n,w] To now the items that mae this maximum value, an addition to this algorithm is necessary. 8 How to find actual Knapsac Items All of the information we need is in the table. n,w] is the maximal value of items that can be placed in the Knapsac. Let i=n and =W if i,] i,] then mar the i th item as in the napsac i = i, = w i i = i // Assume the i th item is not in the napsac // Could it be in the optimally paced napsac? 9 i\w i= : (,) : (,) : (,) : (,6) = b i =6 w i = i,] = i,] = Finding the Items while i, > if i,] i,] then mar the i th item as in the napsac i = i, = w i i = i i\w i= : (,) : (,) : (,) : (,6) = b i =6 w i = i,] = i,] = Finding the Items () while i, > if i,] i,] then mar the i th item as in the napsac i = i, = w i i = i i\w i= : (,) : (,) : (,) : (,6) = b i =6 w i = i,] = i,] = Finding the Items () while i, > if i,] i,] then mar the i th item as in the napsac i = i, = w i i = i
8 Finding the Items () i\w i= while i, > if i,] i,] then mar the i th item as in the napsac i = i, = w i i = i : (,) : (,) : (,) : (,6) = b i = w i = i,] = i,] = w i = Finding the Items () i\w i= while i, > if i,] i,] then mar the i th item as in the napsac i = i, = w i i = i : (,) : (,) : (,) : (,6) = b i = w i = i,] = i,] = w i = Finding the Items (6) i\w while i, > if i,] i,] then mar the n th item as in the napsac i = i, = w i i = i i= = : (,) : (,) : (,) : (,6) The optimal napsac should contain {, } Finding the Items () i\w while i, > if i,] i,] then mar the n th item as in the napsac i = i, = w i i = i : (,) : (,) : (,) : (,6) The optimal napsac should contain {, } 6 Review: The Knapsac Problem And Optimal Substructure Solving The Knapsac Problem Both variations exhibit optimal substructure To show this for the  problem, consider the most valuable load weighing at most W pounds» If we remove item j from the load, what do we now about the remaining load?» A: remainder must be the most valuable load weighing at most W  w j that thief could tae, excluding item j The optimal solution to the fractional napsac problem can be found with a greedy algorithm» Do you recall how?» Greedy strategy: tae in order of dollars/pound The optimal solution to the  problem cannot be found with the same greedy strategy» Example: items weighing,, and pounds, napsac can hold pounds Suppose item is worth $. Assign values to the other items so that the greedy strategy will fail 8 8
9 The Knapsac Problem: Greedy Vs. Dynamic Memoization The fractional problem can be solved greedily The  problem can be solved with a dynamic programming approach Memoization is another way to deal with overlapping subproblems in dynamic programming» After computing the solution to a subproblem, store it in a table» Subsequent calls just do a table looup With memoization, we implement the algorithm recursively:» If we encounter a subproblem we have seen, we loo up the answer» If not, compute the solution and add it to the list of subproblems we have seen. Must useful when the algorithm is easiest to implement recursively» Especially if we do not need solutions to all subproblems. 9 Conclusion Dynamic programming is a useful technique of solving certain ind of problems When the solution can be recursively described in terms of partial solutions, we can store these partial solutions and reuse them as necessary (memoization) Running time of dynamic programming algorithm vs. naïve algorithm:»  Knapsac problem: O(W*n) vs. O( n ) 9
If A is divided by B the result is 2/3. If B is divided by C the result is 4/7. What is the result if A is divided by C?
Problem 3 If A is divided by B the result is 2/3. If B is divided by C the result is 4/7. What is the result if A is divided by C? Suggested Questions to ask students about Problem 3 The key to this question
More informationOPRE 6201 : 2. Simplex Method
OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2
More informationSizeConstrained Weighted Set Cover
SizeConstrained Weighted Set Cover Lukasz Golab 1, Flip Korn 2, Feng Li 3, arna Saha 4 and Divesh Srivastava 5 1 University of Waterloo, Canada, lgolab@uwaterloo.ca 2 Google Research, flip@google.com
More informationThe Set Data Model CHAPTER 7. 7.1 What This Chapter Is About
CHAPTER 7 The Set Data Model The set is the most fundamental data model of mathematics. Every concept in mathematics, from trees to real numbers, is expressible as a special kind of set. In this book,
More informationMEP Y9 Practice Book A
1 Base Arithmetic 1.1 Binary Numbers We normally work with numbers in base 10. In this section we consider numbers in base 2, often called binary numbers. In base 10 we use the digits 0, 1, 2, 3, 4, 5,
More information1 Sets and Set Notation.
LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most
More informationMining Data Streams. Chapter 4. 4.1 The Stream Data Model
Chapter 4 Mining Data Streams Most of the algorithms described in this book assume that we are mining a database. That is, all our data is available when and if we want it. In this chapter, we shall make
More informationFindTheNumber. 1 FindTheNumber With Comps
FindTheNumber 1 FindTheNumber With Comps Consider the following twoperson game, which we call FindTheNumber with Comps. Player A (for answerer) has a number x between 1 and 1000. Player Q (for questioner)
More informationMEP Pupil Text 12. A list of numbers which form a pattern is called a sequence. In this section, straightforward sequences are continued.
MEP Pupil Text Number Patterns. Simple Number Patterns A list of numbers which form a pattern is called a sequence. In this section, straightforward sequences are continued. Worked Example Write down the
More informationRecall this chart that showed how most of our course would be organized:
Chapter 4 OneWay ANOVA Recall this chart that showed how most of our course would be organized: Explanatory Variable(s) Response Variable Methods Categorical Categorical Contingency Tables Categorical
More informationSUBGROUPS OF CYCLIC GROUPS. 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by
SUBGROUPS OF CYCLIC GROUPS KEITH CONRAD 1. Introduction In a group G, we denote the (cyclic) group of powers of some g G by g = {g k : k Z}. If G = g, then G itself is cyclic, with g as a generator. Examples
More informationRevised Version of Chapter 23. We learned long ago how to solve linear congruences. ax c (mod m)
Chapter 23 Squares Modulo p Revised Version of Chapter 23 We learned long ago how to solve linear congruences ax c (mod m) (see Chapter 8). It s now time to take the plunge and move on to quadratic equations.
More informationSelecting a Subset of Cases in SPSS: The Select Cases Command
Selecting a Subset of Cases in SPSS: The Select Cases Command When analyzing a data file in SPSS, all cases with valid values for the relevant variable(s) are used. If I opened the 1991 U.S. General Social
More informationPRINCIPAL COMPONENT ANALYSIS
1 Chapter 1 PRINCIPAL COMPONENT ANALYSIS Introduction: The Basics of Principal Component Analysis........................... 2 A Variable Reduction Procedure.......................................... 2
More informationThe Set Covering Machine
Journal of Machine Learning Research 3 (2002) 723746 Submitted 12/01; Published 12/02 The Set Covering Machine Mario Marchand School of Information Technology and Engineering University of Ottawa Ottawa,
More informationHow many numbers there are?
How many numbers there are? RADEK HONZIK Radek Honzik: Charles University, Department of Logic, Celetná 20, Praha 1, 116 42, Czech Republic radek.honzik@ff.cuni.cz Contents 1 What are numbers 2 1.1 Natural
More informationINTEGER PROGRAMMING. Integer Programming. Prototype example. BIP model. BIP models
Integer Programming INTEGER PROGRAMMING In many problems the decision variables must have integer values. Example: assign people, machines, and vehicles to activities in integer quantities. If this is
More informationTopic: Greedy Approximations: Set Cover and Min Makespan Date: 1/30/06
CS880: Approximations Algorithms Scribe: Matt Elder Lecturer: Shuchi Chawla Topic: Greedy Approximations: Set Cover and Min Makespan Date: 1/30/06 3.1 Set Cover The Set Cover problem is: Given a set of
More informationFrequent item set mining
Frequent item set mining Christian Borgelt Frequent item set mining is one of the best known and most popular data mining methods. Originally developed for market basket analysis, it is used nowadays for
More informationWHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT?
WHAT ARE MATHEMATICAL PROOFS AND WHY THEY ARE IMPORTANT? introduction Many students seem to have trouble with the notion of a mathematical proof. People that come to a course like Math 216, who certainly
More informationPrimes. Name Period Number Theory
Primes Name Period A Prime Number is a whole number whose only factors are 1 and itself. To find all of the prime numbers between 1 and 100, complete the following exercise: 1. Cross out 1 by Shading in
More informationJust want the standards alone? You can find the standards alone at http://corestandards.org/thestandards
4 th Grade Mathematics Unpacked Content For the new Common Core State Standards that will be effective in all North Carolina schools in the 201213 school year. This document is designed to help North
More informationCMA. Financial Reporting, Planning, Performance, and Control
Sixth Edition CMA Preparatory Program Part 1 Financial Reporting, Planning, Performance, and Control Manufacturing Input Variances Sample Brian Hock, CMA, CIA and Lynn Roden, CMA HOCK international, LLC
More informationA terminology model approach for defining and managing statistical metadata
A terminology model approach for defining and managing statistical metadata Comments to : R. Karge (49) 306576 2791 mail reinhard.karge@runsoftware.com Content 1 Introduction... 4 2 Knowledge presentation...
More informationGuide for Texas Instruments TI83, TI83 Plus, or TI84 Plus Graphing Calculator
Guide for Texas Instruments TI83, TI83 Plus, or TI84 Plus Graphing Calculator This Guide is designed to offer stepbystep instruction for using your TI83, TI83 Plus, or TI84 Plus graphing calculator
More informationSupport Vector Machines
CS229 Lecture notes Andrew Ng Part V Support Vector Machines This set of notes presents the Support Vector Machine (SVM) learning algorithm. SVMs are among the best (and many believe are indeed the best)
More informationPreferencebased Search using ExampleCritiquing with Suggestions
Journal of Artificial Intelligence Research 27 (2006) 465503 Submitted 04/06; published 12/06 Preferencebased Search using ExampleCritiquing with Suggestions Paolo Viappiani Boi Faltings Artificial
More informationOn SetBased Multiobjective Optimization
1 On SetBased Multiobjective Optimization Eckart Zitzler, Lothar Thiele, and Johannes Bader Abstract Assuming that evolutionary multiobjective optimization (EMO) mainly deals with set problems, one can
More informationMaximizing the Spread of Influence through a Social Network
Maximizing the Spread of Influence through a Social Network David Kempe Dept. of Computer Science Cornell University, Ithaca NY kempe@cs.cornell.edu Jon Kleinberg Dept. of Computer Science Cornell University,
More informationShort Cycles make Whard problems hard: FPT algorithms for Whard Problems in Graphs with no short Cycles
Short Cycles make Whard problems hard: FPT algorithms for Whard Problems in Graphs with no short Cycles Venkatesh Raman and Saket Saurabh The Institute of Mathematical Sciences, Chennai 600 113. {vraman
More information