Simulated Annealing and the Knapsack Problem

Transcription

1 Simulated Aealig ad the Kapsack Problem Bejami Misch December 19, The Kapsack Problem The kapsack problem is a classic ad widely studied computatioal problem i combiatorial optimizatio. We are give objects deoted by x i (i =1, 2,...,) each with correspodig weight w i. We ca imagie a perso carryig a backpack who has access to each of these objects. Ufortuately, this perso ca oly hold some maximum weight W i his backpack. The problem follows: what is the optimal assortmet of objects such that the perso maximizes the value of objects i his pack with the costrait of weight W? The applicatios of this simple problem ca be foud i may fields. I ecoomics, the kapsack problem is aalogous to a simple cosumptio model give a budget costrait. I other words, we choose from a list of objects to buy, each with a certai utility, subject to the budget costrait. Because the kapsack problem is a very geeral problem i combiatorial optimizatio, it has applicatios i almost every field. Oe of the most iterestig applicatios is i solvig the so-called Orego Trail problem. 2 Orego Trail Problem The Orego Trail game, a staple of elemetary school educatio i the 1990s, ivolves a player as a pioeer attemptig to travel from Missouri to the West coast i the 1848 while keepig his family safe ad alive. The game takes place i two stages. First, the player chooses his load out before headig out oto the Orego Trail, ad secod, the player the goes o the trail. The kapsack problem applies to the first part. Give several thousad dollars, the player first chooses a size of wago. Each wago ca carry a certai amout of weight. From here, i additio the budget costrait, we have a weight costrait. Next, we pick two travelig compaios from a large assortmet (icludig a baker, a chef, a fisherma, etc.); each choice is uique with a certai amout of weight, moey, ad utility value. The we choose which supplies to take. This process icludes food, weapos, medicie, comforts, ad more. The problem is to choose from all of these combiatios i order to maximize your chace of survivig to the West coast. This is a combiatorial optimizatio problem subject to multiple costraits ad may specific utility fuctios with regard to each item. Clearly this problem is a very advaced ad complicated versio of the kapsack problem, but I thik it paits a good picture as to how the simple kapsack problem ca be exted to somethig of very high complexity. 1

2 3 The 0-1 Kapsack Problem We will focus o the most basic ad commo versio of the kapsack problem. This versio limits each object to a umber of copies of 0 or 1. We ca mathematically formulate the problem as follows: Let there be items, x 1 to x where each x i has a value v i ad weight w i. The maximum weight that we ca carry i the bag is W. It is commo to assume that all values ad weights are greater tha zero. Maximize: 1 if the item is i the bag v i x i x i = 0 otherwise (1) subject to the costrait w i x i W, x i 0, 1 (2) Maximize the sum of the values of the items i the kapsack so that the sum of the weights must be less tha the kapsack s capacity. We ca approach this problem i two ways: a simple determiistic model ad a simulated aealig model. 4 Algorithm The algorithm solvig the Kapsack Problem is as follows. Imagie you are a thief lootig a house. You see several items aroud the house that you would like to steal, but you ca oly carry a certai amout of weight or you will be caught ruig away. You start with zero weight i your bag. 0. Record your curret assortmet of objects. We will call this ξ 1. ξ = {x 1,x 2,...,x }, x i 0, 1 1. You pick a item at radom with equal probability. 2. Holdig it i your had, you assess whether to add it to your backpack. If addig this ew object ito your bag pushes you over the weight limit, you select a item at radom with equal probability from those i your bag ad the oe from your had. You drop that item. You repeat this process util the objects i your bag are below the weight limit. Otherwise, add the item to your bag. Record the outcome as a trial assortmet. We will call this ξ Compare the value of the trial assortmet with the origial. These values are V 1 ad V 1,respectively. V (ξ) = v i x i, i =1, 2,...,. With some probability P, we accept the ew trial assortmet as the ew assortmet (We will aalyze two methods for determiig this probability). I other words, ξ 2 = ξ 1. Otherwise, we discard the trial assortmet ad set the origial assortmet as the ew assortmet. ξ 2 = ξ 1. 2

3 Method 1: The Determiistic Model. If the differece betwee the trial ad the origial is greater tha zero, set the trial assortmet as the ew assortmet. I other words, if V1 V 1 > 0, set P(Accept)=1, else set P(Accept)=0. This model does ot coverge to ay sigle optimum; rather it alterates betwee several local optima. Method 2: Simulated Aealig Model. For the theory behid Simulated Aealig, I refer the reader to Homework 10 ad 11. I quick review, simulated aealig ivolves a coolig schedule determied by β ad a statioary probability π β (ξ) = 1 Z(β) e βv (ξ). If we cool slowly eough, we ca avoid gettig trapped i local optima. We determie the probability P by mi{1, e (β ) } where = V1 V 1. β is determied more or less by trial ad error. 5 MATLAB Simulatio Now that we have the theory, we ca test it o a example. Let s assume that you have the followig objects. Our backpack ca oly carry a total weight of 20 kg. v w x 1 $50 5 kg x 2 $40 4 kg x 3 $30 6 kg x 4 $50 3 kg x 5 $30 2 kg x 6 $24 6 kg x 7 $36 7 kg The code for simulated aealig is as follows. This code ca be reappropriated easily for the determiistic model value=[ ]; weight=[ ]; TotalWeight=20; beta=0:.01:1; =1000; Kapsack( value, weight, TotalWeight, beta, ) fuctio X = Kapsack( value, weight, TotalWeight, beta, ) % Iput: value = array of values associated with object i. % weight = array of weights associated with object i. % TotalWeight = the total weight oe ca carry i the kapsack. % beta = vector of beta values for simulated aealig. % = umber of simulatios per beta value. % Output: FialValue = maximum value of objects i the kapsack. % FialItems = list of objects carried i the kapsack. % Etries i the vector correspod to object i % beig preset i the kapsack. 3

4 v=legth(value); W=zeros(1,v); Value=0; VW=0; a=legth(beta); =*oes(1,a); for :a b=beta(i); for j=2:(i) m=0; while m==0 c=ceil(rad*v); if W(c)==0 m=1; TrialW=W; TrialW(c)=1; while sum(trialw.*weight)>totalweight e=0; while e==0 d=ceil(rad*v); if TrialW(d)==1 e=1; TrialW(d)=0; f=sum(trialw.*value)-sum(w.*value); g=mi([1 exp(b*f)]); accept=(rad<=g); %Determiistic Model %if f>=0 if accept W=TrialW; VW(j)=sum(W.*value); else VW(j)=VW(j-1); Value=[Value VW(2:legth(VW))]; FialValue=Value(legth(Value)) x=0; for k=1:legth(w) if W(k)==1 x=[x k]; FialItems=x(2:legth(x)) 4

5 It is a good exercise to try ad solve this problem by had first. The fial solutio is $200 with objects 1, 2, 3, 4, 5. The simulated aealig code solved this correctly i every oe of my trials, but the determiistic model would sometimes get stuck at $176 with objects 1, 2, 4, 7. 6 Compariso of Models I theory the simulated aealig model should give us the correct optimum far more ofte tha the determiistic model. To test this hypothesis, we ca compare the two with a paired t-test. I this case, the ull hypothesis is that the differece betwee the two models is zero. I order to gather the data for the test, we ca ru each simulatio may times ad compare the two. I modified my code slightly to ru each simulatio 100 times. I the catalogued the FialValue for each model. I the used MATLAB to ru the paired t-test. [h,p,ci,stats]=ttest(fvsa,fvdet,left) FVSA correspods to the fial values of the simulated aealig, ad FVDet to those of the determiistic model. I received the followig output: h = 1 p = e-09 ci = stats = tstat: df: 99 sd: This shows that there is a sigificat differece betwee the two models. Because this is a oe-tailed test, we see that the simulated aealig model is sigificatly greater tha the determiistic model, meaig that the simulated aealig model provides us with a higher value o average. 5