The Power of Free Branching in a General Model of Backtracking and Dynamic Programming Algorithms

The Power of Free Brachig i a Geeral Model of Backtrackig ad Dyamic Programmig Algorithms SASHKA DAVIS IDA/Ceter for Computig Scieces Bowie, MD sashka.davis@gmail.com RUSSELL IMPAGLIAZZO Dept. of Computer Sciece UCSD russell@cs.ucsd.edu September 24, 2015 JEFF EDMONDS Dept. of Computer Sciece York Uiversity jeff@cs.yorku.ca Abstract The Prioritized brachig trees (pbt) model was defied i [ABBO + 05] to model recursive backtrackig ad a large sub-class of dyamic programmig algorithms. We exted this model to the prioritized Free Brachig Tree (pfbt) model by allowig the algorithm to also brach i order to try differet priority orderigs without eedig to commit to part of the solutio. I additio to capturig a much larger class of dyamic programmig algorithms, aother motivatio was that this class is closed uder a wider variety of reductios tha the origial model, ad so could be used to traslate lower bouds from oe problem to several related problems. First, we show that this free brachig adds a great deal of power by givig a surprisig ad geeral moderately expoetial (2 c for c < 1) upper boud i this model. This upper boud holds wheever the umber of possible data items is moderately expoetial, which icludes cases like k-sat with a restricted umber of clauses. We also give a 2 Ω( ) lower boud for 7-SAT ad a strogly expoetial lower boud for Subset Sum over vectors mod 2 both of which match our upper bouds for very similar problems. 1 Itroductio Algorithms are usually grouped pedagogically i terms of basic paradigms such as greedy algorithms, divide-ad-coquer algorithms, or dyamic programmig. There have bee may successful efforts over the years to formalize these paradigms ([BNR03, AB04, DI07, ABBO + 05, BODI11]). The models defied capture most of the algorithms ituitively i the give paradigm; lower bouds prove that these algorithms caot be improved (i terms of approximatio ratio or simpler form) without usig a stroger paradigm; ad lower bouds prove that other problems that ituitively should ot be solvable i the paradigm provably require expoetial size algorithms withi the model. This gives a formal distictio betwee the power of these differet paradigms. The geeral framework of all these model is that such a algorithm views the iput oe data item at a time, ad after each is forced to make a irrevocable decisio about the part of solutio related to the received data item, (for example, should the data item be icluded or ot i the solutio). The techique for lower bouds are iformatio-theoretic i that the algorithm does ot kow the rest of the iput before eedig to commit to part of the solutio. These techiques borrowed from those dealig with olie algorithms 1

[FW98], where a adversary chooses the order i which the items arrive. However, the differece here is that, while iputs arrive i a certai order, the algorithm determies that order, ot a adversary. A very successful formalizatio of greedy-like algorithms was developed i [BNR03] with the priority model. I this model, the algorithm is able to choose a priority order (greedy criteria) specifyig the order i which the data items are processed. [ABBO + 05] exteds these ideas to the prioritized brachig tree (pbt) model which allows the algorithm whe readig a data item to brach so that it ca make, ot just oe, but may differet irrevocable decisio about the read data item. Beig adaptive, each such brach is also allowed to specify a differet order i which to read the remaiig data items. The width of the program is the maximum umber of braches that are alive at ay give time. This relates to the ruig time of the algorithm. I order to keep this width dow the algorithm is allowed to later prue off a brach (ad back track) whe this computatio path does ot seem to be workig well. Surprisigly, the resultig model ca also simulate a large fragmet of dyamic programmig algorithms. This model was exteded further i [BODI11] to the prioritized brachig program model (pbp) as a way of ituitively capturig more of the power of dyamic programmig. Oe weakess of both the pbt ad pbp models is that they are ot closed uder reductios that itroduce dummy data items. I a typical reductio, we might traslate each data item for oe problem ito a gadget cosistig of a small set of correspodig data items for aother. However, may reductios also have gadgets that are always preset, idepedet of the actual iput. For example, whe we reduce 3-SAT to exactly-3-out-of-five SAT, we might itroduce two ew variables per clause, ad if the clause is satisfied by t 1 literals, we ca set 3 t of these ew variables to true. The set of ew variables ad which clauses they are i depeds oly o the umber of clauses i the origial formula, ot the structure of these clauses. Settig the value of oe of these dummy variables does costrai the origial satisfyig assigmet, but ot ay particular origial variable. Such a data item i the extreme is what we call a free data item. These are data items such that the irrevocably decisio that must be made about it does ot actually affect the solutio i ayway. The reaso addig these free data items to the pbt model icreases its power is because whe readig them the computatio is allowed to brach without eedig to read a data item that maters. I hidsight, ot allowig this is ot such a atural restrictio o the model ayway. We exted the pbt model to the prioritized Free Brachig Tree (pfbt) model by allowig the algorithm to also brach i order to try differet priority orderigs without readig a data time ad hece eedig to fix the part of the solutio relatig to this data item. I additio to closig the model uder reductios, it also allows for a much wider rage of atural algorithms. For example, oe such brach a algorithm for the Kapsack Problem might sort the data items accordig to price, i aother accordig to weight, ad i aother accordig to price per weight. The cotributios of this paper are both to develop a ew geeral proof techique for provig lower bouds i this seemigly ot much more powerful model ad for fidig ew surprisig algorithms showig that this chage i fact gives the model great deal of additioal (ufair) power. While the kow lower bouds for 3-SAT i the brachig program model are the tight 2 Ω(), we are oly able to prove a 2 Ω(1/2 ) lower boud (ad for 7 SAT ). The upper bouds show that this is due to the model s surprisig stregth, rather tha our iability to prove tight lower bouds. More precisely, ay problem whose set D of possible data items is relatively small, has a upper boud i the model of roughly expoetial i (). This covers the case of ay sparse SAT formula, where the umber of occurreces of each variable is bouded. There is some formal evidece that this is actually the most difficult case of SAT ([IPZ01]). The idea behid the ureasoable algorithm is as follows. Whe the algorithm reads data item D, it iadvertetly lears that all data items appearig before D i the priority orderig are ot i the istace. Beig allowed to try may such orderigs ad abado those that are ot lucky, the algorithm is able to lear all of the data items from the domai D that are ot i the istace ad i so doig effectively lear what the iput is. The algorithm is all-powerful with what it kows ad hece istatly kows the aswer. Havig 2

m log D free data item, i.e. those whose decisio does ot affect the solutio, further improves this upper boud i the free brachig model ad havig m = O( log D ), brigs the tree width dow to O( log D ) 3 eve without free brachig, i.e. i pbt. Figure 1 gives a summary of those results. Computatioal Problem D W i pbt W i pfbt W with m f.d.i. Geeral d 2 O() 2 O( log d) 2 O log d m+ log d k-sat O(1) clauses per var O(1) 2 O( log ) k-sat m clauses 2 k 1 2 O(k 1 2 3 1 m 1 3 log ) Matrix Iversio O(1) 2 Ω() 2 Ω( ) 2 Ω 7-SAT O(1) clauses per var Matrix Parity 2 2 Ω() 2 Ω Subset Sum 2Ω(/log ) SS o carries or i pfbt 2 Ω() m+ 2 m+ Figure 1: The summery of the results i the paper. Here pbt ad pfbt stad for the two models of computatio prioritized Brachig Trees ad prioritized Free Brachig Tree. The prioritizig allows the model to specify a order i which to read the data items defiig the iput istace. Brachig allow the algorithm to make differet irrevocable decisios about the data items read ad to try differet such orderigs. The free braches allows the tree to brach without readig ad hece eedig to fix the part of the solutio relatig to this data item. Free data items, f.d.i., are data items whose irrevocable decisio does ot effect the solutio. Here D is the full domai of all possible data items that potetially could be used to defie the problem. The algorithm that uses free brachig ad/or free data items i a surprisig ad ufair way depeds o this domai beig small. W is short for the width of the brachig tree. This is a measure of how much time the algorithm uses. The problem SS o carries is the Subset Sum problem with the o-stadard chage that the sums are doe over GF-2. A slightly better lower boud ca be obtaied for this because the reductio to it does ot eed dummy/free data items. Similarly the lower boud is obtaied i the model pfbt which restrict the use of such dummy data items. The secod way of gettig aroud the challeges of doig reductios is to restrict the model pfbt to pfbt. At ay poit durig the curret computatio path, a adversary is allowed to completely reveal ay data items that the algorithm may use as a free data item makig them ito kow data items. Kowig that this kow data item is i the actual iput istace, the algorithm o loger has ay direct eed to receive the data item. Besides if the algorithm did receive it, it would be forced to make a irrevocable decisio about it, which it may or may ot be prepared to do. The oly remaiig purpose of receivig a kow data item is to iadvertetly lear what data items are ot i the istace (because they would appear before the kow data items i the priority order). The pfbt model is a restrictio of the pfbt model that does ot allow the priority orderig to mix kow ad ukow data items together. Havig removed their power, the 2 Ω() width lower boud for the matrix parity problem goes through i the pfbt model o matter how may kow/free data items there are. The reductio the gives the 2 Ω() width lower boud for Subset Sum i pfbt model. The paper is orgaized as follows. Sectio 2 formally defies the computatioal problems ad the pfbt model. It also motivates the model because of its ability to express olie, greedy, recursive backtrackig, ad dyamic programmig algorithms. Sectio 3 provides the upper bouds arisig from havig free brachig ad/or free data items ad small domai size. We also exted this algorithm to obtai the 3

subexpoetial algorithms whe there are few clauses. I Sectio 4 we preset the reductios from the hard problems 7-SAT ad Subset Sum to the easy problems Matrix Iversio ad Matrix Parity. I Sectio 5 we provide a geeral techique for provig lower bouds withi pfbt ad the use the techique to obtai the results for the matrix problems. The remaiig sectios after Sectio 2, ca be read i ay order. 2 Defiitios Here we give a formal defiitio of a computatioal problem ad also state the k-sat, the Matrix Iversio, the Matrix Parity Problem, ad the Subset Sum problem i the priority algorithm framework: A Geeral Computatioal Problem: A computatioal problem with priority model (D, Σ) ad a family of objective fuctios f : D Σ R is defied as follows. The iput I = D 1, D 2,...,D D is specified by a set of data items D i chose from the give domai D. A solutio S = σ 1, σ 2,...,σ Σ cosists of a decisio σ i Σ made about each data item D i i the istace. For a optimizatio problem, this solutio must be oe that maximizes f(i, S). For a search problem, f(i, S) {0, 1}. k-sat: The iput cosists of the AND of a set of clauses, each cotaiig at most k literals ad the output is a satisfyig assigmet, assumig there is oe. A data item cotais the ame x j of the variable ad the list of all clauses i which it participates. Urestricted: The umber of potetial data items i the domai is D = 2 2(2)k 1, because there are 2(2) k 1 clauses that the give variable x i might be i. m clauses: Restrictig the iput istace to cotai oly m clauses, does ot restrict the domai of data items D. O(1) clause per variable: If k-sat is restricted so that each variable is i at most O(1) clauses, the D = [2(2) k 1 ] O(1) = O(1). The Matrix Iversio Problem: This problem was itroduced i [ABBO + 05] ad modified by us slightly. The matrix M {0, 1} is fixed ad kow to the algorithm. It is defied by a fixed matrix M {0, 1} that is o-sigular, has exactly seve oes i each row, at most K oes i each colum, ad is a (r, 7, c)-boudary expader (defied i Sectio 5.2), with K Θ(1), c = 3 ad r Θ(). The iput to this problem is vector b {0, 1}. The output is x {0, 1} such that Mx = b over GF-2. Each data item cotais the ame x j of a variable ad the value of the at most K bits b i from b ivolvig this variable, i.e. those for which M i,j = 1. Note D = 2 K. The Matrix Parity Problem (MP): The iput cosists of a matrix M {0, 1}. The output x i, for each i [], is the parity of the i th row, amely x i = j [] M i,j. This is easy eough, but the problem is that this data is distributed i a awkward way. Each data item cotais the ame x j of a variable ad the j th colum of the matrix. Note D = 2. Subset Sum (SS): The iput cosists of a set of -bit itegers, each stored i a data item. The goal is to accept a subset of the data items that adds up to a fixed kow target T. Agai D = 2. Subset Sum No Carries: Aother versio of SS does the GF-2 sum bit-wise so that there are o carries to the ext bits. 4

2.1 Defiitios of Olie, Greedy, Recursive Back-Trackig, ad Dyamic Programmig Algorithms Ideally a formal model of a algorithmic paradigm should be able to capture the complete power ad weakesses of the paradigm. I this sectio, we focus o the defiig characteristics of olie, greedy, recursive back-trackig, ad dyamic programmig. The formal defiitio of the pfbt model appears i Sectio 2.2. To make the discussio cocrete, cosider the Kapsack Problem specified as follows. A data item D i = p i, w i specifies the price ad the weight of the i th object. The decisio optio is σ i Σ = {0, 1}, where 1 ad 0 are the two optios, meaig to accept ad ot to accept the object, respectively. The goal is to maximize f(i, S) = i p iσ i as log as i w iσ i W. A determiistic olie algorithm for such a problem would receive the data items D i oe at a time ad must make a irrevocable decisio σ i about each as it arrives. For example, a algorithm for the Kapsack Problem may choose to accept the icomig data item as log as it still fits i the kapsack. We will assume that the algorithm has ubouded computatioal power based o what it kows from the data items it has see already. The algorithm s limitatio of power arises because it does ot kow the data items that it has ot yet see. After it commits to a partial solutio PS i = σ 1, σ 2,...,σ k by makig decisios about the partial istace PI i = D 1, D 2,...,D k see so far, a adversary ca make the future items PI future = D k+1, D k+2,...,d be such that there is o possible way to exted the solutio with decisios PS future = σ k+1, σ k+2,...,σ so that the fial solutio S = PS i, PS future is a optimal solutio for the actual istace I = PI i, PI future. This lower boud strategy is at the core of all the lower bouds i this paper. A greedy algorithm is give the additioal power to specify a priority orderig fuctio (greedy criteria) ad is esured that it will see the data items of the istace i this order. For example, a algorithm for the Kapsack Problem may choose to sort the data items by the ratio p i w i. I order to prove lower bouds as doe with the olie algorithm, [BNR03] defied the priority model. It allows the algorithm to specify the priority order without allowig it to see the future data items by havig it specifyig a orderig π O(D) of all possible data items. The algorithm the receives the ext data item D i i the actual iput accordig this order. A strogly adaptive algorithm is allowed to reorder the data items every time it sees a ew data item (also kow as fully adaptive priority algorithm i [BNR03, DI07]). A added beefit of receivig data item D is that the algorithm iadvertetly lears that every data item D before D i the priority orderig is ot i the iput istace. The set of data items leared ot to belog to the istace is deoted as PI out. It is a restrictio o the power of the greedy algorithm to require it to make a sigle irrevocable decisios σ i about each data item before kowig the future data items. I cotrast, backtrackig algorithms search for a solutio by first explorig oe decisio optio about a data item ad the if the search fails, the the algorithm backs up ad searches for a solutio with a differet decisio optio. The computatio forms a tree. To model this [ABBO + 05] defies prioritized Brachig Trees (pbt). Each state of the computatio tree specifies oe priority orderig, receives oe data item D i, ad makes zero, oe, or more decisios σ i about it. If more tha oe decisio is made, the tree braches to accommodate these. If zero decisios are made the the computatio path termiates. The computatio returs the solutio S that is best from all the otermiatig paths. A computatio of the algorithm o a istace I dyamically builds a tree of states as follows. Alog the path from the root state to a give state u i the tree, the algorithm has see some partial istace PI u = D1 u, Du 2,...,Du k ad committed to some partial solutio PS u = σ1 u, σu 2,...,σu k about it. At this state (kowig oly this iformatio) the algorithm specifies a orderig π u O(D) of all possible data items. The algorithm the receives the ext usee data item Dk+1 u i the actual iput accordig this order. The algorithm the ca make oe decisio σk+1 u that is irrevocable for the duratio of this path, to fork makig a umber of such decisios, or to termiate this path of the search all together. The computatio returs the solutio S that is best from all the otermiatig paths. 5

A curious restrictio of this model is that it does ot allow the followig completely reasoable kapsack algorithm. Retur the best solutio after ruig three parallel greedy algorithms, oe with the priority fuctio beig largest p i, aother smallest w i, ad aother largest p i w i. Beig fully adaptive, each state alog each brach is allowed to choose a differet orderig π O(D) for its priority fuctio. However, the computatio is oly allowed to fork whe it is makig differet decisios about a ewly see data item ad this oly occurs after seeig a data item. We geeralize their model by allowig the algorithm to fork without seeig a data item. This is facilitated by, i additio to the iput readig states described above, havig free brachig states which do othig except brach to some umber of childre. This ability itroduces a surprisig additioal power to the algorithms. We call this ew model prioritized Free Brachig Tree (pfbt) algorithms. See Sectio 2.2 for the formal defiitio. Though it is ot iitially obvious how, pfbt (pbt) are also quite good at expressig dyamic programmig algorithms. Dyamic programmig algorithms derive a collectio of sub-istaces to the computatioal problem from the origial istace such that the solutio to each sub-istace is either trivially obtaied or ca be easily computed from the solutios to the subsubistaces. Geerally, oe thiks of these sub-istaces beig solved from the smallest to largest, but oe ca equivaletly cotiue i the recursive backtrackig model where braches are prued whe the same subistace has previously bee solved. Each state u i the pfbt computatio tree ca be thought of the root of the computatio o the subistace PI future = D k+1, D k+2,...,d cosistig of the yet usee data items, whose goal is to fid a solutio PS future = σ k+1, σ k+2,...,σ so that S = PS u, PS future is a optimal solutio for the actual istace I = PI i u, PI future. Here, PI i u = D u,1,...,d u,k is the partial istace see alog the path to this state ad PS u = σ u,1,...,σ u,k is the partial solutio committed to. It is up to the algorithm to decide which pairs of states it cosiders to be represetig the same computatio. For example, i the stadard dyamic programmig algorithm for the Kapsack Problem, the data items are always viewed i the same order. Hece, all states at level k have the same subistace PI future = D k+1, D k+2,...,d yet to be see. However, because differet partial solutios PS u = σ u,1,...,σ u,k have bee committed to alog the path to differet states u, their computatio tasks are differet. As far as the future computatio is cocered, the oly differece betwee the differet subproblems is the amout W = W i k w iσ u,i of the kapsack remaiig. Hece, for each such W, the algorithm should kill off all but oe of these computatio paths. This reduces the width from 2 k to beig the rage of W. For a give value W, which of the paths should be kept is clearly the oe that made the best iitial partial solutio PS u = σ u,k+1,...,σ u, accordig to maximizig i k p iσ u,i. A additioal challege i implemetig the stadard dyamic programmig algorithm for the Kapsack Problem i the pbt model is the followig. I the model, the algorithm does ot kow which path should live util it kows partial istace PI i u, however, after this partial istace is received, the pbt model does ot allow cross talk betwee these differet paths. This is where the ubouded computatioal power of the pbt model comes i hady. Idepedetly, each path, kowig PI i u, kows what every other paths i the computatio is doig. Hece, it kows whether or ot it should be termiated. If required, it quietly termiates itself. I this way, pbt is able to model may dyamic programmig algorithms. However, Bellma-Ford s dyamic programmig algorithm for the shortest paths problem caot be expressed i the model. Each state i the computatio tree at depth k will have traversed a sub-path of legth k i the iput graph from the source ode s, ad the future goal is to fid to a shortest path from the curret ode to the destiatio ode t. States i the computatio tree whose sub-path ed i the same iput graph ode v are cosidered to be computig the same subproblem. However, because the differet states u have read differet paths PI i u = D u,1,...,d u,k of the iput graph, they caot kow what the other computatio states are doig. Without cross talk, they caot kow whether they are to termiate or ot. [BODI11] formally proves that pbt requires expoetial width to solve shortest paths ad defies aother model called prioritized Brachig Programs (pbp) which captures this otio of memoizatio by mergig computatio states that are computig the same subproblem. The computatio the forms a DAG istead 6

of a tree. 2.2 The Formal Defiitio of pfbt We proceed with a formal defiitio of a pfbt algorithm A. O a istace I, a pfbt algorithm builds a computatio tree T A (I) as follows. Cosider a state u p of the tree alog a path p. It is labeled with PI p, PS p, f p, costitutig the partial istace, the partial solutio, ad the free brachig see alog this path. The partial iformatio about the istace is split ito two types PI p = PIp i, PIp out. Here PIp i = D p,1,...,d p,k ad PSp = σ p,1,...,σ p,k, where D p,i D is the data item see ad σ p,i Σ is the decisio made about it i the i th iput readig state alog path to p. PIp out D deotes the set of data items iadvertetly leared by the algorithm to be ot i the iput istace. Fially, f p = f p,1,...,f p,k, where fi N idicates which brach is followed i the i th free brachig state. Each such state u p is either a iput readig or free brachig state. A iput readig state u p first specifies the priority fuctio with the orderig π A (u p ) 2 D of the possible data items. Suppose D p,k+1 is the first data item from I \ PIp i accordig to this total order. The iput readig state u p must the specify the decisios c A (u p, D p,k+1 ) Σ to be made about D p,k+1. 1 For each σ p,k+1 c A (u p, D p,k+1 ), the state u p has a child u p with D p,k+1 added to PIp i, each data item D appearig before D p,k+1 i the orderig π A (u p ) added to PIp out, ad σ p,k+1 added to PS p. A free brachig state u p must specify oly the umber F A (u p ) of braches it will have. For each f p,k +1 F A (u p ), the state u p has a child u p with f p,k +1 added to f p. The width W A () of the algorithm A is the maximum over istaces I with data items, of the maximum over levels k, of the umber of states u p i the computatio tree at level k. Though this completes the formal defiitio of the model pfbt, i order to give a better uderstadig of it, we will ow prove what such a algorithm kows about the iput istace I whe i a give state u p. This requires uderstadig how the computatio tree T A (I) chages for differet istaces I. If these T A (I) were allowed to be completely differet for differet istaces I, the for each I, T A (I) could simply kow the aswer for I. Lemma 1 proves that the partial istace PI p = PIp i, PIp out costitutes the sum kowledge that algorithm A kows about the istace. Namely, if we switched algorithm A s iput istace from beig I to beig aother istace I cosistet with this iformatio, the A would remai i the same state u p. Defie I PI p to mea that istace I is cosistet with p, i.e. cotais the data items i PI i p ad ot those i PI out p. Lemma 1. Suppose that two iputs I ad I are both cosistet with the partial istace PI, i.e. I PI ad I PI. If there exists a path p T A (I) idetified with PI p, PS p, f p, with PI p = PI, the there exists a path p T A (I ) idetified with the same PI p, PS p, f p = PIp, PS p, f p. Proof. The proof uses the fact that pfbt cosiders determiistic algorithms that explore a ukow istace ad give the same kowledge they always make idetical decisios. This could be prove by lookig at the iductive structure of the computatio tree T A (I). Recall that T A (I) is the tree of the thigs algorithm A tries o iput I. Istead, we will cosider every actio A will do over all possible istaces. We will refer to this as the super decisio tree view the algorithm A. Just as with T A (I), its states u p are idetified with PI p, PS p, f p. As with the computatio tree T A (I), the super decisio tree braches whe the algorithm tries differet irrevocable decisios PS p ad whe free brachig i order to try differet priority orderigs. I additio, the super decisio tree also braches depedig o which ext data item D the algorithm receives. The set of paths i this super decisio tree is the the uio {p = PI p, PS p, f p I such that p is a path i T A (I)} of the paths from the differet computatio trees T A (I). 1 If oe wated to measure the time util a solutio is foud, the oe would wat to specify the order i which these decisios σ p,k+1 c A(u p, D p,k+1 ) were tried. 7

2.3 The Formal Defiitio of pfbt We ow formally defie the restricted model pfbt. Recall that free data items are defied to be additioal data items that have o bearig o the solutio of the problem. The reaso addig these free data items to the pbt model icreases its power is that whe the algorithm reads data item D, it iadvertetly lears that all data items appearig before D i the priority orderig are ot i the istace. This allows a pbt algorithm to solve ay computatioal problem with width W = ( log D ) 3. Though this makes the provig of lower bouds i the model eve more impressive, this aspect of the model is clearly ot practical. The pfbt model restricts the algorithm i a way that excludes the possibility of it implemetig this impractical algorithm while still allowig ay practical algorithm that is i pfbt to still be i pfbt. A data item is said to be kow at some poit i a computatio path if the algorithm kows that it is i the actual iput istace eve though it has ot yet received it or made a irrevocable decisio about it. Such items ca arise i a umber of ways: 1) It might be a free data item defied i the computatioal problem. 2) It might be a dummy data item ad the algorithm kows that the iput istace is comig from the reductio. 3) A lower boud adversary, worried that the data item may be used as free data item, may iform the algorithm that this data item is i the iput istace. I cotrast, a data item is said to be ukow if at this poit i a computatio path, the algorithm still does ot kow if it is i the iput istace. Classify iput readig states u p of the computatio tree ito three types. It is said to be a ukow readig state if it puts all of the possible ukow data items before the kow oes. It is said to be a kow readig state if it puts all of the kow oes before the possible ukow oes. Fially, it is said to be a mixed readig state if it itertwies the two types of data items together i the orderigs. The model pfbt is defied to be the same as pfbt except that mixed readig states are ot allowed. We argue that ay practical algorithm that is i pfbt is still i pfbt by arguig that there is o (direct) reaso that a algorithm should wat to receive a kow data item. Already kowig that it is i the istace, it gais o ew iformatio, but it is still required to make a irrevocable decisio about it, which it may or may ot be prepared to do. After the adversary makes a data item kow, the algorithm should readjust its strategy so that it either uses kow readig states to get these data items out of the way, or uses ukow readig states to avoid receivig them util the ed whe kowig the etire iput istace is easy to make ad irrevocable decisio about them. The flaw i this argumet is that mixed readig states are useful as described above. The surprisig algorithm from Theorem 2 uses mixed readig states to lear every data item that is ot i the istace without receivig a sigle real ukow data item. No real poly-time algorithm, however, is able to do this. First, it requires the algorithm to keep track of a expoetial amout of iformatio. Secod, it requires the algorithm to have ubouded computatioal power with which to istatly compute the solutio for the ow kow istace. 3 Upper Bouds I this sectio, we provide a surprisig algorithm for ay computatioal problem where the algorithm has access either to free brachig ad/or free data items, ad the computatioal problem has a small set of possible data items D. Next we will exted the geeric algorithm to a algorithm for the k-sat problem with oly m clauses. Theorem 2 (Algorithm with Free Brachig ad/or Free Data Items). Cosider ay computatioal problem with kow iput size ad D = d potetial data items. There is a pfbt algorithm for this problem with width W A () = 2 O( log d). Whe m free data items are added to the problem, the required width decreases to 2 O( log d m+ log d ). Whe m = O( log d), it decreases to O( log d) 3 eve without free brachig, i.e. i pbt. 8

Proof. We first cosider the case without free data items. We costruct the determiistic algorithm by provig that for every fixed iput istace, the probability of our radomized algorithm fidig a correct solutio is less tha oe over the umber ( d) d of istaces. Hece, by the uio boud we kow that the probability that a radomly chose settig of all to coi flips fails to work for all iput istaces is less tha oe. Hece, there must exist a algorithm that solves every iput istace correctly. I its first stage, the algorithm lears what the iput istace is, ot by learig each of the data items i the istace but by learig the complete set of D data items that are ot i the iput istace. It does this without havig to commit to values for more tha l = O( log d) variables. The algorithm braches whe makig a decisio about each of these variables. Hece, the width is Σ l = 2 O( log d). I its secod stage, the algorithm uses its ubouded computatioal power to istatly compute the solutio for the ow kow istace. This solutio will be cosistet with oe of its Σ l braches. We ow focus o the how the algorithm ca use free braches to lear lots of data items that are ot i the istace. The algorithm radomly orders the potetial data items ad lears the first oe that is i the iput istace. The items earlier i the orderig are leared to be ot i the istace ad are added to PI out. This is expected to cosist of a 1 fractio of the remaiig potetial items. However, if istead, the algorithm free braches to repeat this experimet F = 2 l = 2 O( log d) times, the we will see below that with high probability at least oe of these braches will result a i Θ( l(f) ) = Θ( log d ) fractio of the remaiig potetial data items beig added to PI out. The algorithm might wat to keep the oe brach that performs the best, but this would require cross talk betwee the braches. Istead, it keeps ay brach that performs sufficietly well. The threshold q i is carefully set so that the expected umber of braches alive stays the costat r from oe iteratio to the ext. algorithm pf BT(, d) pre cod : is the umber of data items i the iput istace ad d is the umber of potetial data items post cod : Forms a pfbt tree such that whp oe of the leaves kows the solutio begi l = O( log d) F = 2 l r 0 = r = O(l 2 log d) = O( log d) 2 Width (2r) F Σ l = 2 O( log d) Free brach to form r 0 braches Loop: i = 1, 2,...,l loop ivariat : The curret states i the pfbt tree form a r i 1 Σ i 1 rectagle. The r i 1 rows were formed usig free braches. For each such row, the states kows a set PIi 1 i of i 1 of the data items kow to be i the iput istace. The colum of Σ i 1 states withi this row were formed from decisio braches i order to make all Σ i 1 possible decisios o these i 1 variables. Except for makig differet decisio, these states are idetical. Each of the r i 1 rows of states also kows a set PIi 1 out of the data items kow to be ot i the istace. PIi 1 out has grow large eough so that there are oly d i 1 = d PIi 1 i PIout i 1 remaiig potetial data items. Each state free braches with degree F formig a total of r i 1 F rows For each of these r i 1 F rows of states 9

Radomly order the d i 1 remaiig potetial data items D = the first data item that is i the iput q = # of data items before D i the order q i = Θ( d i l(f) ), set so that Pr(q q i ) = 1 F if(q q i ) the add D to PI i i 1 ad these q earlier data items to PIout i 1 d i = d i 1 q i 1 remaiig potetial data items Make a decisio brach to decide each possibility i Σ about D else Abado this brach ed if ed for r i = the total # of these r i 1 F rows that survive if( r i [1, 2r] ) abort ed loop We kow what the iput istace is by kowig all the data items that are ot i it. We use our ubouded power to compute the solutio. This solutio will be cosistet with oe of our Σ l states. ed algorithm The first thig to esure is that d l = d PIl i PIout l becomes l so the remaiig possible data items must all be i the istace. At the begiig of the i th iteratio, there are d i 1 remaiig potetial data items. Let q deote the umber of these data items, chose before the first appears that is i the istace. The algorithm radomly chooses oe of these to be first i the orderig. It is i our fixed iput istace with probability at most d i 1. Coditioal o this oe ot beig i the istace, the ext has probability at most d i 1 1 of beig i the istace, while the last of cocer has probability at most d i 1 q 1 = d i. Exp[q] d i. Pr[q q i] (1 d i ) q i e q i/d i. (This approximatio is good util the ed game. See below for how the time for that is bouded.) Because we have F chaces ad we wat the expected umber of these to survive to be oe, we set q i = d i l(f) so that this probability is 1 F. For the states that achieve q q i, the umber of remaiig potetial data items is d i = d i 1 q i 1 d i 1 d i l(f) = d i 1 d il = d i 1. 1 l Hece, i the ed d l = d d (1 l )l e l2 d = 2 l d e < 1. Before this occurs, the iput istace is kow. As is ofte the case, the last 20% of the work takes 80% the time. The approximatio (1 i d i ) q i e iq i /d i fails i the ed game whe the umber of remaiig possible data items d i gets close to the umber of usee iput items i, say d i 2 i. This meas the umber of data items that must still be elimiated is at most i. Not beig doe meas that d i i +1 givig that Pr[q q i ] (1 i d i ) q i 1 q i d i. Settig this to F 1 ad solvig gives that q i log 2 (F). If this may data items are elimiated log 2 (d i ) = O( log d) log 2 (d i ) each roud the the umber of required rouds to elimiate the remaiig i data elemets is at most i q i O( log d). This at most doubles the umber of rouds already beig cosidered. We must also boud the probability that the algorithm aborts to be less tha oe over the umber d of istaces. It does so if r l [1, 2r] because it eeds at least oe brach to survive ad we do t wat the computatio tree to grow too wide. Lemma 3 proves that this failure probability is at most 2 Θ(r/(m+l)2), which is d as log as r = Θ((m + l) 2 log d). This is r = Θ( log d) 2 whe m = 0 ad grows to Θ( log d) 3 whe m = O( log d). Free data items provide eve more power tha free brachig does. Just as doe above the set PIi out of data items kow ot to be i the istace grows every time oe of these data items is received, however, because they have o bearig o the solutio of the problem, the algorithm eed ot brach makig all 10

possible decisios Σ m about them. The algorithm is idetical to the oe above, except that m = mi(m, ) of the free data items are radomly ordered ito the remaiig real data items. The ext data item received will either be oe of the real data items or oe of these m free data items. As before, let q deote the umber of data items before this received data item i the priority orderig, i.e. the umber added to PIi 1 out. Pr[q q i ] (1 +m d i ) q i e 2q i/d i. Durig this first stage, the algorithm wats to receive all m of the free data items ad oly receive l of the real data items. If a real data item is read after already havig received l of them, the the brach dies. Pr[q q i ad the data item received is a free oe] e 2q i/d i m +m. Still havig F = 2 l chaces ad watig the expected umber of these to survive to be +m )) 2 oe, we set q i = d i(l(f) l( m d il 3 so that this probability is 1 log d F. (l(f) = l will be set to O( m ) l( m +m ).) Hece, d i = d i 1 q i 1 d i 1 d, givig i the ed d 1 l m+l = d < 1, whe (1 l 3 3 )m+l e l(m+l) 3 log d l = Ω( m+ log d ). This gives a total width of W A() (2r) F Σ l = 2 O( log d m+ log d ) as required. The remaiig poit is that this ca be achieved without free brachig, i.e. i pbt, whe m = O( log d) ad W A () 2r = O( log d) 3. I this case, a brach termiates every time a real data item is received, i.e. l = 0. Oly F = 2 braches are eeded each time a free data item is received. Though we do ot have free braches, this ca be achieved by havig the algorithm brach o the Σ differet possible decisios for this received free data item. We ow boud the probability that the umber of rows startig at r goes to zero or expads past 2r. Lemma 3. Start with r rabbits. Each iteratio i [l], all the rabbits curretly alive have F babies (ad dies). Each baby lives idepedetly with probability 1 F. Hece, the expected umber of rabbits remaiig is agai r. The game succeeds if the populatio does ot die out ad there are ever more tha 2r rabbits. The probability of failure is at most 2 Θ(r/l2). r/2 Proof. Set h = l. Exp[r i ] = r i 1 ad usig Cheroff bouds the probability that it deviates by more tha h r i 1 is at most 2 Θ(h2). The probability that this does ot occur i ay of the l iteratios is at most l2 Θ(h2) = 2 Θ(r/l2). I such cases, assumig r i is always below 2r, each of the l iteratios chages r i by at most h 2r = r l. I reality, these chages perform a radom walk, because each of these chages is i a radom directio. But eve if they are all i the same directio, the total chage is at most r, which either zeros, or doubles r i. The width 2 O( log D ) of the algorithm described i the last sectio is 2 O( log ) for k-sat whe restricted to havig oly O(1) clauses per variable, because the the domai D of data items is o bigger tha O(1). However, for geeral k-sat problem, this result does ot directly improve the width because there are D = 2 2(2)k 1 potetial data items. Despite this, this algorithm ca surprisigly be exteded to 2 O(1 3 m 1 3 log ) whe the istace is restricted to havig oly m clauses. Theorem 4 (k-sat m-clauses). Whe there are oly m clauses, the k-sat ca be doe with 2 O(k 1 2 1 3 m 1 3 log ) width. For k = 3, this is 2 O(2 3 log ) whe there are oly m = Θ() clauses ad 2 o() whe there are oly m = o( 2 ) clauses. The problem is that there may be m = Θ( 3 ) clauses. Proof. The first step is to ask for all variables that appear i at least l = k 1 2 1 3m 2 3 clauses. There are at most km l = k 1 23m 1 1 3 of these. Brachig o each of these requires width 2 O(k 2 1 3 1 m 1 3 ) width. For each of the remaiig variables, oly kllog(2) bits are eeded to specify the k variables i each of the at most l clauses that they are i. Hece, the umber of possible data items for each of these is at most d = (2) kl. 11

Theorem 2 the gives a pfbt algorithm for this problem with width w = 2 O( log d) = 2 O( kllog ) = 2 O(k 1 2 1 3 m 1 3 log ). 4 Reductios from Hard to Easy Problems This sectio does reductios to show that if pbt or pfbt caot do well o the simple matrix problems the they caot do well o the hard problems either. Theorem 5 (Lower Bouds for Hard Problems). 7-SAT requires width 2 Ω() i pbt ad 2 Ω( ) i pfbt. Subset Sum requires width 2 Ω(/log ) i pfbt ad 2 Ω() i pfbt. Fially, Subset Sum without carries requires width 2 Ω() i pfbt. Proof. Lemma 6 gives a reductio to 7-SAT from the Matrix Iversio Problem i both the pbt ad the pfbt models. Lemma 7 gives a reductio i pfbt to Subset Sum (with ad without carries) from the Matrix Parity Problem with free data items. Lemma 8 gives a reductio from Subset Sum i the pfbt model to the Matrix Parity Problem with o free data items i the pfbt model. The results the follow from the lower bouds for the matrix problems give i Theorem 9. Lemma 6. [Reductio to 7-SAT] If 7-SAT ca be solved with width W i the pbt (pfbt) model, the the Matrix Iversio Problem (with O(log) free data items) ca be solved with the same width. Recall that the Matrix Iversio Problem is defied by a fixed ad kow matrix M {0, 1} with at most seve oes i each row ad K O(1) i each colum. The iput is a vector b {0, 1}. The output is x {0, 1} such that Mx = b. Each data item cotais the ame x j of a variable ad the value of the at most K bits b i from b ivolvig this variable, i.e. those for which M i,j = 1. Proof. Give a pbt (pfbt) algorithm for 7-SAT, we desig a pbt (pfbt) algorithm for the Matrix Iversio Problem as follows. Cosider a matrix iversio data item. Let x j be the variable specified ad b i be oe of the at most K bits from b ivolvig this variable, i.e. M i,j = 1. Because the i th row of M has at most seve oes, its cotributio to Mx = b is that the parity of these correspodig seve x variables must be equal to b i. This parity ca be expressed as the AND of 64 clauses ivolvig these seve variables. These clauses for each of the K bits b i are icluded i the 7-SAT data item correspodig to this matrix iversio data item. Note that as required for the 7-SAT problem, this costructed data item cotais a variable x j ad the list of clauses cotaiig this variable. Each clause cotais at most seve variables. As with a matrix iversio data item, the oly ew iformatio that a algorithm who is aware of the reductio lears from a ew 7-SAT data item is the value of the at most K bits b i from b ivolvig this variable. Fially, ote that the output of both the matrix iversio problem ad of 7-SAT is x {0, 1} such that Mx = b. Lemma 7. [Reductio to Subset Sum with free data items] If Subset Sum ca be solved with width W i pfbt, the the Matrix Parity Problem with the itroductio of m = O( log ) data items ca be solved with the same width. If the versio of SS without carries ca be solved the oly m = O() data items eed to be added to the Matrix Parity Problem. Recall that the Matrix Parity Problem is give as iput a matrix M {0, 1}. For each j [], there is a data item cotaiig the ame x j of a variable ad the j th colum of the matrix. The output x i, for each i [], is the parity of the i th row, amely x i = j [] M i,j. Proof. Give a pfbt algorithm for SS (Subset Sum), we desig a pfbt algorithm for the MP (Matrix Parity Problem) as follows. We map the matrix M istace for MP to a 2 + O( log ) iteger istace for SS. If you write these itegers i biary i separate rows j liig up the 2 i bit places ito colums tha 12

at least part of this will be the traspose of M. Let D j deote the MP data item cotaiig the variable ame x j ad the j th colum of the matrix M. This data item will get mapped to two SS data items D j, ad D j,+. Each such iteger is 4 log 2 bits log. Of these 2 are special, bit idex c j for each colum j ad r i for each row i of the matrix M. Cosecutive special bits are separated by at least 2 log 2 zeros so that the sum over all the umbers of the oe special bit does ot carry ito the ext special bit. More formally, Let c j = j 2 log ad r i = 2 log + i 2 log. To idetify the colum-iteger relatio, the c j th bit of D j, will be oe iff j = j. To store the j th colum of the matrix, for i [], the r th i bit is M i,j. The secod iteger D j,+ associated with data item D j is exactly the same as D j, except that the r th j bit of D j, storig the diagoal etry M j,j is complemeted. The SS istace will have aother O( log ) itegers i dummy data items, i order to deal with carries from oe bit to the ext. For each bit idex k [1, log ] ad each row i [], the iteger E i,k will be oe oly i the r i +k th bit. The target T will be oe i the c th j bit ad the r i + log th bit for each i, j []. Havig mapped a istace of the MP problem to a istace of the SS problem, we must ow prove that the solutios to these two istaces correspod to each other. Let S sum be a solutio to the SS istace described above. We first ote that for each j, S SS must accept exactly oe of D j,+ ad D j, because these are the oly itegers with a oe i the c th j bit ad the target T requires a oe i this bit. Let S MP be the solutio to the MP istace i which x j = 1 if ad oly if D j,+ is accepted by S SS. We ow prove that this is a valid solutio, i.e. i [], x i = j [] M i,j. Cosider some idex i. Because S SS is a valid solutio, we kow that its itegers sum up to T ad hece the r th i bit of the sum is zero. Because there is o carry to the r th i bit, we kow that the parity of the r th i bit of the itegers i S SS is zero. The dummy itegers E i,k are zero i this bit. For j i, both D j,+ ad D j, have M i,j i this bit ad as see exactly oe of them is i S SS ad hece these itegers cotribute M i,j to the parity. For j = i, it is slightly trickier. D i,, if it is i S SS, also cotributes M i,i to the parity, while D i,+ cotributes the complimet M i,i 1. I this first case, x i = 0 ad i the secod x i = 1. Hece, we ca simplify this by sayig that together D i, ad D i,+ cotribute M i,i x i. Combiig these gives that the parity of the r th i bit of the [ itegers i S SS is 0 = j i i,j ] M [ ] M i,i x i. This gives as required that xi = j [] M i,j. Ad hece, S MP is a valid solutio. Now we must go i the opposite directio. Let S MP be a solutio for our MP problem. Put D j,+ i S SS if x j = 1, otherwise put D j, i it. For each row i, let N = 2 log ad N i = N (x i + j [] M i,j ) ad let the biary expasio of N i be [N i ] 2 = N i, log,...,n i,0, so that Ni = k [0, log ] N i,k 2 k. For k 0, iclude the iteger E i,k i S SS if ad oly if N i,k = 1. We must ow prove that the itegers i S SS add up to T. The same argumet as above proves that the c th j ad the r th i bits of the sum are as eeded for T. What remais is to deal with the carries. The c th j bits will ot carry. The r th i bits i the D j,+ or D j, itegers of S SS add up to x i + j [] M i,j. For k [1, log ], the iteger i E i,k is oe oly i the r i +k th bit ad hece ca be thought of cotributig 2 k to the r th i bit. Together, they cotribute k [1, log ] N i,k 2 k, which by costructio is equal to N i N i,0. Because S MP is a solutio, N i = N (x i + j [] M i,j ) is eve, makig N i,0 = 0. The total cotributio to the r th i bit is the (x i + j [] M i,j )+N i = N = 2 log, which carries to be zeros everywhere except i the r i + log th bit. This agrees with the target T. What remais to prove is that the MP computatio tree ca state-by-state mirror the SS computatio tree. It is the m = O( log ) free data items that the MP problem has that allows this reductio to be straightforward. For each i [], the MP problem will have a free data item labeled D i,2 d ad for each i [] ad k [1, log ], it will have oe labeled E i,k. I each state of the computatio tree, the SS algorithm specifies a priority orderig of its data items D i,+, D i,, ad E i,k. The simulatig MP algorithm costructs as follows its orderig of its data items D i, D i,2 d, ad E i,k. If either D i,+ or 13

D i, have bee see yet, the the first occurrece of them i this orderig is replaced by D i. The secod occurrece of them ca be igored because it will ever come ito play. If at least oe of D i,+ or D i, has bee see already, the the occurrece of the other oe i this SS orderig is replaced by the free data item D i,2 d. Ay dummy data item E i,k i the SS orderig are replaced by the correspodig free data item E i,k i the MP orderig. If accordig to this SS priority orderig, SS receives the first of D i,±, the MP accordig to its mirrored orderig will receive D i. Note that MP lears the same iformatio about its istace that SS does. If o the other had, SS receives the secod of D i,± or receives a dummy data item E i,k, the MP will receive the correspodig free data item. The MP algorithm (ad we ca assume the SS algorithm) kows that its istace is comig from this reductio ad hece they kew before receivig it that this dummy/free data item is i the istace ad hece either gais ay ew iformatio from this fact whe it is received. The iformatio of iterest that they both iadvertetly lear is that all data items D i the orderig before this received data item are leared to ot be i the istace ad as such are added to PI out. Because the MP algorithm always lears the same iformatio that SS does, MP is able to cotiue simulatig SS s algorithm. If we are reducig from the carry free versio of Subset Sum the the dummy data items E i,k are ot eeded. Lemma 8. [Reductio to Subset Sum i pfbt ] If Subset Sum ca be solved with width W i the pfbt model, the the Matrix Parity Problem without free data items ca be solved i pfbt with the same width. Proof. Give a pfbt algorithm for SS, our goal is to costruct a pfbt algorithm for MP. Despite this, we will eed to talk about a adversary. Recall that a data item is said to be kow at some poit i a computatio path if the algorithm kows that it is i the actual iput istace eve though it has ot yet received it. Oe way that such a item ca arise is that a adversary, worried that the data item may be used as free data item, may iform the algorithm that this data item is i the iput istace. We map the matrix M istace for MP to a istace for SS as before with itegers D j,, D j,+, ad E i,k. We must assume that the Subset Sum algorithm, SSA, kows that it is receivig a iput from the reductio. Hece, whe it receives the first of D i,±, it does i fact kow that the secod of the pair is i the iput. Hece, at this poit i the curret computatio, the adversary desigates that this secod data item is a kow data item. The E i,k data items are cosidered to be kow from the begiig. The pfbt model the does ot allow SSA to have mixed readig states that itertwie the ukow ad kow data item together i its priority orderigs. Cosider a ukow readig state of SSA, i.e. oe that puts all of the possible ukow data items before the kow oes. The simulatig MP algorithm costructs its orderig of its data items D i by replacig the first occurrece of D i,± with D i. If SSA receives oe of D i,±, the MP receives D i. If SSA receives a kow data item, the this meas that the curret computatio path is doe because the iput istace is completely kow. Now cosider a kow readig state of SSA, i.e. oe that puts all of the kow oes before the possible ukow oes. Both SSA ad MP kow that these kow data items are i SSA s istace ad hece kow that SSA will receive the first oe i this order. I fact, there was o poit i this read state at all. If SSA forks i order to make more tha oe irrevocable decisio about this kow data item the MP makes this same braches usig a free brach. This free brach was oe of the iitial motivators of havig free braches. 5 Lower Bouds for the Matrix Problems This sectio will prove the followig lower bouds for the matrix problems. 14

Theorem 9 (Lower Bouds for the Matrix Problems). The Matrix Iversio Problem requires width 2 Ω() i the pbt model ad width 2 Ω( ) i the pfbt model. The Matrix Parity Problem requires width 2 Ω() i the pfbt model. If icluded i these problems are m free data items, the widths 2 Ω m+ ad 2 Ω 2 m+ are still eeded. This Sectio will be orgaized as follow. We will begi by developig i Sectio 5.1 a geeral techique for provig lower bouds o the width of the computatio tree of ay algorithm solvig a give problem i either the pbt or the pfbt models. Theorem 10 formally states ad proves that this techique works. We will the show how it is applied to get all of the results i Theorem 9. This proof still depeds o four lemmas specific to the problems at had. Sectio 5.2 proves Lemmas 12 ad 13 for the Matrix Parity Problem ad Sectio 5.2 proves Lemmas 16 ad 17 for the Matrix Iversio Problem. 5.1 The Lower Boud Techique As said, we begi developig a geeral techique for provig lower bouds for pbt ad pfbt models usig it to prove Theorem 9 subject to four lemmas proved later. For each iput istace I, the computatio tree T A (I) has height = I measured i terms of the umber of data items received. Let l be a parameter, Θ() or Θ( ) depedig o the result. We will focus our attetio o partial paths p from the root to a state at level l i T A (I). Recall that such states are uiquely idetified with PIp i, PIp out, PS p, f p. The partial istace PIp = PIp i, PIp out cosists of the l data items kow to be i the istace ad those iadvertetly leared to be ot i the istace. Lemma 1 proves that PI p costitutes the sum kowledge that algorithm A kows about the istace I i this state. With oly this limited kowledge, it is ulikely that the partial solutio PS p that the algorithm has irrevocably decided about the data items i PIp i are correct. Formally, we prove that Pr I [S(I) PS p I PI p ] is small, where I PI p is defied to mea that istace I is cosistet with p, i.e. cotais the data items i PIp i ad ot those i PIp out ad S(I) PS p is defied to mea that the decisios PS S are cosistet with the/a solutio for I. With free brachig, i depth oly l upper = O( log d), the ureasoable upper boud i Theorem 2 maages to lear the etire iput I ot by havig PIp i cotai all of its data items but by havig PIp out cotai all of the data items ot i it. This demostrates how havig PIp out extra ordiarily large, ca cause I PI p to give the algorithm sufficiet iformatio about the istace I that it ca deduce a correct partial solutio PS p causig Pr I [S(I) PS p I PI p ] to be far too big. Hece, we defie a path p to be bad if this is the case. A pfbt algorithm A is allowed to fork both i a free brachig state i order to try differet priority orderigs ad i a iput readig state i order to try differet irrevocable decisios. Each of the computatio paths p for a give tree T A (I) ca be uiquely idetified by a tuple of forkig idexes f = σ 1,...,σ l, f 1,...,f l, where σ i Σ is idicates the decisio made i the i th iput readig state ad f i N idicates which brach is followed i the i th free brachig state. Note that if the width of the computatio tree T A (I) is bouded by W A (), the each f i is at most W A (). This allows us to defie Forks pfbt = [Σ [W A ()]] l be the set of possible tuples of forkig idexes. I cotrast, a pbt algorithm is ot allowed free brachig, ad hece Forks pbt = Σ l. The oly differece betwee these two models that we will use is the sizes of these sets. Oce we fix such a tuple f (idepedet of a iput I), we ca the defie the algorithmic strategy A f used by the brachig program A to be the mappig betwee the iput I ad the PI i p, PI out p, PS p, f p idetifyig the state at the ed of the path idetified by tuple f. This ext theorem outlies the steps sufficiet to prove that the width of the algorithm must be high. Theorem 10 (The Lower Boud Techique). 1. Defie a probability distributio P o a fiite family of hard istaces. 15

For both the Matrix Parity ad Matrix Iversio Problems the distributio is uiform o the legal istaces with data items. 2. Oly cosider partial paths p from the root to a state at level l i the tree T A (I). l = Θ() or Θ( ) depedig o the result. 3. A path p is defied to be good if its set PI out p is good. For the Matrix Iversio Problems, PIp out is defied to be good if it is sufficietly big. For the Matrix Parity Problems, it oly must be sufficietly big with respect to at least oe variable x j. 4. Prove that if p is good path at level key level l, the Pr I [S(I) PS p I PI p ] pr good, amely that the iformatio about the istace I gaied from PI p = PIp i, PIp out is ot sufficiet to effectively make the irrevocable decisios PS p about the data items i i If the algorithm simply guesses the decisio PS p for each of the l data items i PI i p, the it is correct with probability Σ l = 2 l. For both the Matrix Parity ad the Matrix Iversio problems, we are able to boud pr good 2 Ω(l). See Lemmas 13 ad 17. 5. Prove that with probability at least 1 2, every set PIout p i T A (I) is sufficietly small so that it is cosidered good. Cosider a tuple of forkig idexes [ f Forks. Cosider some algorithmic strategy A f alog these forkig idexes. Prove that Pr I Af produces a PI out that is bad ] 1 pr bad 1 Note that 2 Forks gives us the probability 1 2 after doig the uio boud. p. 2 Forks. For the Matrix Iversio Problem, we are able to boud pr bad 2 Ω(). For the Matrix Parity Problem, we are able to do much better boudig it by 2 Ω(2). See Lemmas 16 ad 12. These three step gives that ay pfbt (pbt) algorithm A requires width W A () 1 2pr good. We ow prove that the techique works. Proof. of Theorem 10: Step 5 i the techique proves that for a radom istace I, the probability that T A (I) cotais a bad partial path is at most 1 2. Hece, ay workig algorithm A must solve the problem with probability at least 1 2 usig a full path whose first l levels is a good partial path. This requires that there exists a partial path p T A (I) such that the irrevocable decisios PS p made alog it are cosistet with the/a solutio S(I) of the istace, amely 1 2 Pr I [ p T A (I) l, such that p is good ad S(I) PS p ] This probability deals oly with paths i T A (I), which i tur depeds o the radomly selected istace I. I cotrasts, the Step 4 probability, Pr I [S(I) PS p I PI p ] pr good, talks about fixed sets PI p ad PS p that ca deped o the algorithm A as a whole but ot o our curret choice of I. To uderstad the algorithm A idepedet of a particular istace I, defie Paths = {p = PI p, PS p, f p I such that p is a partial path of legth l i T A (I ) } ad Paths g those that are good. Note that p T A (I) [p Paths ad I PI p ]. It follows that the previously metioed probability is bouded as follows. Pr I [ p Paths g, such that I PI p ad S(I) PS p ] The uio boud, coditioal probabilities, ad pluggig i the probability from Step 4 traslates this probability ito the followig. 16

Pr I [I PI p ad S(I) PS p ] p Paths g = Pr I [S(I) PS p I PI p ] Pr I [I PI p ] p Paths g pr good Pr I [I PI p ] p Paths g Traslatig from the algorithm A as a whole back to the just those paths i T A (I) requires uderstadig how the computatio tree T A (I) chages for differet istaces I. Lemma 11 proves that if p = PI p, PS p, f p is a path that algorithm A follows for some istace I ad I is cosistet with what is leared i this path, the this same p will be followed whe A is give istace I, amely p Paths ad I PI p, the p T A (I). This bouds the previous probability with the followig. pr good p P aths Pr I [p T A (I)] The width of T A (I) at level l is equal to the umber of paths of legth l i T A (I). This width is bouded by W A (). Hece, the above sum of probabilities is ca be iterpreted as follows. = pr good Exp I [width of T A (I) at level l] pr good W A () Together this gives us W A () 1 2pr good as required. We ow state a prove the lemma eeded i the above proof that cosiders how the computatio tree T A (I) chages for differet istaces I. Lemma 11. If p = PI p, PS p, f p is a path that algorithm A follows for some istace I ad I is cosistet with what is leared i this path, the this same p will be followed whe A is give istace I, amely p Paths ad I PI p, the p T A (I). Proof. Suppose p Paths ad I PI p. Recall p Paths meas that there exists a istace I such that p = PI p, PS p, f p is the label of a partial path of legth l i T A (I ). It follows that I PI p. This gives us everythig eeded for Lemma 1, amely we have two iputs I ad I that are both cosistet with the partial istace PI p, i.e. I PI p ad I PI p ad there exists a path p T A (I ) idetified with PI p, PS p, f p. This lemma the gives that p T A (I). Our lower boud results follow from this techique ad the four Lemmas specific to the problems at had. Proof. of Theorem 9: I all cases, the techique gives that W A () 1 2pr good 2 Ω(l). See Lemmas 13 ad 17. What remais is to esure that the probability pr bad of a bad path is at most 1 2 Forks. Lemmas 16 states that for the Matrix Iversio Problem, pr bad 2 Ω(). I the model pbt, Forks pbt = 2 l, givig Forks pbt pr bad 1 2 eve whe l Θ(). I the pfbt model, however, Forks pfbt = [2W A ()] l = [2 Θ(l) ] l = 2 Θ(l2), givig Forks pfbt pr bad 1 2 oly whe l Θ( ). Lemmas 12 states that for the Matrix Parity Problem pr bad 2 Ω(2). Hece we have o problem settig l Θ() either way. If icluded i this problem are m free data items the what chages is that istead of just cosiderig l levels of the computatio, we cosider the computatio up to the poit at which l real data items have bee received ad ay umber of free data items have bee received. This ivolves up to l + m levels i total. This icreases Forks pfbt from [2W A ()] l = 2 Θ(l2) to [2W A ()] l+m = 2 Θ(l(l+m)). Hece, whe pr bad 2 Ω(), l Θ ( m+ ) is eeded ad whe pr bad 2 Ω(2), l Θ ( 2 m+ ) is eeded. 17

5.2 The Matrix Parity Problem This sectio proves the required good ad bad path lemmas eeded for the Matrix Parity Problem. Here is a remider the relevat defiitios. MP with distributio P: The distributio P uiformly chooses a matrix M {0, 1} for the iput istace. The j th data item cotais the ame x j of a variable ad the j th colum of the matrix. The output x i, for each i [], is the parity of the i th row, amely x i = j [] M i,j. Height l: Defie l = ǫ to be the level to which the partial paths p are cosidered. Partitioig D: For each j [], let D j D deote the set of possible data items labeled by x j ad hece cotaiig the j th colum of the matrix. Similarly, for a give path p, PI out ca be partitioed ito PIj out D j. Defie PI j? D j to be the set of data items for which it remais ukow whether or ot it is i the istace I. If j PI i (i.e. PI i does ot cotai a data item from D j ), the PI j? = D j PIj out. If j PI i, the PI j? =, because havig received oe data item from D j, the algorithm kows that o more from this domai are possible. Good Path: Such a computatio path p ad the set PI out arisig from it are cosidered to be good if j PI i, PI? j q D j, where q is set to be 2 (1 ε)l = 2 (1 ε)ǫ. Because a 2 (1 ε)ǫ fractio of D j effectively meas that at most (1 ε)ǫ about the j th colum have bee revealed. The ext lemma the bouds the probability that a computatio path is bad. Cosider a tuple of forkig idices f Forks. Cosider some algorithmic strategy A f alog these forkig idexes haltig after l levels. Lemma 12. pr bad = Pr I [A f is a q-bad path] (3q) l. This gives pr bad 2 1 2 ǫ2 for the matrix parity problem as claimed. Proof. The algorithmic strategy A f follows oe path p util l data items have bee read. Durig each of these l steps, it chooses a permutatio of the remaiig data items ad the lears which of the items i the radomly chose iput appears first i this orderig. The item revealed to be i the iput is put ito PI i ad all those appearig before it i the specified orderig are put ito PI out. We will, however, break this process ito sub-steps, both i terms of defiig the orders ad i terms of delay fully choosig the iput util the iformatio is eeded. I each sub-step, the computatio A f specifies oe data item D to be ext i the curret orderig. The the coditioed distributio P radomly decides whether to put D ito the iput. Above we partitioed the data items ito D j D based o which variable x j it is about. Though the algorithm deals with all such D j i parallel, they really are idepedet. Whe takig a step i the j th game, A f selects oe data item D from PI j? = D j PIj out to be ext i the curret orderig. Give A f kows that exactly oe of the data items i this set are i the iput ad they are equally likely to be, P decides that D is i the iput with probability 1/ PI j?. If it is, the this jth game is doe. Otherwise, D is removed from PI j?. Give the statemet of the lemma beig proved bouds the probability that this path p followed is bad, we eed to assume that the algorithm A f is doig everythig i its power to make this the case. Hece, we say that A f lose this game if this path is good, i.e. if after PI i cotais l data items it is the case that j PI i, PI j? q D j. It order to make the game less cofusig ad to give A f more chace of wiig, we will ot stop the game whe PI i cotais l data items, but istead we will let him play all games to completio. He wis the j th game if he ca maage to shrik PI j? to be smaller tha q D j without addig j to PI i. If he accidetal does add j to PI i, the he loses this j th game. It is ok for him to lose l 18

of these games, because, durig the part of his computatio we cosider, we allow PI i to grow to be of size l. However, if he loses l+1 of the games, the we claim that this computatio path is good. At least oe of these lost games had its j added to PI i after we stopped cosiderig his computatio ad at the poit i time j PI i ad PI? j q D j. Now let us boud the probability that he is able to wi the j th game. Effectively what he is able to do i this game is to specify the oe order i which it will ask about the data items i D j. The algorithm wis this game if ad oly if the data item D j from D j that is i the iput lays i the last q fractio of this orderig. Because D j is selected uiformly at radom, the probability of the algorithm wiig this game is q. For j [], let X j be the radom variable idicatig whether A f wis the j th game ad let X = j [] X j deote the umber of games wo. We proved that Pr[X j = 1] = q. We also proved that Pr[path is bad] Pr[X l]. Defie µ = Exp[X] = q ad 1 + δ = 1 q (1 l ( ) ) so µ e that (1 + δ)µ = l. Cheroff bouds gives that Pr[X (1 + δ)µ] < δ (1+δ) (3q) l. 1+δ We ow prove that the partial iformatio about the iput I gaied from a good PI = PI i, PI out is ot sufficiet to effectively make irrevocable decisios PS about the data items i PI i. This is the oly result i the lower bouds sectio that depeds o aythig specific about the computatio problem ad it does ot deped o the specifics of the model of computatio. Lemma 13. If PI, PS, f is the label of a q-good path p of legth l, the pr good = Pr I [S(I) PS I PI] 1 q 2 l. Here q = 2 (1 ε)l, givig pr good 2 εl as eeded. Proof. Assume the data items i PI i have told the algorithm the first l colums of M, but required him to make irrevocable guesses PS of the parities x 1,...,x l of the first l rows of M. PI out beig q-good meas that j PI i, PI? j q D j, where D j PI? j D j = {0, 1} is a vector that remais possible for the j th colum of M. I order to be ice to the algorithm, we will give the algorithm all of M except for this colum j. The for each row i [1, l], we ow kow the parity j j M i,j of these colums ad kow the parity x i = j [] M i,j that the algorithm has promised for the etire row. Hece, we ca compute the required value for M i,j i order for the algorithm to be correct. Let A be the set of vectors α for the j th colum such that the algorithm is correct, amely A = {α {0, 1} i [1, l], α i = the required M i,j }. Note that A = 2 l because there is such a α for each way of settig the remaiig l bits of this colum. The adversary ow radomly selects α PI? j that will be the jth colum i the iput M. If the selected α is i A, the the algorithm wis. The probability of this, however, is A / PI? j [ 2 l] / [q2 ] = 1 q 2 l. This completes the lower boud for the Matrix Parity Problem. 5.3 The Matrix Iversio Problem This sectio proves the required good ad bad path lemmas eeded for the Matrix Iversio Problem. Here is a remider the relevat defiitios. Fixed Matrix M: The problem MI is defied by a fixed matrix M {0, 1} that is o-sigular, has exactly seve oes i each row, at most K oes i each colum, ad is a (r, 7, 3)-boudary expader, with K Θ(1), ad r Θ(). Boudary Expader: M is a (r, 7, c)-boudary expader if it has exactly seve oes i each row ad if for every set R [] of at most r rows, the boudary M (R) has size at least c R. The boudary is defied to cotai the colum j if there is a sigle oe i the rows R of this colum. Sectio 5.4 sets K = Θ(1), c = 3, ad r = Θ() ad proves that such a matrix exits. 19

MI with distributio P: The distributio P uiformly chooses a vector b {0, 1}. The j th data item cotais the ame x j of a variable ad the value of the at most K bits b i from b ivolvig this variable, i.e. those for which M i,j = 1. The output is x {0, 1} such that Mx = b over GF-2. Height l: Defie l to be the level to which the partial paths p are cosidered. If we are provig the 2 Ω() width lower boud for the pbt model, the set l = = Θ(). If we are provig 2 Ω( ) for 2 K+3 the pfbt, the set l = Θ( ). Good Path: Such a computatio path p ad the set PI out arisig from it are cosidered to be good if PI out Q. Here Q = r Kl = Θ(), which is reasoable give that the umber of possible data items is D = 2 K = Θ(). At each poit alog the computatioal path p, the algorithmic strategy A f kows that the iput is cosistet with the curret partial istace P I. Kowig what the algorithm lears from this is complex. Hece, we will chage the game by havig the adversary reveal extra iformatio to the algorithms so that at each poit i his computatio, what he kows is oly some r bits of the iput b. If the path is good, the this umber does ot exceed r, i.e. the first parameter of expasio. Lemma 14. Let PI = PI i, PI out be arbitrary sets of data items. I PI reveals at most r = K PI i + PI out bits of b. More formally, let B = {b {0, 1} I PI} be the set of istaces b I cosistet with the partial istace P I. The adversary ca reveal more iformatio to the algorithm, restrictig this set to B B, so that the vectors i B have fixed values i r of its bits ad all 2 r possibilities i the remaiig r bits. Note that whe PI out is good, the algorithm lears at most K PI i + PI out K l + (r Kl) = r bits of b. Proof. Recall Lemma 12 gives a sub-step iterpretatio of the algorithmic strategy A f followig oe path p. I each of these sub-steps, the algorithm specifies oe data item D from those that are still possible ad is told whether or ot D is i the iput. If it is the D is added to PI i ad if ot it is added to PI out. Whe the algorithm lears that data item D is i PI i, he lears lears the value of at the most K bits of b {0, 1} correspodig to the j th colum of M. Whe he lears that D is i PI out, he lears a settig i {0, 1} k that k K of bits i b do ot have. If k is large, the the immediate usefuless of this iformatio is ot clear. The adversary chooses oe of the k bits of b that the data item is about ad that the algorithm does ot yet kow the value of ad reveals that this bit of b has the opposite value from that give i the data item. The algorithm, seeig this as the reaso the data item is ot i the iput, lears othig more about the iput. I the ed, what the algorithm lears from I PI is K PI i + PI out bits of b. Lemma 15. Let PI = PI i, PI out deotes the curret state computatio. Let D be a data item that is still possibly i the iput I. It follows that Pr I [D I I PI] q = 1 2 K. Proof. The proof of Lemma 14 chages the game so that what the algorithm kows at each step of the computatio is some r of bits of the vector b {0, 1}. The coditioal distributio P uiformly at radom chooses the remaiig r bits. Each data item D specifies a variable x j ad up to K bits of the vector b. If this D disagrees with some of the r revealed bits of b, the D i ot still possible. Otherwise, 1 the probability that D is i the iput is 1, where K K is the umber of the specified bits that 2 K 2 K have ot bee fixed. The ext lemma the bouds the probability that the computatio path give by A f is bad. Lemma 16. pr bad = Pr I [A f is a Q-bad path] 2 Ω(). 20

Proof. Geerally, we cosider A f s computatio util it receives l data items. However, istead, let us cosider it for Q of these sub-steps. For each t [Q], let X t be the radom variable idicatig whether the t th such data item is added to PI i. By Lemma 15, Pr[X t = 1] q. Note that X = t [Q] X t deotes the resultig size of PI i. If X l, the by this poit i the computatio PI i l, so the part of the computatio we ormally cosider has already stopped, but PI out = Q PI i Q ad hece this computatio is cosidered to be good. It follows that Pr[path is bad] Pr[X < l]. Defie µ = Exp[X] = qq ad δ = 1 l qq so that (1 δ)µ = l. Cheroff bouds gives that Pr[X < (1 δ)µ] < e µδ2 /2 = e qq(1 l qq )2/2. We defied l r, Q = r Kl, q = 1 l, givig that 2 K+3 2 K qq 1 2. This gives pr bad e Q 2 K+3 2 Ω(). Lemma 14 proves that for good paths, I PI reveals at most r bits of b. We ow prove that this is ot sufficiet to effectively make irrevocable decisios PS about the data items i PI i. Lemma 17. If p is a good path, the Pr I [S(I) PS I PI] = pr good 2 Ω(l). Proof. Let R [] deote the idexes i of the at most r bits of b that have bee fixed by Lemma 14 from I PI ad let b R {0, 1} R be the values that these b R bits have bee fixed to. Let M R deote the correspodig rows of the matrix M. These rows impose the requiremet M R x = b R. I additio, PS commits the algorithm to the values of exactly l variables. Let Fixed [] deote the idexes j of these variables x j ad let x Fixed deote the values committed to them. Our probability distributio P o iputs I is uiform o the settigs of b. Because there is a bijectio mappig x = M 1 b betwee the settigs of b ad the settigs of x, it is equivalet to istead cosider the uiform distributio o settig of x. Our requiremet I PI o our iput whe cosiderig the distributio o b was that b R = b R. But whe cosiderig the distributio o x, this traslates over to the requiremet M R x = b R. The requiremet S(I) PS i both settigs is that x fixed = x fixed. A iterestig effect of doig this traslatio is that the equatios M ([] R) x = b ([] R) o loger matter. To coclude, this traslatio gives that [ pr good = Pr I [S(I) PS I PI] = Pr xfixed = x fixed M R x = b ] R x u{0,1} = {x x fixed = x fixed ad M R x = b R } {x M R x = b R } What we ow eed to compute is both the umber of solutios x to the subsystem of equatios M R x = b R ad the umber of these cosistet with x fixed = x fixed. Towards this goal, we will ru the followig Gaussia-like elimiatio algorithm. It selects, satisfies, ad elimiates equatios i R oe at a time by settig ad elimiatig a variable x j. We easily establish the loop ivariat by settig R 0 = R ad C 0 = []. GAUSSIAN-LIKE ELIMINATION R 0 R ad C 0 [] while (c R t > l) % Loop Ivariat: LI1: We have a set R t R of rows ad a set C t [] of colums with Fixed C t. Let M Rt,C t deote the submatrix with the selected rows ad colums. Let x Ct deote the vector of selected variables. LI2: pr good = {x C t x fixed = x fixed ad M Rt,C t x Ct = b Rt }. {x Ct M Rt,C t x Ct = b Rt } LI3: M Rt,C t is a (r, 7, 3)-boudary expader. 1. Pick x j (R t ) Fixed 21

2. Let i R t be the row i which x j appears. 3. x j value that satisfies M i x = b i 4. R t+1 R t i ad C t+1 C t j 5. t = t + 1 We prove as follows that this code maitais the loop ivariat. Assume by way of iductio that it is true for t. We ow prove that it is true for t+1. By the defiitio of M Rt,C t beig a (r, 7, 3)-boudary expader ad from the fact that R t R r, we kow that the boudary M (R t ) C t has size at least c R t, which by ot havig exited more is tha l. Hece, lie 1 is able to pick a variable x j (R t ) Fixed. The boudary M (R t ) is defied to cotai the colum j if there is a sigle oe i the rows R t of this colum. Let i R t be the row i which x j appears. The equatios M Rt i,c t x Ct = b (Rt i) do ot use this variable x j ad hece are ot effected by how it is set. The equatio M i,ct x Ct = b i ca be solved ad satisfied by settig the value of x j to be this liear combiatio of the variables i R t i. Beig satisfied, the equatio i is removed from R t+1 = R t i. Not effectig the remaiig equatios, the variable x j ca be removed from them, C t+1 = C t j. Beig set, the variable x j adds o degrees of freedom ad hece does ot effect the size of either {x Ct x fixed = x fixed ad M Rt,C t x Ct = b Rt } or of {x Ct M Rt,C t x Ct = b Rt }. Hece, the ratio of their sizes remais beig equal to pr good, maitaiig LI2. Deletig row i from M Rt,C t does ot effect it beig a (r, 7, 3)-boudary expader because the defiitio cosiders every subset of rows. Neither does deletig colum j because oce row i has bee deleted, this j th colum is all zeros. This proves that all the loop ivariats have bee satisfied. The Gaussia-like elimiatio algorithm termiates whe c R t l. The size of {x Ct x fixed = x fixed ad M Rt,C t x Ct = b Rt } is at most 2 Ct l because C t is the set of remaiig variables ad x fixed = x fixed fixes the value of l of them. The size of {x Ct M Rt,C t x Ct = b Rt } is at least 2 Ct l/3 because oly R t l 3 equatios restrict the values of the C t variables. LI2 the gives that pr good 2 C t l 2 C t l/3 = 2 2/3 l, completig the proof. 5.4 Boudary Expader Matrix Here we eed to prove that a square matrix A {0, 1} exists such that 1. Each row has exactly 7 oes. 2. Each colum has at most 7 oes, for some costat. 3. A is of full rak ad hece is o-sigular. 4. A is a (r, 7, 3) boudary expader with r θ(). This meas that for ay set I of at most r rows, the boudary A (I) has size is at least 3 I. The boudary is defied to cotai colum j if there is a sigle oe i the rows I of this colum. First we defie a distributio o biary matrices A, such that each row has exactly 7 oes ad each colum exactly 7 oes. The we show that the probability that A is ot a good boudary expader is less tha 1 2. Next we argue that the probability that A is ot full rak is also less tha 1 2. Hece, there must exist such a A that is both a good boudary expader ad of full rak. The we are doe because ay liearly idepedet rows of that matrix will iherit the expasio of the big matrix, is o-sigular, ad the umber of oes i each colum will be at most 7. The biary matrix is radomly chose by radomly choosig a perfect matchig i the followig bipartite graph. O the left side we have equatios, each with 7 slots. O the right side we have variables, each with 7 slots. A edge from a slot of equatio i to a slot of variable j puts a oe at matrix etry A[i, j]. The graph beig a perfect matchig esures that there are exactly 7 oes i each row ad 22

exactly 7 oes i each colum. This perfect matchig is chose radomly by fillig oe at a time each slot o the left by radomly pickig a umatched slot o the right ad match them. Lemma 18. For ay sufficietly large ad costat, the probability that matrix A {0, 1} is ot a (r, 7, 3) boudary expader is less tha 1 2. Proof. Let I deote a arbitrary set of rows I, I = t r, defiig t equatios. Each such equatio radomly chooses 7 of the variable slots for a total of 7t. Suppose that 0 i 7t of these variables slots have bee chose already. Let B deote the curret umber of boudary variables, i.e. the umber for which exactly oe of its slots have bee selected. Similarly, let D deote the umber that have duplicate slots filled. This leaves B D variables with oe of its slots filled. We will watch these values dyamically as more slots are chose. We eed to boud the probability of the bad evet that after i = 7t such steps B < 3t. We boud the probability that the ext step will icrease B by oe to be p + = 7 ( B D) 7 i 1 B + D This is because there are B D variables with all 7 of there slots empty ad fillig oe of these will make such a variable a boudary variable. Here 7 i is the total umber of slots curretly ufilled. We will ow prove that we may assume that at all times B + D 5t. Other tha B icreasig, the other two optios are as follows. If the ext slot chose is that of a boudary variable, B decreases ad D icreases, keepig B + D costat. If it is of a duplicate variable either chages. O the other had, if B ever icreases durig more tha 5t of the steps, the i the remaiig 7t 5t = 2t steps, B be uable to decrease below 3t as required for A to fail. We simplify this process by assumig that each step B either icreases or decreases by oe each step with the probability of decrease beig p = 1 p + B+D 5t. Remember that t r O(), so this is a arbitrarily small costat. The probability that this radom walk of legth 7t eds with B < 3t the is bouded by the probability that this decrease occurs at least 2t times which is p tbad ( ) ( ) 7t 7t p k (1 p ) 7t k 2 p 2t. k 2t k 2t We the do the uio boud to boud the probability that there exists a set of size t r whose boudary is at most 3t. r ( ) r ( ) e t ( ) 7e 2t ( ) 5t 2t r ( ) 6152 t t p = p tbad t t 2 t=1 t=1 t=1 This is strictly less tha 1 6152 t 2 as log as 1 4, which we ca assure by settig r = this case, such a expader exists. 24,608 Θ(). I Lemma 19. For ay sufficietly large ad costat, the probability that matrix A {0, 1} does ot have full rak is less tha 1 2. Proof. Suppose A is ot full rak. The there exits a ew equatio that is ot i the spa of the equatios give by the rows of A. Let F [] be the subset of variables such that this ew equatio is i F x i = 0. This F does i fact witess the fact that A is ot full rak iff each row of A, viewed as the values of the variables, satisfies this equatio. Whe F r Θ(), we will show that F likely fails to be such a witess because there is a row of A that cotais exactly oe variable from F ad hece i F x i = 1 for this row. I cotrast, whe F > r, we will show that F likely fails because there is a row that cotais 23

exactly seve variables from F ad hece whe doig Boolea algebra we also get that i F x i = 1. If all such F fail to be a witess the A is full rak. The amusig thig is that the case with small F is exactly the same boudary expasio property as proved i Lemma 18 ad hece the same proof with slightly differet umbers works. The oly differece is that we switch the roles of rows ad the colums ad the boudary does ot eed to be of size 3t but oly of size oe. Repeatig the statemet is that for ay set F of at most r colums, the boudary A (F) has size is at least 1. The boudary is defied to cotai row j if there is exactly oe oe i the colums F of this row. The required chages i the umbers are as follows. 7( B D) p + = 1 B + D 7 i p tbad ( 7 t k t=1 k 3.5 t t=1 1 3.5 t = 1 3.5t = 1 p ) p k (1 p ) 7 t k 2 ( ) 7 t p 3.5 t. 3.5 t r ( ) r ( e ) ( ) t p = p tbad 2 7 t 3.5t 3.5 t r ( ) 20t 3 t t t This is strictly less tha 1 20t 4 as log as 1 8, which we ca assure by settig r = 160 Θ(). I this case, all F smaller tha r fail to be witesses that A is ot full rak. We ow switch to provig that for F > r = c, F likely fails to be a witess because there is a row (equatio) such that all seve of its variables are from F. Cosider oe such row. Whe radomly selectig a variable for this row, the probability that it is i F is 1 c. The probability that all seve variables fall i F is 1. The probability that this fails to happe for all rows is ( 1 1 ). c 7 c The umber of differet sets 7 F [] is at most 2. Hece, probability that there exists a large witess F is at most 2 (1 1 c 7 ) < 1 4 for ay c 7. By Lemmas 18 ad 19 for reasoably large there exists a o-sigular biary matrix which is (r, 7, 3) boudary expader i which each colum sums up to a costat 7. t=1 6 Coclusios We defie the prioritized Free Brachig Tree (pfbt) model extedig the Prioritized Brachig Trees (pbt) model use to model recursive back-trackig ad a large sub-class of dyamic programmig algorithms. We give some surprisig upper bouds ad matchig lower bouds. Refereces [AB04] Spyros Agelopoulos ad Alla Borodi. O the power of priority algorithms for facility locatio ad set cover. Algorithmica, 40(4):pp.271 291, 2004. [ABBO + 05] Michael Alekhovich, Alla Borodi, Joshua Buresh-Oppeheim, Russell Impagliazzo, Aver Mage, ad Toia Pitassi. Toward a model for backtrackig ad dyamic programmig. I IEEE Coferece o Computatioal Complexity, pages 308 322, 2005. [AHI04] Michael Alekhovich, Edward A. Hirsch, ad Dmitry Itsykso. Expoetial lower bouds for the ruig time of DPLL algorithms o satisfiable formulas. I ICALP 2004, Turku, Filad., pages 84 96, July 2004. 24

[AS92] Noga Alo ad Joel Specer. The Probabilistic Method. Joh Wiley, 1992. [BBL04] [BCM05] [BEY98] [BNR03] Alla Borodi, Joa Boyar, ad Kim S. Larse. Priority algorithms for graph optimizatio problems. I WAOA, pages 126 139, 2004. Alla Borodi, David Cashma, ad Aver Mage. How well ca primal-dual ad local-ratio algorithms perform?. I ICALP, pages 943 955, 2005. Alla Borodi ad Ra El-Yaiv. Olie Computatio ad Competitive Aalysis. Cambridge Uiversity Press, 1998. Alla Borodi, Morte N. Nielse, ad Charles Rackoff. (Icremetal) Priority Algorithms. Algorithmica, 37(4):pp.295 326, 2003. [BODI11] Joshua Buresh-Oppeheim, Sashka Davis, ad Russell Impagliazzo. A stroger model of dyamic programmig algorithms. Algorithmica, 60(4):938 968, 2011. [Bol01] Bela Bollobas. Radom Graphs. Cambridge Uiversity Press, 2001. [CLRS01] [DI07] [FW98] [Hoe63] [IKR85] [IPZ01] Thomas H. Corme, Charles E. Leiserso, Roald L. Rivest, ad Clifford Stei. Itroductio to Algorithms, Secod Editio. The MIT Press ad McGraw-Hill Book Compay, 2001. Sashka Davis ad Russell Impagliazzo. Models of greedy algorithms for graph problems. Algorithmica, Novemeber 2007. Amos Fiat ad Gerhard Woegiger. Olie Algorithms: The State of the Art. Spriger, Septemer 1998. Wassily Hoeffdig. Probability iequalities for sums of boudded radom variables. J. of Amer. Statist. Assoc., 58(301):13 30, March 1963. Oscar H. Ibarra, Sam M. Kim, ad Louis E. Rosier. Some characterizatios of multihead fiite automata. Iformatio ad Cotrol, 67(1-3):114 125, 1985. Russell Impagliazzo, Ramamoha Paturi, ad Fracis Zae. Which problems have strogly expoetial complexity? J. Comput. Syst. Sci., 63(4):512 530, 2001. [KT05] Jo Kleiberg ad Eva Tardos. Algorithm Desig. Addiso Wesley, 2005. [MR95] [MU05] [MV04] Rajeev Motwai ad Prabhakar Raghava. Radomized Algorithms. Cambridge Uiversity Press, 1995. Michael Mitzemacher ad Eli Upfal. Probability ad Computig. Cambridge Uiversity Press, 2005. Jiri Matousek ad Ja Vodrak. The Probabilistic Method, Lecture Notes. Mauscript, August 2004. [PPZ99] Ramamoha Paturi, Pavel Pudlák, ad Fracis Zae. Satisfiability codig lemma. Chicago J. Theor. Comput. Sci., 1999. [RSZ02] Alexader Russell, Michael E. Saks, ad David Zuckerma. Lower bouds for leader electio ad collective coi-flippig i the perfect iformatio model. SIAM J. Comput., 31(6):1645 1662, 2002. 25

[Woe99] [Woe01] Gerhard J. Woegiger. Whe does a dyamic prgrammig formulatio guaratee the existace of fptas? I SODA, pages 820 829, 1999. Gerhard J. Woegiger. Exact Algorithms for NP-Hard Problems: A survey. I Combiatorial Optimizatio, pages 185 208, 2001. 26