5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected by a edge. How hard is it to decide whether G is coected? Specifically, how may etries i the adjacecy matrix do we have to examie? s usual, we wat the worst-case ruig time of the best possible algorithm, as a fuctio of, the umber of vertices: D() = mi T (, G) G Here I ll use D() istead of T () to emphasize that we are lookig at determiistic algorithms. Clearly D() (, sice the adjacecy matrix is symmetric ad has zeros o the diagoal. Oe way to derive a lower boud for ay decisio problem is to cosider the size of the smallest proof or certificate that verifies the result. To prove that a graph is coected, it is clearly both ecessary ad sufficiet to reveal the edges i a arbitrary spaig tree of G. This tree costitutes a certificate that G is coected, or a 1-certificate for short. Coversely, if we wat to prove that a graph is discoected, we eed to demostrate a cut with o crossig edges, that is, a partitio of the vertices ito two disjoit subsets, such that o edge has oe edpoit i each subset. Such a cut is called a 0-certificate. Clearly, ay algorithm that tests coectedess must check all the edges i some 1-certificate before returig True ad all the edges i a 0-certificate before returig False. Let C 0 () ad C 1 () respectively deote the imum size of a 0- or 1-certificate i a -vertex graph, ad let C() = {C 0 (), C 1 ()}. We immediately have the followig geeral result. Theorem 1. D() C(). I the case of graph coectivity, we have C 0 () = / / = ( ( mod )/4 ad C 1 () = 1, so D() C() = ( ( mod )/4 = Ω( ). I other words, the trivial algorithm check every edge is optimal up to a small costat factor. Surprisigly, this algorithm is actually exactly optimal! Theorem. D() = ( ) Proof: I ll describe a adversary strategy that requires ay algorithm to examie every edge. The adversary maitais two graphs, Y ad M, each with vertices; iitially, Y is empty ad M is a clique. Y ( yes ) cotais the edges that the algorithm has examied ad foud to be preset i the fictioal iput graph. M ( maybe ) cotais ay edge that might be i the fictioal iput graph; i other words, a edge (i, j) is abset from M if the algorithm kows that a ij = 0. Note that Y is a subgraph of M. The adversary uses the followig simple strategy whe the algorithm examies a potetial edge: retur False uless that aswer would force the fictioal iput graph to be discoected. 1
Examie(i, j): if (i, j) Y derisively retur True else if (i, j) M mockigly retur False else if M \ (i, j) is discoected add (i, j) to Y grudgigly retur True else remove (i, j) from M sigh ad retur False We easily observe that with this strategy M is always coected ad Y is always acyclic. Moreover, wheever the adversary adds a edge betwee two compoets of Y, every other edge joiig those two compoets of Y has already bee queried ad removed from M. It follows that Y becomes coected oly after ( queries; at that momet, both Y ad M cosist of the same spaig tree, ad the algorithm ca safely retur True. Before the ( th query, M is coected ad Y is discoected. Sice both graphs are cosistet with the adversary s aswers, either could serve as the fictioal iput graph, which meas the algorithm caot possibly determie the correct output. graph property like coectivity that requires lookig at every possible edge to detect is called evasive. We will retur to evasive graph properties i a future lecture. 5. Strig Properties (Boolea fuctios, laguages, whatever... ) Let s look at these ideas i a little more geerality. strig property 1 is ay fuctio of the form F : {0, 1} {0, 1}. Let T (, s) deote the umber of bits i a -bit iput strig s that a algorithm examies before correctly returig F (s). The determiistic decisio tree complexity of a strig property F is, as usual, the worst-case ruig time of ay algorithm to compute it, as a fuctio of the iput size : D() = mi T (, s) s = The model of computatio is the same boolea decisio tree that we saw i the very first lecture. For each iput size, we ca model ay algorithm by a rooted biary tree, where each iteral ode stores the idex of the ext bit to examie, ad each leaf stores a output value. I this model, T (, s) is the depth of the path traversed by the iput strig s i tree, ad the determiistic complexity of ay strig property is the miimum depth of ay tree that correctly computes it. strig property is evasive if D() =. We ca defie the certificate complexity of a strig property F as follows. Ituitively, a 1- certificate is a subset of the bits i the iput s that forces F (s) = 1, ad a 0-certificate is a subset of bits that forces F (s) = 0. The certificate complexity C(s) of a strig s is the size of the smallest certificate cosistet with s. Fially, the certificate complexity C() of F is the imum certificate complexity of ay -bit iput strig. Equivaletly, we have C() = mi T (, s) s = 1 dmittedly, this is a rather bizarre ame, but it is aalogous to graph properties such as coectedess (which we just saw), acyclicity, plaarity, ad the like. strig property is just a boolea fuctio with argumets, or equivaletly, a set of -bit strigs, or equivaletly, a laguage.
where (as above) the mi is take over all algorithms that correctly compute F for all iputs. Origially, certificate complexity was kow as o-determiistic decisio tree complexity, sice it correspods to a odetermiistic variat of the boolea decisio tree model. The best way to thik of a odetermiistic decisio tree is as a family of determiistic decisio trees, where for each iput, we use the best decisio tree i the set for that iput. We ve see oe example of these defiitios already. For the fuctio F : {0, 1} ( {0, 1} that idicates the coectivity of a -vertex graph, we have D( ( ( ) = ( ad C( ) ) = mod. s aother example, let F : {0, 1} {0, 1} deote the exactly half fuctio: F (s) = 1 if ad oly if s cotais exactly 1s ad exactly 0s. For this fuctio, we easily observe that D() = C() =. 5.3 Blum s Theorem Oe of the most geeral results about decisio tree complexity was proved by Mauel Blum. Theorem 3 (Blum). For ay strig property, C() D() C(). Proof: The first iequality C() D() is almost trivial, sice ay algorithm must examie all the bits i a certificate before returig its output. lterately, the iequality x mi y f(x, y) mi y x f(x, y) is easy to prove for ay fuctio f(x, y). To prove the other iequality, we describe a algorithm that examies C() bits. Let π s deote the smallest certificate for ay iput strig s. Let S 0 = {π s F (s) = 0} ad S 1 = {π s F (s) = 1} be the sets of all miimal 0- ad 1-certificates, respectively. To simplify otatio, let f = C(). Our algorithm works i k phases, examiig at most k bits i each phase. t each phase, we keep oly the certificates that are cosistet with the bits we ve examied so far. Let S0 i deote the subset of S 0 cosistet with the bits see i the first i phases, ad defie S1 i similarly. I particular, we have S0 0 = S 0 ad S1 0 = S 1. However, sice it is poitless to carry aroud ay bit whose value we already kow, the algorithm oly maitais the bits i each certificate that have ot yet bee queried. Thus, after each iput bit x j is queried, ay certificate π that is defied i that bit positio is either removed (if π j x j ) or its legth is decreased by oe (if π j = x j ). I the ith phase, the algorithm simply chooses a arbitrary 0-certificate π S i 1 0 ad queries all its (previously uqueried) bits. If all the queries agree with π, the algorithm correctly returs False; otherwise, it cotiues with the ext phase. Now I claim that each phase reduces the legth of every survivig 1-certificate by at least 1. Let σ be a arbitrary 1-certificate i S i 1 1. Sice every iput strig cotais either a 0-certificate or a 1-certificate, but ot both, there must be at least oe bit positio that appears i both σ ad the chose 0-certificate π. Moreover, this bit positio was ot queried i ay earlier phase, because otherwise, at most oe of them would have survived. If the iput strig agrees with π at that commo bit positio, σ is discarded; otherwise, the legth of σ decreases, as claimed. It follows that after at most k phases, we are left with either o 1-certificates, i which case the output must be False, or a sigle empty 1-certificate (i.e., a 1-certificate that matches the iput i every bit positio), i which case the output must be True. Sice each phase examies at most k bits, the algorithm examies at most k bits altogether. Nodetermiism meas ever havig to admit you re wrog. 3
We have already see examples where the first iequality is tight. The secod iequality ca be tight as well. Cosider the fuctio x 1 if k = 0 F k (x 1, x,..., x k) = F k 1 (x 1, x,..., x k 1) F k 1 (x k 1 +1, x k 1 +,..., x k) if k is eve F k 1 (x 1, x,..., x k 1) F k 1 (x k 1 +1, x k 1 +,..., x k) if k is odd This fuctio F k models a d-or tree of depth k: a boolea circuit i the form of a complete biary tree, where the leaves are iputs, gates alterate betwee d ad Or at each level, ad the value at the root is the output. (Do t cofuse the d-or tree with the decisio tree that evaluates it!) x 1 x 1 x x 1 x x 3 x 4 x 1 x x 3 x 4 x 5 x 6 x 7 x 8 d-or trees of depth 0 through 3 It s easy to show that this fuctio is evasive, but that it has much smaller certificate complexity: D(F k ) = k C(F k ) = k/ I fact, it s so easy that I ll leave it as a homework exercise. 5.4 Radomized complexity Recall that a radomized decisio tree is just a probability distributio over a set of determiistic decisio trees, ad that the radomized complexity of a strig property is the worst-case expected ruig time of the fastest radomized decisio tree: R() = mi P s Pr P [] T (, s) gai, I m usig R() istead of the earlier otatio T () to emphasize the radomizatio. We ve already see examples where radomess helps a little bit, but a more extreme example might be more helpful. Let M : {0, 1} 3 {0, 1} deote the boolea media (or majority) fuctio x + y + z M(x, y, z) =. The we ca defie the iterated media fuctio M k : {0, 1} 3k {0, 1} as M 0 (x 1 ) = x 1 ad M k (x 1,..., x 3 k) = M(M k 1 (x 1,..., x 3 k 1), M k 1 (x 3 k 1 +1,..., x 3 k 1), M k 1 (x 3 k 1 +1,..., x 3 k)) for all k 1. 3 3 Douglas Hofstadter defied a three-player game called Hruska based o iterated medias. I a level-0 Hruska game, each player chooses a iteger betwee 0 ad 5; the player with the media choice wis the game, ad adds his chose umber to his level-1 score. For ay i 1, a level-i game cosists of six level-(i 1) games, startig with level-i scores of zero; the wier is the player with the media level-i score at the ed of the game, which is the added to that player s score at level i + 1. I order to make the game fair (ad well-defied), each roud uses a differet permutatio of the players to break ties. I oce wo a level-3 Hruska game by havig the media level-3 score, by havig the lowest level- score, by havig the media level-1 score, by choosig the umber 5 o my 16th level-0 tur. I do t recommed playig a level-4 game, uless you wat your brai to explode. 4
It is ot hard to prove that D(M k ) = 3 k ad C(M k ) = k ; this is a good warm-up for the previous homework exercise. The radomized complexity fits betwee these two bouds. Cosider the followig radomized algorithm for computig M(x, y, z): Query two of x, y, z at radom. If they are equal, retur their commo value. Otherwise, query the remaiig variable ad retur its value. The probability of two variables that agree is at least 1/3, so the expected ruig time of this algorithm is at most 8/3. We ca use this idea recursively to evaluate M k quickly as well. Radomly choose two of the three istaces of M k 1 ad evaluate them recursively. If they retur the same value, we re doe; otherwise (with probability at most /3) we have to evaluate the third istace. We have the recurrece R(M k ) 8 3 R(M k 1), which has the obvious solutio R(M k ) (8/3) k. s it turs out, this algorithm is optimal so R(M k ) = (8/3) k but we do t have the tools to prove this yet. Soo. Promise. Notice that i this case, the radomized complexity falls strictly betwee the certificate complexity ad the determiistic complexity. This should ot be a surprise; eve radomized algorithms have to ru log eough to certify their aswers. I geeral, we have the followig trivial extesio of Blum s theorem for ay boolea fuctio. C() R() T () C() For the d-or tree fuctio F k, the radomized complexity is ( 1+ 33 4 ) k 1.68614 k 0.753. The upper boud follows from a careful radomized algorithm, which I will leave as the third part of the homework exercise. The matchig lower boud is due to Saks ad Widgerso 4. I fact, Saks ad Widgerso cojecture that this is a lower boud o the radomized complexity of ay evasive boolea fuctio. s far as I kow, this cojecture is still ope. 4 Proc. STOC 1986 5