Which Codes Have CycleFree Tanner Graphs?


 Madeleine Tamsyn Harrell
 1 years ago
 Views:
Transcription
1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER Which Coes Have CycleFree Taer Graphs? Tuvi Etzio, Seior Member, IEEE, Ari Trachteberg, Stuet Member, IEEE, a Alexaer Vary, Fellow, IEEE Abstract If a liear block coe of legth has a Taer graph without cycles, the maximumlikelihoo softecisio ecoig of ca be achieve i time O( ). However, we show that cyclefree Taer graphs caot support goo coes. Specifically, let be a (; k; ) liear coe of rate R = k= that ca be represete by a Taer graph without cycles. We prove that if R :5 the, while if R<:5 the is obtaie from a coe of rate :5 a istace by simply repeatig certai symbols. I the latter case, we prove that + +1 < R : Furthermore, we show by meas of a explicit costructio that this bou is tight for all values of a k. We also prove that biary coes which have cyclefree Taer graphs belog to the class of graphtheoretic coes, kow as cutset coes of a graph. Fially, we iscuss the asymptotics for Taer graphs with cycles, a preset a umber of ope problems for future research. Iex Terms Iterative ecoig, liear coes, miimum istace, Taer graphs. I. INTRODUCTION Iterative ecoig algorithms o factor graphs [15] have become a subject of much active research i recet years [1], [], [4], [5], [9], [15] [18], [], [9], a [3]. For example, the wellkow turbo coes a turbo ecoig methos [5], [4] costitute a special case of this geeral approach to the ecoig problem. Factorgraph represetatios for turbo coes were itrouce i [9] a [3], where it is also show that turbo ecoig is a istace of a geeral ecoig proceure, kow as the sumprouct algorithm. Aother extesively stuie [8], [7] special case is trellis ecoig of block a covolutioal coes. It is show i [9] a [3] that the Viterbi algorithm o a trellis is a istace of the misum iterative ecoig proceure, whe applie to a simple factor graph. The forwar backwar algorithm o a trellis, ue to Bahl, Cocke, Jeliek, a Raviv [3], is agai a special case of the sumprouct ecoig algorithm. More geeral iterative algorithms o factor graphs, collectively terme the geeralize istributive law or GDL, were stuie by Aji a McEliece [1], []. These algorithms ecompass maximumlikelihoo ecoig, belief propagatio i Bayesia etworks [1], [], a fast Fourier trasforms as special cases. Mauscript receive December 15, 1997; revise October 8, This work was supporte by the Davi a Lucile Packar Fouatio, the Natioal Sciece Fouatio, a the U.S. Israel Biatioal Sciece Fouatio uer Grat The work of A. Trachteberg a A. Vary was also supporte by the Computatioal Sciece a Egieerig Program at the Uiversity of Illiois. The material i this correspoece was presete i part at the IEEE Iteratioal Symposium o Iformatio Theory, MIT, Cambrige, MA, August 16 1, T. Etzio is with the Departmet of Computer Sciece, Techio Israel Istitute of Techology, Haifa 3, Israel. A. Trachteberg is with Digital Computer Laboratory, Uiversity of Illiois at UrbaaChampaig, Urbaa, IL 6181 USA. A. Vary is with the Uiversity of Califoria at Sa Diego, La Jolla, CA USA. Commuicate by F. R. Kschischag, Associate Eitor for Coig Theory. Publisher Item Ietifier S (99) It is prove i [], [15], [6], a [3] that the misum, the sumprouct, the GDL, a other versios of iterative ecoig o factor graphs all coverge to the optimal solutio if the uerlyig factor graph is cyclefree. If the uerlyig factor graph has cycles, very little is kow regarig the covergece of iterative ecoig methos. This work is cocere with a importat special type of factor graphs, kow as Taer 1 graphs. The subject ates back to the work of Gallager [11] o lowesity paritycheck coes i 196. Taer [6] extee the approach of Gallager [11], [1] to coes efie by geeral bipartite graphs, with the two types of vertices represetig coe symbols a checks (or costraits), respectively. He also itrouce the misum a the sumprouct algorithms, a prove that they coverge o cyclefree graphs. More recetly, coes efie o sparse (regular) Taer graphs were stuie by Spielma [], [5], who showe that such coes become asymptotically goo if the uerlyig Taer graph is a sufficietly strog expaer. These coes were stuie i a ifferet cotext by MacKay a Neal [16], [18], who emostrate by extesive experimetatio that iterative ecoig o Taer graphs ca approach chael capacity to withi about 1 B. Latest variats [17] of these coes come withi about.3 B from capacity, a outperform turbo coes. I geeral, a Taer graph for a coe of legth over a alphabet A is a pair (G; L), where G =(V; E) is a bipartite graph a L = fc1; C; 111; C rg is a set of coes over A, calle behaviors or costraits. We eote the two vertex classes of G by X a Y, so that V = X[Y. The vertices of X are calle symbol vertices a jx j =, while the vertices of Y are calle check vertices a jyj = r. There is a oetooe correspoece betwee the costraits C 1; C; 111; Cr i L a the check vertices y 1;y; 111;yr i Y; so that the legth of the coe C i Lis equal to the egree of the vertex y i Y, for all i =1; ; 111;r.Acofiguratio is a assigmet of a value from A to each symbol vertex x 1;x; 111;x i X. Thus a cofiguratio may be thought of as a vector of legth over A. Give a cofiguratio =( 1 ; ; 111; ) a a vertex y Y of egree, we efie the projectio y of o y as a vector of legth over A obtaie from by retaiig oly those values that correspo to the symbol vertices ajacet to y. Specifically, if fx i ;x i ; 111;x i gx is the eighborhoo of y i G; the y = ( i ; i ; 111; i ). A cofiguratio is sai to be vali if all the costraits are satisfie, amely, if y C i for all i =1; ; 111;r. The coe represete by the Taer graph (G; L) is the the set of all vali cofiguratios. While the foregoig efiitio of Taer graphs is quite geeral, the theory a practice of the subject [7], [16] [18], [], [6], is focuse almost exclusively o the simple special case where all the costraits are sigleparitycheck coes over IF. This work is o exceptio, although we will provie for the represetatio of liear coes over arbitrary fiels by cosierig the zerosum coes over IF q rather tha the biary sigleparitycheck coes. It seems appropriate 1 Note o termiology. The term Taer graph was first use by Wiberg, Loeliger, a Kötter [3] to refer to the more geeral graphs itrouce i [3]. These were later terme TWL graphs by Forey [9], although TWLK graphs woul have bee more appropriate. By ow, the term factor graphs is almost uiversally use i this cotext, which leaves Taer graphs available to refer to the ki of factor graphs actually stuie by Taer [6]. The emphasis i this paper (as i all of the literature [7], [16], [18], [], a [6] o the subject) is o a special type of Taer graphs that come with simple paritycheck costraits. These Taer graphs iclue the graphs uerlyig Gallager s lowesity paritycheck coes [11], [1] /99$ IEEE
2 174 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 to call the correspoig Taer graphs simple. Notice that i the case of simple Taer graphs, the set of costraits L is implie by efiitio, so that oe ca ietify a simple Taer graph with the uerlyig bipartite graph G. All of the Taer graphs cosiere i this correspoece, except i Sectio VC, are simple. Thus for the sake of brevity, we will heceforth omit the quatifier simple. Istea, whe we cosier the geeral case i Sectio VC, we will use the term geeral Taer graphs. We ca thik of a (simple) Taer graph for a biary liear coe of legth as follows. Let H be a r paritycheck matrix for. The the correspoig Taer graph for is simply the bipartite graph havig H as its X ; Y ajacecy matrix. It follows that the umber of eges i ay Taer graph for a liear coe of legth is O( ). Thus if we ca represet by a Taer graph without cycles, the maximumlikelihoo ecoig of ca be achieve i time O( ), usig the misum algorithm, for istace. However, both ituitio a experimetatio (cf. [16]) suggest that powerful coes caot be represete by cyclefree Taer graphs. The otio that cyclefree Taer graphs ca support oly weak coes is, by ow, wiely accepte. Our goal i this correspoece is to make this folk kowlege precise. We provie rigorous aswers to the questio: Which coes ca have cyclefree Taer graphs? Our results i this regar are twofol: we erive a characterizatio of the structure of such coes a a upper bou o their miimum istace. The upper bou (Theorem 5) shows that coes with cyclefree Taer graphs provie extremely poor traeoff betwee rate a istace for each fixe legth. This iicates that at very high sigaltooise ratios these coes will perform baly. I geeral, however, the miimum istace of a coe oes ot ecessarily etermie its performace at sigaltooise ratios of practical iterest. Iee, there exist coes for example, the turbo coes of [4] a [5] that have low miimum istace, a yet perform very well at low sigaltooise ratios. The evelopmet of aalytic bous o the performace of cyclefree Taer graphs uer iterative ecoig is a challegig problem, which is beyo the scope of this work. Nevertheless, our results o the structure of the correspoig coes iicate that they are very likely to be weak: their paritycheck matrix is much too sparse to allow for a reasoable performace eve at low sigaltooise ratios. The rest of this correspoece is orgaize as follows. We start with some efiitios a auxiliary observatios i the ext sectio. I Sectio III, we show that if a (; k; ) liear coe ca be represete by a cyclefree Taer graph a has rate R =k= :5; the. We furthermore prove that if R < :5, the is ecessarily obtaie from a coe of rate :5 a miimum istace by simply repeatig certai symbols i each coewor. Theorem 5 of Sectio IV costitutes our mai result: this theorem gives a upper bou o the miimum istace of a geeral liear coe that ca be represete by a cyclefree Taer graph. Furthermore, the bou of Theorem 5 is exact. This is also prove i Sectio IV by meas of a explicit costructio of a family of (; k; ) liear coes that attai the bou of Theorem 5 for all values of a k. Asymptotically, for!1, the upper bou takes the form b1=rc (1) a a immeiate cosequece of (1) is that asymptotically goo coes with cyclefree Taer graphs o ot exist. We show i Sectio V that the same is true for Taer graphs with cycles, uless the umber of cycles icreases expoetially with the legth of the coe. We also show i Sectio V that for every biary coe that ca be represete by a cyclefree Taer graph, there exists a graph G such that is the ual of the cycle coe of G. This establishes a iterestig coectio betwee coes with cycle (a) (b) Fig. 1. (a) Taer graph with cycles a (b) cyclefree Taer graph for the same coe. free Taer graphs a the wellkow [6], [15], [13], [1], [4] class of graphtheoretic cutset coes. Fially, we coclue this correspoece with a partial aalysis of geeral Taer graphs. II. PRELIMINARIES Let H =[h ij ] be a r matrix, with etries raw from the fiite fiel IF q of orer q. We let 3 eote a ozero etry i H. Give H; we efie a bipartite graph T as follows: the vertex set of T cosists of the set X = fx 1;x ; 111;x g of symbol vertices a the set Y = fy 1 ;y ; 111;y r g of check vertices; there is a ege (y i ;x j ) i T if a oly if h ij = 3. Thus the eighborhoo of the vertex y i Y correspos to the ith row of H, a the eighborhoo of the vertex x j X correspos to the jth colum of H. We say that T is the Taer graph of H, a eote T = T (H). It is obvious that every matrix efies a uique Taer graph. Over IF, the coverse is also true: every bipartite graph T efies a uique biary matrix H such that T = T (H), which is the X ; Y ajacecy matrix of T. We say that a bipartite graph T represets a liear coe,or simply that T is a Taer graph for, if there exists a paritycheck matrix H for such that T is the Taer graph of H. I geeral, a give liear coe ca be represete by may istict Taer graphs. O the other ha, over IF, a give Taer graph represets a uique biary coe. We say that a matrix H is cyclefree if the correspoig Taer graph T (H) is cyclefree. Notice that every submatrix of a cyclefree matrix is also cyclefree. We say that a liear coe over IF q is cyclefree if there exists a cyclefree paritycheck matrix for. Observe that if the matrices H a H iffer by a permutatio of rows a colums the the Taer graphs T (H) a T (H ) are isomorphic. O the other ha, if H a H iffer by a sequece of elemetary row operatios the T (H) a T (H ) are geerally ot isomorphic. Thus it is possible to have two paritycheck matrices for the same coe, oe of which is cyclefree while the other is ot. It is also possible to have two cyclefree Taer graphs for the same coe that are ot isomorphic. Example: Suppose that a paritycheck matrix H for the (5; ; 3) biary liear coe is give by H = The correspoig Taer graph T (H) is show i Fig. 1(a). Notice that H is ot cyclefree, sice the sequece of eges (x 1 ;y 1 ); (y 1;x 3); (x 3;y ); a (y ;x 1) costitutes a cycle. However, the coe is, i fact, cyclefree sice aig the first row of H to the seco row prouces a cyclefree paritycheck matrix H for. The graph T (H ) show i Fig. 1(b) is a cyclefree Taer graph for. :
3 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER To show that the bou of Theorem is tight for all a k, with = k 1, we may start with the sigleparitycheck coe E k+1 a repeat ay symbol (or symbols) i E k+1 util a coe of legth is obtaie. The followig lemma shows that this always prouces a (; k; ) cyclefree coe for k =. Fig.. coe. (a) (b) Taer graphs for (a) E a (b) for a geeral lowrate cyclefree The followig simple lemma will serve as our startig poit. This lemma is well kow i graph theory see, for istace, West [8, p. 5] a we omit the proof. Lemma 1: A graph G = (V;E) is cyclefree if a oly if jej = jv j!(g), where!(g) eotes the umber of coecte compoets i G. A cyclefree graph cosistig of a sigle coecte compoet is calle a tree, a thus a multiplecompoet cyclefree graph is also kow as a forest. For trees, we have jej = jv j1 by Lemma 1. Sice every forest cotais at least oe tree, we have jej jv j1 () for ay cyclefree graph. If M is a m matrix, the the umber of vertices i T (M ) is m + a the umber of eges i T (M ) is equal to wt (M ) the total umber of ozero etries i M. Thus if M is cyclefree, the wt (M ) m + 1 i view of (). III. THE STRUCTURE OF CYCLEFREE CODES We start with a simple theorem, which gives a tight upper bou o the miimum istace of highrate cyclefree liear coes. Theorem : Let be a (; k; ) cyclefree liear coe of rate k= :5. The. Proof: Let H be the r cyclefree paritycheck matrix for : We assume without loss of geerality (w.l.o.g.) that H has full rowrak a r = k, sice otherwise we ca remove the liearly epeet rows of H while preservig the cyclefree property. Let i eote the umber of colums of weight i i H. If 6=the =1, a we are oe. Otherwise, we have 1 +( 1)= 1 +( r) wt (H) + r 1 (3) i view of (). Substitutig r = k ito (3), this iequality reaily reuces to 1. Sice k= :5, it follows that k r a 1 r +1. This meas that the umber of weightoe colums i H is greater tha the umber of rows i H. Hece H cotais at least two colums of weight oe that are scalar multiples of each other, a =. Theorem implies that the (; 1; ) sigleparitycheck coe E is, i a sese, the optimal cyclefree coe of rate :5, sice all such coes have istace a E has the highest rate. The cyclefree Taer graph for E is epicte i Fig. (a). Lemma 3: Let be a cyclefree coe of legth a imesio k: Fix a positive iteger i, with i, a let be the coe obtaie from by repeatig the ith symbol i each coewor. The is a cyclefree coe of legth +1a imesio k. Proof: The legth a imesio of are obvious. To see that is cyclefree, observe that a Taer graph T for ca be obtaie from the cyclefree Taer graph T for by itroucig two ew vertices x a y a two ew eges: (x ;y ) a (x i ;y ). It is easy to see that this proceure oes ot create ew cycles. Let be a cyclefree coe of legth, a let 3 be the coe of legth + obtaie from by iteratively applyig times the proceure of Lemma 3, while possibly choosig a ifferet value of i at ifferet iteratios. We the say that 3 is a coe obtaie by repeatig symbols i. To make our termiology precise, we further exte the otio of coes obtaie by repeatig symbols i to also iclue the coes obtaie from 3 by appeig allzero cooriates. The followig propositio shows that every lowrate cyclefree liear coe has this structure. Propositio 4: Let be a (; k; ) cyclefree liear coe over IF q of rate k= :5. The, up to scalig by costats i IF q at certai positios, is obtaie by repeatig symbols i a cyclefree coe of rate >:5. Proof: Let H be the fullrak r cyclefree paritycheck matrix for, with r = k. The by Lemma 1 a (), we have wt (H) + r 1 3r 1, where the seco iequality follows from the fact that k= :5. This implies that H cotais at least oe row of weight. If this row is of weight oe, the the correspoig cooriate of, say the th cooriate, is allzero. Otherwise, assume without loss of geerality that this row is of the form h = (; ; 111; ; 3; 3). The, up to scalig the last two colums of H by costats i IF q, we may further assume that h =(; ; 111; ; 1; 1). This woul mea that the th symbol i is a repetitio of the preceig symbol. I both cases, we ca pucture out the th cooriate of, a iteratively repeat the argumet util a cyclefree coe of rate >:5 is obtaie. Loosely speakig, Propositio 4 implies that every cyclefree coe of rate :5 ca be represete by a Taer graph whose structure is show i Fig. (b). The ashe lie i Fig. (b) ecloses a cyclefree Taer graph for a coe of rate >:5 a istace. It follows that to establish a bou o the miimum istace of lowrate cyclefree coes, we ee to etermie a optimal choice for i Fig. (b) a a optimal sequece of symbol repetitios. This problem is cosiere i etail i the ext sectio. Specifically, we will show i the ext sectio that the sigleparitycheck coe costitutes a optimal choice for, a every symbol shoul be repeate equally ofte. IV. THE MINIMUM DISTANCE OF CYCLEFREE CODES The followig theorem gives a upper bou o the miimum istace of cyclefree liear coes. Later i this sectio, we will show that this bou is exact for all values of a k. Theorem 5: Let be a (; k; ) cyclefree liear coe over IF q. The + +1 : (4)
4 176 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 (a) (b) (c) Fig. 3. (a) A cyclefree matrix, (b) its Taer graph, a (c) its rowgraph G. Observe that for k= :5, the bou i (4) reuces to. This simple special case was ealt with i Theorem. The proof of Theorem 5 for geeral a k is cosierably more ivolve. This proof will be presete i Sectio IVB, after we establish a series of auxiliary lemmas i the ext subsectio. A. Grouwork: Auxiliary Lemmas For the sake of brevity, we will cosier oly biary coes, although our proof reaily extes to coes over a arbitrary fiite fiel. Furthermore, with a slight abuse of otatio, we will ot istiguish betwee equivalet coes: amely, give a paritycheck matrix H for a coe, we will ofte freely permute the colums of H while still referrig to the resultig matrix as a paritycheck matrix for. Let be a (; k; ) cyclefree biary liear coe, a let H be a r cyclefree paritycheck matrix for, where r = k. We say that H is i scaoical form, if this matrix has the followig structure: H = A B I s (5) where all the rows of B have weight 1, a I s is the s s ietity matrix, for some s i the rage s r. Notice that if s = the (5) reuces to H = A (which meas that every matrix is i caoical form), while if s = r the the correspoig caoical form is H = [B j I r ]. We will use the shortha H = Ak s B to eote the scaoical form i (5). Lemma 6: Let H =Ak s B be a cyclefree biary matrix i scaoical form, a suppose that s<r. The at least oe of the followig statemets is true. The matrix A cotais a row of weight two or less; 5 The matrix A cotais three ietical colums of weight oe;? The matrix A cotais two ietical colums of weight oe, a furthermore the row of A which cotais the ozero etries of these two colums has weight three. Proof: Let T (A) be the Taer graph of A. Evietly T (A) is a subgraph of T (H), obtaie by retaiig oly the first s symbol vertices x 1 ;x ; 111;x s, the first r s check vertices y 1 ;y ; 111;y rs, a all the eges betwee these vertices. Sice T (H) is cyclefree by assumptio, so is T (A). We ow costruct aother graph G, calle the rowgraph of A, whose vertex set y 1;y ; 111;y rs correspos to the rows of A. The ege set of G is erive from the colums of A of weight, so that a colum of weight w i A cotributes w 1 eges to G. A example illustratig the costructio of the rowgraph G for a 6 8 cyclefree matrix is epicte i Fig. 3. Specifically, there is a ege betwee y i a y j i G iff i<ja there exists a colum (a 1 ;a ; 111;a rs) t i A, such that a i = a j =1 while a i+1 = a i+ = 111 = a j1 =. Notice that each such ege (y i ;y j ) i G correspos to a path of legth two i T (A): amely (y i ;x p ); (x p ;y j ), where p eotes the positio at which the colum (a 1;a ; 111;a rs) t is to be fou i A. It follows that if there is a cycle i G, the there is a cycle i T (A). Sice T (A) is cyclefree, the so is G. As such, G ecessarily cotais at least two vertices of egree 1, i view of (). Let y 3 be oe such vertex i G, a let a 3 =(a 1 ;a ; 111;a s ) be the correspoig row of A. If wt (a 3 ), the () is true. If wt (a 3 )=3a eg y 3 =, the (5) is true. If wt (a 3 )=3a eg y 3 =1, the (?) is true. Fially, if wt (a 3 ) 4, the (5) is true, regarless of whether eg y 3 =or eg y 3 =1. We will say that a r matrix H is i reuce caoical form, if H = Ak s B a either s = r or all the rows of A have weight 3. Lemma 7: Let be a (; k; ) cyclefree biary liear coe. The there exists a cyclefree paritycheck matrix for, which is i reuce caoical form. Proof: Let H be a arbitrary cyclefree paritycheck matrix for. We first put H i scaoical form, for the highest possible s, by meas of row a colum permutatios. This is achieve by cosierig all the rows of H of weight oe, for which the ozero etry 3 is cotaie i a colum of weight oe, a all the rows of H of weight two such that at least oe of the two 3 is cotaie i a colum of weight oe. Uer a appropriate colum permutatio, these rows of H will form the submatrix [B j I s ] i (5). If there are o such rows, the s =a H = A. Sice row a colum permutatios preserve the cyclefree property of H, this proceure prouces a cyclefree paritycheck matrix H = Ak s B for, which is i caoical form, although ot ecessarily reuce. Sice H = Ak s B is fullrak by assumptio, all the rows of A have weight 1. The key observatio is that certai elemetary operatios o the rows of H allow us to elimiate rows of weight oe a two i A, while still preservig the cyclefree property.
5 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER to the Appeix. The proof is by iuctio o the legth of the coe. Thus we first trasform (4) ito the form Fig. 4. Part of the Taer graph for before a after the row operatio. Iee, suppose that A has a row (a 1 ;a ; 111;a rs) of weight oe, with the sigle 3 i positio i. The we ca a this row to all the rows of H that are ozero at positio i. This proceure is equivalet to eletig all but oe of the eges iciet at the symbol vertex x i i T (H ), which certaily oes ot create ew cycles. Followig this proceure, the sigle 3 i (a 1 ;a ; 111;a rs ) is cotaie i a colum of weight oe. Hece we ca trasform the resultig cyclefree paritycheck matrix for ito the form A k s+1b, by meas of row a colum permutatios, thereby elimiatig the row of weight oe i A. Now suppose that A has a row (a 1;a ; 111;a rs) of weight two, with the two 3 i positios i a j, a let y 3 be the correspoig check vertex i T (H ). We agai a this row to all the rows of H that are ozero at positio i. Let (b 1;b ; 111;b ) be such a row, a let y eote the correspoig check vertex of T (H ). If b j = b i = 3, the T (H ) cotais the cycle (x i;y 3 ); (y 3 ;x j); (x j;y ); (y ;x i). Sice T (H ) is cyclefree, we coclue that b i = 3 while b j =. Hece, aig (a 1 ;a ; 111;a ) to (b 1 ;b ; 111;b ) correspos to eletig the ege (y ;x i) i T (H ) while itroucig a ew ege (y ;x j ), as illustrate i Fig. 4. However, this ew ege caot close a cycle, sice T (H ) is cyclefree a a path from y to x j alreay exists i T (H ): iee, (y ;x i), (x i ;y 3 ), (y 3 ;x j ) is such a path. Followig all these elemetary row operatios, the 3 at positio i i (a 1 ;a ; 111;a ) is cotaie i a colum of weight oe. Thus we ca agai trasform the resultig cyclefree paritycheck matrix for ito the form A k s+1 B, thereby elimiatig the row of weight two i A. We iteratively repeat the process escribe i the foregoig two paragraphs, util either s = r or all the rows of A have weight 3. This proceure prouces a cyclefree paritycheck matrix for, which is i reuce caoical form. If the reuce caoical form i Lemma 7 is achieve i the extreme case s = r, the it is easy to prove the claim of Theorem 5. Lemma 8: Let be a (; k; ) cyclefree biary liear coe. If there is a paritycheck matrix for of the form H =[B j I r ], where r = k a the rows of B have weight 1, the the miimum istace of satisfies the upper bou of Theorem 5. Proof: If B cotais a colum of weight w, the clearly w +1. Sice B is a r k matrix, a all the rows of B have weight 1, we have wt(b) k +1 r = k : (6) As is a iteger, this implies that b=kc. It is easy to see that b=kc b=()c, uless k =1a is o. But, i the latter case, both (4) a (6) reuce to. B. Proof of the Mai Result We are ow i a positio to procee with the proof of Theorem 5. Part of this proof ivolves teious calculatios, which will be eferre ; if +16 mo() +1; if +1 mo() that is more coucive to iuctio o. It ca be easily see by irect verificatio that (4) a (7) are equivalet. As the iuctio basis, we may cosier coes of legth =, for which the bou of Theorem 5 hols trivially. As the iuctio hypothesis, we assume that the miimum istace of every cyclefree liear coe of legth <satisfies the bou of Theorem 5. The iuctio step is establishe as follows. Let be a (; k; ) cyclefree biary liear coe. We may assume that k 1, sice for k =1the bou of (4) reuces to, while if k = the =1a (4) obviously hols with equality. By Lemma 7, there exists a r cyclefree paritycheck matrix H = Ak s B for, which is i reuce caoical form. If s = r, the the iuctio step follows immeiately from Lemma 8. Otherwise, Lemma 6 implies that either (5) or (?) is true. Observe that case () of Lemma 6 oes ot occur, sice by the efiitio of a reuce caoical form, the matrix A oes ot have rows of weight. Furthermore, both (5) a (?) imply that A cotais at least two ietical colums of weight oe. Let i a j eote the positios at which these two colums are fou i A. Further, let w i a w j eote the weight of the correspoig colums of B. Let w = w i + w j +. It follows from the caoical form structure of H = Ak sb that the ith bit, respectively jth bit, of is repeate w i times, respectively w j times, i the last s positios. Further observe that the sum of the ith a the jth colums of H together with the correspoig w i + w j colums of the ietity matrix prouces the allzero rtuple. Hece there is a coewor of weight w = w i + w j +i, a w. We ow shorte at positios i a j to obtai a ( ;k ; ) coe. That is, we cosier the subcoe of cosistig of all the coewors that are zero o positios i a j a efie to be the coe obtaie by pucturig out the w =+(w i + w j) zero positios i this subcoe. Notice that shorteig at positios i a j is equivalet to eletig w i +w j + colums of H a w i +w j rows of H, as illustrate i Fig. 5. It is easy to see that the parameters of the resultig coe satisfy (7) = w k k : (8) Furthermore, sice H is cyclefree by assumptio, the paritycheck matrix for which results by eletig rows a colums of H is also cyclefree. It follows that is a cyclefree coe of legth <, a we ca ivoke the iuctio hypothesis. We istiguish betwee two cases. Case 1: +1 6 mo(k +1): I this case, the iuctio hypothesis implies that b =(k +1)c. Takig ito accout the relatios (8) betwee the parameters of a, we obtai k +1 w k 1 k 1 where the thir iequality follows from the fact that w. It is show i the Appeix that the relatio betwee ; k; a i (9) implies (7). (9)
6 178 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 k 3 = k, a 3 =. Applyig the iuctio hypothesis 3 to, we get 3 3 w 1 k 3 +1 k 1 k 1 (13) where the seco iequality follows from the fact that if +1 mo(k +1) the mo(k 3 +1). The rightha sie of (13) is the same as (9), which was alreay cosiere i Case 1. It remais to cosier the case where = fg, amely, k =. But i this case k i view of (8), a the upper bou of Theorem 5 follows irectly from the Griesmer bou [19, p. 547]. Sice we have ow exhauste all the possibilities, this establishes the iuctio step, a completes the proof of Theorem 5. Fig. 5. Deletig rows a colums of H to shorte a cyclefree coe. Case : +1 mo(k +1): We agai apply the iuctio hypothesis. Notice that i this case, the upper bou of Theorem 5 may be rewritte as +1 k +1 +1= k +1 1 (1) where ( +1)=(k +1)is a positive iteger. Suppose that is a eve iteger. The, sice the rightha sie of (1) is a o iteger, we have +1 k (11) k 1 where the seco iequality i (11) follows from (8) alog with the fact that w. It is show i the Appeix that (11) implies (7). Now suppose that is o. I this case, the bou of (1) oes ot suffice to establish (7), a we ee to use the aitioal structure preset i statemets (5) a (?) of Lemma 6. Suppose that (5) is true, a the matrix A cotais three ietical colums of weight oe, at positios ; ;. Let w ;w ;w eote the weight of the correspoig colums of B. Notice that at least oe of w + w ;w + w ;w + w is a eve iteger. Hece we ca choose the two positios i a j i Fig. 5 from the three positios ; ;, i such a way that w = w i + w j +is eve. Sice w a is o, it follows that w 1. I cojuctio with (1), we thus obtai +1 k +1 1 w +1 1 k 1 k 1 1: (1) It is show i the Appeix that if is o, the (1) implies (7). Now suppose that (?) is true, a the matrix H cotais a row of weight three, with the three 3 at positios h; i; j. The after eletig w i +w j + colums of H a w i + w j rows of H, as illustrate i Fig. 5, we are left with a row of weight oe, with the sigle 3 at positio h. This meas that the hth positio i is etirely zero; this positio ca be pucture out without ecreasig the imesio or the miimum istace. We thus obtai a ( 3 ;k 3 ; 3 ) coe 3, with 3 = 1; C. Optimal CycleFree Coes While provig the upper bou of Theorem 5 require cosierable effort, showig that this bou is exact is easy. We ow costruct a family of cyclefree coes that attai the bou of Theorem 5 with equality, for all values of a k. The costructio is quite simple: as i Sectio III, we start with the sigleparitycheck coe E k+1 of imesio k, a repeat the symbols of E k+1 util a coe of legth is obtaie. It is obvious that the imesio of is k, a by Lemma 3 this coe is cyclefree. The miimum istace of will epe o the sequece of symbol repetitios. The iea is to repeat each symbol i E k+1 equally ofte, i as much as possible. For example, for =13a k =3, we obtai the followig paritycheck matrix i reuce caoical form: H = (14) which efies a (13; 3; 6) cyclefree coe. I geeral, the umber of symbols to be repeate is, while the umber of positios available is (). Write ()=a()+b where a; b are itegers, a b k. This ecompositio of the umber of available positios meas that i our costructio exactly k b +1symbols of E k+1 will be repeate a = () = 1 times, while the remaiig b symbols of E k+1 will be repeate a +1 times. If b k 1, the at least two symbols of E k+1 are repeate exactly a times. Sice E k+1 cotais a coewor of weight i every two positios, the miimum istace of the resultig coe is =+a + a = : (15) If b = k, the oly oe symbol i E k+1 is repeate a times, while all the other symbols are repeate a +1 times. I this case, the miimum istace of is =+a +(a+1) = +1: (16)
7 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER Fig. 7. (a) Two iequivalet cyclefree cutset coes. (b) Fig. 6. (a) (b) Two alterative Taer graphs for optimal cyclefree coes. Notice that b = k if a oly if +1 mo(). Hece it follows from (15) a (16) that the coe costructe i this maer attais the bou of Theorem 5 with equality. Fig. 6 schematically shows two alterative cyclefree Taer graphs for coes resultig from this costructio (compare the Taer graph i Fig. 6(a) with Fig. (b)). We poit out that although cyclefree coes obtaie by repeatig symbols i E k+1 have the highest possible miimum istace, they are ot the oly coes with this property. For example, cosier the followig paritycheck matrix i reuce caoical form: H = : (17) It is easy to see that this matrix efies a (13; 3; 6) cyclefree coe, whose istace attais the bou of Theorem 5 with equality. This coe was obtaie by repeatig symbols i a (5; 3; ) coe. It ca be reaily verifie that is ot equivalet to the (13; 3; 6) cyclefree coe, efie by the paritycheck matrix i (14) a obtaie by repeatig symbols i E 4. For istace, cotais the alloe coewor, while oes ot. V. FURTHER RESULTS AND OPEN PROBLEMS I this sectio, we iscuss three ifferet topics: a coectio betwee biary cyclefree coes a cutset coes of a graph, asymptotic behavior of Taer graphs with cycles, a the extesio of the results of the previous sectio to geeral Taer graphs. I each case, we provie a umber of ope problems for future research. A. CycleFree Coes a GraphTheoretic Coes There is a iterestig coectio betwee cyclefree coes a cutset coes of a graph. Let G =(V; E) be a multigraph (a graph that may cotai multiple eges with both epoits the same) with = jej eges a m = jv j vertices. A cutset i G is a set of eges which cosists of all the eges havig oe epoit i some set X V a the other epoit i V X. Uer the operatio of symmetric ifferece, the cutsets i G form a subspace of the biary vector space of all subsets of E. Hece replacig subsets of E by their characteristic vectors i IF prouces a biary liear coe (G), calle the cutset coe of G. The ual coe of (G) is the cycle coe of G, efie as the liear spa of the characteristic vectors of cycles i G. Graphtheoretic coes, amely, cutset coes a cycle coes of a graph, have bee extesively stuie see [6], [14], [13], [3], a [4], for istace. The coectio betwee cyclefree coes a cutset coes of a graph ca be summarize as follows. Theorem 9: Let be a cyclefree biary liear coe of legth. The there exists a graph G with eges, such that is a cutset coe of G. Proof: Let H be a r cyclefree paritycheck matrix for, a let T = T (H) be the correspoig cyclefree Taer graph that represets. The followig proceure coverts T ito a graph G, such that is the cutset coe of G. We will escribe this proceure assumig that T is a tree, i which case G is coecte. I case T is a forest cosistig of! trees, the same proceure shoul be carrie out iepeetly for each tree i T, a G will have! coecte compoets. Let Y = fy 1 ;y ; 111;y r g be the set of check vertices i T, a let X i X eote the eighborhoo of y i Yfor i =1; ; 111;r. Further efie X 3 i = X 1 [ X [111[X i. Sice T is a tree, it is always possible to eumerate the check vertices i T i such a way that X i itersects X 3 i1 i oe a oly oe symbol vertex for all i. Give such eumeratio y 1;y ; 111;y r, we costruct G iteratively, checkvertex by checkvertex. First, we represet y 1 a its eighborhoo X 1 by a cycle G 1 cosistig of jx 1 j eges a jx 1j vertices. Now suppose that X \ X 1 = fx g. The we create G from G 1 by appeig jx j1 eges oe for each symbol vertex i X except x a jx j vertices, i such a way that the eges correspoig to the symbol vertices i X form a cycle i G. A so forth: if X i \ X 3 i1 = fx ig, we create G i from G i1 by appeig jx ij1 eges a jx ij vertices, i such a way that the eges correspoig to the symbols i X i form a ew cycle. It is easy to see that G = G r will cotai exactly eges a (r 1) vertices. Furthermore, the coe? geerate by H is precisely the cycle coe of G. Sice the cutset coe of G is the ual of its cycle coe, our proof is complete. For example, the cyclefree coes efie by the paritycheck matrices i (14) a (17) are cutset coes of the graphs epicte i Fig. 7(a) a (b), respectively. For cutset coes, it is well kow [14], [1] that m, where m is the umber of vertices i the uerlyig graph G. Iee, this follows immeiately from the fact that if the miimum istace of (G) is, the every vertex of G must have egree at least, otherwise, the cutset that isolates this vertex will have less tha eges. It is also well kow that im (G) =m!(g), where
8 18 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 asymptotically, coes that have Taer graphs of prescribe miimum girth. Fig. 8. A cutset coe which is ot cyclefree.!(g) is the umber of coecte compoets i G. Thus we obtai the followig cutset bou o the miimum istace of cyclefree coes: k +!(G) : (18) If it is kow that is eve, the the cutset bou of (18) obviously implies Theorem 5. I geeral, however, Theorem 5 is stroger tha the cutset bou base o Theorem 9. Iee, there exist cutset coes that are ot cyclefree. As a simple example, cosier the (6; 3; 3) cutset coe of the graph epicte i Fig. 8, a otice that the miimum istace of this coe violates the upper bou of Theorem 5. I summary, we have prove that every cyclefree biary liear coe is a cutset coe. We pose the coverse as a ope problem: which cutset coes are cyclefree? A aswer to this questio may follow from a closer look at the costructio of the graph G from a cyclefree Taer graph T i the proof of Theorem 9. We observe that G is plaar, a that ay two regios i G, except for the outer regio, itersect i at most oe ege. Furthermore, if we remove from the ual graph of G the vertex correspoig to the outer regio of G a all the eges ajacet to this vertex, the resultig graph is a tree (or a forest). While we believe that the cutset coe of ay graph with these properties is cyclefree, we will ot pursue a proof of this claim herei. B. Asymptotics for Taer Graphs with Cycles It is obvious from Theorem 5 that Taer graphs without cycles caot support asymptotically goo coes. Startig with Theorem 5, it is ot ifficult to show that the same is true for Taer graphs with cycles, uless the umber of cycles icreases expoetially with the legth of the coe as!1. To see this, suppose the the cycle rak of a Taer graph T =(V;E) represetig a (; k; ) coe is c = jejjv j +!(T ). This meas that T cotais c cycles a uios of isjoit cycles (cf. [1, p. 137]). Now let x i be a symbol vertex that lies o a cycle i T. The removig x i a all the eges iciet o x i from T prouces a graph whose cycle rak is strictly less tha c. This proceure is equivalet to shorteig at the ith positio to obtai a ( ;k ; ) coe with = 1, k k1, a. Sice the cycle rak strictly ecreases each time we cut a cycle i T i this way, after repeatig this proceure t c times we obtai a cyclefree coe 3. Clearly, 3 is a ( 3 ;k 3 ; 3 ) coe with 3 = t, k 3 k t, a 3. Thus Theorem 5 implies 3 3 k t +1: (19) k t +1 Now let = c=, a otice that t=. Hece if lim!1 =, the (19) asymptotically reuces to =R, as i (1). Thus to support a asymptotically goo sequece of coes, c must grow liearly with, which meas that the umber of cycles c grows expoetially with. It woul be useful to fi out how the parameter = c=, which has to o with the umber of cycles, traes off versus the traitioal asymptotic parameters = = a R = k= as! 1. It woul be also iterestig to ivestigate, at least C. Geeral Taer Graphs Without Cycles We ow retur to the case of geeral Taer graphs, as efie i Sectio I, a observe that every geeral Taer graph (G; L) ca be coverte ito a simple Taer graph for the same coe through a vertexsplittig proceure. Iee, let y Ybe a check vertex i G, let fx i ;x i ; 111;x i gx be the eighborhoo of y, a let C be the correspoig costrait coe of legth. Ifim C =, we split y ito vertices y1;y ; 111;y a create eges betwee x i ;x i ; 111;x i a y1;y ; 111;y accorig to a paritycheck matrix H for C. A obvious but importat observatio is this: if H is cyclefree, the this proceure oes ot create ew cycles. Thus we have prove the followig statemet. Propositio 1: If a liear coe ca be represete by a geeral Taer graph (G; L) such that G is cyclefree a all the costraits i L are cyclefree, the ca be represete by a simple Taer graph without cycles. A immeiate cosequece of Propositio 1 is that all the results erive so far for simple Taer graphs, icluig the bou of Theorem 5, straightforwarly exte to geeral Taer graphs with cyclefree costraits. I the geeral case, where check costraits are ot ecessarily cyclefree, it appears to be very ifficult to say aythig about the structure/properties of the coe beig represete. As a example, cosier a geeral Taer graph for which cotais a sigle check vertex Y = fyg with the correspoig costrait coe beig itself. The existece of this cyclefree represetatio for obviously oes ot provie ay iformatio whatsoever about. Notwithstaig the trivial couterexample iscusse above, it is plausible that if the uerlyig Taer graph is cyclefree, the istace of shoul be limite by the istaces of the costrait coes i some maer. Furthermore, if simple ecoig is sought, simple costrait coes must be use. It thus appears that the rage of coe parameters that are possible with cyclefree Taer graphs will epe o the ecoig complexity tolerate. We leave further ivestigatio of this relatio as a ope problem. APPENDIX We will show that each of the three relatios (9), (11), a (1) betwee ; k, a erive i Sectio IVB implies (7), proviig is a iteger i (9), (11) a is a o iteger i (1). I orer to make the appeix selfcotaie, we ow restate these iequalities k 1 +1 (11) k 1 1: (1) k 1 Notice that what we are tryig to establish has othig to o with graphs or coes; this is just maipulatio of iteger iequalities. I particular, we have the followig simple lemma. Lemma 11: If a b=c a a; b; c are positive itegers, the a (b + a)=(c +1). The proof of Lemma 11 is straightforwar, a is left to the reaer. We first eal with (1), assumig is o. Takig the commo eomiator a applyig (twice) Lemma 11, we see that (1) implies (k 1) = ( +1) (9) 1: ()
9 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER Sice ( +1)= is a iteger for o, it follows from () that ( + 1)= b(+1)=(k+1)c. This may be rewritte as +1 1: (1) If +16 mo()the b( +1)=()c = b=()c, a (1) clearly implies (7). If ( +1)=()is a iteger, the (1) is precisely the equivalet form of (7) give i (1). It is easy to see that if is a o iteger, the (9) implies (1). Sice this case was alreay establishe above, it remais to prove (9) for eve. Usig oce agai Lemma 11, we see that (9) implies =(), or equivaletly = =(). Sice = is a iteger for eve, we ca take the iteger part of =()i the above expressio. It follows that for eve, we have which clearly implies (7). Fially, it ca be reaily see that if is a iteger, the (11) implies (9). Hece, our proof of Theorem 5 is ow complete. ACKNOWLEDGMENT The authors are grateful to Jeff Erickso, Amir Khaai, a Ralf Kötter for stimulatig iscussios. The authors woul also like to thak the aoymous referees for their valuable commets, which improve the presetatio of this correspoece. REFERENCES [1] S. M. Aji a R. J. McEliece, A geeral algorithm for istributig iformatio i a graph, i Proc. IEEE It. Symp. Iformatio Theory (Ulm, Germay, 1997), p. 6. [] S. M. Aji a R. J. McEliece, The geeralize istributive law, IEEE Tras. Iform. Theory, submitte for publicatio, July [3] L. R. Bahl, J. Cocke, F. Jeliek, a J. Raviv, Optimal ecoig of liear coes for miimizig symbol error rate, IEEE Tras. Iform. Theory, vol. IT, pp , [4] C. Berrou a A. Glavieux, Near optimum error correctig coig a ecoig: Turbocoes, IEEE Tras. Commu., vol. 44, pp , [5] C. Berrou, A. Glavieux, a P. Thitimajshima, Near Shao limit errorcorrectig coig a ecoig: Turbo coes, i Proc. IEEE It. Cof. Commuicatios (Geeva, Switzerla, 1993), pp [6] J. Bruck a M. Blaum, Neural etworks, errorcorrectig coes, a polyomials over the biary cube, IEEE Tras. Iform. Theory, vol. 35, pp , [7] M. Esmaeili a A. K. Khaai, Acyclic Taer graphs a maximumlikelihoo ecoig of liear block coes, preprit, May [8] J. Feigebaum, G. D. Forey Jr., B. H. Marcus, R. J. McEliece, a A. Vary, Es., Special Issue o Coes a Complexity, IEEE Tras. Iform. Theory, vol. 4, pp , Nov [9] G. D. Forey Jr., The forwarbackwar algorithm, i Proc. 34th Allerto Cof. Commuicatio, Cotrol, a Computig (Moticello, IL., Oct. 1996), pp [1] B. J. Frey, Bayesia etworks for patter classificatio, ata compressio, a chael coig, Ph.D. issertatio, Uiv. Toroto, Toroto, Ot., Caaa, July [11] R. G. Gallager, Lowesity paritycheck coes, IRE Tras. Iform. Theory, vol. IT8, pp. 1 8, 196. [1] R. G. Gallager, LowDesity ParityCheck Coes. Cambrige, MA: MIT Press, [13] S. L. Hakimi a J. Breeso, Graph theoretic errorcorrectig coes, IEEE Tras. Iform. Theory, vol. IT14, pp , [14] S. L. Hakimi a H. Frak, Cutset matrices a liear coes, IEEE Tras. Iform. Theory, vol. IT11, pp , [15] F. R. Kschischag, B. J. Frey, a H.A. Loeliger, Factor graphs a the sumprouct algorithm, IEEE Tras. Iform. Theory, submitte for publicatio, July [16] D. J. C. MacKay, Goo errorcorrectig coes base o very sparse matrices, IEEE Tras. Iform. Theory, vol. 45, pp , Mar [17], Gallager coes that are better tha turbo coes, i Proc. 36th Allerto Cof. Commuicatio, Cotrol, a Computig (Moticello, IL., Sept. 1998). [18] D. J. C. MacKay a R. M. Neal, Near Shao limit performace of lowesity paritycheck coes, Electro. Lett., vol. 3, pp , [19] F. J. MacWilliams a N. J. A. Sloae, The Theory of ErrorCorrectig Coes. New York: NorthHolla, [] J. Pearl, Probabilistic Reasoig i Itelliget Systems: Networks of Plausible Iferece. Sa Mateo, CA: Kaufma, [1] W. W. Peterso a E. J. Welo Jr., ErrorCorrectig Coes, e. Cambrige, MA: MIT Press, [] M. Sipser a D. A. Spielma, Expaer coes, IEEE Tras. Iform. Theory, vol. 4, pp , [3] P. Solé a T. Zaslavsky, The coverig raius of the cycle coe of a graph, Discr. Appl. Math., vol. 45, pp. 63 7, [4], A coig approach to sige graphs, SIAM J. Discr. Math., vol. 7, pp , [5] D. A. Spielma, Lieartime ecoable a ecoable coes, IEEE Tras. Iform. Theory, vol. 4, pp , [6] R. M. Taer, A recursive approach to lowcomplexity coes, IEEE Tras. Iform. Theory, vol. IT7, pp , [7] A. Vary, Trellis structure of coes, i Habook of Coig Theory, V. S. Pless a W. C. Huffma, Es. Amsteram, The Netherlas: Elsevier, 1998, pp [8] D. B. West, Itrouctio to Graph Theory Eglewoo Cliffs, NJ: PreticeHall, [9] N. Wiberg, Coes a ecoig o geeral graphs, Ph.D. issertatio, Dep. Elec. Eg., Uiv. Liköpig, Liköpig, Swee, Apr [3] N. Wiberg, H.A. Loeliger, a R. Kötter, Coes a iterative ecoig o geeral graphs, Euro. Tras. Telecommu., vol. 6, pp , TimeVaryig Perioic Covolutioal Coes With LowDesity ParityCheck Matrix Alberto Jiméez Felström, Member, IEEE, a Kamil Sh. Zigagirov, Seior Member, IEEE Abstract We preset a class of covolutioal coes efie by a lowesity paritycheck matrix a a iterative algorithm of the ecoig of these coes. The performace of this ecoig is close to the performace of turbo ecoig. Our simulatio shows that for the rate R = 1= biary coes, the performace is substatioally better tha for oriary covolutioal coes with the same ecoig complexity per iformatio bit. As a example, we costructe covolutioal coes with memory M = 15; 49; a 497 showig that we are about 1 B from the capacity limit at a biterror rate (BER) of 1 5 a a ecoig complexity of the same magitue as a Viterbi ecoer for coes havig memory M =1. Iex Terms Covolutioal coes, iterative ecoig, lowesity paritycheck coes. Mauscript receive Jue 19, 1997; revise March, The material i this correspoece was presete i part at the Iteratioal Symposium o Iformatio Theory, Cambrige, MA, August 16 1, The authors are with the Telecommuicatio Theory Group, Departmet of Iformatio Techology, Lu Uiversity, S1, Lu, Swee ( Commuicate by N. Seshari, Associate Eitor for Coig Techiques. Publisher Item Ietifier S (99) /99$ IEEE
Consistency of Random Forests and Other Averaging Classifiers
Joural of Machie Learig Research 9 (2008) 20152033 Submitted 1/08; Revised 5/08; Published 9/08 Cosistecy of Radom Forests ad Other Averagig Classifiers Gérard Biau LSTA & LPMA Uiversité Pierre et Marie
More informationSOME GEOMETRY IN HIGHDIMENSIONAL SPACES
SOME GEOMETRY IN HIGHDIMENSIONAL SPACES MATH 57A. Itroductio Our geometric ituitio is derived from threedimesioal space. Three coordiates suffice. May objects of iterest i aalysis, however, require far
More informationHOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1
1 HOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1 Brad Ma Departmet of Mathematics Harvard Uiversity ABSTRACT I this paper a mathematical model of card shufflig is costructed, ad used to determie
More information4. Trees. 4.1 Basics. Definition: A graph having no cycles is said to be acyclic. A forest is an acyclic graph.
4. Trees Oe of the importat classes of graphs is the trees. The importace of trees is evidet from their applicatios i various areas, especially theoretical computer sciece ad molecular evolutio. 4.1 Basics
More informationType Less, Find More: Fast Autocompletion Search with a Succinct Index
Type Less, Fid More: Fast Autocompletio Search with a Succict Idex Holger Bast MaxPlackIstitut für Iformatik Saarbrücke, Germay bast@mpiif.mpg.de Igmar Weber MaxPlackIstitut für Iformatik Saarbrücke,
More informationStéphane Boucheron 1, Olivier Bousquet 2 and Gábor Lugosi 3
ESAIM: Probability ad Statistics URL: http://wwwemathfr/ps/ Will be set by the publisher THEORY OF CLASSIFICATION: A SURVEY OF SOME RECENT ADVANCES Stéphae Bouchero 1, Olivier Bousquet 2 ad Gábor Lugosi
More informationON THE EVOLUTION OF RANDOM GRAPHS by P. ERDŐS and A. RÉNYI. Introduction
ON THE EVOLUTION OF RANDOM GRAPHS by P. ERDŐS ad A. RÉNYI Itroductio Dedicated to Professor P. Turá at his 50th birthday. Our aim is to study the probable structure of a radom graph r N which has give
More informationEverything You Always Wanted to Know about Copula Modeling but Were Afraid to Ask
Everythig You Always Wated to Kow about Copula Modelig but Were Afraid to Ask Christia Geest ad AeCatherie Favre 2 Abstract: This paper presets a itroductio to iferece for copula models, based o rak methods.
More informationSystemic Risk and Stability in Financial Networks
America Ecoomic Review 2015, 105(2): 564 608 http://dx.doi.org/10.1257/aer.20130456 Systemic Risk ad Stability i Fiacial Networks By Daro Acemoglu, Asuma Ozdaglar, ad Alireza TahbazSalehi * This paper
More informationSignal Reconstruction from Noisy Random Projections
Sigal Recostructio from Noisy Radom Projectios Jarvis Haut ad Robert Nowak Deartmet of Electrical ad Comuter Egieerig Uiversity of WiscosiMadiso March, 005; Revised February, 006 Abstract Recet results
More informationDryad: Distributed DataParallel Programs from Sequential Building Blocks
Dryad: Distributed DataParallel Programs from Sequetial uildig locks Michael Isard Microsoft esearch, Silico Valley drew irrell Microsoft esearch, Silico Valley Mihai udiu Microsoft esearch, Silico Valley
More informationCrowds: Anonymity for Web Transactions
Crowds: Aoymity for Web Trasactios Michael K. Reiter ad Aviel D. Rubi AT&T Labs Research I this paper we itroduce a system called Crowds for protectig users aoymity o the worldwideweb. Crowds, amed for
More informationWhich Extreme Values Are Really Extreme?
Which Extreme Values Are Really Extreme? JESÚS GONZALO Uiversidad Carlos III de Madrid JOSÉ OLMO Uiversidad Carlos III de Madrid abstract We defie the extreme values of ay radom sample of size from a distributio
More informationJ. J. Kennedy, 1 N. A. Rayner, 1 R. O. Smith, 2 D. E. Parker, 1 and M. Saunby 1. 1. Introduction
Reassessig biases ad other ucertaities i seasurface temperature observatios measured i situ sice 85, part : measuremet ad samplig ucertaities J. J. Keedy, N. A. Rayer, R. O. Smith, D. E. Parker, ad M.
More informationNo Eigenvalues Outside the Support of the Limiting Spectral Distribution of Large Dimensional Sample Covariance Matrices
No igevalues Outside the Support of the Limitig Spectral Distributio of Large Dimesioal Sample Covariace Matrices By Z.D. Bai ad Jack W. Silverstei 2 Natioal Uiversity of Sigapore ad North Carolia State
More informationBOUNDED GAPS BETWEEN PRIMES
BOUNDED GAPS BETWEEN PRIMES ANDREW GRANVILLE Abstract. Recetly, Yitag Zhag proved the existece of a fiite boud B such that there are ifiitely may pairs p, p of cosecutive primes for which p p B. This ca
More informationHow Has the Literature on Gini s Index Evolved in the Past 80 Years?
How Has the Literature o Gii s Idex Evolved i the Past 80 Years? Kua Xu Departmet of Ecoomics Dalhousie Uiversity Halifax, Nova Scotia Caada B3H 3J5 Jauary 2004 The author started this survey paper whe
More informationTeaching Bayesian Reasoning in Less Than Two Hours
Joural of Experimetal Psychology: Geeral 21, Vol., No. 3, 4 Copyright 21 by the America Psychological Associatio, Ic. 963445/1/S5. DOI: 1.7//963445..3. Teachig Bayesia Reasoig i Less Tha Two Hours Peter
More informationWhich Networks Are Least Susceptible to Cascading Failures?
Which Networks Are Least Susceptible to Cascaing Failures? Larry Blume Davi Easley Jon Kleinberg Robert Kleinberg Éva Taros July 011 Abstract. The resilience of networks to various types of failures is
More informationTesting for Welfare Comparisons when Populations Differ in Size
Cahier de recherche/workig Paper 039 Testig for Welfare Comparisos whe Populatios Differ i Size JeaYves Duclos Agès Zabsoré Septembre/September 200 Duclos: Départemet d écoomique, PEP ad CIRPÉE, Uiversité
More informationCatalogue no. 62557XPB Your Guide to the Consumer Price Index
Catalogue o. 62557XPB Your Guide to the Cosumer Price Idex (Texte fraçais au verso) Statistics Caada Statistique Caada Data i may forms Statistics Caada dissemiates data i a variety of forms. I additio
More informationThe Unicorn, The Normal Curve, and Other Improbable Creatures
Psychological Bulleti 1989, Vol. 105. No.1, 156166 The Uicor, The Normal Curve, ad Other Improbable Creatures Theodore Micceri 1 Departmet of Educatioal Leadership Uiversity of South Florida A ivestigatio
More informationA Kernel TwoSample Test
Joural of Machie Learig Research 3 0) 73773 Subitted 4/08; Revised /; Published 3/ Arthur Gretto MPI for Itelliget Systes Speastrasse 38 7076 Tübige, Geray A Kerel TwoSaple Test Karste M. Borgwardt Machie
More informationThe Arithmetic of Investment Expenses
Fiacial Aalysts Joural Volume 69 Number 2 2013 CFA Istitute The Arithmetic of Ivestmet Expeses William F. Sharpe Recet regulatory chages have brought a reewed focus o the impact of ivestmet expeses o ivestors
More informationAdverse Health Care Events Reporting System: What have we learned?
Adverse Health Care Evets Reportig System: What have we leared? 5year REVIEW Jauary 2009 For More Iformatio: Miesota Departmet of Health Divisio of Health Policy P.O. Box 64882 85 East Seveth Place, Suite
More informationTurning Brownfields into Greenspaces: Examining Incentives and Barriers to Revitalization
Turig Browfields ito Greespaces: Examiig Icetives ad Barriers to Revitalizatio Juha Siikamäki Resources for the Future Kris Werstedt Virgiia Tech Uiversity Abstract This study employs iterviews, documet
More informationSpinout Companies. A Researcher s Guide
Spiout Compaies A Researcher s Guide Cotets Itroductio 2 Sectio 1 Why create a spiout compay? 4 Sectio 2 Itellectual Property 10 Sectio 3 Compay Structure 15 Sectio 4 Shareholders ad Directors 19 Sectio
More informationare new doctors safe to practise?
Be prepared: are ew doctors safe to practise? Cotets What we foud 02 Why we ve writte this report 04 What is preparedess ad how ca it be measured? 06 How well prepared are medical graduates? 08 How has
More informationNo One Benefits. How teacher pension systems are failing BOTH teachers and taxpayers
No Oe Beefits How teacher pesio systems are failig BOTH teachers ad taxpayers Authors Kathry M. Doherty, Sadi Jacobs ad Trisha M. Madde Pricipal Fudig The Bill ad Melida Gates Foudatio ad the Joyce Foudatio.
More informationTrying Juveniles as Adults: An Analysis of State Transfer Laws and Reporting Patrick Griffin, Sean Addie, Benjamin Adams, and Kathy Firestine
U.S. Departmet of Justice Office of Justice Programs Office of Juveile Justice ad Deliquecy Prevetio Natioal Report Series September 2011 This bulleti is part of the Juveile Offeders ad Victims Natioal
More information