Decomposition Methods for Large Scale LP Decoding

Transcription

1 Decomposton Methos for Large Scale LP Decong Sharth Barman Xshuo Lu Stark Draper Benjamn Recht Abstract Felman et al. IEEE Trans. Inform. Theory, Mar. 2005) showe that lnear programmng LP) can be use to ecoe lnear error correctng coes. The bt-error-rate performance of LP econg s comparable to state-of-theart BP ecoers, but has sgnfcantly stronger theoretcal guarantees. However, LP econg when mplemente wth stanar LP solvers oes not easly scale to the block lengths of moern error correctng coes. In ths paper we raw on ecomposton methos from optmzaton theory to evelop effcent strbute algorthms for LP econg. The key enablng techncal result s a nearly lnear tme algorthm for two-norm projecton onto the party polytope. Ths allows us to use LP econg, wth all ts theoretcal guarantees, to ecoe large-scale error correctng coes effcently. I. INTRODUCTION Felman et al. n [4] propose a novel econg algorthm for low ensty party check LDPC) coes base on lnear programmng LP). They thereby pose the econg problem n the framework of convex optmzaton. They show that the bt-error-rate performance of ths LP-base ecoer s comparable to teratve econg an can correct a constant number of errors. The performance of ths econg algorthm can accurately be establshe usng pseuo-coewors. Upon success the algorthm proves a certfcate of correctness ML certfcate). Stanar LP solvers can be employe to solve the unerlyng econg program n poly-tme but o not mmeately have a strbute nature. In contrast, econg algorthms that follow the belef propagaton framework o not have strong theoretcal convergence guarantees but o posses a strbute nature. Ths s a esrable trat as t leas to scalablty va parallel mplementatons. In ths work we explot the rch structure nherent to the LP formulaton to evelop effcent strbute algorthms for solvng the econg problem. The result s a theoretcally strong metho for effcently econg moern error correctng coes wth large block lengths. Over the years there has been sgnfcant work on ecomposton methos an scalable algorthms n the optmzaton communty. One establshe technque s the alternatng recton metho of multplers ADMM) see Boy et al. [1] an the references theren). ADMM s well sute for strbute convex optmzaton. Strong convergence guarantees have been establshe for ADMM. ADMM s a robust Department of Computer Scences, Unversty of Wsconsn - Mason. s@cs.wsc.eu. Department of Electrcal an Computer Engneerng, Unversty of Wsconsn - Mason. xlu94@wsc.eu. Department of Electrcal an Computer Engneerng, Unversty of Wsconsn - Mason. sraper@ece.wsc.eu. Department of Computer Scences, Unversty of Wsconsn - Mason. brecht@cs.wsc.eu. metho that has successfully been apple to a number of large-scale problems n machne learnng an statstcs. In ths paper we apply ADMM to evelop an effcent ecouple econg algorthm. The feasble regon of the econg LP s state usng the so-calle party polytope. In the ADMM framework projectng onto the party polytope s a nontrval step. We evelop a new characterzaton of the party polytope an base on that evelop a fast nearly lnear tme projecton algorthm, whch allows us to solve the econg LP va ADMM. II. BACKGROUND AND RELATED WORK In ths paper we conser a bnary lnear LDPC coe C of length N efne by a M N party-check matrx H. Each of M party checks, nexe by J = {1, 2,..., M}, correspons to a row n the party check matrx H. Coewor symbols are nexe by the set I = {1, 2,..., N}. The neghborhoo of a check j, enote by N c j), s the set of nces I that partcpate n the jth party check,.e., N c j) = { H j, = 1}. Smlarly for a component I, N v ) = {j H j, = 1}. Gven a vector x {0, 1} N, the jth party-check s sa to be satsfe f N x cj) s even. In other wors, the bts assgne to x for N c j) have even party. We say that a length-n bnary vector x s a coewor, x C, f an only f ff) all party checks are satsfe. In a regular LDPC coe there s a fxe constant, such that for all checks j J, N c j) =. Also for all components I, N v ) s a fxe constant. We focus on regular LDPC coes but our technques an results exten to general LDPC coes. Let P j be the bnary N matrx that selects out the components of x that partcpate n the jth check. For example, say the neghborhoo of the jth check, N c j) = { 1, 2,... }, where 1 < 2 <... <. Then, for all k [] the k, k )th entry of P j s one, the remanng entres are zero. For any coewor x C, P j x s an even party vector of menson for all j. We begn by escrbng maxmum lkelhoo econg an the LP relaxaton propose by Felman et al. Say vector x s receve over a bnary symmetrc channel BSC), wth cross over probablty p. Maxmum lkelhoo ML) econg selects a coewor x C that maxmzes p x x), the probablty that x was receve gven that x was sent. For the channel at han we have p x x) = I p x x ). Equvalently, we select a coewor that maxmzes I log p x x ). Let ) γ be the negatve log-lkelhoo ) p x 0) p x 1) p 1 p rato, γ := log. Then, γ = log f x = 1 ) an γ = log 1 p p f x = 0. Snce log p x x ) = γ x + log p x 0), ML econg reuces to etermnng

2 an x C that mnmzes γ x. Thus, ML econg requres mnmzng a lnear functon over the set of coewors. The feasble regon n the econg LP s escrbe usng the party polytope, PP = conv{e {0, 1} e 1 }), the convex hull of all -mensonal bnary vectors wth an even number of 1s. Note that for all j, P j x s a vertex of PP. The relaxaton propose by Felman et al. enforces that for all checks j J, P j x PP nstea of beng a vertex. Puttng these ngreents together yels the LP: mnmze γ T x s.t. P j x PP j J 1) The unerlyng structure of the formulaton was use by Vontobel an Koetter n [10] to evelop strbute messagepassng type algorthms to solve the econg LP. Ther algorthms are base on the coornate-ascent metho whch, when matche wth the approprate scheulng etermne by Burshten n [2], converge to the optmal soluton. In Yea et al [12] fference-map BP s evelope, a smple strbute algorthm whch seems to recover the performance of LP econg, but ose not have convergence guarantees. In ths paper we frame the LP econg problem n the template of an ADMM problem. ADMM s strbute, has strong convergence guarantees an, n general, s more robust than coornate ascent. We gve the general formulaton of ADMM problems an specalze t to the LP econg problem n Sec. III. When evelopng the ADMM upate steps we fn that one of the steps requre projectng onto the party polytope. Thus, n Sec. IV we evelop the effcent projecton algorthm requre. We present numercal results n Sec. V. III. DECOUPLED RELAXATION AND OPTIMIZATION ALGORITHMS In ths secton we present the ADMM formulaton of the LP econg problem an summarze our contrbutons. In Sec, III-A we ntrouce the general ADMM template. We specalze the template to our problem n Sec. III-B. We state the algorthm n Sec. III-C. A. ADMM formulaton To make the LP 1) ft nto the ADMM template we relax x to le n the hypercube, x [0, 1] N, an a the auxlary replca varables z j R for all j J. We work wth the followng parameterzaton of the econg LP. mnmze γ T x subject to P j x = z j z j PP j J j J x [0, 1] N 2) The alternatng recton metho of multples works wth an augmente Lagrangan whch, for ths problem, s L µ x, z, λ) := γ T x + λ T j P j x z j ) j J + µ P j x z j j J Here λ j R for j J are the Lagrange multplers an µ > 0 s a fxe penalty parameter. We use λ an z to succnctly represent the collecton of λ j s an z j s respectvely. Note that the augmente Lagrangan s obtane by ang the two norm term of the resual to the Lagrangan. Say X an Z are the feasble regons for varables x an z respectvely nuce by [0, 1] N an the PP ), ADMM conssts of the followng teratons: x k+1 := argmn x X L µ x, z k, λ k ) z k+1 := argmn z Z L µ x k+1, z, λ k ) λ k+1 j := λ k j + µ P j x k+1 z k+1 ) j The ADMM upate steps nvolve fxng one varable an mnmzng the other. In partcular, x k an z k are the kth terate an the upates to the x an z varable are performe n an alternatng fashon. We use ths framework to solve the LP relaxaton propose by Felman et al. an hence evelop a strbute econg algorthm. B. ADMM Upate Steps The x-upate correspons to fxng z an λ obtane from the prevous teraton or ntalze at the begnnng) an mnmzng L µ x, z, λ) subject to x [0, 1] N. For the augmente Lagrangan at han the x-upate smplfes to x = Π [0,1] N P 1 j P T j z j 1 ) µ λ j 1 µ γ. Here P = j P j T P j an Π [0,1] N ) correspons to projectng onto the hypercube [0, 1] N. The latter can easly be accomplshe by nepenently projectng the components onto [0, 1]. Note that for any j, Pj T P j s a N N agonal bnary matrx wth non-zero entres at, ) ff N c j). Ths mples that j P j T P j s a agonal matrx wth the, )th entry equal to N v ). Hence P 1 = j P j T P j) 1 s a agonal matrx wth 1/ N v ) as the th agonal entry. Component-wse, the upate rule correspons to takng the average of the corresponng replca values, z j, ajuste by the the scale ual varable, λ j /µ, an takng a step n the negatve log-lkelhoo recton. For any j N v ) let z ) j enote the component of z j that correspons to the th component of x, n other wors the th component of Pj T z j. Smlarly let λ ) j be the th component of Pj T λ j. Wth ths notaton the upate rule for the th component of x s x = Π [0,1] 1 z ) j 1 ) N v ) µ λ) j 1 µ γ. j N v) Component-wse the x-upate reuces to a type of averagng, each of whch can be one n parallel.

3 The z-upate correspons to fxng x an λ an mnmzng L µ x, λ, z) subject to z j PP for all j J. The relevant observaton here s that the augmente Lagrangan s separable wth respect to z j s an hence the mnmzaton step splts nto J separate problems that can each be solve nepenently. Ths ecouples the unerlyng algorthm makng t scalable. For each j J the upate s to fn the z j that mnmzes µ 2 P jx z j 2 2 λ T j z j s.t. z j PP. Snce the values of x an λ are fxe so are P j x an λ j /µ. Settng v = P j x + λ j /µ an completng the square we get that the esre upate z j s z j = argmn z PP v z 2 2. Thus, the z-upate correspons to projectng onto the party polytope. Recall that the party polytope PP s the the convex hull of all -mensonal bnary vectors wth even Hammng weght. In [6] Jeroslow gves an explct representaton of the party polytope. Later, n [11] Yannakaks mproves ths to prove a quaratc n terms of the menson ) representaton,.e., the total number of nvolve constrants n the econg LP s quaratc n see [4]). Whle ths LP can be solve wth stanar solvers n polynomal tme, the quaratc sze of the LP mght be prohbtve n real-tme or embee econg applcatons. In Sec. IV we evelop a new characterzaton of the party polytope. We show that for all vectors u PP there exsts an even nteger r < such that u can be expresse as a convex combnaton of -mensonal bnary vectors of Hammng weght r or r + 2. Of course, any vector n the party polytope s a convex combnaton of bnary vectors of even Hammng weght. Our characterzaton shows that, n fact, vectors of only two weghts are requre. We term the lower weght, r, the consttuent party of the vector. The consttuent party s trvally solve for. Base on ths representaton we evelop a near-lnear tme projecton algorthm. Roughly, our approach s as follows. Gven a vector v R we frst compute r, the consttuent party of ts projecton. Let PP r PP r+2 ) enote the convex hull of all -mensonal bnary vectors of Hammng weght r r + 2). Gven our characterzaton, projectng onto the polytope s equvalent to etermnng an α [0, 1], a vector a PP r, an a vector b PP r+2 such that the l 2 norm of v αa 1 α)b s mnmze. We evelop an algorthm n Sec. IV-D) that, for a fxe α, etermnes the optmal a PP r an b PP r+2. The functon mn a PP r, b PP r+2 v αa 1 α)b 2 2 s convex n α. Hence we can perform perform a one-mensonal lne search usng, for example, the secant metho) to etermne the optmal value for α an thence the esre projecton. The algorthm that, for a gven α, solves for the optmal a an b, frst projects the gven vector onto αpp r an then projects the resual onto 1 α)pp r+2 ; αpp r s a scale verson of PP r. Projecton onto αpp r cf. Sec. IV-D) can be performe n O log ) tme usng a type of reverse waterfllng algorthm. Thus, our approach gves an effcent metho for projectng onto the party polytope. C. ADMM Decong Algorthm The complete ADMM-base algorthm s specfe below. We eclare convergence when the replcas ffer form the optmal x varable by less than some tolerance ɛ > 0. Algorthm 1 Gven a bnary N-mensonal vector x {0, 1} N, party check matrx H, an parameters µ an ɛ, solve the econg LP specfe n 2) 1: Construct the negatve log-lkelhoo vector γ base on receve wor x. 2: Construct the N matrx P j for all j J. 3: Intalze z j an λ j as the all zeros vector for all j J. 4: repeat ) 1 N v) j N v) 5: Upate x [0,1] for all I. 1 µ γ 6: for all j J o 7: Set v j = P j x + λ j /µ. 8: Upate z j Π PP v j ) where Π PP ) means project onto the party polytope. 9: Upate λ j λ j + µ P j x z j ). 10: en for 11: untl max j P j x z j < ɛ return x. z ) j 1 µ λ) j IV. PROJECTING ONTO THE PARITY POLYTOPE In ths secton we evelop our effcent projecton algorthm. We set notaton n Sec. IV-A. We evelop our twoslce representaton of any pont n PP n Sec. IV-B. Gven any u R, n Sec. IV-C we connect the weght of the projecton of u onto PP to the easly compute) consttuent party of the projecton of u onto the unt hypercube. Fnally, n Sec. IV-D we evelop the projecton algorthm. A. Notaton Let P = { e {0, 1} e 1 }. The party polytope PP = convp ), the convex hull of P. Note that v PP ff there exst e P such that v = α e where α = 1 an α 0. We enote the set of -mensonal bnary vectors wth party r as P r = {e {0, 1} e 1 = r} an efne PP r = convp r ). Gven a R + by a even we enote the smallest even nteger greater than or equal to a an by a even the largest even nteger less than or equal to a. We enote the projecton of a vector v onto a convex set Ω by Π Ω v). B. Structural Characterzaton Let v an w be -vectors sorte n ecreasng orer. The vector w s sa to majorze v f q q v k w k 1 q <, v k = w k. ))

4 We make use of the followng Theorem see [7] an references theren). Theorem 1: Suppose v an w are -vectors sorte n ecreasng orer. Then v s n the convex hull of all permutatons of w f an only f w majorzes v. From ths theorem we conclue that a sorte vector v [0, 1] s n the PP s ff q v k mnq, s) 1 q <, 3) v k = s. 4) The relatons follow by notng the frst sum s less than q an the secon sum must equal s for v to be n PP s. By efnton, any v PP can be expresse as a convex combnaton of the vertces of PP. Usng e s) th vertex of party s an γ s) weghtng, we wrte v = γs) we have to enote the the corresponng non-negatve γs) e s). Defnng µ s = µ s = 1, µ 0. 5) Summng the frst q coornates of v an efnng e s),k to be the kth component of e s), we apply 3) to get q q γ s) v k = µ s e s),k µ s µ s mnq, s) 1 q <. 6) In aton, snce the e s) are all of weght s, when we sum over all coornates we get v k = µ s s. 7) Lemma 1: Two-slce lemma) Suppose v PP an r s the even nteger satsfyng r v k r + 2, then v can be expresse as a convex combnaton of vectors of party r an r + 2. Let α, 0 α 1 be such that v 1 = αr + 1 α)r + 2) = r + 21 α). In the followng proof α plays the role of µ r an 1 α) plays the role of µ r+2. To show Lemma 1 what we nee to show s that there s a representaton of any v PP such that µ s = 0 for all s except r an r + 2. Proof: Gven the efnton of α an the entfcatons µ r = α an µ r+2 = 1 α), 6) an 7) smplfy to q v k α mnq, r) + 1 α) mnq, r + 2) 1 q <, 8) v k = αr + 1 α)r + 2). 9) By the efnton of α, 9) s satsfe. In 8) the cases q r an q r + 2 are rather straghtforwar. For any q < r, snce there are only q terms n 8) an v k 1 for all k then, e.g., mn{q, r} = q an 8) must hol. For any q r +2 we use 9) to wrte q v k = αr+1 α)r+2) q+1 v k αr + 1 α)r + 2) snce v k 0. So to prove contanment n PP, t remans to verfy only one more nequalty n 8). Namely, we nee to show that r+1 v k αr + 1 α)r + 1) = r + 1 α). By assumpton, v an µ satsfy 5) an 6). Thus r+1 v k µ s mns, r + 1) 10) must hol. We now show that the largest value attanable for the rght han se s precsely r + 1 α. To see ths, conser the lnear program µ s mns, r + 1) maxmze subject to µ ss = r + 21 α) µ s = 1 µ s 0. The ual program s mnmze r + 2 2α)u 1 + u 2 subject to u 1 s + u 2 mns, r + 1). Settng µ r = α, µ r+2 = 1 α), u 1 = 1/2, u 2 = r/2, an all other prmal varables to zero satsfes the Karush-Kuhn- Tucker KKT) contons for ths prmal/ual par of LPs. The assocate optmal cost s r + 1 α. Thus, the rght han se of 10) s at least r + 1 α, completng the proof. Another useful consequence of Theorem 1 s the followng corollary. Corollary 1: Let v be a vector n [0, 1]. If =1 v s an even nteger then v PP. Proof: Let v = s. Snce v s majorze by a sorte bnary vector of party s then, by Theorem 1, v PP s whch, n turn, mples v PP. We call the even nteger v 1 even the consttuent party of vector v. C. Consttuent Party of the Projecton In ths secton we prove a useful boun on the l 1 norm of the projecton of any u R. Ths wll proves us the consttuent party of the projecton. Lemma 2: For any vector u R, enote by ω the projecton of u onto [0, 1] an enote by π the projecton of u onto the party polytope. The followng boun hols: ω 1 even π 1 ω 1 even. Proof: Let ρ U = ω 1 even an ρ L = ω 1 even. We prove the followng fact: gven any y PP wth y 1 > ρ U there exts a vector y [0, 1] such that y 1 = ρ U, y PP, an u y 2 2 < u y 2 2. The mplcaton of ths fact wll be that any vector n the party polytope wth

5 l 1 norm strctly greater that ρ U cannot be the projecton of u. Smlarly we can also show that any vector wth l 1 norm strctly less than ρ L cannot be the projecton on the party polytope. Frst we construct the vector y base on y an w. Defne the set of hgh values to be the coornates on whch y s greater than w,.e., H := { [] y > ω }. Snce by assumpton y 1 > ρ U ω 1 we know that H 1. Conser the test vector t efne component-wse as { ω f H t = y else Note that t 1 ω 1 ρ U < y 1. The vector t ffers from y only n H, thus by changng reucng) components of y n the set H we can obtan a vector y such that y 1 = ρ U. In partcular there exsts a vector y wth y 1 = ρ U such that for H : y y ω an for / H : y = y. Snce the l 1 norm of y an s n [0, 1] we have by Corollary 1 that y PP. We next show that for all H, u y u y. The nequalty wll be strct for at least one yelng u y 2 2 < u y 2 2 an thereby provng the clam. We start by notng that y PP so y [0, 1] for all. Hence, f for some, ω < y we must also have ω < 1, n whch case u ω snce ω s the projecton of u onto [0, 1]. In summary, ω < 1 ff u < 1 an when ω < 1 then u ω. Combnng wth the fact that y [0, 1] we have that f y > ω then ω u. Thus for all H we get y y ω u where the frst nequalty s strct for at least one. Snce y = y for / H ths means that u y u y for all where the nequalty s strct for at least one value of. Overall then, u y 2 2 < u y 2 2 an both y PP by constructon) an y PP by assumpton). Thus, y cannot be the projecton of u onto PP. Thus the l 1 norm of the projecton of u, π 1 ρ U. A smlar argument shows that π 1 ρ L an so π 1 must le n [ρ L, ρ U ] D. Projecton Algorthm In ths secton we evelop the projecton algorthm. Gven a vector v R enote by ω the projecton of v on [0, 1] an set r = ω 1 even. From Lemma 2 we know that the consttuent party of the projecton of v onto PP s r. In orer to etermne the projecton usng the representaton of Sec. IV-B we nee to solve the followng quaratc program mn α [0,1] = mn α [0,1] mn t PP r+2 mn v αs 1 α)t 2 2 s PP r mn t 1 α)pp r+2 mn s αpp r v s t 2 2, 11) where αpp r = {αy y PP r }. In other wors αpp r s the convex hull of the set {e {0, α} e 0 = r}. Projectng onto PP r : To evelop our projecton onto PP r we note that, for any α [0, 1], a vector y αpp r f an only f 0 y α an y = αr. =1 The problem of projectng v onto αpp r s thus: 1 mn y 2 v y 2 2 subject to 0 y α an y = αr. 12) Ths problem s equvalent to projectng onto the surface of a l 1 ball of raus αr wth box constrants. We evelop an algorthm smlar to the one n [5] to accomplsh ths task. The key fference between projectng onto αpp r an the problem consere n [5] s that the l 1 norm constrant s enforce as an nequalty n [5] whereas we wsh to mpose an equalty,.e., we nee to etermne a vector wth l 1 norm exactly equal to αr. Recallng that Π Ω v) enotes the projecton of v onto Ω n the followng we use Π αpp r ) an Π 1 α)pp r+2 ) for concseness. To evelop ntuton for the quaratc program 12) we wrte own the KKT contons. Snce the objectve functon an the nequalty constrants are convex an the equalty constrant s affne the KKT contons are not only necessary but are also suffcent. We assocate ual varables γ, µ an θ wth the constrants an wrte the corresponng Lagrangan as Ly, µ, γ, θ) = 1 2 v y ) 2 θ µ α y ) rα γ y. y ) The KKT contons state that an optmal soluton, y = Π αpp r v), satsfes Ly ) = 0 whch, for the above Lagrangan, mples that for all v y = θ + µ γ. 13) Furthermore µ α y ) = 0 an γ y = 0. Hence for all such that 0 < y < α we must have µ = 0 an γ = 0. In other wors, for all such that y 0 an y α, the fference v y s exactly the same: v y = θ. We efer the etals of the algorthm along wth a proof of correctness an an analyss of ts tme complexty to the appenx. The algorthm guarantees that the constructe vector satsfes the KKT contons. The suffcency of KKT contons mples that the constructe vector s optmal. The algorthm s a reverse waterfllng type algorthm consstng of two passes. The algorthm works wth three sets: the clppe set C := { y = α}, the actve set A := { 0 < y < α} an the zero set Z = { y = 0}. Usng the KKT contons we now argue that the largest components of v belong to C, the smallest to Z an the rest to A. Frst, conser any C an j A. We show that v > v j. By the KKT contons, for any C we can express v as v = θ + µ + y = θ + µ + α where γ = 0. In aton,

6 for any j A we have alreay note that v j = θ + yj. Then, snce yj < α because j A an snce µ 0, we know that v > v j. Next, for any nex k Z by the KKT contons we can wrte v k = θ γ so, snce γ 0 we see that v j > v k. What the above tells us s that we shoul sort the nput vector v component-wse n non-ncreasng orer at the begnnng of the algorthm. If we o ths the nces n C wll be strctly smaller than the nces n A, an the nces n A wll be strctly smaller that the nces n Z. However, we stll nee to etermne the change ponts between sets. To see how to etermne these refer to Fg. 1. There s a lower waterfllng level θ an an upper level θ +α. Components of v larger than the upper level are n C an components of v smaller than the lower level are n Z. The total waterfllng buget s consume by the C terms n C an the contrbuton of the terms n A. Ths s equal to the total areas of the shae bars n the fgure at levels θ an θ + α. Conceptually, our algorthm starts wth θ hgh an lowers t untl the relaton α C + A v θ ) = αr s satsfe, whch follows from the fact that y = αr. Numercally we can test all possble sets of break-ponts. As there are only 2 choces ths s a lnear search. Once the sets have been etermne we use the same relaton α C + A v θ ) = αr to solve for θ. Wth the nex sets an θ at han we can rectly etermne nvual components of the projecton y. The algorthmc complexty s omnate by the ntal sort, hence t s O log ). Fg. 1. Clppe set y *, C Actve set Zero set y v y 0 * y, A *, Z v values sorte)... Water-fllng Algorthm for etermnng Π αpp r v) Projectng onto PP : We now evelop an effcent algorthm to solve the quaratc program n 11),.e., to project a vector v R onto PP. The algorthm executes a bnary search over α, for each frst projectng the v onto αpp r an then projectng the resual onto 1 α)pp r+2. Let Ω α = αpp r + 1 α)pp r+2, n other wors, Ω α enotes the followng convex set: {w+w w αpp r, w 1 α)pp r+2 }. For a gven α [0, 1] let y α = Π Ωα v),.e., the program state n 11) wth α fxe. We show that Algorthm 2, below, etermnes y α. Fnally, Π PP v) = argmn {yα} α [0,1] v y α 2 2 gves the esre result. Algorthm 2 Gven v R an α [0, 1] etermne Π Ωα v) 1: Set s Π αpp r v). 2: Set t Π 1 α)pp r+2v s ). 3: Return s + t. Frst we make a comment on the computatonal complexty of the algorthm. Usng the waterfllng approach the projectons n the frst two steps of the algorthm can be performe n O log ) tme. Snce there are only two projectons, Algorthm 2 executes n O log ) tme. To prove the correctness of the algorthm we begn by provng a useful fact about t, where t s etermne n the secon step of the algorthm. Lemma 3: For any vector t 1 α)pp r+2 we have v s t ) T t t ) 0. Proof: Snce t s the projecton of v s on 1 α)pp r+2 we get the esre nequalty by the projecton theorem. Next we show that for any s αpp r we have v s t ) T s s ) 0. Combnng ths wth the prevous lemma we get that v s t ) T s + t s t ) 0 for any s αpp r an t 1 α)pp r+2. Hence for any u Ω α we have v s t ) T u s t ) 0. An agan, by the projecton theorem, we get that s + t = Π Ωα v). Lemma 4: For any vector s αpp r we have v s t ) T s s ) 0. Proof: We show that s = Π αpp r v t ), that s s s the projecton of v t on αpp r, whch n turn gves us the requre nequalty. When we project v onto αpp r we get that, for some θ 1, the followng contons hol: s = 0 f v θ 1 α f v α θ 1. 14) v θ 1 f v α < θ 1 < v These contons are erve from the KKT contons of the corresponng quaratc program as n 13) cf. Secton 4.1 n [5] for a general scusson). The relatons n 14) mply that the nex set {1, 2,..., } s parttone nto three parts: U, M an L. In the frst s = α, n the secon 0 < s < α, an n the thr s = 0. Ths s agramme n Fg. 2. Defnng = v s, the above contons mply that θ 1 for all U; = θ 1 for all M an θ 1 for L. Note that t = Π 1 α)pp r+2 ). Defnng ρ = v s t, we have that for all nces, j M the followng equalty hols: t = t j. Hence, there exsts λ R such that for all M ρ = λ. Ths follows from wrtng contons smlar to 14) for t an nput vector. In partcular, for some θ 2 we have the followng contons on t.

7 v s s = α s 0,α) s =0 = v s θ 1 = θ 1 U M L Fg. 2. KKT Contons θ 1 0 f θ 2 t = 1 α f 1 α) θ 2. 15) θ 2 f 1 α) < θ 2 < A useful consequence s that ρ = t s component-wse sorte n the same orer as. That s, for we have ρ 1 ρ 2... ρ. Fnally we prove that s satsfes the KKT conton of the followng quaratc program wth nput vector w = v t. That s s s the optmal soluton of mnmze 1 2 w y 2 2 subject to 0 y α an y = αr. Note that ρ = w s. For all M we have by efnton that s 0, α) an, as state above, ρ = λ. Snce ρ s sorte component-wse we have that for all U, ρ λ. Along smlar lnes, for all L, ρ λ. Hence s satsfes the KKT contons for the above quaratc program, n partcular 13) wth θ = λ. Overall ths mples that s = Π αpp r w). Hence for all s αpp r we have w s ) T s s ) 0, whch proves the lemma. We establsh the correctness of Algorthm 2 n the followng lemma. Lemma 5: Gven vector v R an scalar α [0, 1] let s an t be the projectons etermne by Algorthm 2. Then s + t = Π Ωα v). Proof: By efnton for any u Ω α there exsts s αpp r an t 1 α)pp r+2 such that u = s+t. Applyng the prevous two lemmas we get that v s t ) T s+t s t ) 0 for any s αpp r an t 1 α)pp r+2. Therefore for any u Ω α we have v s t ) T u s t ) 0. Hence by the projecton theorem we get that s + t = Π Ωα v). We note that, for a fxe vector v, the functon fα) = v Π Ωα v) 2 2 s convex, an hence we can perform bnary search over α [0, 1] wth Algorthm 2 as a subroutne an n accorance wth the level of accuracy etermne Π PP v). Ths mples the followng theorem. Theorem 2: Gven vector v R we can etermne Π PP v) wth δ [0, 1] precson n tme O log log1/δ)). V. NUMERICAL RESULTS In ths secton, we present smulaton results for our ADMM econg algorthm on two fferent coes. The frst coe s the 155,64) LDPC coe esgne by Tanner et al [9]. The other coe s a 1057,244) LDPC coe stue by Yea et al. n [12]. In Fg. 3 we plot the the error performance of ADMM econg for the 155,64) coe. The parameters use n ths smulaton are: µ = 1.5, ɛ = 1e-4, T max = 200 an δ = 1e-6. For comparson we plot the WER performance of LP econg usng the smplex metho, mplemente wth aaptve LP econg [8], results rawn from [3]. The performance of the two ecoers matches closely. wor error rate LP econg ADMM econg Cross Over Probablty Fg. 3. Error performance comparson between ADMM econg an LP econg for the 155,64) LDPC coe. The WER s plotte as a functon of crossover probablty. In Fg. 4 we plot results for the 1057,244) coe. We agan smulate the bnary symmetrc channel but, to facltate comparson wth [12], we plot the WER performance as a functon of the equvalent sgnal-to-nose rato SNR) γ. Ths s the SNR of a bnary-moulate sgnal transmtte over the atve whte Gaussan nose AWGN) channel couple wth har-ecson econg. The relatonshp between crossover probablty p an γ s p = Q 2γ), where Q ) s the Q-functon. The results from [12] are labele E-BP-LP, whch s the ecoer evelope n [12] to get estmates of LP econg at very low WERs. See [12] for etals. In ths set of smulatons we nvestgate the convergence rate of the algorthm by testng performance uner fferent maxmum number of teratons: T max = 50, T max = 100 an T max = 300. The other system parameters are µ = 2, ɛ = 1e-4 an δ = 1e-6. We also nvestgate the error performance uner fferent values of the µ parameter. The error performance when µ = 10 an T max = 300 s very close to the performance foun n [12]. Note also that n both smulatons, we accumulate more than 200 econg errors for each ata pont. We woul lke to emphasze two mportant mplementatonal aspects of our ADMM ecoer. Frst, the one mensonal searchng metho use to optmze α has a sgnfcant mpact on the overall effcency of the ecoer. We compare the golen secton search wth the secant metho an results

8 wor error rate ADMM econg, Tmax = 50, µ = 2 ADMM econg, Tmax = 100, µ = 2 ADMM econg, Tmax = 300, µ = 2 ADMM econg, Tmax = 300, µ = 10 E BP LP acheves WER of LP econg) Eb/N0B) Fg. 4. Error performance comparson between ADMM econg an LP econg for the 1057,244) LDPC coe. The WER s plotte as a functon of SNR. show that the secant metho can be three-to-four tmes faster than the golen secton search. Ths makes the secant metho our choce for the smulatons. Secon, parameters n the algorthm play an mportant role n the ecoer. These parameters nclue termnaton precson δ use n the secant metho, µ an ɛ n Algorthm 1 an the maxmum number of teratons T max allowe for ADMM. Tunng these parameters can affect both error performance an program effcency. We have not fully optmze all of these parameters, an our expermental results mght be further mprove after careful tunng. We efer ths tunng an experments on enser coes for future work. VI. CONCLUSION In ths paper we apply the ADMM template to the LP econg problem ntrouce n [4]. A man techncal hurle was the evelopment of an effcent metho of projectng a vector onto the party polytope. We accomplshe ths n three steps. We frst ntrouce a new representaton of ponts n the party polytope. We then use the representaton to show that projecton can be one n a two-step manner. Fnally we showe that each step conssts of an effcent waterfllng-type algorthm. We emonstrate the effectveness of our econg technque on two coes, on the 155, 64) LDPC coe ntrouce n [9], the secon a 1057, 244) LDPC coe stue n [12]. In those papers the LP econg performance of these coes was foun usng the smplex metho as the LP solver. We reprouce those results usng our ADMM base metho. In contrast to smplex-base methos our ADMM-base metho s strbute n nature an scales to larger block lengths. REFERENCES [1] S. Boy, N. Parkh, E. Chu, B. Peleato, an J. Ecksten. Dstrbute optmzaton an statstcal learnng va the alternatng recton metho of multplers. Machne Learnng, 31):1 123, [2] D. Burshten. Iteratve approxmate lnear programmng econg of LDPC coes wth lnear complexty. Informaton Theory, IEEE Transactons on, 5511): , [3] S.C. Draper, J.S. Yea, an Y. Wang. ML econg va mxenteger aaptve lnear programmng. In Informaton Theory, ISIT IEEE Internatonal Symposum on, pages IEEE, [4] J. Felman, M.J. Wanwrght, an D.R. Karger. Usng lnear programmng to ecoe bnary lnear coes. Informaton Theory, IEEE Transactons on, 513): , [5] M.D. Gupta, S. Kumar, an J. Xao. L 1 projectons wth box constrants. Arxv preprnt arxv: , [6] RG Jeroslow. On efnng sets of vertces of the hypercube by lnear nequaltes. Dscrete Mathematcs, 112): , [7] A.W. Marshall, I. Olkn, an Arnol B.C. Inequaltes: theory of majorzaton an ts applcatons. Sprnger, [8] M.H.N. Taghav an P.H. Segel. Aaptve methos for lnear programmng econg. Informaton Theory, IEEE Transactons on, 5412): , [9] R. M. Tanner, D. Srhara, an T. Fuja. A class of group-structure LDPC coes. In Proc. ICSTA, Amblese, UK, [10] P.O. Vontobel an R. Koetter. On low-complexty lnear-programmng econg of LDPC coes. European transactons on telecommuncatons, 185): , [11] M. Yannakaks. Expressng combnatoral optmzaton problems by lnear programs. Journal of Computer an System Scences, 433): , [12] J.S. Yea, Y. Wang, an S.C. Draper. Dve an concur an fference-map BP ecoers for LDPC coes. IEEE Transactons on Informaton Theory, 572): , 2011.