A Fast Distributed Algorithm for Decomposable Semidefinite Programs

Transcription

1 A as Disribued Agorihm for Decomposabe Semidefinie Programs Abdurahman Kaba and Javad Lavaei Deparmen of Eecrica Engineering, Coumbia Universiy Absrac his paper aims o deveop a fas, paraeizabe agorihm for an arbirary decomposabe semidefinie program SDP. o formuae a decomposabe SDP, we consider a muiagen canonica form represened by a graph, where each agen node is in charge of compuing is corresponding posiive semidefinie marix subec o oca equaiy and inequaiy consrains as we as overapping consisency consrains wih regards o he agen s neighbors. Based on he aernaing direcion mehod of muipiers, we design a numerica agorihm, which has a guaraneed convergence under very mid aspions. Each ieraion of his agorihm has a simpe cosed-form souion, consising of marix muipicaions and eigenvaue decomposiions performed by individua agens as we as informaion exchanges beween neighboring agens. he cheap ieraions of he proposed agorihm enabe soving rea-word arge-scae conic opimizaion probems. I. INRODUCION Aernaing direcion mehod of muipiers ADMM is a firs-order opimizaion agorihm proposed in he mid-970s by [] and []. his mehod has araced much aenion receny since i can be used for arge-scae opimizaion probems and aso be impemened in parae and disribued compuaiona environmens [], [4]. Compared o secondorder mehods ha are abe o achieve a high accuracy via expensive ieraions, ADMM reies on ow-compex ieraions and can achieve a modes accuracy in ens of ieraions. Inspired by Neserov s scheme for acceeraing gradien mehods [5], grea effor has been devoed o acceeraing ADMM and aaining a high accuracy in a reasonabe number of ieraions [6]. Since ADMM s performance is affeced by he condiion number of he probem s daa, diagona rescaing is proposed in [7] for a cass of probems o improve he performance and achieve a inear rae of convergence. he O n wors-case convergence rae of ADMM is proven in [8], [9] under he aspions of cosed convex ses and convex funcions no necessariy smooh. In [0], he O n convergence rae is obained for an asynchronous ADMM agorihm. he recen paper [] represens ADMM as a dynamica sysem and hen reduces he probem of proving he inear convergence of ADMM o verifying he sabiiy of a dynamica sysem []. Semidefinie programs SDP are aracive due in par o hree reasons. irs, posiive semidefinie consrains appear in many appicaions []. Second, SDPs can be used o sudy and approximae hard combinaoria opimizaion probems []. hird, his cass of convex opimizaion probems incudes inear, quadraic, quadraicay-consrained quadraic, Emai: [email protected] and [email protected]. his work was suppored by he ONR YIP Award, NS CAREER Award 579, and NS EECS Award and second-order cone programs. I is known ha sma- o medium-sized SDP probems can be soved efficieny by inerior poin mehods in poynomia ime up o any arbirary precision [4]. However, hese mehods are ess pracica for arge-scae SDPs due o compuaion ime and memory issues. However, i is possibe o somewha reduce he compexiy by expoiing any possibe srucure in he probem such as sparsiy. he pressing need for soving rea-word arge-scae opimizaion probems cas for he deveopmen of efficien, scaabe, and parae agorihms. Because of he scaabiiy of ADMM, he main obecive of his work is o design a disribued ADMM-based parae agorihm for soving an arbirary sparse arge-scae SDP wih a guaraneed convergence, under very mid aspions. We consider a canonica form of decomposabe SDPs, which is characerized by a graph of agens nodes and edges. Each agen needs o find he opima vaue of is associaed posiive semidefinie marix subec o oca equaiy and inequaiy consrains as we as overapping consrains wih is neighbors more precisey, he marices of wo neighboring agens may be subec o consisency consrains. he obecive funcion of he overa SDP is he maion of individua obecives of a agens. rom he compuaion perspecive, each agen is reaed as a processing uni and each edge of he graph specifies wha agens can communicae. We propose a disribued agorihm, whose ieraions comprise oca marix muipicaions and eigenvaue decomposiions performed by individua agens as we as informaion exchanges beween neighboring agens. his paper is organized as foows. An overview of ADMM is provided in Secion II. he disribued mui-agen SDP probem is formaized in Secion III. An ADMM-based parae agorihm is deveoped in Secion IV, by firs sudying he -agen case and hen invesigaing he genera mui-agen case. Simuaion resus on randomy-generaed arge-scae SDPs wih a few miion variabes are provided in Secion V. inay, some concuding remarks are drawn in Secion VI. Noaions: R n and S n denoe he ses of n rea vecors and n n symmeric marices, respecivey. Lower case eers e.g., x represen vecors, and upper case eers e.g., W represen marices. r{w } denoes he race of a marix W and he noaion W 0 means ha W is symmeric and posiive semidefinie. Given a marix W, is, m enry is denoed as W, m. he symbos, and denoe he ranspose, -norm for vecors and robenius norm for marices operaors, respecivey. he ordering operaor a, b reurns a, b if a < b and reurns b, a if a > b. he noaion X represens he cardinaiy or size of he se X. he finie

2 sequence of variabes x,..., x n is denoed by {x i } n i. or an m n marix W, he noaion W X, Y denoes he submarix of W whose rows and coumns are chosen form X and Y, respecivey, for given index ses X {,..., m} and Y {,..., n}. he noaion G V, E defines a graph G wih he verex or node se V and he edge se E. he se of neighbors of verex i V is denoed as Ni. o orien he edges of G, we define a new edge se E + {i, i, E and i < }. II. ALERNAING DIRECION MEHOD O MULIPLIERS Consider he opimizaion probem min fx + gy a x R n, y Rm subec o Ax + By c b where fx and gy are convex funcions, A, B are known marices, and c is a given vecor of appropriae dimension. he above opimizaion probem has a separabe obecive funcion and inear consrains. Before proceeding wih he paper, hree numerica mehods for soving his probem wi be reviewed. he firs mehod is dua decomposiion, which uses he Lagrangian funcion Lx, y, λ fx + gy + λ Ax + By c fx + λ Ax + gy + λ By λ c } {{ } } {{ } h x,λ h y,λ where λ is he Lagrange muipier corresponding o he consrain b. he above Lagrangian funcion can be separaed ino wo funcions h x, λ and h y, λ. Inspired by his separaion, he dua decomposiion mehod is based on updaing x, y and λ separaey. his eads o he ieraions x : argmin x y : argmin y h x, λ h y, λ λ : λ + α Ax + By c a b c for 0,,,..., wih an arbirary iniiaizaion x 0, y 0, λ 0, where α is a sep size. Noe ha argmin denoes any minimizer of he corresponding funcion. Despie is decomposabiiy, he dua decomposiion mehod has robusness and convergence issues. he mehod of muipiers coud be used o remedy hese difficuies, which is based on he augmened agrangian funcion L µ x, y, λ fx + gy + λ Ax + By c + µ Ax + By c where µ is a nonnegaive consan. Noice ha 4 is obained by augmening he Lagrangian funcion in wih a quadraic erm in order o increase he smaes eigenvaue of he Hessian of he Lagrangian wih respec o x, y. However, his augmenaion creaes a couping beween x and y. he ieraions corresponding o he mehod of muipiers are x, y : argmin x,y L µ x, y, λ λ : λ + µax + By c 4 5a 5b I i I i i n W W i W W n ig. : A graph represenaion of he disribued mui-agen SDP. where 0,,,... In order o avoid soving a oin opimizaion wih respec o x and y a every ieraion, he aernaing direcion mehod of muipiers ADMM can be used. he main idea is o firs updae x by freezing y a is aes vaue, and hen updae y based on he mos recen vaue of x. his eads o he -bock ADMM probem wih he ieraions [4]: Bock : Bock : x : argmin x y : argmin y L µ x, y, λ L µ x, y, λ 6a 6b Dua: λ : λ + µax + By c 6c ADMM offers a disribued compuaion propery, a high degree of robusness, and a guaraneed convergence under very mid aspions. In he reminder of his paper, we wi use his firs-order mehod o sove arge-scae decomposabe SDP probems. III. PROBLEM ORMULAION Consider a simpe, conneced, and undireced graph G V, E wih he node se V : {,..., n} and he edge se E V V, as shown in igure. In a physica conex, each node coud represen an agen or a machine or a processor or a hread and each edge represens a communicaion ink beween he agens. In he conex of his paper, each agen is in charge of compuing a posiive semidefinie marix variabe W i, and each edge i, E specifies an overap beween he marix variabes W i and W of agens i and. More precisey, each edge i, is accompanied by wo arbirary ineger-vaued index ses I i and I i o capure he overap beween W i and W hrough he equaion W i I i, I i W I i, I i. igure iusraes his specificaion hrough an exampe wih hree overapping marices, where every wo neighboring submarices wih an idenica coor mus ake he same vaue a opimaiy. Anoher way of hinking abou his seing is ha igure represens he sparsiy graph of an arbirary sparse arge-scae SDP wih a singe goba marix variabe W, which is hen reformuaed in erms of cerain marices of W, named W,..., W n, using he Chorda exension and marix compeion heorems [5]. he obecive of his paper is o sove he decomposabe SDP probem inerchangeaby referred o as disribued mui-agen SDP given beow.

3 4 5 I,, 4, 5 I,,, W W I, 5 I, 4 I 6, 7, 8 I,, W 5 5 ig. : An iusraion of he definiions of I i and I i for hree overapping submarices W, W and W Decomposabe SDP: min ra i W i i V subec o : 7a rb i W i c i,..., p i and i V 7b rd i W i d i,..., q i and i V 7c W i 0 i V 7d W i I i, I i W I i, I i i, E + 7e wih he variabes W i S ni for i,..., n, where he superscrip in i is no a power bu means ha he expression corresponds o agen i V. n i denoes he size of he submarix W i, and p i and q i show he numbers of equaiy and inequaiy consrains for agen i, respecivey. and d i denoe he h and h eemens of he vecors c i R pi and d i R qi for agen i, as defined beow: c i c i [c i,..., ci p i ], he marices A i, B i, and D i d i [d i,..., di q i ] are known and correspond o agen i V. he formuaion in 7 has hree main ingrediens: Loca obecive funcion: each agen i V has is own oca obecive funcion ra i W i wih respec o he oca marix variabe W i. he maion of a oca obecive funcions denoes he goba obecive funcion in 7a. Loca consrains: each agen i V has oca equaiy and inequaiy consrains 7b and 7c, respecivey, as we as a oca posiive semidefinieness consrain 7d. Overapping consrains: consrain 7e saes ha cerain enries of W i and W are idenica. he obecive is o design a disribued agorihm for soving 7, by aowing each agen i V o coaborae wih is neighbors Ni o find an opima vaue for is posiive semidefinie submarix W i whie meeing is own consrains as we as a overapping consrains. his is accompished by oca compuaions performed by individua agens and oca communicaion beween neighboring agens for informaion exchange. here are wo scenarios in which 7 coud be used. In he firs scenario, i is ased ha he SDP probem of ineres is associaed wih a mui-agen sysem and maches he formuaion in 7 exacy. In he second scenario, we consider an arbirary sparse SDP probem in he cenraized sandard form, i.e., an SDP wih a singe posiive semidefinie marix W, and hen conver i ino a disribued SDP wih muipe bu smaer posiive semidefinie marices W i o mach he formuaion in 7 noe ha a dense SDP probem can be pu in he form of 7 wih n. he conversion from a sandard SDP o a disribued SDP is possibe using he idea of chorda decomposiion of posiive semidefinie cones in [6], which expois he fac ha a marix W has a posiive semidefinie compeion if and ony if cerain submarices of W, denoed as W,..., W n, are posiive semidefinie [7]. In his work, we propose an ieraive agorihm for soving he decomposabe SDP probem 7 using he firs-order ADMM mehod. We show ha each ieraion of his agorihm has a simpe cosed-form souion, which consiss of marix muipicaion and eigenvaue decomposiion over marices of size n i for agen i V. Our work improves upon some recen papers in his area. [8] is a specia case of our work wih n, which does no offer any paraeizabe agorihm for sparse SDPs and may no be appicabe o arge-scae sparse SDP probems. [6] uses he cique-ree conversion mehod o decompose sparse SDPs wih chorda sparsiy paern ino smaer sized SDPs, which can hen be soved by inerior poin mehods bu his approach is imied by he arge number of consisency consrains for he overapping pars. Receny, [9] soves he decomposed SDP creaed by [6] using a firs-order spiing mehod, bu i requires soving a quadraic program a every ieraion, which again imposes some imiaions on he scaabiiy of he proposed agorihm. In conras, he agorihm o be proposed here is paraeizabe wih ow compuaions a every ieraion, wihou requiring any iniia feasibe poin unike inerior poin mehods. IV. DISRIBUED ALGORIHM OR DECOMPOSABLE SEMIDEINIE PROGRAMS In his secion, we design an ADMM-based agorihm o sove 7. or he convenience of he reader, we firs consider he case where here are ony wo overapping marices W and W. Laer on, we derive he ieraions for he genera case wih an arbirary graph G. A. wo-agen Case Ase ha here are wo overapping marices W and W embedded in a goba SDP marix variabe W as shown in igure, where * submarices of W are redundan meaning

4 4 ha here is no expici consrain on he enries of hese pars. he SDP probem for his case can be pu in he canonica form 7, by seing V {, }, E + {, } and V : min W S n W S n ra W + ra W 8a s.. rb W c,..., p 8b rb W c,..., p 8c rd W d,..., q 8d rd W d,..., q 8e W, W 0 8f W I, I W I, I 8g where he daa marices A, B,D S n, he marix variabe W S n and he vecors c R p and d R q correspond o agen, whereas he daa marices A, B,D S n, he marix variabe W S n and he vecors c R p and d R q correspond o agen. Consrain 8g saes ha he I, I submarix of W overaps wih he I, I submarix of W. Wih no oss of generaiy, ase ha he overapping par occurs a he ower righ corner of W and he upper ef corner of W, as iusraed in igure. he dua of he -agen SDP probem in 8 can be expressed as min c z + d v + c z + d v 9a subec o : p z B p z B H, H, v, v 0 R, R 0 q v D q v D +R R A 0 H 9b, H, 0 A 0 0 9c 9d 9e 9f wih he variabes z, z, v, v, R, R, H,, H,, where z R p, z R p, v R q and v R q are he Lagrange muipiers corresponding o he equaiy and inequaiy consrains in 8b-8e, respecivey, and he dua marix variabes R S n and R S n are he Lagrange muipier corresponding o he consrain 8f. he dua marix variabe H, is he Lagrange muipier corresponding o he submarix W I, I of W, whereas H, is he Lagrange muipier corresponding o he submarix W I, I of W. Since he overapping enries beween W and W are equa, as refeced in consrain 8g, he corresponding Lagrange muipiers shoud be equa as we, eading o consrain 9d. If we appy ADMM o 9, i becomes impossibe o spi he variabes ino wo bocks of variabes associaed wih agens and. he reason is ha he augmened Lagrangian funcion of 9 creaes a couping beween H, and H,, which hen requires updaing H, and H, oiny. his issue can be resoved by inroducing a new auxiiary variabe H, in order o decompose he consrain H, H, ino wo consrains H, H, and H, H,. Simiary, o make he updae of v and v easier, we do no impose posiiviy consrains direcy on v and v as in 9e. Insead, we impose he posiiviy on wo new vecors u, u 0 and hen add he addiiona consrains v u and v u. By appying he previous modificaions, 9 coud be rewrien in he decomposabe form min c i z i + d i v i + I + R i + I + v i 0a i subec o : p z B p z B H, H, H, H, v u v u q v D q v D +R R A 0 H 0b, H, 0 A 0 0 0c 0d 0e 0f 0g wih he variabes z, z, v, u, v, u, R, R, H,, H,, H,, where I + R i is equa o 0 if R i 0 and is + oherwise, and I + v i is equa o 0 if v i 0 and is + oherwise. o sreamine he presenaion, define and B i p i z i B i, Di q i H, fu 0 0, H 0 H, fu, v i D i, i, H, Noe ha Bi, Di, H, fu and Hfu, are funcions of he variabes z i, v i, H, and H,, respecivey, bu he argumens are dropped for noaiona simpiciy. he augmened Lagrangian funcion for 0 can be obained as L µ, M c i z i + d i v i + I + R i + I + v i i + µ B D + R H, fu A + G µ + µ B D + R H, fu A + G µ + µ H, H, + G, µ + µ H, H, + G, µ + µ v u + λ µ + µ v u + λ µ where z, z, v, v, u, u, R, R, H,, H,, H, is he se of opimizaion variabes and M

5 5 W W I, I W I, I W ig. : Posiive semidefinie marix W wo bocks G, G, G,, G,, λ, λ is he se of Lagrange muipiers whose eemens correspond o consrains 0b - 0g, respecivey. Noe ha he augmened Lagrangian in is obained using he ideniy r [ X A B ] + µ A B µ A B + X µ + consan 4 In order o proceed, we need o spi he se of opimizaion variabes ino wo bocks of variabes. o his end, define X { u, u, R, R, H,} and Y {z, z, v, v, H,, H, }. Using he mehod deineaed in Secion II, he wo-bock ADMM ieraions can be obained as Bock X argmin X Bock Y argmin G G + µ G G + µ G, G, + µ G, G, + µ Y B L µ X, Y, M L µ X, Y, M D + R B D + R H, H,, H, H λ λ + µ v u λ λ + µ v u fu H, A fu H, A 5a 5b 5c 5d 5e 5f 5g 5h for 0,,,... he above updaes are derived based on he fac ha ADMM aims o find a sadde poin of he augmened agrangian funcion by aernaivey performing one pass of Gauss Seide over X and Y and hen updaing he Lagrange muipiers M hrough Gradien ascen. I is sraighforward o show ha he opimizaion over X in Bock is fuy decomposabe and amouns o 5 separae opimizaion subprobems wih respec o he individua variabes u, u, R, R, H,. In addiion, he opimizaion over Y in Bock is equivaen o separae opimizaion subprobems wih he variabes z, v, H, and z, v, H,, respecivey. Ineresingy, a hese subprobems have cosedform souions. he corresponding ieraions ha need o be aken by agens and are provided in 6 and 7 given in he nex page. Noe ha hese agens need o perform oca compuaion in every ieraion according o 6 and 7 and hen exchange he updaed vaues of he pairs H,, G, and H,, G, wih one anoher. o eaborae on 6 and 7, he posiive semidefinie marices R and R are updaed hrough he operaor +, where X + is defined as he proecion of an arbirary symmeric marix X ono he se of posiive semidefinie marices by repacing is negaive eigenvaues wih 0 in he eigenvaue decomposiion [8]. he posiive vecors u and u are aso updaed hrough he operaor x +, which repaces any negaive enry in an arbirary vecor x wih 0 whie keeping he nonnegaive enries. Using he firs-order opimaiy condiion H,L µ 0, one coud easiy find he cosed-form souion for H, as shown in 6c and 7c. By combining he condiions z L µ 0, v L µ 0 and H, L µ 0, he updaes of z, v, H, and z, v, H, reduce o a no necessariy unique inear mapping, denoed as Lin in 6d and 7d due o non-uniqueness, we may have muipe souions, and any of hem can be used in he updaes. he Lagrange muipiers in M are updaed hrough Gradien ascen, as specified in 6e-6g for agen and in 7e- 7g for agen. B. Mui-Agen Case In his par, we wi sudy he genera disribued muiagen SDP 7. he dua of his probem, afer considering a modificaions used o conver 9 o 0, can be expressed in he decomposabe form min c i z i + d i v i + I + R i + I + v i 8a i V subec o : B i D i + R i k Ni H fu i,k A i i V 8b H i, H i, i, E + 8c H,i H i, i, E + 8d v i u i i V 8e wih he variabes z i, v i, u i, R i, H i,, H,i, H i, for every i V and i, E +, where Bi p i zi B i, Di q i vi D i and Hi k Ni Hfu i,k. Noe ha z i R pi and v i R qi are he Lagrange muipiers corresponding o he equaiy and inequaiy consrains in 7b and 7c, respecivey, and ha R i S ni is he Lagrange muipier corresponding o he consrain 7d. Each eemen h fu i,k a, b of Hfu i,k is eiher zero or equa o he Lagrange muipier corresponding o an overapping eemen W i a, b beween W i and W k. or a beer undersanding of he difference beween Hi, fu, H i, and Hi, an exampe is given in igure 4 for he case where agen is overapping wih agens and. he ADMM ieraions for he genera case can be derived simiary o he -agen case, which yieds he oca compuaion 9 for each agen i V. Consider he parameers defined in 0 for every i V, i, E +, and ime {,,,...}. Define V as

6 6 Ieraions for Agen Ieraions for Agen R B + D + Hfu, + A G u v + λ H, H, + H, + G, G, µ z, v, H, Lin u, R G B + µ G G, G, + µ λ λ + µ, H,, G, G,, λ D + R H fu, A H, H, v u 6a 6b 6c 6d 6e 6f 6g R B + D + Hfu, + A G u v + λ H, H, + H, + G, G, µ z, v, H, Lin u, R G B + µ G G, G, + µ λ λ + µ, H,, G, G,, λ D + R H fu, A H, H, v u 7a 7b 7c 7d 7e 7f 7g Ieraions for Agen i V R i B i + D i + H i u i vi + λ i + A i G i 9a 9b i,k H Hi,k + H k,i + G i,k G k,i k Ni 9c µ { } z i, v i, H i,k Lin u i, R i, k Ni { } } i,k H, G i {G, i,k, k Ni λ i 9d k Ni G i G i B + µ i D i + R i H i A i 9e G i,k G i,k + µ H i,k H i,k k Ni 9f λ i λ i + µ v i u i 9g V i V + i, E + p + i p4 + i d + i d i p + i, p + i, d i, Noe ha p, p, p, p4, d, d, d, and V are he prima residues, dua residues and aggregae residue for he decomposed probem 8. I shoud be noiced ha he dua residues are ony considered for he variabes in he bock X { u i, R i, H i,}. Since H i, appears wice in 8, he norm in he residue d is muipied by. he main resu of his paper wi be saed beow. heorem. Ase ha Saer s condiions hod for he decomposabe SDP probem 7. Consider he ieraive agorihm given in 9. he foowing saemens hod: p i B i + D i + H i + A i Ri p i, Hi, H i, p i, H,i H i, p4 i v i u i R d i i R i u d i i u i d i, H i, Hi, 0a 0b 0c 0d 0e 0f 0g he aggregae residue V aenuaes o 0 in a nonincreasing way as goes o +. or every i V, he imi of G, G,..., G n a + is an opima souion for W, W,..., W n. Proof. Afer reaizing ha 9 is obained from a wo-bock ADMM procedure, he heorem foows from [0] ha sudies he convergence of a sandard ADMM probem. he deais are omied for breviy. Since he proposed agorihm is ieraive wih an asympoic convergence, we need a finie-ime sopping rue. Based on [], we erminae he agorihm as soon as max{p, P, D, D, D, D 4, Gap} becomes smaer han a pre-

7 7 I bue,, 5 I orange 5, 7, H H, fu + H, fu H, H, ig. 4: An iusraion of he difference beween Hi, fu, H i, and Hi. Agen is overapping wih agens and agen a he enries specified by I and I. he whie squares in he ef marix H, fu + Hfu, represen hose enries wih vaue 0, and he coor squares carry Lagrange muipiers. specified oerance, where B i W i c i + max D i W i d i, 0 P i a + c i W i I i, I i W I i, I i P i, b + W i I i, I i + W I i, I i D i B i Di + R i Hi A i c + A i Hi, H i, D i, + H i, + H i, d H,i H i, D i, + H,i + H i, e v i u i D 4 i f + v i + u i i V c Gap i z i + d i v i r A i W i + i V c i z i + d i v i + i V r A iw i g for every i V and i, E +, where he eers P and D refer o he prima and dua infeasibiiies, respecivey. W i is he vecorized version of W i obained by sacking he coumns of W i one under anoher o creae a coumn vecor. B i and D i are marices whose coumns are he vecorized versions of B i and D i for,..., p i and,..., q i, respecivey. he sopping crieria in are based on he prima and dua infeasibiiies as we as he duaiy gap. V. SIMULAIONS RESULS he obecive of his secion is o eucidae he resus of his work on randomy generaed arge-scae srucured SDP probems. A prooype of he agorihm was impemened in MALAB and a of he simuaions beow were run on a apop wih an Ine Core i7 quad-core.5 GHz CPU and 8 GB RAM. or every i V, we generae a random insance of he probem as foows: Each marix A i is chosen as Ω + Ω + n i I, where he enries of Ω are uniformy chosen from he ineger se {,,, 4, 5}. his creaes reasonaby we-condiioned marices A i. Each marix B or D is chosen as Ω + Ω, where Ω is generaed as before. Each marix variabe W i is ased o be 40 by 40. he marices W,..., W n are ased o overap wih each oher in a banded srucure, associaed wih a pah graph G wih he edges,,,,..., n, n. One can regard W i s as submarices of a fu-scae marix variabe W in he form of igure bu wih n overapping bocks, where 5% of he enries of every wo neighboring marices W i and W i+ eading o a 0 0 submarix overaps. In order o demonsrae he proposed agorihm on argescae SDPs, hree differen vaues wi be considered for he oa number of overapping bocks or agens: 000, 000 and o give he reader a sense of how arge he simuaed SDPs are, he oa number of enries of W i s in he decomposed SDP probem N Decomp and he oa number of enries of W in he corresponding fu-sdp probem N u are ised beow: 000 agens: N u 0.9 biion, N Decomp.6 miion 000 agens: N u.6 biion, N Decomp. miion 4000 agens: N u 4.4 biion, N Decomp 6.4 miion he simuaion resus are provided in abe I wih he foowing enries: P ob and D ob are he prima and dua obecive vaues, ier denoes he number of ieraions needed o achieve a desired oerance, CPU and ier are he oa CPU ime in seconds and he ime per ieraion in seconds per ieraion, and Opimaiy in percenage is cacuaed as: Opimaiy Degree % 00 P ob D ob P ob 00 As shown in abe I, he simuaions were run for hree cases: p i 5 and q i 0: each agen has 5 equaiy consrains and no inequaiy consrains. p i 0 and q i 5: each agen has no equaiy consrains and 5 inequaiy consrains. p i 5 and q i 5: each agen has 5 equaiy consrains and 5 inequaiy consrains. A souions repored in abe I are based on he oerance of 0 and an opimaiy degree of a eas 99.9%. he aggregaive residue V is poed in igure 5 for he agen case wih p i q i 5, which is a monoonicay decreasing funcion. Noe ha he ime per ieraion is beween.66 and 8.0 in a MALAB impemenaion, which can be reduced significany in C++. Efficien and compuaionay cheap precondiioning mehods coud dramaicay reduce he number of ieraions, bu his is ouside he scope of his paper. VI. CONCLUSION his paper deveops a fas, paraeizabe agorihm for an arbirary decomposabe semidefinie program SDP. o formuae a decomposabe SDP, we consider a mui-agen canonica form represened by a graph, where each agen node is

8 8 Cases P ob 4.897e e e+6 D ob 4.896e e+5.966e+6 p i 5 ier q i 0 CPU sec ier sec per ier Opimaiy 99.98% 99.98% 99.98% P ob 8.6e e e+6 D ob 8.6e e e+6 p i 0 ier q i 5 CPU sec ier sec per ier Opimaiy % % % P ob 5.e+5.067e+6.84e+6 D ob 5.e+5.067e+6.8e+6 p i 5 ier q i 5 CPU sec ier sec per ier Opimaiy 99.98% 99.99% 99.99% ABLE I: Simuaion resus for hree cases wih 000, 000 and 4000 agens. Aggregae Residue Ieraions ig. 5: Aggregae residue for he case of 4000 agens wih p i q i 5. in charge of compuing is corresponding posiive semidefinie marix. he main goa of each agen is o ensure ha is marix is opima wih respec o some measure and saisfies oca equaiy and inequaiy consrains. In addiion, he marices of wo neighboring agens may be subec o overapping consrains. he obecive funcion of he opimizaion is he of a obecives of individua agens. he moivaion behind his formuaion is ha an arbirary sparse SDP probem can be convered o a decomposabe SDP by means of he Chorda exension and marix compeion heorems. Using he aernaing direcion mehod of muipiers, we deveop a disribued agorihm o sove he underying SDP probem. A every ieraion, each agen performs simpe compuaions marix muipicaion and eigenvaue decomposiion wihou having o sove any opimizaion subprobem, and hen communicaes some informaion o is neighbors. By deriving a Lyapunov-ype non-increasing funcion, i is shown ha he proposed agorihm converges as ong as Saer s condiions hod. Simuaions resus on arge-scae SDP probems wih a few miion variabes are offered o eucidae he efficacy of his work. REERENCES [] D. Gabay and B. Mercier, A dua agorihm for he souion of noninear variaiona probems via finie eemen approximaion, Compuers and Mahemaics wih Appicaions, vo., no., pp. 7 40, 976. [] R. Gowinski and A. Marroco, Sur approximaion, par mens finis d ordre un, e a rsouion, par pnaisaion-duai d une casse de probmes de diriche non inaires, ESAIM: Mahemaica Modeing and Numerica Anaysis - Modisaion Mahmaique e Anayse Numrique, vo. 9, no. R, pp. 4 76, 975. [Onine]. Avaiabe: hp://eudm.org/doc/969 [] J. Ecksein and W. Yao, Augmened agrangian and aernaing direcion mehods for convex opimizaion: A uoria and some iusraive compuaiona resus, RUCOR Research Repors, vo., 0. [4] S. Boyd, N. Parikh, E. Chu, B. Peeao, and J. Ecksein, Disribued opimizaion and saisica earning via he aernaing direcion mehod of muipiers, oundaions and rends R in Machine Learning, vo., no., pp., 0. [5] Y. Neserov, A mehod of soving a convex programming probem wih convergence rae O/k, Sovie Mahemaics Dokady, vo. 7, no., pp. 7 76, 98. [6]. Godsein, B. O Donoghue, S. Sezer, and R. Baraniuk, as aernaing direcion opimizaion mehods, SIAM Journa on Imaging Sciences, vo. 7, no., pp , 04. [7] P. Gisesson and S. Boyd, Diagona scaing in Dougas Rachford spiing and ADMM, 5rd IEEE Conference on Decision and Conro, 04. [8] B. He and X. Yuan, On he O/n convergence rae of he Dougas Rachford aernaing direcion mehod, SIAM Journa on Numerica Anaysis, vo. 50, no., pp , 0. [9] R. D. C. Moneiro and B.. Svaier, Ieraion-compexiy of bockdecomposiion agorihms and he aernaing direcion mehod of muipiers, SIAM Journa on Opimizaion, vo., no., pp , 0. [0] E. Wei and A. Ozdagar, On he O/k Convergence of Asynchronous Disribued Aernaing Direcion Mehod of Muipiers, ArXiv e-prins, Ju. 0. [] R. Nishihara, L. Lessard, B. Rech, A. Packard, and M. I. Jordan, A genera anaysis of he convergence of ADMM, arxiv preprin arxiv: , 05. [] J. Lavaei and S. H. Low, Zero duaiy gap in opima power fow probem, IEEE ransacions on Power Sysems, vo. 7, no., pp. 9 07, 0. [] M. X. Goemans and D. P. Wiiamson, Improved approximaion agorihms for maximum cu and saisfiabiiy probems using semidefinie programming, Journa of he ACM JACM, vo. 4, no. 6, pp. 5 45, 995. [4] L. Vandenberghe and S. Boyd, Semidefinie programming, SIAM Review, vo. 8, pp , 994. [5] R. Madani, G. azenia, S. Sooudi, and J. Lavaei, Low-rank souions of marix inequaiies wih appicaions o poynomia opimizaion and marix compeion probems, IEEE Conference on Decision and Conro, 04. [6] M. ukuda, M. Koima, K. Muroa, and K. Nakaa, Expoiing sparsiy in semidefinie programming via marix compeion I: Genera framework, SIAM Journa on Opimizaion, vo., no., pp , 00. [7] R. Grone, C. R. Johnson, E. M. Sá, and H. Wokowicz, Posiive definie compeions of paria hermiian marices, Linear agebra and is appicaions, vo. 58, pp. 09 4, 984. [8] Z. Wen, D. Godfarb, and W. Yin, Aernaing direcion augmened agrangian mehods for semidefinie programming, Mahemaica Programming Compuaion, vo., no. -4, pp. 0 0, 00. [9] Y. Sun, M. S. Andersen, and L. Vandenberghe, Decomposiion in conic opimizaion wih pariay separabe srucure, SIAM Journa on Opimizaion, vo. 4, no., pp , 04. [0] B. He and X. Yuan, On non-ergodic convergence rae of Dougas Rachford aernaing direcion mehod of muipiers, Numerische Mahemaik, pp., 0. [] H. Miemann, An independen benchmarking of SDP and SOCP sovers, Mahemaica Programming, vo. 95, no., pp , 00. [Onine]. Avaiabe: hp://dx.doi.org/0.007/s