CONSIDER a connected network of n nodes that all wish

Transcription

1 36 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 Coded Cooperatve Data Exhange n Multhop Networks Thomas A. Courtade, Member, IEEE, and Rhard D. Wesel, Senor Member, IEEE Abstrat Consder a onneted network of n nodes that all wsh to reover k desred pakets. Eah node begns wth a subset of the desred pakets and exhanges oded pakets wth ts neghbors. Ths paper provdes neessary and suffent ondtons that haraterze the set of all transmsson strateges that permt every node to ultmately learn (reover) all k pakets. When the network satsfes ertan regularty ondtons and pakets are randomly dstrbuted, ths paper provdes tght onentraton results on the number of transmssons requred to aheve unversal reovery. For the ase of a fully onneted network, a polynomal-tme algorthm for omputng an optmal transmsson strategy s derved. An applaton to serey generaton s dsussed. Index Terms Coded ooperatve data exhange, unversal reovery, network odng. I. INTRODUCTION CONSIDER a onneted network of n nodes that all wsh to reover k desred pakets. Eah node begns wth a subset of the desred pakets and broadasts messages to ts neghbors over dsrete, memoryless, and nterferene-free hannels. Furthermore, every node knows whh pakets are already known by eah node and knows the topology of the network. How many transmssons are requred to dssemnate the k pakets to every node n the network? How should ths be aomplshed? These are the essental questons addressed. We refer to ths as the Coded Cooperatve Data Exhange (CCDE) problem, or just the Cooperatve Data Exhange problem. Ths work s motvated n part by emergng ssues n dstrbuted data storage. Consder the problem of bakng up data on servers n a large data enter. One ommonly employed method to protet data from orrupton s replaton. Usng ths method, large quanttes of data are replated n several loatons so as to protet from varous soures of orrupton (e.g., equpment falure, power outages, natural dsasters, et.). As the quantty of nformaton n large data Manusrpt reeved Marh 5, 202; revsed May 3, 203; aepted July 5, 203. Date of publaton November 4, 203; date of urrent verson January 5, 204. Ths work was supported n part by Rokwell Collns under Contrat and n part by the NSF Center for Sene of Informaton under Grant CCF Ths paper was presented at the 200 Mltary Communatons Conferene [] and the Allerton Conferene on Communaton, Control, and Computng [2], [3]. T. A. Courtade was wth the Department of Eletral Engneerng, Unversty of Calforna, Los Angeles, CA USA. He s now wth the Department of Eletral Engneerng, Stanford Unversty, Stanford, CA USA (e-mal: ourtade@stanford.edu). R. D. Wesel s wth the Eletral Engneerng Department, Unversty of Calforna, Los Angeles, CA USA (e-mal: wesel@ee.ula.edu). Communated by M. Franeshett, Assoate Edtor for Communatons. Color versons of one or more of the fgures n ths paper are avalable onlne at Dgtal Objet Identfer 0.09/TIT enters ontnues to nrease, the number of fle transfers requred to omplete a perod replaton task s beomng an nreasngly mportant onsderaton due to tme, equpment, ost, and energy onstrants. The results ontaned n ths paper address these ssues by haraterzng the mnmum number of multast transmssons that must take plae n order to suessfully replate the data olletvely held by the servers at all loatons. Ths model also has natural applatons n the ontext of tatal networks, and we gve one of them here. Consder a senaro n whh an arraft fles over a group of nodes on the ground and tres to delver a vdeo stream. Eah ground node mght only reeve a subset of the transmtted pakets due to nterferene, obstrutons, and other sgnal ntegrty ssues. In order to reover the transmsson, the nodes are free to ommunate wth ther nearby neghbors, but would lke to mnmze the number of transmssons n order to onserve battery power (or avod deteton, et.). How should the nodes share nformaton, and what s the mnmum number of transmssons requred so that the entre network an reover the vdeo stream? The results n the present paper provde nsght nto how a ooperatve protool at the applaton-layer (so that ommunaton lnks appear noseless and error-free) an be optmzed. Beyond the examples mentoned above, the results presented heren an also be appled to pratal serey generaton amongst a olleton of nodes. We onsder ths applaton n detal n Seton IV. A. Related Work Dstrbuted data exhange problems have reeved a great deal of attenton over the past several years. The powerful tehnques afforded by network odng [4], [5] have paved the way for ooperatve ommunatons at the paket-level. The oded ooperatve data exhange problem (also alled the unversal reovery problem n [] [3]) was orgnally ntrodued by El Rouayheb et al. n [6], [7] for a fully onneted network (.e., a sngle-hop network). For ths speal ase, a randomzed algorthm for fndng an optmal transmsson strategy was gven n [8], and the frst determnst algorthm was reently gven n [9]. In the onludng remarks of [9], the weghted unversal reovery problem (n whh the objetve s to mnmze the weghted sum of transmssons by nodes) was posed as an open problem. However, ths was solved usng a varant of the same algorthm n [0], and ndependently by the present authors usng the submodular optmzaton algorthm presented heren (orgnally appearng n [3]) IEEE. Personal use s permtted, but republaton/redstrbuton requres IEEE permsson. See for more nformaton.

2 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 37 The oded ooperatve data exhange problem s related to the ndex odng problem orgnally ntrodued by Brk and Kol n []. Spefally, generalzng the ndex odng problem to permt eah node to be a transmtter (nstead of havng a sngle server) and further generalzng so that the network need not be a sngle hop network leads to a lass of problems that nludes our problem as a speal ase n whh eah node desres to reeve all pakets. Also losely related to the ooperatve data exhange problem s the extensve work on gossp algorthms for nformaton dssemnaton (f. [2, Chapter 3] and the referenes theren). Gossp algorthms are defned as a lass of deentralzed network ommunaton protools whh an be employed to dssemnate data wthn a network. Gossp algorthms for data dssemnaton assume eah node ntally begns wth a unque pee of data and that nodes ommunate n a unast fashon. In ontrast, the oded ooperatve data exhange problem allows for nodes to ntally possess ommon data and permts multast transmssons. Moreover, typal results n the lterature on gossp algorthms show that a gossp protool an dssemnate data n order-optmal tme, whereas n the oded ooperatve data exhange problem, we seek to haraterze the exat number of transmssons whh must take plae n order to suessfully dssemnate data. Despte these fundamental dfferenes, we would lke to pont out that the work on gossp algorthms that s most losely related to the present paper s [3]. In [3], the authors show that a gossp algorthm employng random lnear network odng an attan order optmal dssemnaton tme. Smlarly, n the present paper, we show that lnear network odng s suffent to optmally solve the oded ooperatve data exhange problem. Ths paper apples prnples of ooperatve data exhange to generate serey n the presene of an eavesdropper. In ths ontext, the serey generaton problem was orgnally studed n [4]. In [4], Csszar and Narayan gave sngle-letter haraterzatons of the seret-key and prvate-key apates for a network of nodes onneted by an error-free broadast hannel. Whle general and powerful, these results left two pratal ssues as open questons. Frst, (as wth many nformaton-theoret nvestgatons) the results requre the nodes to observe arbtrarly long sequenes of..d. soure symbols, whh s generally not pratal. Seond, no onstrutve algorthm s provded n [4] whh aheves the respetve serey apates. More reent work n [5], [6] addressed the latter pont. Also related, though to a lesser extent, s the body of work on seret-sharng shemes. We refer the reader to the survey [7] and the referenes theren for further detals. B. Our Contrbutons In ths paper, we provde neessary and suffent ondtons for ahevng unversal reovery n arbtrarly onneted multhop networks. We spealze these neessary and suffent ondtons to obtan prese results n the ase where the underlyng network topology satsfes some modest regularty ondtons. In ths paper, we use the term unversal reovery to refer to the ultmate ondton where every node has suessfully reovered all pakets. For the ase of a fully onneted network, we provde an algorthm based on submodular optmzaton whh solves the ooperatve data exhange problem. Ths algorthm s unque from the others prevously appearng n the lterature (f. [8] [0]) n that t explots submodularty 2. As a orollary, we provde exat onentraton results when pakets are randomly dstrbuted n a network. In ths same ven, we also obtan tght onentraton results and approxmate solutons when the underlyng network s d-regular and pakets are dstrbuted randomly. Furthermore, f pakets are dvsble (allowng transmssons to onsst of partal pakets), we prove that the tradtonal ut-set bounds an be aheved for any network topology. In the ase of d-regular and fully onneted networks, we show that splttng pakets does not typally provde any sgnfant benefts. Fnally, for the applaton to serey generaton, we leverage the results of [4] n the ontext of the ooperatve data exhange problem for a fully onneted network. In dong so, we provde an polynomal-tme algorthm that aheves the serey apaty wthout requrng any quanttes to grow asymptotally large. C. Organzaton Ths paper s organzed as follows. Seton II formally ntrodues the problem and provdes bas defntons and notaton. Seton III presents our man results. Seton IV dsusses the applaton of our results to serey generaton by a olleton of nodes n the presene of an eavesdropper. Seton V ontans the relevant proofs. Seton VI delvers the onlusons and dsusses dretons for future work. II. SYSTEM MODEL AND DEFINITIONS Before we formally ntrodue the problem, we establsh some notaton. Let N = 0,, 2,... denote the set of natural numbers. For two sets A and B, the relaton A B mples that A s a proper subset of B (.e., A B and A = B). If t s lear from ontext that A s a subset of a ground set B, the shorthand notaton A wll be used to denote B\A. Foraset A, the orrespondng power set s denoted 2 A := {B : B A}. We use the notaton [m] to denote the set {,...,m}. Ths paper onsders a network of n nodes. The network must be onneted, but t need not be fully onneted (.e., t need not be a omplete graph). A graph G = (V, E) desrbes the spef onnetons n the network, where V s the set of vertes {v : {,...,n}} (eah orrespondng to a node) and E s the set of edges onnetng nodes. We assume that the edges n E are undreted. Eah node wshes to reover the same k desred pakets, and eah node begns wth a (possbly empty) subset of the desred pakets. Formally, let P {p,...,p k } be the (ndexed) set of pakets orgnally avalable at node, and{p } n = satsfes n= P = {p,...,p k }. Eah p j F, wheref s some 2 We note that [8], whh appeared around the same tme as [3], proposed an algorthm explotng submodularty for omputng optmal rate alloatons n the losely related ommunaton for omnsene problem (see [4]). We omment on the relatonshp n Seton III-D and Appendx A.

3 38 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 Fg.. For the gven graph, a set of vertes S and ts neghborhood Ɣ(S) are depted. The set (S) (.e., the boundary of S) onssts of the four vertes n Ɣ(S) whh are not n S. fnte feld 3 (e.g.f = GF(2 m )). For our purposes, t suffes to assume F 2n. The set of pakets ntally mssng at node s denoted P := {p,...,p k }\P. Throughout ths paper, we assume that eah paket p {p,...,p k } s equally lkely to be any element of F. Moreover, we assume that pakets are ndependent of one another. Thus, no orrelaton between dfferent pakets or pror knowledge about unknown pakets an be exploted. To smplfy notaton, we wll refer to a gven problem nstane (.e., a graph and orrespondng sets of pakets avalable at eah node) as a network T ={G, P,...,P n }. When no ambguty s present, we wll refer to a network by T and omt the mplt dependene on the parameters {G, P,...,P n }. Let the set Ɣ() be the neghborhood of node. There exsts an edge e E onnetng two vertes v,v j V ff Ɣ( j). For onvenene, we put Ɣ(). Node sends (possbly oded) pakets to ts neghbors Ɣ() over dsrete, memoryless, and nterferene-free hannels. In other words, f node transmts a message, then every node n Ɣ() reeves that message. If S s a set of nodes, then we defne Ɣ(S) = S Ɣ(). In a smlar manner, we defne (S) = Ɣ(S)\S to be the boundary of the vertes n S. An example of sets S, Ɣ(S), and (S) s gven n Fgure. Ths paper seeks to determne the mnmum number of transmssons requred to aheve unversal reovery (when every node has learned all k pakets). Wth the exepton of the results gven n Seton III-D, we fous exlusvely on the ase where pakets are deemed ndvsble, motvatng the followng defntons: Defnton : A transmsson by node onssts of sendng a paket (.e., somez F) to all nodes j Ɣ(). Defnton 2: Gven a network T, the mnmum number of transmssons requred to aheve unversal reovery s denoted M (T ). To larfy these defntons, we brefly onsder two examples: Example (Lne Network): Suppose T s a network of nodes onneted along a lne as follows: V ={v,v 2,v 3 }, 3 We restrt pakets to be an element of a feld F for notatonal smplty. In prate, pakets are typally omprsed of a vetor of feld elements, and arthmet n the vetor spae s performed usng the operatons nherted from F. Fg. 2. An llustraton of the transmsson strategy employed n Example. Durng the frst tme nstant, Nodes and 3 broadast pakets p and p 2, respetvely. Durng the seond tme nstant, Node 2 broadasts the XOR of pakets p and p 2. Ths strategy requres three transmssons and aheves unversal reovery. E ={(v,v 2 ), (v 2,v 3 )}, P ={p }, P 2 =,andp 3 ={p 2 }. Note that eah node must transmt at least one n order for all nodes to reover {p, p 2 }, hene M (T ) 3. Suppose node transmts p and node 3 transmts p 2. Then (upon reept of p and p 2 from nodes and 3, respetvely) node 2 transmts p p 2,where ndates addton n the fnte feld F. Ths strategy requres 3 transmssons and allows eah node to reover {p, p 2 }. Hene M (T ) = 3. Example demonstrates a transmsson shedule that uses two rounds of ommunaton. The transmssons by node n a partular round of ommunaton an depend only on the nformaton avalable to node pror to that round (.e.p and prevously reeved transmssons from neghborng nodes). In other words, the transmssons are ausal. The transmsson strategy employed n Example s llustrated n Fgure 2. Example 2 (Fully Conneted Network): Suppose T s a 3-node fully onneted network n whh G s a omplete graph on 3 vertes, and P ={p, p 2, p 3 }\p. Clearly one transmsson s not suffent, thus M (T ) 2. It an be seen that two transmssons suffe: let node transmt p 2 whh lets node 2 have P 2 {p 2 } = {p, p 2, p 3 }. Now, node 2 transmts p p 3, allowng nodes and 3 to eah reover all three pakets. Thus M (T ) = 2. Sne eah transmsson was only a funton of the pakets orgnally avalable at the orrespondng node, ths transmsson strategy an be aomplshed n a sngle round of ommunaton. In the above examples, we note that the transmsson strateges are partally haraterzed by a shedule dtatng whh nodes transmt durng whh round of ommunaton. We formalze ths noton wth the followng defnton: Defnton 3 (Transmsson Shedule): A set of ntegers {b j : [n], j [r], b j N} s alled a transmsson shedule for r rounds of ommunaton f node makes exatly b j transmssons durng ommunaton round j. When the parameters n and r are lear from ontext, a transmsson shedule wll be denoted by the shorthand notaton {b j j }. Note that b an exeed one, mplyng that multple transmssons per ommunaton round are permtted. However, any transmsson shedule wth multple

4 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 39 transmssons per round an easly be transformed to a transmsson shedule wth one or fewer transmssons per round by smply nreasng r. Although fndng a transmsson shedule that permts unversal reovery s relatvely easy (e.g., eah node transmts all pakets n ther possesson at eah tme nstant), fndng one that permts unversal reovery wth M (T ) transmssons an be extremely dffult. Ths s demonstrated by the followng example: Example 3 (Optmal CCDE s NP-Hard.): Suppose T s a network wth k = orrespondng to a bpartte graph wth left and rght vertex sets V L and V R respetvely. Let P ={p } for eah V L,andletP = for eah V R. In ths ase, M (T ) s gven by the mnmum number of sets n {Ɣ()} VL whh over all vertes n V R. Thus, fndng M (T ) s at least as hard as the Mnmum Set Cover problem, whh s NP-omplete [9]. Reently, Gonen and Langberg have studed the hardness of the ooperatve oded data exhange problem n greater detal. Sne we do not address ths ssue beyond the example above, we refer the reader to ther work [20] for a detaled treatment. Several of our results are stated n the ontext of randomly dstrbuted pakets. Assume 0 < q < s gven. Our model s essentally that eah paket s avalable ndependently at eah node wth probablty q. However, we must ondton on the event that eah paket s avalable to at least one node. Thus, when pakets are randomly dstrbuted, the underlyng probablty measure s gven by Pr p P j ( q) S = ( q) n () j S for all [k] and all nonempty S V =[n]. Fnally, we ntrodue one more defnton whh lnks the network topology wth the number of ommunaton rounds, r. Defnton 4: For a graph G = (V, E) on n vertes, defne S (r) (G) (2 V ) r+ as follows: (S 0, S,...,S r ) S (r) (G) f and only f the sets {S } r =0 satsfy the followng two ondtons: S V for eah 0 r, and S S Ɣ(S ) for eah r. In words, any element n S (r) (G) s a nested sequene of subsets of vertes of G. Moreover, the onstrant that eah set n the sequene s ontaned n ts predeessor s neghborhood mples that the sets annot expand too qukly relatve to the topology of G. Note that, for any graph G, thesets (r) (G) (2 V ) r+ s unque. To make the defnton of S (r) (G) more onrete, we have llustrated a sequene (S 0, S, S 2 ) S (2) (G) for a partular hoe of graph G n Fgure 3. III. MAIN RESULTS In ths seton, we present our man results. Proofs are delayed untl Seton V. Fg. 3. An example of a sequene (S 0, S, S 2 ) S (2) (G) for a partular hoe of graph G. A. Neessary and Suffent Condtons for Unversal Reovery Frst, we provde neessary and suffent ondtons for ahevng unversal reovery n a network T. It turns out that these ondtons are haraterzed by a partular set of transmsson shedules R r (T ) whh we defne as follows: Defnton 5: For a network T ={G, P,...,P n },defne the regon R r (T ) N n r to be the set of all transmsson shedules {b j } satsfyng: r j= S j Ɣ(S j ) (r+ j) b P S r for eah (S 0,...,S r ) S (r) (G). For a gven network T, the regon R r (T ) haraterzes a subset of transmsson shedules. Whle the defnton of R r (T ) may ntally appear somewhat heavyhanded, ts mportane s revealed through the followng theorem. Theorem : For a network T, a transmsson shedule {b j } permts unversal reovery n r rounds of ommunaton f and only f {b j } R r (T ). Theorem reveals that the set of transmsson shedules permttng unversal reovery s haraterzed presely by the regon R r (T ). In fat, gven a transmsson shedule n R r (T ), a orrespondng odng sheme that aheves unversal reovery an be omputed n polynomal tme usng the algorthm n [2] appled to the network odng graph dsussed n the proof of Theorem. Alternatvely, one ould employ random lnear network odng over a suffently large feld sze [22]. If transmssons are made n a manner onsstent wth a shedule n R r (T ), unversal reovery wll be aheved wth hgh probablty. Thus, the problem of ahevng unversal reovery wth the mnmum number of transmssons redues to solvng a ombnatoral optmzaton problem over R r (T ). Asths problem was shown to be NP-hard n Example 3, we do not attempt to solve t n ts most general form. Instead, we apply Theorem to obtan surprsngly smple haraterzatons for several ases of nterest. Before proeedng, we provde a few examples to shed some lght on the regon R r (T ). Frst, we show how the tradtonal ut-set bounds an be reovered from Theorem. (2)

5 40 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 Example 4 (Cut-Set Bounds): Consderng the onstrant defnng R r (T ) n whh the nested subsets that form S (r) (G) are all dental. That s, (S, S,...,S) S (r) (G) for some nonempty S V. We see that any transmsson shedule {b j } R r (T ) must satsfy the famlar ut-set bounds: r j= (S) b j S P. (3) In words, the total number of pakets that flow nto the set of nodes S must be greater than or equal to the number of pakets that the nodes n S are olletvely mssng. As a seond example, we apply Theorem to the lne network T onsdered n Example to fnd the set of permssble transmsson shedules for r = 2 rounds of ommunaton. Although t s a tedous applaton of Theorem, we wll see that t supples the desred result. Example 5: Let T be the lne network onsdered n Example, and let G be the orrespondng graph. Three dfferent hoes of (S 0, S, S 2 ) S (2) (G) are gven below wth the orrespondng onstrants ndued by R 2 (T ). S 0 S S 2 ndued onstrant {} {} {, 2} b2 2 {} {, 2} {, 2} b3 {3} {2, 3} {2, 3} b In words, the above onstrants mply that nodes and 3 must eah transmt one n the frst ommunaton round, and node 2 must transmt one n the seond ommunaton round. Argung along the lnes of Example, these ondtons are both neessary and suffent for a transmsson shedule to permt unversal reovery n r = 2 rounds of ommunaton. Indeed, the reader an hek that any (S 0, S, S 2 ) S (2) (G) not lsted n the table above ndues a onstrant va R 2 (T ) that s a lnear ombnaton of those gven above. B. Fully Conneted Networks When T s a fully onneted network, the graph G s a omplete graph on n vertes. Ths s perhaps one of the most pratally mportant ases to onsder. For example, n a wred omputer network, lents an multast ther messages to all other termnals whh are ooperatvely exhangng data. In wreless networks, broadast s a natural transmsson mode. Indeed, there are protools talored spefally to wreless networks whh support relable network-wde broadast apabltes (f. [23] [26]). It s fortunate then, that the ooperatve data exhange problem an be solved n polynomal tme for fully onneted networks: Theorem 2: For a fully onneted network T, a transmsson shedule requrng only M (T ) transmssons an be omputed n polynomal tme. Neessary and suffent ondtons for unversal reovery n ths ase are gven by the ut-set onstrants (3). Moreover, a sngle round of ommunaton s suffent to aheve unversal reovery wth M (T ) transmssons. For the fully onneted network n Example 2, we remarked that only one round of transmsson was requred. Theorem 2 states that ths property extends to any fully onneted network. Moreover, an optmal transmsson shedule requrng M (T ) transmssons an be omputed usng the algorthm derved n Appendx A. Although Theorem 2 apples to arbtrary sets of pakets P,...,P n, t s nsghtful to onsder the ase where pakets are randomly dstrbuted n the network aordng to (). In ths ase, the mnmum number of transmssons requred for unversal reovery onverges n probablty to a smple funton of the (random) sets P,...,P n. Theorem 3: If T s a fully onneted network and pakets are randomly dstrbuted, then M (T ) = n n = P. (4) wth probablty approahng as the number of pakets k. We remark that, f pakets are randomly dstrbuted aordng to (), then M (T ) s a random varable. Hene, Theorem 3 shows that ths random varable agrees wth n= P wth hgh probablty n k. n At frst glane, t appears that the result of Theorem 3 s ndependent of the parameter q, whh dtates how random pakets are dstrbuted n (). However, q s mpltly present n the result sne t parameterzes the dstrbuton of the random varables P. C. Networks Whh Are d-regular Gven that prese results an be obtaned for fully onneted networks, t s natural to ask whether these results an be extended to a larger lass of networks whh nludes fully onneted networks as a speal ase. In ths seton, we partally answer ths queston n the affrmatve. To ths end, we defne d-regular networks. Defnton 6 (d-regular Networks): AnetworkT s sad to be d-regular f () = d for eah V and (S) d for eah nonempty S V wth S n d. Inotherwords,a network T s d-regular f the assoated graph G s d-regular and d-vertex-onneted. Immedately, we see that the lass of d-regular networks nludes fully onneted networks as a speal ase wth d = n. Further, the lass of d-regular networks nludes many frequently studed network topologes (e.g., yles, grds on tor, et.). Unfortunately, the determnst algorthm of Theorem 2 does not appear to extend to d-regular networks. However, a slghtly weaker onentraton result smlar to Theorem 3 an be obtaned when pakets are randomly dstrbuted. Before statng ths result, onsder the followng Lnear Program (LP) wth varable vetor x R n defned for a network T : n mnmze x (5) subjet to: = (j) x P j for eah j V. (6)

6 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 4 Let M LP (T ) denote the optmal value of ths LP. Interpretng x as j b j, the onstrants n the LP are a subset of the ut-set onstrants of (3) whh are a subset of the neessary onstrants for unversal reovery gven n Theorem. Furthermore, the nteger onstrants on the x s are relaxed. Thus M LP (T ) ertanly bounds M (T ) from below. Surprsngly, f T s a d-regular network and the pakets are randomly dstrbuted, M (T ) s very lose to ths lower bound wth hgh probablty: Theorem 4: If T s a d-regular network and the pakets are randomly dstrbuted, then M (T )<M LP (T ) + n (7) wth probablty approahng as the number of pakets k. We make two mportant observatons. Frst, M (T ) [M LP (T ), M LP (T ) + n) wth hgh probablty. Hene, even though the number of pakets k may be extremely large, M (T ) an be estmated aurately by omputng the soluton to LP (5-6) for a gven nstantaton of the paket dstrbuton {P } n =. Seond, as k grows large, M (T ) s domnated by the loal topology of T. Ths s readly seen sne the onstrants defnng M LP (T ) orrespond only to nodes mmedate neghborhoods. The mportane of the loal neghborhood was also seen n [27] where network odng apaty for ertan random networks s shown to onentrate around the expeted number of nearest neghbors of the soure and the termnals. D. Large (Dvsble) Pakets We now return to general networks wth arbtrarly dstrbuted pakets. However, we now onsder the ase where pakets are large and an be dvded nto several smaller pees (e.g., pakets atually orrespond to large fles). To formalze ths, assume that eah paket an be parttoned nto t hunks of equal sze, and transmssons an onsst of a sngle hunk (as opposed to an entre paket). In ths ase, we say the pakets are t-dvsble. To llustrate ths pont more learly, we return to Example 2, ths tme onsderng 2-dvsble pakets. Example 6 (2-Dvsble Pakets): Let T be the network of Example 2 and splt eah paket nto two halves: p (p (), p (2) ). Denote ths new network T wth orrespondng sets of pakets: P ={p (), p(2) p () 2, p(2) 2, p() 3, p(2) 3 }\{p(), p (2) }. (8) Three hunk transmssons allow unversal reovery as follows: Node transmts p (2) 2 p (2) 3. Node 2 transmts p () p () 3. Node 3 transmts p(2) p () 2. It s readly verfed from (3) that 3 hunk-transmssons are requred to permt unversal reovery. Thus, M (T ) = 3. Hene, f we were allowed to splt the pakets of Example 2 nto two halves, t would suffe to transmt 3 hunks. Normalzng the number of transmssons by the number of hunks per paket, we say that unversal reovery an be aheved wth.5 paket transmssons. Motvated by ths example, defne Mt (T ) to be the mnmum number of (normalzed) paket-transmssons requred to aheve unversal reovery n the network T when pakets are t-dvsble. For the network T n Example 2, we saw above that M2 (T ) =.5. It turns out, f pakets are t-dvsble and t s large, the ut-set bounds (3) are nearly suffent for ahevng unversal reovery. To see ths, let Mut-set(T ) be the optmal value of the LP: n mnmze x (9) subjet to: = (S) x S P for all S V. (0) Clearly Mut-set(T ) Mt (T ) for any network T wth t-dvsble pakets beause the LP produng Mut-set(T ) relaxes the nteger onstrants and s onstraned only by (3) rather than the full set of onstrants gven n Theorem. However, there exst transmsson shedules whh an approah ths lower bound. Stated more presely: Theorem 5: For any network T, the mnmum number of (normalzed) paket-transmssons requred to aheve unversal reovery wth t-dvsble pakets satsfes lm t M t (T ) = M ut-set(t ). () Presely how large t s requred to be n order to approah Mut-set(T ) wthn a spefed tolerane s not lear for general networks. Sne the nequaltes (0) have nteger oeffents, Cramer s rule and Hadamard s nequalty mply that t n n/2 s suffent. However, we should antpate a muh better estmate n general sne an mmedate onsequene of Theorem 3 s that t = n s suffent to aheve Mut-set(T ) wth hgh probablty when pakets are randomly dstrbuted n a fully onneted network. We note that [8] presents an algorthm explotng submodularty that solves LP (9) n the speal ase where T s a fully onneted network. Ths s n ontrast to the algorthm we present n Appendx VI (orgnally appearng n [3]), whh solves the ILP verson of ths problem orrespondng to ndvsble pakets. Fnally, we remark that t s a smple exerse to onstrut examples where the ut-set bounds alone are not suffent to haraterze transmsson shedules permttng unversal reovery when pakets are not dvsble (e.g., a 4-node lne network wth pakets p and p 2 at the left-most and rght-most nodes, respetvely). Thus, t-dvsblty of pakets provdes the addtonal degrees of freedom neessary to approah the ut-set bounds more losely. E. Remarks One nterestng onsequene of our results s that splttng pakets does not sgnfantly redue the requred number of paket-transmssons for many senaros. Indeed, at most one transmsson an be saved f the network s fully onneted (under any dstrbuton of pakets). If the network s d-regular, we an expet to save fewer than n transmssons

7 42 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 f pakets are randomly dstrbuted (n fat, at most one transmsson per node). It seems possble that ths result ould be strengthened to nlude arbtrary dstrbutons of pakets n d-regular networks (as opposed to randomly dstrbuted pakets), but a proof has not been found. The lmted value of dvdng pakets has pratal ramfatons sne there s usually some addtonal ommunaton overhead assoated wth dvdng pakets (e.g.addtonal headers, et. for eah transmtted hunk are requred). Thus, f the pakets are very large, say eah paket s a vdeo fle, our results mply that entre oded pakets an be transmtted wthout sgnfant loss, avodng any addtonal overhead nurred by dvdng pakets. IV. AN APPLICATION: SECRECY GENERATION In ths seton, we onsder the setup of the ooperatve data exhange problem for a fully onneted network T, but we onsder a dfferent goal. In partular, we wsh to generate a seret-key among the nodes that annot be derved by an eavesdropper prvy to all of the transmssons among nodes. Also, lke the nodes themselves, the eavesdropper s assumed to know the ndes of the pakets ntally avalable to eah node and the ommunaton protool whh the nodes follow. In partular, the only nformaton unknown to the eavesdropper s the ontents of the nodes pakets. The goal s for the nodes to generate the maxmum amount of serey that annot be determned by the eavesdropper. The theory behnd serey generaton among multple termnals was orgnally establshed n [4] for a very general lass of problems n the usual nformaton-theoret, asymptot settng. Our results should be nterpreted as a pratal applaton of the theory orgnally developed n [4]. Indeed, our proofs essentally follow those n [4], but have been streamlned and adapted to deal wth the non-asymptot and ombnatoral senaro under onsderaton. The am of the present seton s to show how serey an be generated n a pratal senaro. By pratal, we mean that t s possble to generate the maxmum amount of serey (as defned n [4]) among nodes n a fully onneted network T = {G, P,...,P n } usng a polynomal tme algorthm. Moreover, ths s possble n the non-asymptot regme (.e., there are no ɛ s and we don t requre the number of pakets or nodes to grow arbtrarly large), and t s possble to generate perfet serey nstead of ɛ-serey wthout any sarfe. A. Pratal Serey Results In ths subseton, we state two results on serey generaton n fully onneted networks T. Proofs are agan postponed untl Seton V. We begn wth some defntons 4.LetF denote the set of all transmssons (all of whh are avalable to the eavesdropper by defnton). A funton K of the pakets {p,...,p k } n the network s alled a seret key (SK) f K 4 We attempt to follow the notaton of [4] where approprate. s reoverable by all nodes after observng F, and t satsfes the (perfet) serey ondton I (K ; F) = 0, (2) and the unformty ondton Pr (K = key) = for all key K, (3) K where K s the alphabet of possble keys. We defne C SK (P,...,P n ) to be the seret-key apaty for a partular dstrbuton of pakets. We wll drop the notatonal dependene on P,...,P n where t doesn t ause onfuson. By ths we mean that a seret-key K an be generated f and only f K = F C SK. In other words, the nodes an generate at most C SK pakets worth of seret-key. Implt n ths defnton s the requrement that C SK s an nteger. That s, we are nterested n the maxmum nteger number of pakets of seret key whh an be generated. Our frst result of ths seton s the followng analogue of [4, Theorem ]: Theorem 6: The seret-key apaty s gven by: C SK (P,...,P n ) = k M (T ). Next, onsder the related problem where a subset D V of nodes s ompromsed, and assume all nodes are aware of whh nodes are ompromsed (.e., D s known to all nodes). In ths problem, the eavesdropper has aess to F and P for D. In ths ase, the seret-key should also be kept hdden from the nodes n D (or else the eavesdropper ould also reover t). Thus, for a subset of nodes D, letp D = D P, and all K a prvate-key (PK) f t s a seret-key whh s only reoverable by the nodes n V \D, and also satsfes the stronger serey ondton: I (K ; F, P D ) = 0. (4) Smlar to above, defne C PK (P,...,P n, D) to be the prvatekey apaty for a partular dstrbuton of pakets and subset of nodes D. Agan, we mean that a prvate-key K an be generated f and only f K = F C PK. In other words, the nodes n V \D an generate at most C PK pakets worth of prvatekey. As wth C SK, we requre that C PK s an nteger. That s, we are nterested n the maxmum nteger number of pakets of prvate key whh an be generated. Note that, sne P D s known to the eavesdropper, eah node D an transmt ts respetve set of pakets P wthout any loss of serey apaty. Defne a new network T D ={G D, {P (D) } V \D } as follows. Let G D be the omplete graph on V \D, andletp (D) = P \P D for eah V \D. Thus, T D s a fully onneted network wth n D nodes and k P D pakets. Our seond result of ths seton s the followng analogue of [4, Theorem 2]: Theorem 7: The prvate-key apaty s gven by: C PK (P,...,P n, D) = (k P D ) M (T D ). (5) The bas dea for prvate-key generaton s that the users n V \D should generate a seret-key from {p,...,p k }\P D. By the defntons of the SK and PK apates, Theorem 2 mples that t s possble to ompute these apates effently (.e., n polynomal tme). Moreover, as we wll see n the ahevablty proofs, these apates an be aheved

8 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 43 by performng oded ooperatve data exhange amongst the nodes. Thus, the algorthm developed n Appendx A ombned wth the algorthm n [2] an be employed to solve the serey generaton problem we onsder n polynomal tme. Remark : If one drops the ntegralty requrement for C SK and C PK, ths amounts to determnng the maxmum (possbly fratonal) number of pakets of seret/prvate key that an be generated. In ths ase, we smply replae M (T ) wth M LP (T ) n Theorem 6 and M (T D ) wth M LP (T D ) n Theorem 7. These fratonal seret- and prvate-key apates an be approahed as the feld sze F grows large. The proofs are nearly dental to those of Theorems 6 and 7 (splttng pakets where approprate), and are therefore omtted. We onlude ths subseton wth an example to llustrate the results. Example 7: Consder agan the network of Example 2 and assume F ={0, } (.e., eah paket s a sngle bt). The seretkey apaty for ths network s bt. After Nodes -3 perform unversal reovery as desrbed n Example 2, an eavesdropper would observe p 2 and the party p p 3. A perfet seret-key s K = p (we ould alternatvely use K = p 3 ). If any of the nodes are ompromsed by the eavesdropper, the prvate-key apaty s 0. We remark that the seret-key n the above example an n fat be attaned by all nodes usng only one transmsson (.e., unversal reovery s not a prerequste for seret-key generaton). However, t remans true that only one bt of serey an be generated. V. PROOFS OF MAIN RESULTS A. Neessary and Suffent Condtons for Unversal Reovery Proof of Theorem : Ths proof s aomplshed by redung the problem at hand to an nstane of a sngle-soure network odng problem and nvokng the Max-Flow Mn-Cut Theorem for network nformaton flow [4]. Frst, fx the number of ommunaton rounds r to be large enough to permt unversal reovery. For a network T, onstrut the network-odng graph G NC = (V NC, E NC ) as follows. The vertex set, V NC s defned as: r r V NC ={s, u,...,u k } {v j,...,vj n } {w j,...,wj n}. j=0 The edge set, E NC, onssts of dreted edges and s onstruted as follows: For eah [k], there s an edge of unt apaty 5 from s to u. If p P j, then there s an edge of nfnte apaty from u to v 0 j. For eah j [r] and eah [n], there s an edge of nfnte apaty from v j to v j. For eah j [r] and eah [n], there s an edge of apaty b j from v j to w j. For eah j [r] and eah [n], there s an edge of nfnte apaty from w j to v j ff Ɣ(). j= 5 An edge of unt apaty an arry one feld element z F per unt tme. Fg. 4. The graph G NC orrespondng to the lne network of Example. Edges represented by broken lnes have nfnte apaty. Edges wth fnte apates are labeled wth the orrespondng apaty value. The nterpretaton of ths graph s as follows: the vertex u s ntrodued to represent paket p,thevertexv j represents node after the j th round of ommunaton, and the vertex w j represents the broadast of node durng the j th round of ommunaton. In other words, G NC s the tme-expanded graph orrespondng to the unversal reovery problem (the onstruton of dreted ayl networks va tme-expanson orgnated n [4] n the ontext of network odng). If the b j s are hosen suh that the graph G NC admts a network odng soluton whh supports a multast of k unts from s to {v r,...,vr n }, then ths network odng soluton also solves the unversal reovery problem for the network T when node s allowed to make at most b j transmssons durng the j th round of ommunaton. The graph G NC orrespondng to the lne network of Example s gven n Fgure 4. We now formally prove the equvalene of the network odng problem on G NC and the unversal reovery problem defned by T. Suppose a set of enodng funtons { f j } and a set deodng funtons {φ } desrbe a transmsson strategy whh solves

9 44 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 the unversal reovery problem for a network T n r rounds of ommunaton. Let b j be the number of transmssons made by node durng the j th round of ommunaton, and let I j be all the nformaton known to node pror to the j th round of ommunaton (e.g.i = P ). The funton f j s the enodng funton for user durng the j th round of ommunaton (.e. f j (I j ) j Fb ), and the deodng funtons satsfy: ( φ I r, Ɣ(){ f r (Ir )}) ={p,...,p k }. (6) Note that, gven the enodng funtons and the P s, the I j s an be defned reursvely as: I j+ = I j { f j (I j )}. (7) Ɣ() The funtons { f j } and {φ } an be used to generate a network odng soluton whh supports k unts of flow from s to {v r,...,vr n } on G NC as follows: For eah vertex v V NC,letIN(v) be whatever v reeves on ts nomng edges. Let g v be the enodng funton at vertex v, and g v (e, IN(v)) be the enoded message whh vertex v sends along e (e s an outgong edge from v). If e s an edge of nfnte apaty emanatng from v, let g v (e, IN(v)) = IN(v). Let s send p along edge (s, u ). At ths pont, we have IN(v 0) = P = I. For eah [n], let g v 0((v 0,w ), IN(v0 )) = f (I ). By a smple ndutve argument, defnng the enodng funtons g j v ((v j,wj+ ), IN(v j )) to ( be equal to f j+ yelds the result that IN(v r ) = I r, Ɣ(){ f r (I r )} ). Hene, the deodng funton φ an be used at v r to allow error-free reonstruton of the k-unt flow. The equvalene argument s ompleted by showng that a network odng soluton whh supports a k-unt multast flow from s to {v r,...,vr n } on G NC also solves the unversal reovery problem on T. Ths s argued n a smlar manner as above, and s therefore omtted. Sne we have shown that the unversal reovery problem on T s equvalent to a network odng problem on G NC,the elebrated max-flow mn-ut result of Ahlswede et. al [4] s applable. In partular, a fxed vetor {b j } admts a soluton to the unversal reovery problem where node makes at most b j transmssons durng the j th round of ommunaton f and only f any ut separatng s from some v r n G NC has apaty at least k. What remans to be shown s that the nequaltes defnng R r (T ) are satsfed f and only f any ut separatng s from some v r n G NC has apaty at least k. To ths end, suppose we have a ut (S, S ) satsfyng s S and v r S for some [n]. We wll modfy the ut (S, S ) to produe a new ut (S, S ) wth apaty less than or equal to the apaty of the orgnal ut (S, S ). Defne the set S 0 [n] as follows: S 0 ff v r S (by defnton of S, wehavethats 0 = ). Intally, let S = S. Modfy the ut (S, S ) as follows: M) If Ɣ(S 0 ), then plae w r nto S. M2) If / Ɣ(S 0 ), then plae w r nto S. Modfatons M and M2 are justfed (respetvely) by J and J2: J) If Ɣ(S 0 ), then there exsts an edge of nfnte apaty from w r to some v r S. Thus, movng w r to S (f neessary) does not nrease the apaty of the ut. J2) If / Ɣ(S 0 ), then there are no edges from w r to S, hene we an move w r nto S (f neessary) wthout nreasng the apaty of the ut. Modfatons M and M2 guarantee that w r S ff Ɣ(S 0 ). Thus, assume that (S, S ) satsfes ths ondton and further modfy the ut as follows: M3) If S 0, then plae v r nto S. M4) If / Ɣ(S 0 ), then plae v r nto S. Modfatons M3 and M4 are justfed (respetvely) by J3 and J4: J3) If S 0, then there exsts an edge of nfnte apaty from v r to v r S. Thus, movng v r to S (f neessary) does not nrease the apaty of the ut. J4) If / Ɣ(S 0 ), then there are no edges from v r to S (sne w r / S by assumpton), hene we an move v r nto S (f neessary) wthout nreasng the apaty of the ut. At ths pont, defne the set S [n] as follows: S ff v r S. Note that the modfatons of S guarantee that S satsfes S 0 S Ɣ(S 0 ). Ths proedure an be repeated for eah layer of the graph resultng n a sequene of sets S 0 S r [n] satsfyng S j Ɣ(S j ) for eah j [r]. We now perform a fnal modfaton of the ut (S, S ): M5) If p j Sr P, then plae u j nto S. M6) If p j / Sr P, then plae u j nto S. Modfatons M5 and M6 are justfed (respetvely) by J5 and J6: J5) If p j Sr P, then there s an edge of nfnte apaty from u j to S and movng u j nto S (f neessary) does not nrease the apaty of the ut. J6) If p j / Sr P, then there are no edges from u j to S, hene movng u j (f neessary) nto S annot nrease the apaty of the ut. A quk alulaton shows that the modfed ut (S, S ) has apaty greater than or equal to k ff: r b r+ j j= S j Ɣ(S j ) P S r. (8) Sne every modfaton of the ut ether preserved or redued the apaty of the ut, the orgnal ut (S, S ) also has apaty greater than or equal to k f the above nequalty s satsfed. In Fgure 5, we llustrate a ut (S, S ) and ts modfed mnmal ut (S, S ) for the graph G NC orrespondng to the lne network of Example. By the equvalene of the unversal reovery problem on a network T to the network odng problem on G NC and the max-flow mn-ut theorem for network nformaton flow, f a transmsson strategy solves the unversal reovery problem on T, then the assoated b j s must satsfy the onstrants

10 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 45 ondtons to the settng of a fully onneted network to establsh Theorems 2 and 3. Proof of Theorem 2: In the ase where T s a fully onneted network, we have that S j Ɣ(S j ) = S j for any nonempty S V. Therefore, the onstrants defnng R r (T ) beome: r j= S j b r+ j P S r. (9) Now, suppose a transmsson shedule {b j } R r (T ) and onsder the modfed transmsson shedule { b j } defned by: b r = r j= b j and b j = 0 for j < r. By onstruton, S j+ S j n the onstrants defnng R r (T ). Therefore, usng the defnton of { b j },wehave: r b r b r+ j. (20) S j= S j S r P Fg. 5. The graph G NC orrespondng to the lne network of Example wth orgnal ut (S, S ) and the orrespondng modfed mnmal ut (S, S ).In ths ase, S 0 = S = S 2 ={}. Upon substtuton nto (8), ths hoe of S 0, S, S 2 yelds the nequalty b2 + b2 2. of the form gven by (8). Conversely, for any set of b j s whh satsfy the onstrants of the form gven by (8), there exsts a transmsson strategy usng exatly those numbers of transmssons whh solves the unversal reovery problem for T. Thus the onstrants of (8), and hene the nequaltes defnng R r (T ), are satsfed f and only f any ut separatng s from some v r n G NC has apaty at least k. Remark 2: Sne [n] P = 0, onstrants where S r =[n] are trvally satsfed. Therefore, we an restrt our attenton to sequenes of sets where S r [n]. Remark 3: Our problem formulaton does not requre lnear enodng shemes, however the proof above reveals that lnear enodng shemes are suffent to aheve unversal reovery wth the mnmum number of transmssons. B. Fully Conneted Networks Havng establshed neessary and suffent ondtons for unversal reovery n Theorem, we spealze these Thus the modfed transmsson shedule s also n R r (T ). Sne S P S r P,whenT s a fully onneted network, t s suffent to onsder onstrants of the form: b P for all nonempty S V. (2) S S Ths proves the followng: that the ut-set onstrants are neessary and suffent for unversal reovery when T s a fully onneted network, and that a sngle round of ommunaton s suffent to aheve unversal reovery wth M (T ) transmssons (sne (2) does not depend on r). Wth these results establshed, an optmal transmsson shedule an be obtaned by solvng the followng nteger lnear program: n mnmze b = subjet to: b P for all nonempty S V. (22) S S In order to aomplsh ths, we dentfy B P and set w = for [n] and apply the submodular algorthm presented n Appendx A. We remark brefly that the algorthm presented n Appendx A s suffently general to solve the ILP above wth the objetve (22) replaed by n = w b, where {w } n = are nonnegatve reals. In other words, we an easly ompute transmsson shedules whh () permt unversal reovery, and () mnmze a weghted sum of nodes transmssons. Now we onsder fully onneted networks n whh pakets are randomly dstrbuted aordng to (), whh s parametrzed by q. The proof of Theorem 3 requres the followng tehnal lemma: Lemma : Let 0 < q < be fxed. For eah l {2,..., n }, defne δ l = ( n l n ) ( q ( q) n ) ( q) l ( q) n. (23)

11 46 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 Then, for eah l {2,...,n }: n l n = ( q)l ( q) n q ( q) n ( + δ l ) (24) wth 0 <δ 2 <δ 3 <δ n. Proof: Applyng Jensen s nequalty to the strtly onvex funton f (x) = ( q) x usng the onvex ombnaton l = θ + ( θ) n yelds: n l n > ( q)l ( q) n q ( q) n (25) for l {2,...,n }. Thus t s readly verfed that (24) holds wth δ l > 0 by smply substtutng n the defnton of δ l. It s a smple alulus exerse to verfy that δ l s strtly nreasng n l for l {2,...,n }. Indeed, let the parameter l vary ontnuously n the nterval (, n), and look at the dervatves { } n l l ( q) l ( q) ( n ( q) = ( q) [log l l ) ] ( q) n + ( q) n (26) 2 { } n l l 2 ( q) l ( q) n [ ( ( q) = ( q) l l ) ] log ( q) log ( q) n + < 0. (27) Hene, we an onlude that (23) s onave n l over the nterval (, n), wth a maxmum beng attaned when l n. The proof of Theorem 3 proeeds by showng that the neessary and suffent ondtons of Theorem admt a smply-strutured soluton when pakets are randomly dstrbuted aordng to (). Proof of Theorem 3: We begn by showng that the LP n mnmze b (28) = subjet to: b P for all S V. (29) S S has an optmal value of n= n P wth hgh probablty as k, over random nstantatons of the P s (reall that pakets are randomly dstrbuted aordng to ()). To ths end, note that the nequaltes n b Pj for j n. (30) = = j are a subset of the nequalty onstrants (29). Summng both sdes of (30) over j n reveals that any feasble vetor b R n for LP (28)-(29) must satsfy: n = b n n = P. (3) Ths establshes a lower bound on the optmal value of the LP. We now dentfy a soluton that s feasble wth probablty approahng as k whle ahevng the lower bound of (3) wth equalty. To begn note that b j = n n = P P j (32) s a soluton to the system of lnear equatons gven by (30) and aheves (3) wth equalty. Now, we prove that ( b,..., b n ) s a feasble soluton to LP (28) wth hgh probablty. To be spef, we must verfy that b P (33) S S holds wth hgh probablty for all subsets S V satsfyng 2 S n (the ase S = s satsfed by the defnton of { b } n = ). Substtuton of (32) nto (33) along wth smple algebra yelds that the followng equvalent ondtons must hold: ( ) n n S P P P n k k k. (34) S = To ths end, note that for any S, S P s a random varable whh an be expressed as S P = k j= X S j, where X S j s an ndator random varable takng the value f p j S P and 0 otherwse. From () we have: ( ) ξ S Pr X S j = = ( q) S ( q) n ( q) n. (35) For l = 2, 3,...,n, defne δ l as n Lemma. Now, defne δ 2 ν 2n + δ 2 (2n ), (36) and defne ν l to satsfy: + ν l = ( + δ l ) ( ν (2n )) for l = 2,...,n. By defnton, ν > 0, but also observe that ν l > 0forl 2 sne Lemma mples δ l δ 2, and therefore S + ν l = ( + δ l ) ( ν (2n )) (37) ( + δ 2 ) ( ν (2n )) (38) = + ν. (39) Now, for notatonal onvenene, defne the funton ζ(n, q) of (n, q) as follows: ( { ζ(n, q) = 2 (ν l ξ l ) 2}). (40) mn l [n ] Note that Hoeffdng s nequalty yelds the followng onentraton result: ( ) Pr k P ξ S >ν S ξ S 2exp [ 2k(ν S ξ S ) 2] S (4) 2exp[ k ζ(n, q)]. (42)

12 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 47 Therefore, by the unon of events bound, we have Pr { } k P ξ S >ν S ξ S S:S [n],s = S 2 n+ exp [ k ζ(n, q)]. (43) Thus, wth probablty greater than 2 n+ exp [ k ζ(n, q)], the followng nequaltes hold for S V wth 2 S n : ( ) n n S P n k P k (44) = S ( ) n S nξ ( ν ) n ( ) n S (n )ξ ( + ν ) (45) n ( )( n S ( q) ( q) n ) ( = ν n ( q) n (2n ) ) (46) ( ( q) S ( q) n ) = ( q) n ( + δ S ) ( ν (2n ) ) (47) ( ( q) S ( q) n ) = ( q) n ( + ν S ) (48) = ξ S ( + ν S ) (49) P k, (50) S where (45) follows from (43). (46) follows from elementary algebra and the defnton of ξ. (47)-(49) follow from the defntons of δ S, ν S and ξ S, respetvely. (50) follows from (43). Ths proves that (34) holds, and therefore ( b,..., b n ) s a feasble soluton to LP (28) wth probablty greater than 2 n+ exp [ k ζ(n, q)]. Fnally, Corollary n Appendx A states that the optmal values of the ILP: { n mn b : } b P, S V, S =, b Z S = S and the orrespondng LP relaxaton: { n mn b : } b P, S V, S =, b R S = S dffer by less than. Combnng ths fat wth Theorem 2 ompletes the proof. Remark 4: Observe that we have atually proven a stronger onentraton result than was needed to prove Theorem 3. Indeed, we have shown that f pakets are randomly dstrbuted aordng to (), then M (T ) = n n P = (5) wth probablty greater than 2 n+ exp [ k ζ(n, q)], where ζ(n, q) s an explt funton of n and q. C. d-regular Networks Before we begn, let us defne 2, to be the usual l 2 -andl -norms, respetvely. Also, let denote the olumn vetor of all ones. In order to smplfy the proof of Theorem 4, we requre two tehnal lemmas: Lemma 2: Let A R n n be a symmetr matrx wth nonnegatve entres and all olumn sums equal to d. Let x y be the vetor of mnmum Euldean norm whh mnmzes Ax y y 2. There exsts an optmal soluton x to the lnear program whh satsfes mnmze T x (52) subjet to: Ax y (53) x x y A A x y y 2, (54) where A s a onstant dependng only on A. Proof: See Appendx B. Lemma 3: Assume pakets are randomly dstrbuted n a d-regular network T aordng to (). For any ɛ>0, there exsts an optmal soluton x to LP (5-6) whh satsfes x d E[ P ] <ɛk (55) wth probablty approahng as k,wheree ndates expetaton. Before proeedng wth the proof of Lemma 3, we reall that the random paket model () mples that E[ P j ] = E[ P ] for all j [n]. Hene, there s nothng noteworthy about the appearane of P n the statement of the lemma, any P j would suffe. Proof: Let P = ( P,..., P n )T and let A be the adjaeny matrx of G (.e., a, j = f(, j) E and 0 otherwse). Observe that A s symmetr and A = d, where denotes a olumn vetor of s. Wth ths notaton, LP (5) an be rewrtten as: mnmze T x (56) subjet to: Ax P, (57) where a b for vetors a, b R n means that a b for =,...,n. Let A + denote the Moore-Penrose pseudonverse of A. Observe that the lnear least squares soluton x LS arg mn x Ax P 2 s gven by: x LS = A + P (58) = A + E P ( ) + A + P E P (59) = d E P ( ) + A + P E P. (60) For the last step above, note that E P s an egenvetor of A wth egenvalue d so E P wll also be an egenvetor of A + wth egenvalue d. Hene, x LS d E P ( ) 2 = A + P E P 2 (6) A + 2 P E P 2, (62)

13 48 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 where A + 2 s the operator norm of A + ndued by the 2-norm on vetors (also known as the spetral norm of A + ). Thus, for any vetor y, wehave y d E P y x LS + x LS d E P (63) y x LS + x LS d E P 2 (64) y x LS + A + 2 P E P 2, (65) where (63) follows from the trangle nequalty, (64) follows from the defntons of the l 2 -andl -norms, and (65) follows from (62). Now, Lemma 2 guarantees the exstene of an optmal soluton x to LP (56) (and onsequently LP (5)) whh satsfes x x LS A A x LS P 2,where A s a onstant dependng only on A. Hene, lettng y = x n (65), we have: x d E P x x LS + A + 2 P E P 2 A A x LS P 2 + A + 2 P E P 2 A d AE P P 2 + A + 2 P E P 2 = A E P P 2 + A + 2 P E P 2, where the thrd nequalty follows by defnton of x LS.By the weak law of large numbers, P E P 2 ɛk wth probablty tendng to as k for any ɛ>0. Notng that E P = E[ P ] ompletes the proof. Now, we are n a poston to prove Theorem 4. Roughly speakng, the proof of Theorem 4 s smlar to that of 3. In partular, we show that f pakets are randomly dstrbuted aordng to (), then a smply-strutured transmsson shedule satsfes the ondtons of Theorem. However, are must be taken to handle the more general network topologes. Proof of Theorem 4: We begn wth some observatons and defntons: Frst, reall that our model for randomly dstrbuted pakets () mples that [ ] E P = k ( q) S ( q) n ( q) n (66) S for all nonempty S V. Applyng the dentty (66), we have E [ [ ] ] P ( + q )E P (67) for all S V, S 2, where q > 0sdefnedby q( q) q 2 ( ( q) 2 ( q) n). (68) Next, Lemma mples the exstene of a onstant δ q > 0 suh that for any S V wth 2 S n : n S n ( q) S ( q) n ( q) ( q) n + δ q (69) = E [ S P ] E [ ] P + δ q, (70) where the equalty follows from (66). S Argung n a manner smlar to the dsusson surroundng (43), t s readly verfed that Hoeffdng s nequalty 6 mples ( + mn{δ ) q, q } 4 [ E S P ] S P (7) wth probablty approahng as k. Fnally, for the proof below, we wll take the number of ommunaton rounds suffently large to satsfy r max { 2n dδ q, 2n( + q) d q }. (72) Fx ɛ>0. Lemma 3 guarantees that there exsts an optmal soluton x to LP (5) satsfyng x d E[ P ] <ɛk (73) wth probablty tendng to n k. Now, t s always possble to onstrut a transmsson shedule {b j } whh satsfes j b j = x and r x b j r x for eah, j. Observe that, j b j < n + x. Thus, provng that {b j } R r (T ) wth hgh probablty wll prove the theorem. Sne the network s d-regular, (S) d whenever S n d, and (S ) n S 2 whenever S 2 n d and S S 2. To see that the latter pont s true, observe the followng. If S n d, then (S ) d and hene S 2 n d mples that (S ) n S 2.Onthe other hand, f S n d, then (S ) =n S n S 2, where the nequalty follows sne S S 2. We onsder the ases where 2 S r n d and n d < S r n separately. The ase where S r = ondes presely wth the onstrants (6), and hene s satsfed by defnton of {b j }. Consderng the ase where 2 S r n d, wehavethe followng strng of nequaltes: r j= S j Ɣ(S j ) r (r+ j) b j= S j Ɣ(S j ) r r r x x j= S j Ɣ(S j ) (74) nr (75) r r d E[ P ] nkɛ nr (76) j= S j Ɣ(S j ) = r r d E[ P ] j= (S j ) r d E[ P ] nkɛ nr (77) S r S 0 6 In fat, the weak law of large numbers s suffent here, but Hoeffdng s nequalty an be used to prove exponental onentraton f desred.

14 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 49 r rd E[ P ] (S j ) n nr(kɛ + ) (78) j= + q rd E P r (S j ) n S r j= nr(kɛ + ) (79) + q rd E P (rd n) nr(kɛ + ) (80) S r ( + ) q E 2 P nr(kɛ + ) (8) S r ( + ) q E 4 P (82) S r P S r. (83) The above strng of nequaltes holds wth probablty tendng to as k. They an be justfed as follows: (74) follows by defnton of {b j }. (75) follows sne r x r x and S j Ɣ(S j ) n. (76) follows from (73) and the fat that S j Ɣ(S j ) n. (77) follows from wrtng r j= S j Ɣ(S j ) as ( r j= (S j ))\ ( S r S0) and expandng the sum. (78) s true sne S0 S r n. (79) follows from (67). (80) follows from (S j ) d by d regularty and the assumpton that 2 S r n d. (8) follows from our hoe of r gven n (72). To see ths, reall that we have hosen r 2n(+ q) d q, mplyng the nequalty + q rd (rd n) + q 2. (84) (82) follows sne q 4 E [ S r P ] nr(kɛ + ) for ɛ suffently small and k suffently large. Indeed, ths s equvalent to q 4 E k P = q S r 4 ( q) Sr ( q) n ( q) n ( nr ɛ + ). (85) k Sne the left hand sde of (85) s strtly postve and ndependent of k,ɛ, the lam follows. (83) follows from (7). Next, onsder the ase where n d S r n. Startng from (78), we obtan: r (r+ j) b j= S j Ɣ(S j ) r rd E[ P ] (S j ) n nr(kɛ + ) (86) j= ( ) rd E[ P ] r(n S r ) n nr(kɛ + ) (87) ( n = E[ P ] Sr n ) nr(kɛ + ) (88) d rd ( n E[ P ] Sr n n ) nr(kɛ + ) (89) rd E[ P ] ( E [ S r P E[ P ] ] + δ q n rd nr(kɛ + ) (90) ( [ E E[ P ] S r P ] E[ P ] + δ ) q nr(kɛ + ) (9) 2 ( + δ ) q E 4 P (92) S r. (93) S r P The above strng of nequaltes holds wth probablty tendng to as k. They an be justfed as follows: (86) s smply (78) repeated for onvenene. (87) follows sne n d S r n and hene d-regularty mples that (S j ) (n S r ). (89) follows sne d n. (90) follows from (70). (9) follows from from our defnton of r gven n (72). To see ths, reall that we have hosen r dδ 2n q, mplyng the nequalty δ q rd n δ q 2. (92) follows sne δ q 4 E[P ]nr(kɛ+) for ɛ suffently small and k suffently large. The argument justfyng ths s smlar to (85). (93) follows from (7). Thus, we onlude that, for ɛ suffently small, the transmsson shedule {b j } satsfes eah of the nequaltes defnng R r (T ) wth probablty tendng to. Sne the number of suh nequaltes s fnte, an applaton of the unon bound ompletes the proof that {b j } R r (T ) wth probablty tendng to as k. Remark 5: As was done n the proof of Theorem 3, t s possble to prove an exponental onentraton result for Theorem 4. However, n favor of a smpler presentaton, we omt the detals and ontent ourselves wth the gven asymptot result. D. Dvsble Pakets The proof of Theorem 5 s relatvely straghtforward. In partular, gven a transmsson shedule that satsfes the utset onstrants, we reate a orrespondng transmsson shedule that satsfes the neessary and suffent ondtons of Theorem adapted to t-dvsble pakets. Proof of Theorem 5: Fx any ɛ>0andletx be an optmal soluton to LP (9). Put b = x +ɛ. Note that b s nonnegatve. Ths follows by onsderng the set S\{} n the nequalty onstrant (0), whh mples x 0. )

15 50 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 Now, take an nteger r ɛ n max n b. If pakets are t-dvsble, we an fnd a transmsson shedule {b j } suh that r b b j r b + t for all [n], j [r]. Thus, for any (S 0,, S r ) S (r) (G) we have the followng strng of nequaltes: r j= S j Ɣ(S j ) where r = r = r r (r+ j) b r j= S j Ɣ(S j ) r j= (S j ) r j= (S j ) S 0 S r b r b (94) x + ɛ r S 0 S r b (95) r (S j ) j= b (96) r x + ɛ n r r max b (97) n j= (S j ) r r P j j= S j + ɛ n r max b (98) n, (99) S r P j (94) follows sne b j r b. (95) follows from wrtng r j= S j Ɣ(S j ) as ( r j= (S j ))\ ( S r S0) and expandng the sum. (96) follows from our defnton b = x + ɛ. (97) follows sne (S j ) and S0 S r n. (98) follows sne x s an optmal soluton to LP (9), and therefore must satsfy the lamed nequalty. (99) follows from our hoe or r ɛ n max n b. Hene, Theorem mples that the transmsson shedule {b j } s suffent to aheve unversal reovery. Notng that, j b j n = b + nr t n = ompletes the proof of the theorem. ( r ) x + n t + ɛ (00) E. Serey Generaton In ths subseton, we prove Theorems 6 and 7. We agan remark that our proofs essentally follow those n [4], but have been adapted for the problem at hand. For notatonal onvenene, defne P ={p,...,p k }. We wll requre the followng lemma. Lemma 4: Gven a paket dstrbuton P,...,P n,letk be a seret-key ahevable wth ommunaton F. Then the followng holds: H (K F) = H (P) n x. (0) for some vetor x = (x,...,x n ) whh s feasble for the followng ILP: mnmze subjet to: = n x (02) = x S P S for all S V. (03) Moreover, f K s a PK (wth respet to a set D) and eah node D transmts ts respetve set of pakets P,then H (K F) = H (P P D ) x. (04) V \D for some vetor x = (x,...,x n ) whh s feasble for the ILP: mnmze subjet to: V \D x (05) x S P S S V \D. (06) Remark 6: We remark that (03) and (06) are neessary and suffent ondtons for ahevng unversal reovery n the networks T and T D onsdered n Theorems 6 and 7, respetvely. Thus, the optmal values of ILPs (02) and (05) are equal to M (T ) and M (T D ), respetvely. Proof of Lemma 4: We assume throughout that all entropes are wth respet to the base- F logarthm (.e., nformaton s measured n pakets). For ths and the followng proofs, let F = (F,...,F n ) and F [,] = (F,...,F ),wheref denotes the transmssons made by node. For smplty, our proof does not take nto aount nteratve ommunaton, but an be modfed to do so. Allowng nteratve ommunaton does not hange the results. See [4] for detals. Sne K and F are funtons of P: H (P) = H (F, K, P,...,P n ) (07) n = H (F F [, ] ) + H (K F) = + n H (P F, K, P [, ] ). (08) = Set x = H (F F [, ] ) + H (P F, K, P [, ] ). Then, the substtutng x nto the above equaton yelds: H (K F) = H (P) n x. (09) =

16 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 5 Note that x = (x,...,x n ) s a feasble vetor for the lnear relaxaton of ILP (02), sne: P = H (P S P S ) (0) S = H (F, K, P S P S ) () n = H (K F, P S ) + H (F F [, ], P S ) = + H (P F, K, P [, ], P S [+,n]) (2) S H (F F [, ] ) + H (P F, K, P [, ] ) S S (3) = x. (4) S In the above nequalty, we used the fat that ondtonng redues entropy, the fat that K s a funton of (F, P S ) for any S = V, and the fat that F s a funton of P (by the assumpton that ommunaton s not nteratve). Now, sne C SK s an nteger, (09) mples that n = x s an nteger. Thus, Corollary n Appendx A (whh states that the optmal values of ILP (02) and ts lnear relaxaton dffer by strtly less than ), mples the exstene of an ntegral vetor x = ( x,..., x n ) whh s feasble for ILP (02) and satsfes n n x = x. (5) = Ths proves the frst part of the lemma. To prove the seond part of the lemma, we an assume D = {,...,l}. The assumpton that eah node n D transmts all of the pakets n P mples F = P. Thus, for D we have x = H (P P [, ] ). Repeatng the above argument, we obtan H (K F) = H (P) H (P D ) x (6) V \D = H (P P D ) x. (7) = V \D Exhangng x = (x,...,x n ) for an ntegral vetor x = ( x,..., x n ) as above ompletes the proof of the lemma. Proof of Theorem 6: Converse Part. Suppose K s a seretkey ahevable wth ommunaton F. Then, by defnton of askandlemma4wehave C SK = H (K ) = H (K F) (8) n = H (P) x (9) = H (P) M (T ) (20) = k M (T ). (2) Ahevablty Part. By defnton, unversal reovery an be aheved wth M (T ) transmssons. Moreover, the ommunaton F an be generated as a lnear funton of P (see the proof of Theorem and [2]). Denote ths lnear transformaton by F = LP. Note that L only depends on the ndes of the pakets avalable to eah node, not the values of the pakets themselves (see [2]). Let P F ={P : LP = F} be the set of all paket dstrbutons whh generate F. By our assumpton that the pakets are..d. unform from F, eah P P F s equally lkely gven F was observed. Sne F has dmenson M (T ), P F =F k M (T ). Thus, we an set K = F k M (T ) and label eah P P F wth a unque element n K. The label for the atual P (whh s reonstruted by all nodes after observng F) s the seret-key. Thus, C SK k M (T ). We remark that ths labelng an be aomplshed n polynomal tme by an approprate lnear transformaton mappng P to K. Proof of Theorem 7: Converse Part. Suppose K s a prvatekey. Then, by defnton of a PK and Lemma 4, C PK = H (K ) = H (K F) (22) = H (P P D ) x (23) V \D H (P P D ) M (T D ) (24) = (k P D ) M (T D ). (25) Ahevablty Part. Let eah node D transmt P so that we an update P j P j P D for eah j V \D. Now, onsder the unversal reovery problem for only the nodes n V \D. M (T D ) s the mnmum number of transmssons requred among the nodes n V \D so that eah node n V \D reovers P. At ths pont, the ahevablty proof proeeds dentally to the SK ase. VI. CONCLUDING REMARKS In ths paper, we derve neessary and suffent ondtons for ahevng unversal reovery n an arbtrarly onneted network. For the ase when the network s fully onneted, we provde a polynomal-tme algorthm based on submodular optmzaton whh solves the ooperatve problem. Ths algorthm and ts dervaton yeld tght onentraton results for the ase when pakets are randomly dstrbuted. Moreover, onentraton results are provded when the network s d-regular and pakets are dstrbuted randomly. If pakets are dvsble, we prove that the tradtonal ut-set bounds are ahevable. As a onsequene of ths and the onentraton results, we show that splttng pakets does not typally provde a sgnfant beneft when the network s d-regular. Fnally, we dsuss an applaton to serey generaton n the presene of an eavesdropper. We demonstrate that our submodular algorthm an be used to generate the maxmum amount of serey n a pratal manner. It s onevable that the oded ooperatve data exhange problem an be solved (or approxmated) n polynomal tme f the network s d-regular, but pakets aren t neessarly randomly dstrbuted. Ths s one possble dreton for future work. APPENDIX A AN EFFICIENTLY SOLVABLE INTEGER LINEAR PROGRAM In ths appendx, we ntrodue a speal ILP and provde an polynomal-tme algorthm for solvng t. Ths algorthm

17 52 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 an be used to solve the ooperatve data exhange problem when the underlyng graph s fully-onneted. We begn by ntrodung some notaton 7. Let E ={,...,n} be a fnte set wth n elements. We denote the famly of all subsets of E by 2 E. We frequently use the ompat notaton E\U and U + to denote the sets E U and U {} respetvely. For a vetor x = (x,...,x n ) R n, defne the orrespondng funtonal x : 2 E R as: x(u) := U x, for U E. (26) Throughout ths seton, we let F = 2 E {, E} denote the famly of nonempty proper subsets of E. Let B ={B,...,B n }. No speal struture s assumed for the B sexeptthattheyarefnte. Wth the above notaton establshed, we onsder the followng Integer Lnear Program (ILP) n ths seton: mn w x : x(u) B E E\U, U F, x Z. (27) It s lear that any algorthm that solves ths ILP also solves ILP (22) by puttng B P and w =. A. Submodular Optmzaton Our algorthm for solvng ILP (27) reles heavly on submodular funton optmzaton. To ths end, we gve a very bref ntroduton to submodular funtons here. A funton g : 2 E R s sad to be submodular f, for all X, Y 2 E, g(x) + g(y ) g(x Y ) + g(x Y ). (28) Over the past three deades, submodular funton optmzaton has reeved a sgnfant amount of attenton. Notably, several polynomal tme algorthms have been developed for solvng the Submodular Funton Mnmzaton (SFM) problem mn {g(u) : U E}. (29) We refer the reader to [28] [30] for a omprehensve overvew of SFM and known algorthms. As we wll demonstrate, we an solve ILP (27) va an algorthm that teratvely alls a SFM routne. The most notable feature of SFM algorthms s ther ablty to solve problems wth exponentally many onstrants n polynomal tme. One of the key drawbaks of SFM s that the problem formulaton s very spef. Namely, SFM routnes typally requre the funton g to be submodular on all subsets of the set E. B. The Algorthm We begn by developng an algorthm to solve an equalty onstraned verson of ILP (27). We wll remark on the 7 We attempt to keep the notaton gener n order to emphasze that the results n ths appendx are not restrted to the ontext of the ooperatve data exhange problem. general ase at the onluson of ths seton. To ths end, let M be a postve nteger and onsder the followng ILP: mnmze w T x (30) subjet to: x(u) B for all U F, and (3) E\U x(e) = M. (32) Remark 7: We assume w 0, else n the ase wthout the equalty onstrant we ould allow the orrespondng x + and the problem s unbounded from below. Algorthm A.: SOLVEILP(B, E, M,w) omment: Defne f : 2 E R as n eq. (34). x COMPUTEPOTENTIALX( f, M,w) f CHECKFEASIBLE( f, x) then return (x) else return (Problem Infeasble) Theorem 8: Algorthm A. solves the equalty onstraned ILP (30) n polynomal tme. If feasble, Algorthm A. returns an optmal x. If nfeasble, Algorthm A. returns Problem Infeasble. Proof: The proof s aomplshed n three steps: ) Frst, we show that f our algorthm returns an x, ts feasble. 2) Seond, we prove that f a returned x s feasble, t s also optmal. 3) Fnally, we show that f our algorthm does not return an x, then the problem s nfeasble. Eah step s gven ts own subseton, and our proof roughly follows the development n [30] on rossng famles. Algorthm A. reles on three bas subroutnes gven below: Algorthm A.2: COMPUTEPOTENTIALX( f, M,w) omment: If feasble, returns x satsfyng (3) and (32) that mnmzes w T x. omment: Order elements of E so that w w 2 w n. for n to 2 omment: Defne f (U) := f (U + ) do for U {,...,n}. x SFM( f, {,...,n}) x M n =2 x return (x) Algorthm A.3: CHECKFEASIBLE( f, x) omment: Chek f x(u) f (U) for all U F wth U. omment: Defne f (U) := f (U + ) for U E. f SFM( f, E) <0 then return ( false ) else return ( true )

18 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 53 Algorthm A.4: SFM( f, V ) omment: Mnmze submodular funton f over groundset V. See [28] for detals. v mn { f (U) : U V } return (v) C. Feasblty of a Returned x In ths seton, we prove that f Algorthm A. returns a vetor x, t must be feasble. We begn wth some defntons. Defnton 7: AparofsetsX, Y E s alled rossng f X Y = and X Y = E. Defnton 8: A funton g : 2 E R s rossng submodular f g(x) + g(y ) g(x Y ) + g(x Y ) (33) for X, Y rossng. We remark that mnmzaton of rossng submodular funtons s well establshed, however t nvolves a lengthy reduton to a standard submodular optmzaton problem. However, the rossng famly F admts a straghtforward algorthm, whh s what we provde n Algorthm A.. We refer the reader to [30] for omplete detals on the general ase. For M a postve nteger, defne f (U) := M B x(u), for U F. (34) U Lemma 5: The funton f s rossng submodular on F. Proof: For X, Y F rossng: f (X) + f (Y ) = M B x(x) + M B x(y ) (35) X Y = M B x(x Y ) X +M B x(x Y ) (36) Y M B x(x Y ) X Y +M B x(x Y ) (37) X Y = f (X Y ) + f (X Y ). (38) Observe that, wth f defned as above, the onstrants of ILP (30) an be equvalently wrtten as: f (U) = M B x(u) 0 for all U F, (39) U x(e) = M. (40) Wthout loss of generalty, assume the elements of E are ordered lexographally so that w w 2 w n. At teraton n Algorthm A.2, x j = 0forall j. Thus, settng x mn { f (U)} (4) U {,...,n} = mn { f (U)} (42) U {,...,n}: U { } = mn M B x(u) (43) U {,...,n}: U U and notng that the returned x satsfes x(e) = M, rearrangng (43) guarantees that x(e\u) B, for all U {,...,n}, U (44) U as desred. Iteratng through {2,...,n} guarantees (44) holds for 2 n. Remark 8: In the feasblty hek routne (Algorthm A.3), we must be able to evaluate f (E). The reader an verfy that puttng f (E) = 0 preserves submodularty. Now, n order for the feasblty hek to return true, we must have mn { f (U)} = mn { f (U)} (45) U E U E: U { } = mn M B x(u) (46) U E: U U 0, (47) mplyng that x(e\u) B, for all U E, U. (48) U Combnng (44) and (48) and notng that x(e) = M proves that x s ndeed feasble. Moreover, x s ntegral as desred. D. Optmalty of a Returned x In ths seton, we prove that f Algorthm A. returns a feasble x, then t s also optmal. Frst, we requre two more defntons and a lemma. Defnton 9: A onstrant of the form (39) orrespondng to U s sad to be tght for U f f (U) = M B x(u) = 0. (49) U Lemma 6: If x s feasble, X, Y are rossng, and ther orrespondng onstrants are tght, then the onstrants orrespondng to X Y and X Y are also tght. Proof: Sne the onstrants orrespondng to X and Y are tght, we have 0 = f (X) + f (Y ) f (X Y ) + f (X Y ) 0. (50) The frst nequalty s due to submodularty and the last nequalty holds sne x s feasble. Ths mples the result. Defnton 0: A famly of sets L s lamnar f X, Y L mples ether X Y =, X Y,orY X. At teraton k ( < k n) of Algorthm A.2, let U k be the set where (43) aheves ts mnmum. Note that k U k

19 54 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 60, NO. 2, FEBRUARY 204 {k,...,n}. By onstruton, the onstrant orrespondng to U k s tght. Also, the onstrant x(e) = M s tght. From the U k s and E we an onstrut a lamnar famly as follows: f U j U k = for j < k, then replae U j wth Ũ j U k U j. By Lemma 6, the onstrants orrespondng to the sets n the newly onstruted lamnar famly are tght. Call ths famly L. For eah E, there s a unque smallest set n L ontanng. Denote ths set L.Snek U k {k,...,n}, L = L j for = j. Note that L = E and L L j only f j <. For eah L L there s a unque smallest set L j suh that L L j. We all L j the least upper bound on L. Now, onsder the dual lnear program to (30): maxmze ) π U (M B π E M (5) U F U subjet to: π U + π E + w = 0, for n U F: U π U 0 for U F, and π E free. (52) For eah L L, let the orrespondng dual varable π L = w j w, where L j s the least upper bound on L. By onstruton, π L 0 sne t was assumed that w w n. Fnally, let π E = w and π U = 0for U / L. Now, observe that: π U + π E + w = 0 (53) U F: U as desred for eah. Thus, π s dual feasble. Fnally, note that π U > 0 only f U L. However, the prmal onstrants orrespondng to the sets n L are tght. Thus, (x,π) form a prmal-dual feasble par satsfyng omplementary slakness ondtons, and are therefore optmal. E. No Returned x = Infeasblty Fnally, we prove that f the feasblty hek returns false, then ILP (30) s nfeasble. Note by onstruton that the vetor x passed to the feasblty hek satsfes M B x(u) 0 for all nonempty U {2,...,n}, U and x(e) = M. Agan,letU k be the set where (43) aheves ts mnmum and let L be the lamnar famly generated by these U k s and E exatly as before. Agan, the onstrants orrespondng to the sets n L are tght (ths an be verfed n a manner dental to the proof of Lemma 6). Now, sne x faled the feasblty hek, there exsts some exeptonal set T wth T for whh M B x(t )<0. (54) T Generate a set L T as follows: Intalze L T T. For eah L L, L = E, fl T L =, update L T L T L. Now, we an add L T to famly L whle preservng the lamnar property. We pause to make two observatons: ) By an argument smlar to the proof of Lemma 6, we have that M x(l T )<0. (55) L T B 2) The sets n L whose least upper bound s E form a partton of E. We note that L T s a nonempty lass of ths partton. Call ths partton P L. Agan onsder the dual onstrants, however, let w = 0 (ths does not affet feasblty). For eah L P L defne the assoated dual varable π L = α, andletπ E = α. All other dual varables are set to zero. It s easy to hek that ths π s dual feasble. Now, the dual objetve funton beomes: ) π U ((M B π E M U F U = α ( ) M B x(l) + x(l) + αm L P L L = α M B L T x(l T ) αx(e) + αm = α M B L T x(l T ) + as α. Thus, the dual s unbounded and therefore the prmal problem must be nfeasble. As an mmedate orollary we obtan the followng: Corollary : The optmal values of the ILP: mn { x(e) : x(u) E\U B, U F, x Z } and the orrespondng LP relaxaton: mn { x(e) : x(u) E\U B, U F, x R } dffer by less than. Proof: Algorthm A. s guaranteed to return an optmal x f the nterseton of the polytope and the hyperplane x(e) = M s nonempty. Thus, f M s the mnmum suh M, then the optmal value of the LP must be greater than M. F. Solvng the General ILP Fnally, we remark on how to solve the general ase of the ILP wthout the equalty onstrant gven n (27). Frst, we state a smple onvexty result. Lemma 7: Let pw (M) denote the optmal value of ILP (30) when the equalty onstrant s x(e) = M. We lam that pw (M) s a onvex funton of M. Proof: Let M and M 2 be ntegers and let θ [0, ] be suh that M θ = θ M + ( θ)m 2 s an nteger. Let x () and be x (2) optmal vetors that attan pw (M ) and pw (M 2) respetvely. Let x (θ) = θ x () + ( θ)x (2). By onvexty, x (θ) s feasble, though not neessarly nteger. However, by

20 COURTADE AND WESEL: CODED COOPERATIVE DATA EXCHANGE 55 Fg. 6. Empral omputaton tmes for Algorthm A., where eah data pont s averaged over 0 trals. For the broken lne, the multplatve onstant α and exponent β were hosen to mnmze the MSE n = log(αn β ) log( ˆm n ) 2, where ˆm n s the sample mean of the omputaton tmes for E =n. the results from above, optmalty s always attaned by an ntegral vetor. Thus, t follows that: θ pw (M ) + ( θ)pw (M 2) = θw T x () + ( θ)w T x (2) = w T x (θ) (56) pw (M θ ). (57) Notng that pw (M) s onvex n M, we an perform bseton on M to solve the ILP n the general ase. For our purposes, t suffes to have relatvely loose upper and lower bounds on M sne the omplexty only grows logarthmally n the dfferene. A smple lower bound on M s gven by M max B. G. Complexty Our am n ths paper s not to gve a detaled omplexty analyss of our algorthm. Ths s due to the fat that the omplexty s domnated by the the SFM over the set E n Algorthm A.3. Therefore, the omplexty of Algorthm A. s dtated by the omplexty of the SFM solver employed. Indeed, Algorthm A. sequentally performs SFM over groundsets of sze, 2,...,n, respetvely. Hene the total omplexty of Algorthm A. s O(n SFM(n)), wheresfm(m) s the omplexty of performng SFM over a groundset of sze m.its well-known that SFM(m) s polynomal n m, for nstane the survey [28] desrbes an Ellpsod Algorthm-based approah whh runs n O(m 5 γ +m 7 ) tme, where γ s the omplexty of evaluatng the submodular funton under onsderaton. We remark that the algorthm presented n [8] solves the LP relaxaton of (27) n O(n 3 SFM(n)) tme. Despte the fat that SFM has somewhat large worstase polynomal, we have performed a seres of numeral experments to demonstrate that Algorthm A. performs qute well n prate. In our mplementaton, we ran the Fujshge- Wolfe (FW) algorthm for SFM [3] based largely on a Matlab routne by A. Krause [32]. Whle the FW algorthm has not been proven to run n polynomal tme, t has been shown to work qute well n prate [3] (smlar to the Smplex algorthm for solvng Lnear Programs). Whether or not FW has worst-ase polynomal omplexty s an open problem to date. As mentoned above, there are SFM algorthms that run n strongly polynomal tme whh ould be used f a partular applaton requres polynomally bounded worstase omplexty [28]. In our seres of experments, we hose B F randomly, where F =50. We let n = E range from 0 to 90 n nrements of 0. For eah value of n, we ran 0 experments. The average omputaton tme s shown n Fgure 6, wth error bars ndatng one standard devaton. We onsstently observed that the omputatons run n approxmately O(n.85 ) tme. Due to the teratve nature of the SFM algorthm, we antpate that the omputaton tme ould be sgnfantly redued by mplementng the algorthm n C/C++ nstead of Matlab. However, the O(n.85 ) trend should reman the same. Regardless, we are able to solve the ILP problems under onsderaton wth an astonshng 2 90 onstrants n approxmately one mnute. APPENDIX B PROOF OF LEMMA 2 For onvenene, we repeat the statement of Lemma 2. Lemma 2: Let A R n n be a symmetr matrx wth nonnegatve entres and all olumn sums equal to d. Let x y be the vetor of mnmum Euldean norm whh mnmzes Ax y y 2. There exsts an optmal soluton x to the lnear program whh satsfes mnmze T x (58) subjet to: Ax y (59) x x y A A x y y 2, (60) where A s a onstant dependng only on A. Before movng to the proof of the lemma, we omment brefly on the nterpretaton. If the matrx A n the statement of Lemma 2 s nonsngular, then the soluton x to the lnear program (58) s unque and equal to x y. Indeed, ths an be seen by premultplyng both sdes of (59) by T, mplyng T x d T y for all feasble x. Ths lower bound s aheved only f Ax = y. On the other hand, f A s sngular, there are many optmal solutons to the lnear program (58). More presely, f x s one optmal soluton for the lnear program (58), then so s x + z for any z suh that Az = 0. Thus, Lemma 2 guarantees that we an fnd an optmal soluton x whh s lose to the least squares soluton x y. Proof of Lemma 2: To begn the proof, we make a few defntons. Let λ be the absolute value of the nonzero egenvalue of A wth smallest modulus (at least one exsts sne d s an egenvalue). Defne N (A) to be the nullspae of A, and let N (A) denote ts orthogonal omplement. Fnally, let A + denote the Moore-Penrose pseudonverse of A. Fx x y R n, and note that x s an optmal soluton to LP (58) f and only f x x y s an optmal soluton to the