Information Dimension and the Degrees of Freedom of the Interference Channel

Transcription

1 Inforation Diension and the Degrees of Freedo of the Interference Channel Yihong Wu, Shloo Shaai Shitz and Sergio Verdú Abstract The degrees of freedo of the K-user Gaussian interference channel deterine the asyptotic growth of the axial su rate as a function of the signal-to-noise ratio. Subect to a very general sufficient condition on the cross-channel gains, we give a forula for the degrees of freedo of the scalar interference channel as a function of the deterinistic channel atrix, which involves axiization of a su of inforation diensions over K scalar input distributions. Known special cases are recovered, and even generalized in certain cases with unified proofs. Index Ters Multiuser inforation theory, interference channel, inforation diension, Shannon theory, suset entropy inequalities, degrees of freedo, high-snr channels. I. INTRODUCTION Consider a K-user real-valued eoryless Gaussian Interference Channel IC with a fixed deterinistic channel atrix H = [h i ] known at the encoder and decoder, where at each sybol epoch the i th user transits X i and the i th decoder receives K Y i = snr hi X + N i, where {X i, N i } K i= are independent with E [ ] Xi and N i N 0,. Denote the capacity region of by CH, snr and the surate capacity by { K } CH, snr ax R i : R K CH, snr. i= The degrees of freedo DoF or the ultiplexing gain is the pre-log of the su-rate capacity in the high-snr regie, defined by CH, snr DoFH = li sup. 3 snr log snr This work was partially supported by the National Science Foundation NSF under Grant CCF-0665 and by the Center for Science of Inforation CSoI, an NSF Science and Technology Center, under Grant CCF This work was also supported by the U.S.Israel Binational Science Foundation BSF. The results of this paper were presented in part at the 0 IEEE International Syposiu on Inforation Theory, Saint-Petersburg, Russia, July 3 August 5, 0 []. Y. Wu was with the Departent of Electrical Engineering, Princeton University, Princeton, NJ 08544, USA. He is now with the Departent of Electrical and Coputer Engineering, University of Illinois at Urbana- Chapaign, Urbana, IL 680, USA e-ail: yihongwu@illinois.edu. S. Verdú is with the Departent of Electrical Engineering, Princeton University, Princeton, NJ 08544, USA e-ail: verdu@princeton.edu. S. Shaai Shitz is with the Departent of Electrical Engineering, Technion, IIT, Technion City, Haifa 3000, Israel e-ail: sshloo@ee.technion.ac.il All logariths in this paper are binary unless explicitly stated otherwise. Note that the noralization in the definition of 3 is such that DoFH = in the single-user case. Despite its suggestive terinology, DoFH need not be an integer in the ulti-user case. Deterining the degrees of freedo has been an active and challenging research subect, in view of the fact that the axial su rate of the Gaussian interference channel CH, snr reains unknown. In [] it is shown that DoFH K for fully-connected H, i.e., H with no zero entries. Using Diophantine approxiations, 4 is shown to be achievable for Lebesgue-alost every H [3]. The alost sure achievability of K for the vector interference channel with varying channel gains has been shown in [4] using the technique of interference alignent. Sufficient conditions on individual H that guarantee equality in 4 are also given in [5], [6]. However, equality does not hold in general: for exaple, if all the entries in H are rational, then 4 is strict as shown in [5] based on additive-cobinatorial results and deterinistic channel approxiations. In addition to theoretical investigations, various counication and interference alignents schees have been proposed see, e.g., [7], for a coprehensive review. The goal of this paper is to give a single-letter forula for DoFH via the axiization of a functional involving the inforation diension also known as Rényi diension or entropy diension [8] under a general sufficient condition on the off-diagonals of H, which, in particular, is satisfied alost surely or if the off-diagonals are algebraic. Moreover, the axiization gives a lower bound on DoFH for any channel atrix H. This unified approach allows us to uncover new results, as well as recover previously known results obtained via different ethods. We also give results for the ore general case in which the rates are, unlike, not equally weighted. The rest of the paper is organized as follows: Section II gives the necessary background on inforation diension, which is the cornerstone of the ain results presented in Section III. Our results on the degrees of freedo region are given in Section III-F. In Section IV we discuss self-siilar distributions, which play an iportant role in constructing DoF-achieving input distributions. Proofs of the ain results are given in Section V, while several technical leas are relegated to the appendix. The converse results for rational channel atrices are based on new suset inequalities for linear cobinations of independent group-valued rando variables in Appendix E, which ight be of independent interests. 4

2 II. INFORMATION DIMENSION In this section we suarize the ain properties of inforation diension relevant to the current paper. In particular, the connection between the inforation diension and the high-snr asyptotics of utual inforation with additive noise is presented in Section II-B. Other properties and further discussions can be found in [8], [9], [0], []. A. Definitions A key concept in fractal geoetry, the inforation diension of a rando variable easures the rate of growth of the entropy of its successively finer discretizations [8]. Definition. Let X be a real-valued rando variable. For N, denote the uniforly quantized version of X by X X. 5 The inforation diension of X is defined as H X dx = li log. 6 Useful if the liit in 6 does not exist, the li inf and li sup are called lower and upper inforation diensions of X, respectively, denoted by dx and dx. Definition can be readily extended to rando vectors, where the floor function is taken coponentwise [8, p. 08]. Since dx only depends on the distribution of X, we also denote dp X = dx. Siilar convention is traditionally followed with entropy and other inforation easures. The inforation diension of X is finite if and only if the ild condition H X < 7 is satisfied [8], [0]. A sufficient condition for finite inforation diension is E [log + X ] <, 8 which is ilder than the existence of E [X]. Therefore 7 is satisfied for all rando variables considered in this paper. Equivalent definitions of inforation diension include: For an arbitrary integer M, write the M-ary expansion of X as X = X + i NX i M i. 9 where the i th digit X i M i X M M i X is a discrete rando variable taking values on {0,..., M }. Then dx and dx coincide with the noralized lower and upper entropy rates of the process {X : N}. Denote by Bx, ɛ the open ball 3 of radius ɛ centered at x. Then see [, Definition 4.] and [0, Appendix A] E[log P X BX, ɛ] dx = li. 0 ɛ 0 log ɛ In the sequel, we will use the notation in 5 even if is not an integer. 3 By the equivalence of nors on finite-diensional space, the value of the liit on the right side of 0 is independent of the underlying nor. With Shannon entropy replaced by Rényi entropy of order α in Definition, the generalized notion of diension of order α 4 is defined siilarly [4], [9]: Definition Inforation diension of order α. Let α [0, ]. The inforation diension of X of order α is defined as H α X d α X = li log, where H α Y denotes the Rényi entropy of order α of a discrete rando variable Y with probability ass function P Y defined on Y, defined as log ax H α Y = P Y y, α =, y Y α log y Y P Y y α, α,. The li inf and li sup of the ratio in are called lower and upper inforation diensions of X of order α respectively, denoted by d α X and d α X. Note that d X = dx. As a consequence of the onotonicity of Rényi entropy, d α decreases with α. Moreover, α d α X is continuous except possibly at α = [5]. For instance, if X has a discrete-continuous ixed distribution, then d α X = ˆdX = for all α <, while dx equals to the weight of the continuous part see Theore. If X has a Cantor distribution see [6] or Section IV, d α X = log 3 for all α [0, ]. B. Properties The following are basic properties of inforation diension [8], [0], the last three of which are inherited fro Shannon entropy. Lea. Scale-invariance: For all a 0, 0 dx n n. 3 dax n = dx n. 4 If X n and Y n are independent, then 5 ax{dx n, dy n } dx n + Y n 5 If X n, Y n, Z n are independent, then dx n + dy n. 6 dx n +Y n +Z n +dz n dx n +Z n +dy n +Z n. 7 Proof. Appendix A. To calculate the inforation diension in 6 or, it is sufficient to restrict to an exponential subsequence, as a result of the following lea. 4 Also known as the L α diension or spectru [3]. In particular, d is also called the correlation diension. 5 The upper bound 6 can be iproved by defining conditional inforation diension.

3 3 Lea. Let λ >. If d α X exists, then H α X d α X = li λ l. 8 l l log λ As shown in [8], the inforation diension for the ixture of discrete and absolutely continuous distributions can be deterined as follows see [8, Theore and 3] or [9, Theore, pp ]: Theore. Assue that the distribution of a scalar rando variable X is given by ν = ρν d + ρν c, 9 where ν d is a discrete probability easure, ν c is an absolutely continuous probability easure and 0 ρ. Then dx = ρ. 0 In particular, if X has a density with respect to Lebesgue easure, then dx = ; if X is discrete, then dx = 0. When the distribution of X has a singular coponent, its inforation diension does not adit a siple forula in general. However, for the iportant class of self-siilar singular distributions, the inforation diension can be explicitly deterined, as we show in Section IV-B. C. Inforation diension and high-snr utual inforation The high-snr asyptotics of utual inforation with additive noise is governed by the input inforation diension, as shown by the following result due to Guionnet and Shlyakhtenko [7]. 6 In this paper we give a considerably siplified and self-contained proof for this result based on a non-asyptotic bound on utual inforation in Section V-A, fro which Theore is an iediate consequence. Theore. Let X be independent of N which is standard noral. Denote Then 7 IX, snr IX; snrx + N, IX, snr li snr = dx. log snr Moreover, holds verbati in the vector case. In view of Theore, the inforation diension dx represents the single-user degrees of freedo when the input distribution is constrained to be P X. Naturally, inforation diension, as we will see, also appears in the characterization of degrees of freedo in the ulti-user case. In the singleuser case, indicates that utual inforation is axiized 6 The original results in [7, Theore 3.] are proved assuing 8, which is in fact superfluous as shown in [8, Section.7]. Moreover, Theore holds even if dx =, because IX, snr < if and only if 7 holds [9, Theore 6]. Also, [7] failed to point out that the classical analogue of the free entropy diension they defined is, in fact, the inforation diension. 7 When the liit on the left side of does not exist, continues to hold with li replaced by li inf resp. li sup and d replaced by d resp. d. asyptotically by any absolutely continuous input distribution, in view of Theore in Section II-B. In the interference channel, as we will see in Section III, the situation is far ore intricate since analog inputs are deleterious fro the viewpoint of interference. D. Inforation diension under proection Let A R n with n. Then for any X n, dax n in{dx n, ranka}. 3 Understanding how the diension of a easure behaves under proections is a basic proble in fractal geoetry. It is wellknown that alost every proection preserves the diension, be it Hausdorff diension Marstrand s proection theore [0, Chapter 9] or inforation diension [, Theores. and 4.]. However, coputing the diension for individual proections is, in general, difficult. The preservation of inforation diension of order α, ] under typical proections is shown in [, Theore ], which is relevant to our investigation of the alost sure behavior of degrees of freedo. Lea 3 []. Let α, ] and n. Then for alost every A R n, d α AX n = in {d α X n, }. 4 It is easy to see that 4 fails for inforation diension α = [, p. 04]: Let P X n = ρδ 0 + ρq, where Q is an absolutely continuous distribution on R n, n > and ρ <. Note that alost surely, AX n also has a ixed distribution with an ato at zero of ass ρ. Then dax n = ρ < in {dx n, } = in{ρn, }. Exaples of nonpreservation of d α for 0 α < or α > are given in [, Section 5]. A proble closely related to deterining DoFH is to find the diension difference of a product easure i.e., distribution of a pair of independent rando variables under two proections c.f. Reark 5 in Section III-G. Let p, q, p, q be non-zero real nubers. Then dpx + qy dp X + q Y dpx + qy + dpx + qy dx dy 5, 6 where 5 and 6 follow fro 5 and 6, respectively. Therefore, the diension of a two-diensional product easure under two proections can differ by at ost one half. Moreover, the next result shows that if the coefficients are rational, then the diension difference is strictly less than one half. The proof is based on new entropy inequalities for linear cobinations of independent rando variables developed in Appendix E. Theore 3. Let p, p, q, q be non-zero integers. Then dpx + qy dp X + q Y ɛp q, pq, 7

4 4 where ɛa, b 56 log a + log b In the special cases of p =, q = and p =, q =, the following iproved bounds hold and Proof. Appendix E. dx + Y dx + Y dx Y dx + Y Reark. The constants in 8 are, of course, not optial. However, the logarithic dependence on the coefficients is sharp, as shown by the following exaple: For any p N, there exists independent X and Y, such that dpx + Y dx + Y Θ log p. See the proof of Theore in Section V-E for the construction. A. A general forula III. MAIN RESULTS The degrees of freedo of the K-user interference channel with channel atrix H defined in 3 is given by the following result. Theore 4. Let dofx K, H Then, for any H, 8 K K d h i X d h i X. 3 i= DoFH sup X K dofx K, H, 3 where the supreu is over independent X,..., X K such that H X i < 33 and all inforation diensions appearing in 3 exist. Equality holds in 3 except possibly for H whose off-diagonals are non-algebraic and belong to a set of zero Lebesgue easure. Proof. Section V-B. For a fixed channel atrix H, we have been able to show that equality holds in 3 under the sufficient condition that all cross-channel gains are algebraic e.g., rationals, quadratic irrationals, etc. 9 This ight be an artifact of the proof. We conecture that equality in 3 in fact applies to any atrix H. Nevertheless, in the three-user case, the condition for equality in 3 can be further relaxed. Since DoFH is invariant under 8 In fact, the lower bound 3 holds even if DoFH in 3 is defined with an li inf snr. Therefore the liit exists whenever equality holds in 3. 9 In Section V-B, we give a deterinistic sufficient condition on the crosschannel gains, which is ore general than algebraicity and holds alost surely. See. row and colun scaling, the channel atrix can be reduced to the following standard for [3, p. ]: H = g g. 34 h g 3 where h is the ratio between the product of upper and lower off-diagonals, i.e., i< h h i i> h 35 i Then, 3 holds with equality as long as h is an algebraic nuber. Reark. Condition 33 is uch weaker than the average power constraint, since E [ X ] iplies that H X E [ + X ]. In fact, as we show in Section V-B, in order for 3 to hold, the inputs need not satisfy any power constraint. As long as 33 is satisfied, DoFH does not change even if we allow infinite input power, in which case snr is siply a scale paraeter no longer carrying the operational eaning of signal-to-noise ratio. We proceed to offer soe insight on the forula in Theore 4. By the liiting characterization of the interference channel capacity region [] [, ], the su-rate capacity is given by CH, snr = li n n sup X n,...,xn K K i= IX n i ; Y n i, 36 where Xi n = [X i,,..., X i,n ] is the input of the i th user, and the supreu is over independent X n,..., XK n satisfying the corresponding average cost constraints. Then, IXi n ; Yi n = IX n,..., XK; n Yi n IX n,..., XK; n Yi n Xi n 37 K = I h i X n, snr I h i X n, snr, 38 where I, snr is defined in. Therefore the degrees of freedo adit the following liiting characterization: 0 DoFH = li sup li sup 39 snr n X n n log snr,...,xn K { K K } I h i X n, snr I h i X n, snr, i= Our ain result is the single-letterization of 39. Note that if the liit over snr were inside the supreu, then the desired result 3 would follow iediately in view of the high-snr scaling of utual inforation in Theore. Indeed, eploying i.i.d. input distributions yields 3. Thus the ain difficulty in the converse proof lies in exchanging the supreu with the liits in 39, which aounts to proving that varying the input distribution with increasing SNR does not iprove the degrees of freedo. To this end, we need a non-asyptotic version of Theore given in Section V-A. 0 The second liit in 39 can be replaced by supreu over n N.

5 5 In the rest of the section, we assue that the cross-channel gains are algebraic and we discuss soe of the benefits of the inforation diension characterization in Theore 4. B. Soe siple special cases As we illustrate next, the degrees of freedo of various channels can be obtained by specializing Theore 4. Two-user IC: [ ] 0 a = d = 0 a b DoF = a 0, d 0, b = c = 0 c d otherwise. 40 Many-to-one IC [3]: if h ii 0 for all i and h i 0 for soe i, then h h h 3 h K 0 h 0 0 DoF = K h KK To see this, assuing h 0, we have dofx K, H = d d h X d h X + dx K h X + dx 4 =3 K, 43 where 4 is due to 5. The upper bound K is attained by choosing X,..., X K to be absolutely continuous. One-to-any IC [3]: if h ii 0 for all i, then h h h 0 0 DoF = K 44 h K 0 0 h KK To verify 44 note that dofx K, H = d h X + h X K dx 45 d h X + h X 46 K, 47 attained by choosing X discrete and the rest absolutely continuous. Multiple-access channel MAC: If H is the all-one atrix, then DoFH =, because K K dofx K, H = Kd X d X K d X i= 48 49, 50 where we have used the following additive-cobinatorial result [4, p. 3]: K K K H U H U 5 i= i= with {U i } taking values on an arbitrary group. In view of Lea 9, setting U = X in 5 and sending yields 49. More generally, DoFH = if rankh =. Another way to see that DoFH = is to invoke the last property of DoF in Section III-D: Denote by H the upper-triangular all-one atrix. Then DoFH DoFH = sup X K d X. C. Suboptiality of discrete-continuous ixtures Mixed discrete-continuous input distributions are strictly suboptial in general. In fact, they achieve at ost one degree of freedo in the fully-connected case. To see this, denote for brevity dx i = ρ i. Assue that H is fullyconnected. For each i, K h ix has a discrete-continuous distribution, where the weight of the discrete coponent is equal to K ρ. Then dofx K, H = = K i= K ρ ρ K ρ i ρ 5 i=, 53 Therefore to obtain ore than one degree of freedo, it is necessary to eploy input distributions with singular coponents. As we will show later, singular distributions of inforation diension one half are crucial in achieving the axial degrees of freedo. In Section IV-C we give a faily of such distributions {µ N } N which are self-siilar [6] and DoFachieving in any cases. In fact, µ N can be understood as the distribution of a rando variable whose N-ary expansion has equiprobable even digits and zero odd digits. Then dµ N = in view of 9. Such a distribution is neither discrete nor continuous. D. Properties of DoFH The following are iediate consequences of Theore 4 cobined with the eleentary properties of inforation diension in Lea :

6 6 DoFH is invariant under row or colun scaling [5, Lea ], in view of 4. To be ore precise, for any X K, dofx K, H is invariant under row scaling by 4, but dofx K, H is not invariant under colun scaling. However, by replacing each X i with its scaled version, sup X K dofx K, H is invariant under colun scaling. DoFH K, with equality if and only if H is a diagonal atrix with no diagonal entry equal to zero. To see this, note that dofx K, H = K if and only if K d h i X =, 54 d h i X = for all i. On one hand, iply h ii 0. On the other hand, since dx i d h i X d h i X, 56 we have dx i = for all i, and 55 iplies h i = 0 for all i. Let diagh = [h,..., h KK ]. Then DoFH 0, with equality if and only if diagh = 0. If diagh 0, then DoFH, which can be achieved by choosing one of the inputs to be absolutely continuous unit diension and the rest discrete zero diension. Reoving cross-links increases the degrees of freedo: Let H be obtained fro H by setting soe of the off-diagonal entries to zero. By 7, for any independent X K, dofx K, H dofx K, H. Therefore DoFH DoFH. This can also be seen using a genie-aided converse arguent by providing the interfering essages to the relevant receivers. E. Bounds and exact expressions Next we prove that the nuber of degrees of freedo is upper bounded by K under ore general sufficient conditions than the fully-connected assuption in []. Theore 5. Suppose that H satisfies the assuption in Theore 4. Let π be a fixed-point-free perutation on {,..., K}, i.e., πi i for all i. If h πi,i 0 for each i, then DoFH K. 57 Moreover, if K is odd and π is cyclic, i.e., there exist distinct eleents {n,..., n K } of {,..., K}, such that πn i = n i+ for i =,..., K and πn K = n, then dofx K, H = K if and only if for each i, dx i = 58 d h i X = 59 K d h i X =. 60 Proof. Section V-C. Reark 3. In the second part of Theore 5, the assuption that π is cyclic is not superfluous. To see this, consider K = 5, π =, π =, π3 = 4, π4 = 5, π5 = 3 and h i = 0 except for h ii and h πi,i. Such a channel consists of a two-user IC decoupled with a three-user IC. Therefore as the su of the respective DoFs, DoFH = 5/, which can be achieved by choosing X to be absolutely continuous, X discrete and X 3, X 4, X 5 to be the corresponding DoFachieving distributions for the three-user IC. Clearly in this case 58 is not necessary. The sae also applies to the case of even nuber of users. Next we give various sufficient conditions on H that guarantee DoFH = K. Theore 6 [3]. DoFH = K Proof. Section V-D. for Lebesgue-a.e. H. Recently, Stotz and Bölcskei [5], following up on an earlier version of this paper, obtained deterinistic sufficient conditions on H to guarantee that DoFH = K, which iply Theore 6 as a special case. Using the achievability bound 3 via inforation diension and the self-siilar input distribution in Section IV, the proof in [6] invokes a recent result by Hochan [7] in fractal geoetry, in lieu of the Diophantine approxiation results used in the present paper as well as [5], [3]. In particular, using results fro [7] it is shown in [5, Theore ] that for the self-siilar construction in 73, the inforation diension forula 8 holds for alost every contraction ratio r without requiring the open set condition, which can be difficult to verify. This shortcut offers sipler proofs of achievability results such as the following see [5, Section VIII]: Theore 7. If the off-diagonal entries of H are rational and the diagonal entries are irrational non-zeros, then DoFH = K Ṫheore 7 generalizes the following known results, previously obtained using different Diophantine approxiation results: [5, Theore ], which relies on the Thue-Siegel-Roth theore [8] and requires the diagonal entries to be irrational algebraic nubers. Note that the set of real algebraic nubers is dense but countable. Therefore, alost every real nuber is transcendental. [6, Theore ], which assues that the diagonal and off-diagonal entries are equal to one and h respectively, with h irrational. Upon scaling, this is equivalent to a channel atrix with unit off-diagonal entries and irrational diagonal entries h. Reark 4. Theore 7 requires all diagonal entries to be irrational. Otherwise K degrees of freedo ight be unattainable. For instance, consider the 3 3 channel atrix H λ in 65. Since DoF is invariant under row or colun scaling, DoFH λ

7 7 does not depend on h or h 33 as long as they are both nonzero. Then Theore iplies that DoFH λ < 3 if h is rational, even if h and h 33 are both irrational. For H with non-zero rational coefficients it is known that DoFH < K [5]. Next we give a ore general sufficient condition for the strict inequality in the three-user case. Theore 8. Let K = 3 and let the off-diagonals of H be algebraic. If there exist distinct i,, k, such that h i, h ii, h k and h ki are non-zero rationals, then DoFH < 3. Proof. Section V-E. Exaples of strict iproveent of DoFH < 3 for certain H can be found in Section III-G. The following exaple, which is proved in Appendix F, illustrates that the condition in Theore 8 is not superfluous: 0 DoF 0 = F. Degrees of freedo region The degrees of freedo region of the interference channel is defined as the high-snr liit of noralized capacity regions: DoF H { r K : li ρ r K, snr CH, snr log snr } = 0, 6 where ρx, E inf y E x y denotes the distance between the point x and the set E belonging to an arbitrary Euclidean space. The degrees of freedo region exists and is characterized by the following result, whose proof is alost identical to that of Theore 4: Theore 9. Suppose that H satisfies the assuption in Theore 4. DoF H is the collection of all r K [0, ] K, such that for any probability vector w K, K K r K, w K sup w i d h i X w i d X K i= 63 where the supreu is over independent X,..., X K such that 33 is satisfied and all inforation diensions appearing in 63 exist. Analogous to Theores 5 and 6, the following result shows that the degrees of freedo region coincides with a polyhedron for alost every channel atrix. See Fig. for an illustration in the three-user case. Theore 0. Let H satisfy the assuption in Theore 5. Then DoF H co {e,..., e K, }, 0, 64 where {e,..., e K } is the standard basis, and co denotes the convex hull. Moreover, 64 holds with equality for Lebesguea.e. H. Proof. Section V-C. d 3 Moreover, h i X, d Fig.. DoF region in the three-user case: Outer bound 64. G. An exaple of lower-triangular channel atrix In this section we consider a prototypical exaple of the following lower-triangular channel atrix [5, Section V], which captures the essential difficulty of the three-user interference channel: H λ 0 0 λ Since the off-diagonals of H λ are integers, equality holds in 3, where the axiization can be further siplified as follows: Theore. DoFH λ = + sup X,X dx + λx dx + X. 66 DoFH λ = DoFH T λ. For all λ 0, DoFH λ = DoFH λ DoFH λ with equality if and only if λ = 0 or ; 4 DoFH λ 3 with equality if and only if λ is irrational. 5 For integers λ =, 3, 4,..., 3 λ i λ log λ λ log λ i DoFH λ i= log λ Therefore as λ on N, DoFH λ = 3 + Θ. 70 log λ For λ =, can be sharpened to.7 + log 6 φ DoFH 0 7 d.43, 7

8 8 where φ = + 5 denotes the golden ratio, while for λ = we obtain.0 + Proof. Section V-F. 5 log 5 3 log log 3 DoFH Reark 5. Note that 66 deals with the axial inforation diension difference between proections of a production easure to the lines of angle π 4 and of an arbitrary angle tan λ. In view of this interpretation and the fact that tan λ = π tan λ, 67 becoes apparent. Since λ is the channel gain of the direct link for the second user, one ight expect at first glance that DoFH λ should be increasing in λ. However, 67 shows that this is not the case. IV. SELF-SIMILAR DISTRIBUTIONS In this section we discuss self-siilar distributions, an iportant subclass of singular distributions, which play the central role of DoF-achieving input distributions in the proofs shown in Section V. As shown in Theore 5, input distributions with inforation diension one half are necessary for achieving the axial degrees of freedo K. Although, in view of 0, equal ixtures of discrete and absolutely continuous coponents also have inforation diension one half, they can achieve at ost one degree of freedo as proved in Section III-C. In order to overcoe this bottleneck, it is necessary to eploy singular input distributions. A. Definitions In general, a self-siilar distribution is defined as the invariant easure of iterated function systes, see, e.g., [9], [30]. For our purpose in this paper we focus on the special case of hoogeneous self-siilar distributions defined as follows: Definition 3. A probability easure µ is called hoogeneous self-siilar with siilarity ratio r 0, if µ is the distribution of X = k W k r k, 73 where {W k } are i.i.d. copies of a real-valued nondeterinistic rando variable W taking a finite nuber of values. The distribution of X is self-siilar in view of the fact that X d = rx + W 74 where W is independent of X. It is easy to see that 73 and 74 are equivalent. When W k is binary, 73 is known as the Bernoulli convolution, an iportant obect in fractal geoetry and ergodic theory [3]. As a special case, 73 encopasses distributions defined by M-ary expansion with independent digits, in which case r = M and W is {0,..., M }-valued. For exaple, the Cantor distribution [3], supported on the iddle-third Cantor set, can be defined by 73 via its ternary expansion, which consists of independent digits taking value 0 or equiprobably. B. Inforation diension It is shown in [3] that the inforation diension or diension of order α of self-siilar easures always exists. More general results on the inforation diension of selfsiilar distributions are suarized in [8, Section.8]. Theore [3, Theore. and Lea.]. Let the distribution of X be hoogeneous self-siilar. Then dx exists and satisfies where H[X] dx = sup. 75 Moreover, d α X exists for all α 0. [x] x. 76 In spite of the general existence result in [3], there is no known forula for the inforation diension of selfsiilar distributions unless certain separation conditions are satisfied, the ost coon of which is the open set condition [6, p. 9]: There exists a nonepty bounded open set U R, such that F U U and F i U F U = for i, where we have denoted F x = rx + w for =,...,. The following lea proved in Appendix B gives a sufficient condition for a hoogeneous self-siilar distribution or rando variables of the for 73 to satisfy the open set condition. Lea 4. Let X be defined in 73 with 0 < r < and W {w,..., w } R. Then the open set condition is satisfied if W r W + MW, 77 where the iniu distance and the diaeter of W are defined by W in w i w. 78 w i w W and respectively. MW ax w i w 79 w i,w W The next result gives an upper bound on the inforation diension of a self-siilar hoogeneous easure µ, which holds with equality if the open set condition is satisfied. Theore 3. For hoogeneous self-siilar easure µ with siilarity ratio r defined in Definition 3, d α µ H αw log, α r Moreover, if µ satisfies the open set condition, then Proof. See Appendix C. d α µ = H αw log, α 0. 8 r The results in [3] are considerably ore general, encopassing selfsiilar distributions defined via nonlinear iterative function systes as well.

9 9 C. Self-siilar distributions with half diension In this subsection, we construct a faily of scalar selfsiilar distributions which play an iportant role in the achievability proof of Theores 7 and. Let N be an integer. Let µ N be the distribution of 73 with r = N, w = N and p = N, =,..., N. Then µ N is hoogeneous self-siilar and satisfies the open set condition, in view of Lea 4. By Theore 3, dµ N = for all N. The DoF-achieving distributions in Theores 7 and are slight variants of µ N. Alternatively, µ N can be defined by the N-ary expansions as follows: Let X be distributed according to µ N. Then the odd digits in the N-ary expansion of X are independent and equiprobable, while the even digits are set to zero. In view of 9, the inforation diension of µ N is the noralized entropy rate of the digits, which is equal to. D. Convolutions of self-siilar easures Forula 3 for DoFH deals with convolutions of input distributions. It is therefore of interest to investigate the behavior of self-siilar easures under convolutions. The next lea shows that convolutions of hoogeneous self-siilar easures with coon siilarity ratio are also self-siilar see also [33, p. 34] and the references therein, but with possible overlaps which ight violate the open set conditions. Therefore 8 does not necessarily apply to convolutions. Lea 5. Let X and X be independent with hoogeneous self-siilar distributions of identical siilarity ratio r. Then for any a, a R, the distribution of ax + a X is also hoogeneous self-siilar with siilarity ratio r. Proof. According to 74, X and X satisfy X d = rx + W and X d = rx + W, respectively, where W and W are both finitely valued and {X, W, X, W } are independent. Therefore ax + a X d = rax + a X + aw + a W, which iplies the desired conclusion. A. Auxiliary results V. PROOFS Before proceeding to the proof of Theore 4, we present the following non-asyptotic bounds on utual inforation, which, in view of the alternative definition of inforation diension in 0, iplies the high-snr scaling law of utual inforation in Theore as an iediate corollary. This arguent is considerably sipler and ore general than the original result in [7]. This result also allows us to conclude that the capacity region of the interference channel for non-gaussian noise can differ only by absolute additive constants fro the Gaussian counterpart if the noise has a wellbehaved density such as unifor or Laplace distribution see Reark 6. Lea 6. Let Bx n, ɛ denote the l -ball of radius ɛ centered at x n. For any X n and any snr > 0, 8 n IXn, snr [ n E ] log P X nbx n, / 8 snr. 83 Proof. Appendix D. To gain soe insight on Lea 6, we note that the bracketed ter in 8 is close to the utual inforation between X and itself containated by uniforly distributed noise. Consider the scalar case and let U be uniforly distributed on [, ]. Then the density of Y = X + ɛu is given by p Y y = ɛ P XBy, ɛ/ and it is easy to verify that IX; Y = E[log P X BY,ɛ/], which is provably close to E[log P X BX,ɛ]. The desired result for Gaussian noise follows fro the fact that the Gaussian density can be approxiated by piecewise constant functions incurring at ost an additive constant difference in the utual inforation. Next we present several auxiliary results regarding the Shannon and Rényi entropy of quantized rando variables. The first result states that if X is close to Y and the alphabet of X is well-separated, then the entropy of X is also close to that of Y. Lea 7. Let X and Y be real-valued discrete rando variables such that X Y ɛ alost surely. Suppose X takes values in a set X with iniu distance X δ. Then ɛ HX HY log δ In particular, if ɛ < δ, then HX HY. Moreover, 84 also holds for Rényi entropy H α for all α 0. Proof. Inequality 84 follows fro HX HY HX, Y HY 85 = HX Y = HX Y Y 86 ɛ log +, 87 δ since conditioned on Y the rando variable X takes at ost + ɛ δ values. However, 86 in the above arguent does not directly generalize to Rényi entropy due to the lack of conditioning property of Rényi entropy. Instead, 84 can be deduced fro the following: First note that by Jensen s inequality, if a distribution Q has support cardinality, then u Qα u α resp. α of α < resp. α >. Therefore, H α X H α X, Y 88 = α log PY α y PX Y α =y x y x 89 H α Y + log + ɛ. 90 δ The following lea adits the sae proof as Lea 7. Lea 8. Let U n and V n be integer-valued rando vectors. If B i U i V i A i alost surely, then n HU n HV n log + A i + B i, 9 i=

10 0 which also holds for H α for all α 0. The next lea shows that the entropy of quantized linear cobinations is close to the entropy of linear cobinations of quantized rando variables, as long as the coefficients are integers. The proof is a siple application of Lea 8. Lea 9. For any a K Z K and any X n,..., XK n, K [ H K ] a [X n ] H a X n K n log + a, 9 where [ ] is defined in 76. Lea 0. For any h K R K and any X n,..., XK n, K K H h X n H h X n + K H X n +, 93 where X n Xn X n is the fractional part of X n. Proof. Define Then V = K K h X n, Z = h X n. 94 H V + Z H V H V + Z V 95 + H V + Z V 96 + H Z 97 K + H X n. 98 We state a Diophantine approxiation result on anifolds due to Kleinbock and Margulis [34, Theore A] which is useful for proving Theores 4 and 6. The version below is stated in ters of the Diophantine approxiation exponent introduced by Mahler [35], which is defined as follows: Definition 4. Let, n N. For x R n, denote by x argin x z 99 z Z n denote the l -distance of x to the nearest integer vector. For A R n, define ŵa to be the supreu of w > 0 such that for all sufficiently large real nuber Q > 0, the set {q Z \{0} : Aq Q w, q Q} 00 is non-epty. For any atrix A R n, the Dirichlet approxiation theore [36, Theore VI, p. 3] states that ŵa n, 0 which holds with equality for Lebesgue-a.e. A R n due to the Borel-Cantelli lea [36, Chapter VII]. The Kleinbock- Margulis theore generalizes this result to anifolds. Lea [37, p. 83]. Let M be a non-degenerate anifold of R k. Then wa = k and wat = k for alost every a M. Lea deals with hoogeneous diophantine approxiation approxiating zero by linear fors with integer coefficients. In the course of proving Theore 4, we will also need results on inhoogenous diophantine approxiation approxiating an arbitrary point by linear fors, which are connected to the hoogeneous case via the so-called transference theores see, e.g., [38], [39]. The following transference result is fro [39, Lea 3, p. 750], which is a direct consequence of [38, Theore XVII.B, p. 99] see also [36, Theore VI, p. 38] for a ore general version. Note that this result holds for any fixed atrix G. Lea. Let G R n. Let κ = n + n!. Let X, Y > 0. Suppose the set {y Z n \{0} : G T y < κx, y Y } 0 is epty. Then for all z R n, the set {x Z : Gx + z κy, x X} 03 is non-epty. In the sequel we adopt the following conventional notation of susets and dilations: A + B = {a + b : a A, b B}, 04 λa = {λa : a A}. 05 We need the following lea for iniu distances: Lea 3. Let W {w,..., w } R and let r > 0 satisfy 77. Then for any n N, W + rw + + r n W r n W. 06 Moreover, W +rw + +r n W and W n are in one-to-one correspondence. Proof. Let a n, b n be distinct eleents of W n. Then k in{i : a i b i, i n} is well-defined. Note that n i= a i b i r i a k b k r k n i=k+ ri a i b i. Consequently, W + rw + + r n W 07 { in 0 k n r k W MW n i=k+ r i } = r n W. To see that the above iniu is attained at k = n, note that the assuption that r +M iplies that M r r, which is equivalent to r k M n i=k+ ri r n for all 0 k n.

11 B. General forula In this section we prove Theore 4. As a side product of the proof, we also establish that the capacity regions of the Gaussian interference channel and a particular deterinistic interference channel differ by at ost universal constants See Reark 6. Step. We start by proving the achievability side of the theore, which holds for any channel atrix H. Choosing i.i.d. input distribution, we have CH, snr = K IX i ; Y i 08 i= K K I h i X, snr I h i X, snr. i= In view of Theore, dividing both sides by log snr, sending snr and optiizing over independent X,..., X K subect to 33 yield 3. Step. The rest of the subsection is devoted to proving the converse side of the theore except possibly for H whose off-diagonals are non-algebraic and belong to a set of zero Lebesgue easure, regardless of the diagonal entries. To this end, first we state a nuber-theoretic condition in ters of the Diophantine approxiation exponents, which are satisfied in either of the two cases. Let l KK. Given a atrix H R K K, denote by h the l-diensional vector consisting of the off-diagonals of H. Let b N and define l + b L b 09 b Let P l,b = {f,..., f Lb } 0 denote the collection of all l-variant onoials of degree at ost b, where f. Define the colun vector p b H = f h,..., f Lb h T R L b. Recalling the Diophantine approxiation exponent ŵ in Definition 4, we define w b H ŵp b H, w bh ŵp b H T. We will assue the following condition on the cross-channel gains for the converse proof: w b Hw b+h b. This assuption warrants soe explanation. First of all, as a consequence of 0, for any atrix H, we have w b H L b and w b+ H L b+. Note that L b+ = l+b+ b+ L b and l = KK. Then li inf w bhw L b+ b+h li =. 3 b b L b Therefore the assuption aounts to requiring that the li sup, hence the liit, are also equal to one. As entioned earlier, sufficient conditions for include the following: Algebraic case: For i, let h i be algebraic of degree r i. Consider the finite set { } S = h ki i : k i {0,..., r i }, i. 4 i Then there exists S S such that any point in S can be written as a linear cobination of eleents of S with rational coefficients. Let r denote the rational diension of S, i.e., the sallest cardinality of such S. Then r i r i. Therefore, for all degree b ax r i, p b H has rational diension r. As a consequence of the Schidt subspace theore, we have [40, p. 66]: w b H = in{l b, r}, 5 w b+h = in{l b+, r}. 6 Hence w b Hw b+ H = for all sufficiently large b. Alost sure case: Since {p b G : G R K K } is a nondegenerate anifold of R Lb see, e.g., [37, p. 8], Lea iplies that w b H = L b, 7 w b+h = L b+. 8 hold for a.e. H R K K, which satisfy. As entioned in Section III-A, in the three-user case the channel atrix can be reduced to the standard for 34, where the off-diagonals are either one or h. Then the polynoials of the off-diagonals are in fact univariate polynoials of h. Consequently, we can abbreviate w b H as w b h = wh,..., h b T, which is the exponent for siultaneous approxiation of powers of a real nuber, and w b H as w b h = wh,..., hb, which is the exponent for polynoial approxiation. These are the central obects studied in [39], [40], [4]. In particular, by 0 and [4, Theores 0 and 7], we have b w b h b and b w b h b. Therefore for any h, li inf b w bhw b+h li sup w b hw b+h 4, b and the condition requires that the liit is equal to one. Step 3. We proceed to show that 3 holds with equality by assuing the condition. By the onotonicity of snr CH, snr, we ay restrict the liit in 3 to any increasing unbounded subsequence {snr } as long as the growth is no faster than exponential, i.e., snr + sup <. 9 snr In particular, we choose snr = 4. Consequently, we can restrict to this subsequence in the liiting characterization in 39. Define [ ] gx n, ɛ E log P X nbx n 0, ɛ = E [ log P { X n B X n, ɛ X n} ],

12 where X n is an independent copy of X n. By Lea 6, for any X n and any snr > 0, 8n IX n, snr g X n, snr n. 3 By [0, 94 and 96] or [3, Lea.3], for any N, 0 H[X n ] gx n, 4 3n. 5 Plugging 5 into 39, we have the following equivalent liiting characterization of DoFH: DoFH = li sup li sup 6 n X n n,...,xn K [ ] [ ] K H h i X n H h i X n, i= where X n,..., XK n are independent, such that H X ik C for soe C 0 and for all i =,..., K, k =,..., n. In view of Lea 0, the right side of 6 is unchanged if we replace all inputs by their fractional parts. Therefore, it suffices to consider [0, ]-valued inputs in the axiization of 6. Moreover, by the scale-invariance of the DoF, we can assue, without loss of generality, that H, i.e., h i for all i,. For siplicity, in the reainder of the proof we abbreviate L b and L b+ by L and L, respectively, and suppress the dependence of w b H and w b+ H on H. By Definition 4, for any δ > 0, there exists Q 0 = Q 0 δ > 0 such that for all Q Q 0, {q Z\{0} : p b Hq Q +δw b, q Q} = 7 and { q Z L \{0} :p b+ H T q Q +δw b+, } q Q =. 8 Next we assue that H. Any finite set A induces a quantizer Q A : R A defined by Q A x = argin x a, 9 a A with ties broken arbitrarily. For an arbitrary positive integer N, define the following sets subsets of a lattice { V L } q i f i h: q i Z, q LN, ax N q i N, i L i= 30 V N { L q i f i h: q i Z, q i LN, i =,..., L }. i= 3 As a consequence of 7, we can upper bound the approxiation error of the quantizer Q V as follows. Recall the constant κ defined in Lea, which, in the special case of + n = L, is given by κ = κb = L L!. 3 Let N 0 = N 0 b, δ = κq w b+δ 0 and Q = N κ +δw b. Let N N 0 and, equivalently, Q Q 0. Hence for all integer q Q, we have p b Hq > Q +δw b. Applying Lea with X = N, Y = Q, = L, n = yields the following: For all θ [0, ], there exists x Z L, such that x N and p b H T x + Nθ κy +δw κκ/n b, 33 By the assuption that H, we have f i h for all i, hence p b H T x + Nθ LN. Recall that x = in a Z x a. Therefore 33 iplies that there exists q R L, such that q LN, q i N, i L, such that N L i= q if i h + θ κ/n + +δw b. In other words, for all N N 0, we have sup Q V θ θ κ/n + +δw b. 34 θ [0,] Using 8, we can lower bound the iniu distance of the set V as follows. For all Q Q 0, 8 iplies that for any q Z L such that q i Q, i L, we have L i= q if i h Q +δw b+. Since V V = L q N i f i h : q i Z, q i LN, 35 i= then as long as N Q0 L, we have V N LN +δw b+. 36 Therefore we conclude that 34 and 36 hold siultaneously for all N ax{n 0, Q0 L }. By the ulti-letter characterization 6, for any ɛ > 0 and sufficiently large, there exist n and independent X n,..., XK n with Xn = X t n t= [0, ] n, such that [ ] DoFH ɛ + K H h i X n n i= [ ] H h i X n. 37 The goal is to construct single-letter input distributions { ˆX,..., ˆX K } based on the ulti-letter inputs X n,..., XK n, such that dof ˆX K, H doinates the right side of 37. To this end, fix N to be chosen depending on and b later. Set X t = Q V X t V for all t n. In view of 34, we have X n Xn κ/n + +δw b for all. Next we show that H h X i n and H[ h ix n] are close. First note that [ ] h i Xn h i X n +Kκ/N + +δw b. 38 Moreover, for all t, h X i t takes values in h iv V. We conclude fro 36 that the iniu distance satisfy V N LN +δw b+. 39 h iv

13 3 Set +δw b+ N = L 4K Applying Lea 7 yields [ ] n H h i Xn n H h i X n log + + Kκ/N + +δw b log N LN +δw b+ 3 + KN κ/n+ +δw b LN +δw b+ Cb, δ + + δw b+ + δw b 4 4 log N, 43 where 4 follows fro 40 since N LN +δw b+, and in 43 we have defined Cb, δ log 3 + log6k Lκ + +δw b L +δw b+ which only depends on b and δ, since κ and L are functions of b only. Analogously, a siilar application of Lea 7 yields [ ] n H h i X n n H h i Xn log + + Kκ/N + +δw b 44 Cb, δ + + δw b+ + δw b log N. 45 Now we construct scalar input distributions which are selfsiilar see Section IV and DoF-achieving. First put where W = n k= X k r k, 46 r =. 47 Let {W t : t 0} be a sequence of i.i.d. copies of W. Set ˆX = t 0 W t r nt. 48 Note that each ˆX has a hoogeneous self-siilar distribution with siilarity ratio r n. In view of Theore, d ˆX exists for all. Moreover, by 48, K K h i W t r nt 49 h i ˆX = t 0 is also hoogeneously self-siilar with siilarity ratio r n, in view of Lea 5. Note that M h iv KMV KM[ L, L] = 4KL. Cobining 39, 40 and 47, we have r h iv M h iv. 50 Set W n k= rk V. Since X t V, we have h i W h i W = n k= r k In view of 50, applying Lea 3 yields r n h iw h i V h iv On the other hand, the diaeter satisfies M h iw KMW K n k= rk MV 8KL. Therefore, in view of 50 and 5, our choice of r in 47 satisfies r n h iw M h iw. 53 Invoking the sufficient conditions in Lea 4, we conclude that h ˆX i satisfies the open set condition. Hence, by Theore 3, the inforation diension is given by d H h i ˆX h iw H h iw = log = n r n H h X i n =, 54 n where 54 follows fro the fact that n k= rk h iv and h iv n are in one-to-one correspondence with each other, in view of 46 and Lea 3. Entirely analogously, we have d H h i ˆX h X i n =. 55 n Assebling 40, 45 and 54, we obtain d [ ] h i ˆX n H h i X n Cb, δ + δw b+ log N + δw b, 56 Cobining 40, 43 and 55 yields d [ ] h i ˆX n H h i X n Cb, δ δw b+ log N + δw b. 57 Recall the notation dof defined in 3. Plugging into 37 and suing over i, we obtain DoFH dof ˆX K KCb, δ, H + ɛ δw b+ K log N + δw b.

14 4 Note that according to 40, +δw b+ log N ++δw b+. Sending, then b, and δ 0, and finally ɛ 0, we have DoFH sup dofx K, H + li X K = sup X K dofx K, H + K = sup X K dofx K, H, δ 0 li K b + δ w b w b+ li b w b w b+ where the last step follows fro our assuption. This copletes the proof of the converse. Reark 6. Deterinistic channel approxiation has been eployed to deterine the capacity region of various network counication probles within constant gaps, see, e.g., [4], [43]. As a side product of the proof of Theore 4, we establish a connection between the Gaussian interference channel Y K = snrhx K + N K 59 and the following deterinistic Gaussian interference channel, Ỹ K = [HX K ] 60 where [ ], defined in 76, is taken coponentwise, and E [ ] Xi. It can be shown that for log snr, the capacity regions of and 60 differ by at ost universal constants. To see this, denote the capacity region of 60 by C H. Assebling 5 as well as the ulti-letter characterization 36 and 38, we conclude that there exist a universal constant c which is at ost bits, such that d H CH, snr, C H c 6 holds uniforly for any snr > 0 and any H, where and snr are related through = log snr, and { } d H A, B ax sup inf a b a A, sup inf a b. b B b B a A Moreover, 6 also holds for quantized interference channel with 60 replaced by Y K = [[H] X K ] or Y K = [[H] [X K ] ]. Note that a related one-sided bound was given in [5, Lea 4]. Furtherore, we note that for non-gaussian additive noise with well-behaved densities such as unifor or Laplace distribution, the capacity region is only a constant gap away fro the Gaussian one. This follows fro the ulti-letter characterization and the unifor bound of utual inforation in Lea 6 see also Reark 0 in Appendix D. Reark 7 Rational channel atrices. For channel atrices with rational entries, the general forula can be proved without appealing to diophantine approxiation arguents. Essentially, we can siply construct ˆX K based on a quantized version of X K. An eleentary proof is as follows: Since DoF is scale-invariant, we assue that h i Z without loss of generality. The gist of constructing ˆX is by concatenating the bits fro X n. Let L be an integer such that L > + ax i K log + K h i and let M = n + L. Define ˆX = l 0 Z l Ml, 6 where {Z l } l are independent copies of n Z = L [X k ] +Lk. 63 k= Note that Z is a discrete rando variable whose binary expansion has M bits, fored by concatenating the first bits of each X k preceded by L zeros. Therefore, Z takes values in M {0,..., M L }. Note that each ˆX has a hoogeneous self-siilar distribution with siilarity ratio M. In view of Theore, d ˆX exists for all. Moreover, by 6, K K h i Z l Ml 64 h i ˆX = l 0 is also hoogeneous self-siilar with siilarity ratio M, in view of Lea 5. It is easy to verify that the distribution of 64 also satisfies the open set condition: Let W = M K h i {0,..., M L } 65 denote the alphabet of K h iz. Since h i Z, the iniu distance of W is given by W = M, while the diaeter is upper bounded by MW L M K h i. By Lea 4, 64 satisfies the open set condition. Applying Theore 3, we have K M d h i ˆX K = H h i Z = H n k= K L h i [X ik ] +Lk K K = H h i [X i ],..., h i [X in ] K = H h i [Xi n ] where 67 is due to 63 and 68 follows fro the fact that each + L-bit block in the suation of 67 does not overlap. Since h i Z, by Lea 9, K [ H K ] h i [Xi n ] H h i Xi n K n log + h i 70 nl. 7