Zero error list-decoding capacity of the q/(q 1) channel

Transcription

1 Zero error list-decoding capacity of the q/q 1) channel Sourav Chakraborty 1, Jaikuar Radhakrishnan 2,3, Nandakuar Raghunathan 4, and Prashant Sasatte 5 1 Departent of Coputer Science, University of Chicago, Chicago, USA 2 Toyota Technological Institute at Chicago, USA 3 School of Technology and Coputer Science, Tata Institute of Fundaental Research, Mubai, India 4 Microsoft, Seattle, USA 5 University of Waterloo, Waterloo, Canada Abstract. Let, q, l be positive integers such that l q. A faily H of functions fro [] to [q] is said to be an, q, l)-faily if for every subset S of [] with l eleents, there is an h H such that hs) = [q]. Let, N, q, l) be the size of the sallest, q, l)-faily. We show that for all q, l 1.58q and all sufficiently large, we have N, q, l) = expωq)) log. Special cases of this follow fro results shown earlier in the context of perfect hashing: a theore of Fredan & Kolós 1984) iplies that N, q, q) = expωq)) log, and a theore of Körner 1986) shows that N, q, q + 1) = expωq)) log. We conjecture that N, q, l) = expωq)) log if l = Oq). A standard probabilistic construction shows that for all q, l q and all sufficiently large, N, q, l) = expoq)) log. Our otivation for studying this proble arises fro its close connection to a proble in coding theory, naely, the proble of deterining the zero error listdecoding capacity for a certain channel studied by Elias [IEEE Transactions on Inforation Theory, Vol. 34, No. 5, , 1988]. Our result iplies that for the so called q/q 1) channel, the capacity is exponentially sall in q, even if the list size is allowed to be as big as 1.58q. The earlier results of Fredan & Kolós and Körner cited above iply that the capacity is exponentially sall if the list size is at ost q Introduction Shannon [S56] studied the zero error capacity of discrete finite eoryless noisy channels. Such a channel can be odeled as a bipartite graph V, W, E), Work done while the author was at the University of Chicago. Work done while the author was at TIFR, Mubai.

2 2 Chakraborty, Radhakrishnan, Raghunathan, Sasatte where v, w) E iff the letter w can be received when the letter v is transitted. The goal then, is to encode essages as strings of letters fro the input alphabet V and recover it fro the received essage. The goal naturally is to use as few input letters as possible and still recover the intended essage perfectly. Shannon [S56] and Lovász [L79] deterined the best rate of transission achievable under this odel for several specific channels. We are interested in the list-decoding version of this proble, studied by Elias [E88]. For exaple, consider the channel shown in Figure 1. It is not hard to see that for this channel, no atter how any letters are used in the encoding, it is ipossible to recover an input essage uniquely assuing there are at least two possibilities for the input essage). However, it is not hard to see that one can always encode essages using strings of letters such that based on the received essage one can narrow down the possibilities to just two, that is, we cannot decode exactly but we can list-decode with a list of size two. This otivates the following definition. Fig. 1. The 3/2 channel Definition 1 Code, Rate). Consider a channel C = [q], [q ], E) with q input letters and q output letters. We say that a sequence σ [q] n is copatible with σ [q ] n if for i = 1, 2,..., n, we have σ[i], σ [i]) E. A subset S [q] n is said to be a zero error l-list-decoding code for the channel C if for all σ in [q ] n, {σ S : σ and σ are copatible} l. Let n, C, l) be the iniu n such that there is a zero error l-list-decoding code S for the channel C, such that S. The zero error list-of-l rate of the code S is R C,l S) = 1 ) n log, l and the zero error capacity of the channel C is the least upper bound of the attainable zero error list-of-l rates of all codes.

3 Zero error list-decoding capacity of the q/q 1) channel 3 For the 3/2 channel Elias [E88] proved that the zero error capacity when l = 2 is lower bounded by log3) and upper bounded see 2) below) by log3) In this paper, we study generalization s of the 3/2 channel. The q/q 1) channel corresponds to the coplete bipartite graph K q,q inus a perfect atching. Thus, the transission of any letter can result in all but one of the letters being received. It is easy to see that that it is not possible to design a code where one can always recover the original essage exactly. However, it is possible to design codes that perfor list-decoding with lists of size q 1. In fact a routine probabilistic arguent shows the following. Proposition 1. n, q, q 1) = expoq)) log. The q/q 1) channel is thus a natural and siple channel where exact decoding is not possible, but list-decoding with oderate size lists is possible. The ain point of interest for us is that the rate of the code proised by Proposition 1 is exponentially sall as a function of q. Is this exponentially sall rate the best we can hope for if the list size is restricted to be q 1? Yes, and this follows fro a lower bound on the size of failies of perfect hash functions shown by Fredan and Kolós [FK84]. A generalization of the result of Fredan and Kolós obtained by Körner [K86], iplies that the rate is exponentially sall even if we allow the decoder to produce lists of size q. For what list size, then, can we expect list-decoding codes with constant or inverse polynoial rate? Proposition 2. For all q we have, n, q, q ln q ) = Oq log ). On the other hand, it can be shown that the rate cannot be better than 1 q unless the list size is allowed to depend on. Proposition 3. All functions l : Z + Z + and all q, for all large enough, n, q, lq)) q log. Thus, we know that the rate is exponentially sall when the list size is required to be exactly q, and it is an inverse polynoial when the list size is θq ln q). These observations, however, do not copletely deterine the dependence of the rate on the list size, or even the sallest list size as a function of q) for which there are codes with rate significantly better than an inverse exponential. We conjecture the following. Conjecture 1 The conjecture has two parts. 1. For all constants c > 0, there is a constant ɛ, such that for all large, we have n, q, cq) expɛq) log

4 4 Chakraborty, Radhakrishnan, Raghunathan, Sasatte 2. For all function lq) = oq log q) and for all large we have n, q, lq)) q ω1) log In this paper, we ake progress towards the first part of the conjecture. Theore 1 Main result). For ɛ > 0, there is a delta > 0 such that for all large q and for all large enough, we have n, q, γ ɛ)q)) expδq) log, where γ = e/e 1) Techniques As stated above the inverse exponential upper bounds on the rate when the list size is q 1 or q follow fro results proved earlier in connection with hashing. In this section, we forally state this connection, review the previous techniques, and then outline the arguent we use to obtain our result. 2.1 Connection to hashing Definition 2. Let q, l, be integers such that 1 q l. A faily H of functions fro [] to [q] is said to be an, q, l)-faily of hash functions if for all l-sized subsets S of [], there is a function h H such that hs) = [q]. N, q, l) is the size of the sallest, q, l)-faily of hash functions. For convenience, we will allow, q and l to positive real nubers, and use N, q, l) to ean the size of the sallest, q, l )-faily where, q and l are integers such that, q q and l l. The connection between the faily of hash functions and zero error listdecoding codes for the q/q 1) channel is straightforward. Suppose we have an, q, l)-code C [q] n. Such a code naturally gives rise to n functions h 1, h 2,..., h n fro [] to [q]: where h i j) = k iff the i-th letter of the j- th codeword is k [q]. It is then straightforward to verify that for every set S [] of size l + 1, we have h i S) = [q] for soe i {1, 2,..., n}. This translations works in the other direction as well: if there is an, q, l)-faily of hash functions of size n, then there is an, q, l 1)-code C [q] n. Proposition 4. For all, q, and l, we have n, q, l) = N, q, l + 1). In light of the above, we will concentrate on showing lower bounds for N, q, l). Our ain result can then be reforulated as follows. Theore 2. For all ɛ > 0, there is a δ > 0, such that for all large q, l e e 1 ɛ ) q, and all large enough, N, q, l) expδq) log.

5 Zero error list-decoding capacity of the q/q 1) channel 5 It is easy to see that Theore 1 follows iediately fro this. In Section 4.2 we will forally prove this theore. We now present an overview. 2.2 The lower bound arguent It will be helpful to review the proof of the lower bound shown by Fredan and Kolós [FK84]. Theore 3. For all large q and all large enough, N, q, q) Oq 1/2 expq)) log. First, we note two siple lower bounds on N, q, q). First, any one hash function can perfectly hash at ost q ) q sets of size q. So, N, q, l) q ) q ) q 1 2πq expq). 1) This bound has the required exponential dependence on q but not the logarithic dependence on. A different arguent gives us a logarithic dependence on. If we restrict attention to all eleents of [] that are apped by the first hash function to soe q 1 of the [q] eleents of [], then clearly every q-sized subset of these eleents ust be perfectly hashed by at least one of the reaining ) hash functions. Fro this, we conclude that N, q, q) N q 1 q ), q, q provided q), which iplies N, q, q) log q q 1 ) ) q log. 2) q 1 q 1 [A siilar calculation can be used to justify Proposition 3.] This bound, gives us the required logarithic dependence on but not the exponential dependence on q. Fredan and Kolós devised an ingenious arguent that cobined the erits of 1) and 2). Consider a set T of size q 2. Clearly, if a function aps two of the eleents of T to the sae value in [q], then this hash function is incapable of perfectly hashing any q-eleent superset T T. An averaging arguent shows that for if T is chosen uniforly at rando then all but an exponentially sall fraction of the original faily do ap soe two eleents of T to the sae eleent. Furtherore, for every two eleents of [] T one of the reaining hash functions that are on-to-one on T ) ust ap these two eleents differently. By 2), the nuber of hash functions reaining ust be at least log q + 2). Thus, the size of original faily ust be at expωq)) log q + 2). The arguents used by Fredan and Kolós and Körner are ore sophisticated and yield slightly better bounds.)

6 6 Chakraborty, Radhakrishnan, Raghunathan, Sasatte Our arguent is siilar. Suppose we have an, q, l) faily of hash functions where l = e ɛ for soe ɛ > 0). As in the arguent above, we pick a set T of size q 2. The ain observation now is that for any fixed function h : [] [q], the expected size ht ) as T is chosen at rando) is about q 1 1 e), and there is a sharp concentration of easure near this ean value. Thus, only for an exponentially sall fraction of the hash functions in the faily is the iage of T at least q 1 1 e + ɛ). The ajority of the functions already suffer so any collisions on T, that they cannot cover all of [q] when an additional 1 e ɛ) q eleents are added to T. Using an arguent siilar to the one used to show 2), we conclude that an exponentially sall fraction of the original faily ust be at least log q + 2). The lower bound will follow fro this. This arguent is presented in detail in Section 4.1. It however is soewhat weaker than the bound claied in Theore 2. The stronger result is obtained by applying this idea recursively. The foral proof proceeds by induction, and is presented in Section Preliinaries In this section we develop the tools that will be necessary in the proof of Theore 2 in Section 4.1 and 4.2. Definition 3 Derived function). Let, q, q be integers such that q > q 1 and let T [] Let h : [] [q] be a hash function such that ht ) q q. Then, the function h T,q is defined as follows. Let j 1, j 2,..., j q be the sallest q eleents of [q] ht ). Then, for all i [], let h T,q i) = { k if hi) = jk 1 otherwise. The following proposition follows iediately fro our definition. Proposition 5. Let h : [] [q]. If T, T [] are such that ht ) q q and ht T ) = [q], then h T,q T ) = [q ]. Lea 1. If H is a faily of hash functions fro [] to [2]. Then, there is a subset U [] of size at least /2 H such that hu) = 1, for all h H. Proof. Consider the ap fro [] to {1, 2} H defined by i hi) : h H. The range of this ap has size exactly 2 H. It follows that there are at least /2 H eleents of the doain [] that ap to the sae eleent. Definition 4 Dangerous function). We say that the function h : [] [q] is ɛ-dangerous for the set T [] if ht ) q 1 1 e + ɛ).

7 Zero error list-decoding capacity of the q/q 1) channel 7 Lea 2. Let h : [] [q]. Let T be a rando subset of [] chosen uniforly fro aong all subsets of [] of size q 2. Then, if q, Pr T [h is ɛ-dangerous for T ] 2 exp 2ɛ2 q). To prove Lea 2, we will need the following concentration result due to McDiarid. Lea 3 see McDiarid [M89]). Let X 1, X 2,..., X n be independent rando variables with each X k taking values in a finite set A and let f : A n R. For all k, let f change by at ost c k if only the value of X k is changed, that is, ax x A k fx) fy) c k, when x and y differ only in the kth coordinate. If Y = fx 1, X 2,..., X n ) is the rando variable with expectation E[Y ], then for any t 0, ) 2t 2 Pr[Y E[Y ] t] exp n. i=1 c2 k Proof of Lea 2). Pick q 2 eleents fro [] with) replaceent, let the resulting set be T. With probability ore than 1 q 2 2 ) we have that T = q 2. Now, fix an h H. For j [q], the probability that j ht ) is exactly, ) 1 h 1 j) q 2. Thus, by linearity of expectation, we have E[ [q] ht ) ] = q q j=1 ) q 2 1 h 1 j) 1 1 q = q 1 1 q = q q exp q e. j=1 ) q q 1 q 2 q 1 q h 1 j) ) q 2) The second inequality follows fro Holder s Inequality. Thus, E[ ht ) ] q1 1 e ). We think of ht ) as a function of fx 1, X 2,..., X q ), of q 2 ) q 2

8 8 Chakraborty, Radhakrishnan, Raghunathan, Sasatte independent rando variables X 1, X 2,..., X q 2 each distributed uniforly over the set []). Note that f changes at ost by 1 if the value of any of the the variables is changed leaving the rest unchanged). We ay thus conclude fro Lea 3 that [ Pr ht ) q 1 1e )] + ɛ exp 2 ɛ2 q 2 ) exp 2ɛ 2 q ). q 2 This iplies that if T is chosen to be a rando set of size q 2, then the probability that h is dangerous for T is at ost 1 q 2 2 )) 1 exp 2ɛ 2 q ) 2 exp 2ɛ 2 q ). Corollary 1. Let H be a faily of hash functions fro [] to [q]. Then, there is a set T [] of size q 2 such that at ost 2 exp 2ɛ 2 q) H hash functions in H are ɛ-dangerous for T. Proof. Pick T at rando. By Lea 2, the expected nuber of ɛ-dangerous hash functions for T is at ost 2 exp 2ɛ 2 q) H. There ust be a at least one choice for T with this property. 4 Proof of Theore A weaker bound Our goal in this section is to show the following weaker for of the ain theore, which will serve as the basis for the inductive arguent, when we present the proof of the ain result. Theore 4. For ɛ > 0, large q and all large enough, we have N, q, γ ɛ)q) expδq) log, where γ = e Proof. Let H be an, q, l)-faily with l γ ɛ)q. By Corollary 1, we have a set T of size q 2 such that the nuber of functions that are ɛ-dangerous for T is at ost 2 exp 2ɛ 2 q) H. Fix such a T and consider the derived faily H = {h T,2 : h H is ɛ-dangerous for T }. By Lea 1, there is a set U [] of size /2 H such that h U) = 1 for all h H. We clai that U < q 1 e ɛ). For, otherwise let T be a subset

9 Zero error list-decoding capacity of the q/q 1) channel 9 of U of size q = q 1 e ɛ), and consider the set T T. If h H is not dangerous then ht ) < q q and, therefore, ht T ) < q. On the other hand if h is ɛ-dangerous for T, then our definition of U ensures that ht ) = 1 and, therefore, ht T ) T + 1 < q. Thus, 2 H < q 1 e ɛ ). This, together with H 2 exp 2ɛ 2 q) H iplies our clai. 4.2 The general bound In this section, we will prove the Theore 2. It will be convenient to restate it in a for suitable for an inductive proof. For k 1 and ɛ > 0, let l k q, ɛ) = q 1 + 1e + 1e e ) k ɛ 2k. Theore 5 Version of ain theore). For all k 1, ɛ > 0, q 2k and all large enough, N, q, l k q, ɛ)) 1 2ɛ 2 ) 4k exp q e 2k log. Proof. We will use induction on k. The base case k = 1, follows fro the Theore 4 proved in the previous section. Induction step: Suppose the clai is false, that is, there is an k, q k, l k )-faily H k such that l k l k q k, ɛ) and H k < 1 2ɛ 2 ) 4k exp q k e 2k log. 3) [We use k in the subscript for paraeters associated with the H k to ake ephasize the correspondence with the paraeter k used in the induction.] Fro H k we will derive an k 1, q k 1, l k 1 )-faily H k 1 such that H k 1 H k ; 4) k k k ; 5) 1 q k 1 q k e ɛ ) q k 4 ɛ 2 ; 6) l k 1 l k 1 q k 1, ɛ). 7)

10 10 Chakraborty, Radhakrishnan, Raghunathan, Sasatte Then, using the induction hypothesis, we obtain 1 2ɛ 2 H k H k 1 4k 1) exp q k 1 e 2k 1) 1 2ɛ 2 ) 4k 1) exp q k e 2k 1 1 k = 1 2ɛ 2 ) 4k exp q k e 2k log k, ) log k 1 ) log k ; by the induction hypothesis) contradicting 3). It reains to describe how H k 1 is obtained fro H k. The idea, as outlined in the introduction, is this. In the hope of using induction, we will first pick a subset T of size q k 2, which ost hash functions ap into a sall nuber of eleents. These functions can now be viewed as apping [ k ] T to [q k ], so that the proble reduces to one of covering a large subset of [q k ] with l k q k + 2. However, not all functions are guaranteed to be so well-behaved. For the few functions that do perfor well on T, we need to take evasive action, by restricting attention to a subset of the universe on which these functions are guaranteed to be fail. This idea is ipleented as follows. Using Corollary 1, we first obtain a set T [ k ] of size q k 2 such that at ost 2 exp ) ɛ 2 2 qk H k 2 exp 4) ) ɛ 2 2 qk 4) 1 4k exp 2ɛ 2 ) q k e 2k log 1 2k log k hash functions in H k are ɛ 4) -dangerous for T. Now consider the faily of derived functions H = {h T,2 : h H k is ɛ 4) -dangerous for T }. Using Lea 1, we obtain a set U [ k ] of size at least k q k + 2) 1 2k k such that hu) = 1 for all h H. Our faily H k 1 will be the following set of hash functions fro U to [q k 1 ] where q k 1 = q k 1 e ɛ 4 ) ). H k 1 = {h T,qk 1 : h H k is not ɛ 4) -dangerous for T }. We clai that for all T U of size l k q k 2), there is a function h H k 1 such that ht ) = [q k 1 ]. For, consider the set T T of size l k. By the definition of H k there is an h H k such that ht T ) = [q k ]. Such an h is not ɛ 4) - dangerous for T because our definition of U ensures that ht T ) < q k. So for such an h we have ht ) < q k 1 1 e + ɛ ) q k q k 1e 4 ɛ4 ) = q k q k 1.

11 Zero error list-decoding capacity of the q/q 1) channel 11 Hence, for such an h, by Proposition 5, we have h T,qk 1 T ) = [q k 1 ]. Thus, H k 1 is an k 1, q k 1, l k 1 )-faily for l k 1 = l k q k 2). In particular l k 1 q k 1. We need to verify 4) 7). The definition of H k 1 iediately iplies 4). To verify 5) note that for k q k, U k q k + 2) 1 2k k 1 1 k k. Since q k 1 = q k 1 e ɛ 4 ), 6) holds. Finally, to justify 7), note that 1 l k 1 q k 1, ɛ) q k e ɛ ) e ) e k 1 ɛ 2k 1) [ 1 q k e + 1 e e k ɛ e ɛ e )] e k 1 2k 1) 1 q k e + 1 e ) e k ɛ 2k 1) q k e ) e k 1 ɛ 2k q k + 2 l k q k, ɛ) q k 2) l k q k 2) = l k 1. Acknowledgeent We thank Venkat Guruswai for introducing us to this proble. References E88. P. Elias. Zero Error Capacity Under List Decoding, IEEE Transactions on Inforation Theory, vol. 34, No. 5, 1988): FK84. M. Fredan, J. Kolós. On the Size of Separating Systes and Failies of Perfect Hash Functions, SIAM J. Alg. and Disc. Meth., Vol. 5, No ): K86. J. Körner. Fredan-Kolós bounds and inforation theory, SIAM J. Algebraic and Discrete Methods, ): L79. L. Lovász. On the Shannon Capacity of a Graph, IEEE Trans. Infor. Theory, Vol. IT ): 1-7. M89. C. McDiarid. On the ethod of bounded differences, Surveys in Cobinatorics, vol 141 LMS Lecture Notes Series 1989): S56. C.E. Shannon. The zero error capacity of a noisy channel, IEEE Trans. Infor. Theory, Vol. IT-2, no. 3, 1956): Reprinted in D. Slepian, Ed., Key Papers in the Developent of Inforation Theory. New York: IEEE Press 1974): )