Signal Reconstruction from Noisy Random Projections

Transcription

1 Sigal Recostructio from Noisy Radom Projectios Jarvis Haut ad Robert Nowak Deartmet of Electrical ad Comuter Egieerig Uiversity of Wiscosi-Madiso March, 005; Revised February, 006 Abstract Recet results show that a relatively small umber of radom rojectios of a sigal ca cotai most of its saliet iformatio. It follows that if a sigal is comressible i some orthoormal basis, the a very accurate recostructio ca be obtaied from radom rojectios. We exted this tye of result to show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. We also roose a ractical iterative algorithm for sigal recostructio, ad briefly discuss otetial alicatios to codig, A/D coversio, ad remote wireless sesig. Idex Terms samlig, sigal recostructio, radom rojectios, deoisig, wireless sesor etworks I. INTRODUCTION Recet theory iforms us that, with high robability, a relatively small umber of radom rojectios of a sigal ca cotai most of its relevat iformatio. For examle, the groudbreakig work i [ has show that k radom Fourier rojectios cotai eough iformatio to recostruct iecewise smooth sigals at a distortio level early equivalet to that attaiable from k otimally selected observatios. Similar results hold for radom Gaussia ad Rademacher rojectios i.e., rojectios cosistig of ideedet ad idetically distributed Gaussia or Rademacher radom variables) [, [3. The results reseted i these works ca be roughly summarized as follows. Assume that a sigal f R is comressible i some orthoormal basis i the followig sese. Let f m) deote the best m-term This research was suorted i art by the NSF uder grats CCR ad CCR-03557, ad by the Office of Naval Research, grat N

2 aroximatio of f i terms of this basis ad suose that the average squared error obeys f f m) = f i f m) i ) C A m α for some α 0 ad some costat C A i= > 0. The arameter α govers the degree to which f is comressible with resect to the basis. I a oiseless settig, it ca be show that a aroximatio of such a sigal ca be recovered from k radom rojectios with a average squared error that is uer bouded by a costat times k/ log ) α, early as good as the best k-term aroximatio error. This aer takes this lie of iquiry a ste further by cosiderig the erformace of samlig via radom rojectios i oisy coditios. We show that if the rojectios are cotamiated with zero-mea Gaussia oise, the comressible sigals ca be recostructed with a exected average squared error that is uer bouded by a costat times k/ log ) α α+. For truly sarse sigals with oly a small umber of o-zero terms) a stroger result is obtaied; the exected average squared recostructio error is uer bouded by a costat times k/ log ). These bouds demostrate a remarkable caability of samlig via radom rojectios accurate recostructios ca be obtaied eve whe the sigal dimesio greatly exceeds the umber of samles k ad the samles themselves are cotamiated with sigificat levels of oise. This effect is highlighted by the followig eedle i a haystack roblem. Suose the sigal f is a vector of legth with oe ozero etry of amlitude. If we samle the vector at k radom locatios aki to covetioal samlig schemes), the the robability of comletely missig the o-zero etry is /) k, which is very close to whe k is sigificatly smaller tha. This imlies that the exected average squared error may be almost, or larger if oise is reset. O the other had, by samlig with radomized rojectios our results guaratee that the exected average squared error will be o larger tha a costat times k/ log ), which ca be arbitrarily close to 0 eve whe k, rovided k > log. A closely related roblem is the recostructio of sigals with sarse Fourier sectra from a relatively small umber of o-uiform time samles e.g., radom samles i time) [4 [7. Most of this work cocers oiseless situatios, but [5 addresses the roblem of recostructio from oise-corruted samles. Aother area of work related to our results cocers the recostructio of sigals with fiite degrees of freedom usig a small umber of o-traditioal samles [8, [9. A secial istace of this setu is the case of sigals that are sarse i time the dual of the sectrally sarse case). Recostructio from oise-corruted samles is the focus of [9. I a sese, the samlig ad recostructio roblems addressed i the aers above are secial cases of the class of roblems cosidered here, where we allow sigals that

3 3 are sarse i ay domai. Agai, this more uiversal ersective is recisely the focus of [, [3, which cosider sigal recostructio from oiseless radom rojectios. A iterestig lie of similar work cocers the related roblem of sigal recostructio from radom rojectios corruted by a ukow but bouded erturbatio [0, [. Here we cosider ubouded, Gaussia oise cotamiatio i the samlig rocess. Fially, while this aer was uder review a related ivestigatio was reorted i [ ertaiig to the statistical estimatio of sarse sigals from uderdetermied ad oisy observatios. That work develos quatitave bouds for sarse sigal recostructio similar to ours, but is based o a uiform ucertaity ricile rather tha radomized desigs as here. The aer is orgaized as follows. I Sectio II we state the basic roblem ad mai theoretical results of the aer. I Sectio III we derive bouds o the accuracy of sigal recostructios from oisy radom rojectios. I Sectio IV we secialize the bouds to cases i which the uderlyig sigal is comressible i terms of a certai orthoormal basis. I Sectio V we roose a simle iterative algorithm for sigal recostructio. I Sectio VI we discuss alicatios to ecodig, A/D coversio, ad wireless sesig, ad we make cocludig remarks i Sectio VII. Detailed derivatios are relegated to the Aedix. II. MAIN RESULTS Cosider a vector f = [f f... f T R ad assume that i= f i ) f B for a kow costat B > 0. The assumtio simly imlies that the average er elemet eergy is bouded by a costat. This is a fairly weak restrictio, sice it ermits a very large class of sigals, icludig sigals with eak magitudes as large as O ). Now suose that we are able to make k measuremets of f i the form of oisy, radom rojectios. Secifically, let Φ = {φ i,j } be a k array of bouded, i.i.d. zero-mea radom variables of variace E[φ i,j = /. Samles take the form y j = φ i,j fi + w j, j =,..., k ) i= where w = {w j } are i.i.d. zero-mea radom variables, ideedet of {φ i,j }, with variace σ. The goal is to recover a estimate of f from these observatios. Defie the risk of a cadidate recostructio f to be Rf) = f f + σ where the orm is the Euclidea distace. Next assume that both {φ i,j } ad {y j } are available. The we

4 4 ca comute the emirical risk Rf) = k ) k y j φ i,j f i. j= i= It is easy to verify that E[ Rf) = Rf) usig the facts that {φ i,j } ad {w j } are ideedet radom variables ad E[φ i,j = /. Thus, Rf) is a ubiased estimator of Rf). We will use the emirical risk to obtai a estimator f of f, ad boud the resultig error E[ f f. The estimator is based o a comlexity-regularized emirical risk miimizatio, ad we use the Craig-Berstei cocetratio iequality to cotrol the estimatio error of the recostructio rocess. That iequality etails the verificatio of certai momet coditios, which deed o the ature of Φ ad w. Therefore, i this aer we focus o ormalized) Rademacher rojectios, i which case each φ i,j is ±/ with equal robability, ad assume that w is a sequece of zero-mea Gaussia oises. Geeralizatios to other radom rojectios ad oise models may be ossible followig our aroach; this would oly require oe to verify the momet coditios required by the Craig-Berstei iequality. Suose that we have a coutable collectio F of cadidate recostructio fuctios ad a o-egative umber cf) assiged to each f F such that f F cf). Furthermore, assume that each f F satisfies f B. Select a recostructio accordig to the comlexity-regularized emirical risk miimizatio f k { = arg mi Rf) + f F } cf) log where ɛ > 0 is a costat that deeds o B ad σ. The we have the followig oracle iequality. Theorem Let ɛ = /B + σ) ), the [ E f k f { f f C mi f F + } cf) log + 4, where the costat C is give by with S = B/σ, the sigal-to-oise ratio. C = 9S )S + 3 S )S + A imortat oit regardig the costats above is that they deed oly o σ ad B, the oise ower ad the average sigal ower, resectively. If f is comressible with resect to a certai orthoormal basis, the we ca obtai exlicit bouds o the error i terms of the umber of radom rojectios k ad the degree to which f is comressible.

5 5 Let f m) deote the best m-term aroximatio of f i the basis. That is, if f has a reresetatio f = i= θ iψ i i the basis {ψ i }, the f m) = m i= θ i)ψ i), where coefficiets ad basis fuctios are re-ordered such that θ ) θ ) θ ). Assume that the average squared error f f m) / i= f i f m) i ) satisfies for some α 0 ad some costat C A f f m) C A m α > 0. Power-law decays like this arise quite commoly i alicatios. For examle, smooth ad iecewise smooth sigals ad sigals of bouded variatio exhibit this sort of behavior [, [3. It is also uecessary to restrict oe s attetio orthoormal basis exasios. Much more geeral aroximatio strategies ca be accomodated [3, but to kee the resetatio as simle as ossible we will ot delve further ito such extesios. Now let us take F to be a suitably quatized collectio of fuctios, rereseted i terms of the basis {ψ i } the costructio of F is discussed i Sectio IV). We have the followig error boud. Theorem If cf) = log) {# o-zero coefficiets of f} the there exists a costat C = C B, σ, C A ) > 0 such that [ E f k f ) k α/α+) C C, log where C is as give i Theorem. Note that the exoet α/α + ) is the usual exoet goverig the rate of covergece i oarametric fuctio estimatio. A stroger result is obtaied if the sigal is sarse, as stated i the followig Corollary. Corollary Suose that f has at most m ozero coefficiets. The there exists a costat C = C B, σ) > 0 such that [ E f k f C C ) k, m log where C is as give i Theorem.

6 6 Similar results hold if the sigal is additioally cotamiated with oise rior to the radom rojectio rocess. Corollary Suose observatios take the form y j = φ i,j fi + η i ) + w j, i= j =,..., k where {η i } are i.i.d. zero-mea Gaussia radom variables with variace σ s that are ideedet of {φ i,j } ad {w j }. The Theorems ad ad Corollary hold with slightly differet costats C, C, C, ad ɛ. It is imortat to oit out that all the results above hold for arbitrary sigal legth, ad the costats do ot deed o the size of the roblem. The fact that the rate deeds oly logarithmically o is sigificat ad illustrates the scalability of this aroach. Oe ca iterret these bouds as good idicators of the excetioal erformace of radom rojectio samlig i large- regimes. The deedece o k is show to be olyomial. I aalogy with oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). To drive this oit home, let us agai cosider the eedle i a haystack roblem, this time i a bit more detail. Suose the sigal f is a vector of legth with oe ozero etry of amlitude such that f / =. First, cosider radom satial oit samlig where observatios are oisefree i.e., each samle is of the form y j = f t j ), where t j is selected uiformly at radom from the set {,..., }). The squared recostructio error is 0 if the sike is located ad otherwise, ad the robability of ot fidig the sike i k trials is /) k, givig a average squared error of /) k + k/) 0 = /) k. If is large, we ca aroximate this by /) k e k/, which is very close to whe k is sigificatly smaller tha. O the other had, radomized Rademacher rojectios corruted with oise) yield a average squared recostructio error boud of C k/ log ), as give above i Corollary. This boud may be arbibrarily close to 0 eve whe k, rovided k > log. This shows that eve give the advatage of beig oiseless, the recostructio error from satial oit samlig may be far greater tha that resultig from radom rojectios. III. ORACLE INEQUALITY I this sectio we rove Theorem. For ease of otatio, we adot the shorthad otatio φ j = [φ,j φ,j... φ,j T for the vector corresodig to the j th rojectio. The emirical risk of a vector f

7 7 ca ow be writte as Rf) = k k y j φ T j f). j= We will boud rf, f ) Rf) Rf ), the excess risk betwee a cadidate recostructio f ad the actual fuctio f, usig the comlexity-regularizatio method itroduced i [3. Note that rf, f ) = E [ y j φ T j f y j φ T j f) + φ T j f) φ T j f ) = E [ φ T j f φ T j f ) = f f. Defie the emirical excess risk rf, f ) Rf) Rf ). The rf, f ) = k = k k [ yj φ T j f ) y j φ T j f) j= k j= U j where U j = [y j φ T j f ) y j φ T j f) are i.i.d. Notice that rf, f ) rf, f ) = k k j= U j E [U j ). We will make use of the Craig-Berstei iequality [4, which states that P k U j E [U j ) t ɛ k var k k + k j= j) U e t ζ) j= for 0 < ɛh ζ < ad t > 0, rovided the variables U j satisfy the momet coditio E [ U j E[U j k k! varu j) h k for some h > 0 ad all itegers k. If we cosider vectors f ad estimates f where f B ad f B, Rademacher rojectios, ad Gaussia oises with variace σ, the the momet coditio is satisfied with h = 6B +8 Bσ, as show i the Aedix. Alterative forms of radom rojectios ad oises ca also be hadled usig the aroach outlied ext, rovided the momet coditios are satisfied. To use the Craig-Berstei iequality we also eed a boud o the variace of U j itself. Defiig g = f f, we have ) g varu j ) = E[φ T g) 4 + 4σ g. As show i the Aedix, for itegers k ) g k ) g E[φ T g) k k k)!!,

8 8 where k)!! )3)... k ). Thus we ca boud the variace of U j by ) varu j ) g g + 4σ. Sice g satisfies g 4B ad rf, f ) = f f / = g /, the boud becomes varu j ) 8B + 4σ ) rf, f ). So, we ca relace the term i the Craig-Berstei iequality that deeds o the variace by k var k U j = k varu j ) 8B + 4σ ) rf, f ). k k j= For a give fuctio f, we have P rf, f ) rf, f ) > t 8B + + 4σ ) ) ɛrf, f ) < e t ζ) or, by lettig δ = e t P j= rf, f ) rf, f ) > log δ ) 8B + 4σ ) ) ɛrf, f ) + < δ. ζ) Now assig to each f F a o-egative ealty term cf) such that the ealties satisfy the Kraft Iequality [5 cf) f F ad let δf) = cf) δ. The by alyig the uio boud we have for all f F ad for all δ > 0 rf, f ) rf, f ) cf) log + log ) δ 8B + 4σ ) ɛrf, f ) + ζ) with robability at least δ. Now set ζ = ɛh, defie 8B + 4σ ) ɛ a, ζ) ad choose ɛ < 0B + 8 Bσ + σ. Notice that a < ad ζ < by choice of ɛ. The a)rf, f ) rf, f ) + cf) log + log ) δ holds with robability at least δ for all f F ad ay δ > 0.

9 9 For the give traiig samles, we ca miimize the uer boud by choosig { } f k = arg mi rf, f cf) log ) + f F which is equivalet to { f k = arg mi Rf) + f F f k arg mi f F } cf) log sice we ca igore Rf ) whe erformig the otimizatio. If we defie { } Rf) + cf) log the with robability at least δ a)r f k, f ) r f k, f ) + c f k ) log + log ) δ rf k, f ) + cf k ) log + log ) δ sice f k miimizes the comlexity-regularized emirical risk criterio. Usig the Craig-Berstei iequality agai to boud rf k, f ) rf k, f ) with the same variace boud as before) we get that with robability at least δ rf k, f ) rfk, f ) a rfk, f ) + log ) δ. 3) We wat both ) ad 3) to hold simultaeously, so we use the uio boud to obtai ) r f + a k, f ) rfk a, f ) + cf k ) log + log )) δ a holdig with robability at least δ. Let δ = e t a)/ to obtai P r f k, f ) + a a Itegratig this relatio gives [ E r f k, f ) ) rfk, f ) cf k ) log ) a) t e t a)/. ) + a a rfk, f ) + cf k ) log + 4. a) )

10 0 Now, sice a is ositive, [ E f k f [ = E r f k, f ) where C = + a)/ a). = + a a + a a + a a ) + a mi a f F { f f + = C mi f F ) rfk, f ) + cf k ) log + 4 a) ) rfk, f ) + + a) cf k ) log + 4 a) ) { Rfk ) Rf ) + cf k ) log + 4 { Rf) Rf ) + Tyical values of C ca be determied by aroximatig the costat ɛ < 0B + 8 Bσ + σ. cf) log + 4 } } cf) log + 4 }, Uer boudig the deomiator guaratees that the coditio is satisfied, so let ɛ = / B + σ) ). Now 8B + 4σ ) ɛ a = = ζ) If we deote the sigal to oise ratio by S = B /σ the 4B + σ 5B )Bσ + σ. 4S + a = 5S )S + for which the extremes are a mi = / ad a max = 4/5, givig costats C i the rage of [3/9, 9. IV. ERROR BOUNDS FOR COMPRESSIBLE SIGNALS I this sectio we rove Theorem ad Corollary. Suose that f is comressible i a certai orthoormal basis {ψ i } i=. Secifically, let f m) deote the best m-term aroximatio of f i terms of {ψ i }, ad assume that the error of the aroximatio obeys for some α 0 ad a costat C A > 0. f f m) C A m α Let us use the basis {ψ i } for the recostructio rocess. Ay vector f F ca be exressed i terms of the basis {ψ i } as f = i= θ iψ i, where θ = {θ i } are the coefficiets of f i this basis. Let T deote the trasform that mas coefficiets to fuctios, so that f = T θ ad defie Θ = {θ : T θ B, θ i quatized to levels} to be the set of cadidate solutios i the basis {ψ i }. The ealty term

11 cf) writte i terms of the basis {ψ i } is cf) = cθ) = + ) log) i= I θ i 0 = + ) log θ 0. It is easily verified that f F cf) by otig that each θ Θ ca be uiquely ecoded via a refix code cosistig of + ) log bits er o-zero coefficiet log bits for the locatios ad log bits for the quatized values) i which case the codelegths cf) must satisfy the Kraft iequality [6. The oracle iequality [ E f k f { f f C mi + f F } cf) log + 4 ca be writte i terms of the ew class of cadidate recostructios as [ E f k f { θ θ } cθ) log + 4 C mi + θ Θ where f = T θ. For each iteger m, let θ m) deote the coefficiets corresodig to the best m-term aroximatio of f ad let θ m) q deote the earest elemet i Θ. The maximum ossible dyamic rage for the coefficiet magitudes, ± B, is quatized to levels, givig θ q m) θ m) 4B / = C Q /. Thus, θ m) q θ = θ m) q θ m) + θ m) θ θ m) q θ m) + θ m) q θ m) θ m) θ + θ m) θ C Q CA C Q + m α + C A m α. Now isert θ q m) i lace of θ i the oracle boud, ad ote that cθ q m) ) = + )m log to obtai [ E f { } k f C Q m α CA C Q C mi + θ Θ / + C A m α + )m log log Choosig large eough makes the first two terms egligible. Balacig the third ad fourth terms gives ) ) + ) log α+ k α+ m = ɛc A log so ad sice ) α ) α + ) log C A m α α+ k α+ = C A, ɛc A log k < log k ) α α+

12 whe k > ad > e, [ E f k f ) α k α+ C C, log as claimed i the Theorem, where C = { C A + ) log ɛc A } α ɛ ) α Suose ow that f has oly m ozero coefficiets. I this case, θ m) q θ C Q sice θ m) θ = 0. Now the ealty term domiates i the oracle boud ad [ E f k f ) k C C, m log where { } + ) log + 4 C =. ɛ V. OPTIMIZATION SCHEME Although our otimizatio is o-covex, it does ermit a simle, iterative otimizatio strategy that roduces a sequece of recostructios for which the corresodig sequece of comlexity-regularized emirical risk values is o-decreasig. This algorithm, which is described below, has demostrated itself to be quite effective i similar deoisig ad recostructio roblems [7 [9. A ossible alterative strategy might etail covexifyig the roblem by relacig the l 0 ealty with a l ealty. Recet results show that ofte the solutio to this covex roblem coicides with or aroximates the solutio to the origial o-covex roblem [0. Let us assume that we wish to recostruct our sigal i terms of the basis {ψ i }. Usig the defiitios itroduced i the revious sectio, the recostructio is equivalet to f k = T θ k where f k = arg mi f F θ k = arg mi θ Θ { Rf) + { RT θ) + Thus, the otimizatio roblem ca the be writte as { y P T θ + θ k = arg mi θ Θ } cf) log } cθ) log } log) log) θ 0 ɛ

13 3 where P = Φ T, the trasose of the k rojectio matrix Φ, y is a colum vector of the k observatios, ad θ 0 = i= I {θ i 0}. To solve this, we use a iterative boud-otimizatio rocedure, as roosed i [7 [9. This rocedure etails a two-ste iterative rocess that begis with a iitializatio θ 0) ad comutes:. ϕ t) = θ t) + λ P T )T y P T θ t) ). θt+) i = ϕ t) i if ϕ t) log) log) i λɛ 0 otherwise where λ is the largest eigevalue of P P. This rocedure is desirable sice the secod ste, i which the comlexity term lays its role, ivolves a simle coordiate-wise thresholdig oeratio. It is easy to verify that the iteratios roduce a mootoically o-icreasig sequece of comlexity-regularized emirical risk values [9. Thus, this rocedure rovides a simle iteratio that teds to miimize the origial objective fuctio, ad aears to give good results i ractice [7. The iteratios ca be termiated whe the etries uiformly satisfy θ t+) i θ t) i δ, for a small ositive tolerace δ. The comutatioal comlexity of the above rocedure is quite aealig. Each iteratio requires oly Ok) oeratios, assumig that the trasform T ca be comuted i O) oeratios. For examle, the discrete wavelet of Fourier trasforms ca be comuted i O) ad O log ) oeratios, resectively. Multilicatio by P is the most itesive oeratio, requirig Ok) oeratios. The thresholdig ste is carried out ideedetly i each coordiate, ad this ste requires O) oeratios as well. Of course, the umber of iteratios required is roblem deedet ad difficult to redict, but i our exeriece i this alicatio ad others [7, [9 algorithms of this sort ted to coverge i a reasoably small umber of iteratios, eve i very high dimesioal cases. Oe oit worthy of metio relates to the factor /ɛ = B +σ) i the ealty. As is ofte the case with coservative bouds of this tye, the theoretical ealty is larger tha what is eeded i ractice to achieve good results. Also, a otetial hurdle to calibratig the algorithm is that it deeds o kowledge of B ad σ, either of which may be kow a riori. Strictly seakig, these values do ot eed to be kow ideedetly but rather we eed oly estimate B +σ). To that ed, otice that each observatio is a radom variable with variace equal to f / + σ. Let B = f /, which is the miimum B satisfyig the stated boud f B. The the variace of each observatio is B + σ. Further, it is easy to verify that B +σ ) B +σ). So, a scheme could be develoed whereby the samle variace is used as a surrogate for the ukow quatities i the form i which they aear i the arameter ɛ.

14 4 This would etail usig aother cocetratio iequality to cotrol the error betwee the samle variace ad its mea value, ad roagatig this additioal error through the derivatio of the oracle iequality. While this is relatively straightforward, we omit a comlete derivatio here. To illustrate the erformace of the algorithm above, i Figure we cosider three stadard test sigals, each of legth = Rademacher rojectio samles cotamiated with additive white Gaussia oise) are take for the Blocks, Bums, ad Doler test sigals. The algorithm described above is emloyed for recostructio, with oe slight modificatio. Sice the theoretical ealty ca be a bit too coservative i ractice, the threshold used i this examle is /3 of the theoretical value i.e., a threshold of log) log)/λɛ)/3 was used). The SNR, defied as SNR = 0 log 0 B σ where B = f /, is db for each test sigal. To covey a sese of the oise level, colum a) of Figure shows the origial sigals cotamiated with the same level of oise i.e., the sigal resultig from covetioal oit samles cotamiated with oise of the same ower). Colum b) shows the recostructios obtaied from 600 rojectios; recostructios from 00 rojectios are show i colum c). The Blocks sigal to row) was recostructed usig the Haar wavelet basis Daubechies-), well-suited to the iecewise costat ature of the sigal. The Bums ad Doler sigals middle ad bottom row, resectively) were recostructed usig the Daubechies-6 wavelet basis. Of course, the selectio of the best recostructio basis is a searate matter that is beyod the scoe of this aer. VI. APPLICATIONS Oe immediate alicatio of the results ad methods above is to sigal codig ad A/D coversio. I the oiseless settig, several authors have suggested the use of radom rojectio samlig for such uroses [ [3. Our results idicate how such schemes might erform i the resece of oise. Suose that we have a array of sesors, each makig a oisy measuremet. The oise could be due to the sesors themselves or evirometal factors. The goal of ecodig ad A/D coversio is to rereset the sesor readigs i a comressed form, suitable for digital storage or trasmissio. Our results suggest that k radom Rademacher rojectios of the sesor readigs ca be used for this urose, ad the error bouds suggest guidelies for how may rojectios might be required for a certai level of recisio. Our theory ad method ca also be alied to wireless sesig as follows. Cosider the roblem of sesig a distributed field e.g., temerature, light, chemical) usig a collectio of wireless sesors distributed uiformly over a regio of iterest. Such systems are ofte referred to as sesor etworks. The goal is to obtai a accurate, high-resolutio recostructio of the field at a remote destiatio. Oe

15 5 a) b) c) Fig.. Simulatio examles usig Blocks, Bums, ad Doler test sigals of legth Colum a) shows the origial sigals with a equivalet level of additive er-ixel oise. Colums b) ad c) show recostructios from 600 ad 00 rojectios, resectively. aroach to this roblem is to require each sesor to digitally trasmit its measuremet to the destiatio, where field recostructio is the erformed. Alteratively, the sesors might collaboratively rocess their measuremets to recostruct the field themselves ad the trasmit the result to the destiatio i.e., the odes collaborate to comress their date rior to trasmissio). Both aroaches ose sigificat demads o commuicatio resources ad ifrastructure, ad it has recetly bee suggested that o-collaborative aalog commuicatio schemes offer a more resource-efficiet alterative [ [3. Assume that the sesor data is to be trasmitted to the destiatio over a additive white Gaussia oise chael. Suose the destiatio broadcasts erhas digitally) a radom seed to the sesors.

16 6 Each ode modifies this seed i a uique way kow to oly itself ad the destiatio e.g., this seed could be multilied by the ode s address or geograhic ositio). Each ode geerates a seudoradom Rademacher sequece, which ca also be costructed at the destiatio. The the odes trasmit the radom rojectios to the destiatio hase-coheretly i.e., beamformig). This is accomlished by requirig each ode to simly multily its readig by a elemet of its radom sequece i each rojectio/commuicatio ste ad trasmit the result to the destiatio via amlitude modulatio. If the trasmissios from all sesors ca be sychroized so that they all arrive i hase at the destiatio, the the averagig iheret i the multile access chael comutes the desired ier roduct. After receivig k rojectios, the destiatio ca emloy the recostructio algorithm above usig a basis of choice e.g., wavelet). The commuicatios rocedure is comletely o-adative ad otetially very simle to imlemet. The collective fuctioig of the wireless sesors i this rocess is more aki to a esemble of hase-coheret emitters tha it is to covetioal etworkig oeratios. Therefore, we refer the term sesor esemble istead of sesor etwork i this cotext. A remarkable asect of the sesor esemble aroach is that the ower required to achieve a target distortio level ca be very miimal. Let σ s ad σ c deote the oise variace due to sesig ad commuicatio, resectively. Thus, each rojectio received at the destiatio is corruted by a oise of total ower σ s + σ c. The sesig oise variace is assumed to be a costat ad the additioal variace due to the commuicatio chael is assumed to scale like the iverse of the total received ower σ c P where P is the trasmit ower er sesor. Note that although the total trasmit ower is P, the received ower is a factor of larger as a result of the ower amlificatio effect of the hase-coheret trasmissios [3. I order to achieve rates of distortio decay that we claim, it is sufficiet that the variace due to the commuicatio chael behaves like a costat. Therefore, we require oly that P. This results i a rather surrisig coclusio. Ideal recostructio is ossible at the destiatio with total trasmit ower P tedig to zero as the desity of sesors icreases. If covetioal satial oit samles were take istead e.g., if a sigle sesor is selected at radom i each ste ad trasmits its measuremet to the destiatio), the the ower required er samle would be a costat, sice oly oe sesor would be ivolved i such a trasmissio. Thus, it aears that radom rojectio samlig could be much more desirable i wireless sesig alicatios.

17 7 VII. CONCLUSIONS AND FUTURE WORK We have show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. The squared error bouds for comressible sigals are Ok/ log ) α α+ ), which is withi a logarithmic factor of the usual oarametric estimatio rate, ad Ok/ log ) ) for sarse sigals. We also roosed a ractical iterative algorithm for sigal recostructio. Oe of the most romisig otetial alicatios of our theory ad method is to wireless sesig, wherei oe realizes a large trasmissio ower gai by radom rojectio samlig as oosed to covetioal satial oit samlig. The role of the oise variace i the rates we reseted is worthy of further attetio. As the oise variace teds to zero, oe ituitively exects to attai the fast rates that are kow to be achievable i the oiseless settig. Our theory is based i the oisy regime ad it does ot directly imly the reviously established bouds i the oiseless settig. Simly ut, our aalysis assumes a oise variace strictly greater tha zero. Let us commet briefly o the tightess of the uer bouds give by our theory. I aalogy with classical oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). Moreover, i the secial case of sarse sigals with m o-zero terms, we obtai a error boud of m/k igorig costat ad logarithmic factors). Stadard arametric statistical aalysis suggests that oe should ot exect a rate of better tha m/k degrees-offreedom/ samle-size) i such cases, which agai suorts our ituitio regardig the tightess of the bouds i terms of the covergece rate). However, to our kowledge exlicit miimax lower bouds have ot bee established i the cotext of this roblem, ad the determiatio of such bouds is oe of our future research directios. Although we cosidered oly the case of Gaussia oise i the observatio model ), the same results could be achieved for ay zero-mea, symmetrically distributed oise that is ideedet of the rojectio vector elemets ad satisfies E[w k j k)!!varw j )) k, a result that follows immediately usig the lemmas reseted i the Aedix. Aother extesio would be the cosideratio of other radom rojectios istead of the Rademacher rojectios cosidered here. Most of our basic aroach would go through i such cases; oe would oly eed to verify the momet coditios of the Craig-Berstei iequality for articular cases.

18 8 VIII. ACKNOWLEDGEMENTS The authors would like to thak Rui Castro for his assistace i all stages of the work reseted here. A. The Craig-Berstei Momet Coditio APPENDIX The cetral requiremet of the Craig-Berstei iequality is the satisfactio of the momet coditio E [ X E[X!varX)h for itegers with some ositive costat h that does ot deed o, a coditio that is automatically verified for = with ay fiite value of h. For higher owers this coditio ca be very difficult to verify for several reasos, ot the least of which is the absolute value reset i the momets. Previous work that made use of the Craig-Berstei iequality assumed that the observatios were bouded, forcig a ossibly urealistic case of bouded oise [3. This assumtio is ot sufficiet for the rates of covergece we claim. Ideed, the aïve boud o the observatios is y i B yieldig a costat h that would grow roortioally to. With that motivatio, we develo a framework uder which a boudig costat h ca be determied more directly. First, observe that the momet coditio is usually easier to verify for the eve owers because the absolute value eed ot be dealt with directly. This is sufficiet to guaratee the momet coditio is satisfied for all itegers, as roved i the followig lemma. Lemma Suose the Craig-Berstei momet coditio holds for all eve itegers greater tha or equal to, that is E [ X E[X k k)!varx)hk, k sice the k = case is satisfied trivially for ay h. The the coditio holds also for the odd absolute momets with h = h. Thus A ad B, E [ X E[X k k )!varx) h k 3, k E [ X E[X!varX)h),. Proof: For ease of otatio, let Z = X E[X. Hölder s Iequality states, for ay radom variables E[ AB E[ A / E[ B q /q

19 9 where <, q < ad / + /q =. Take A = Z, B = Z k, ad = q = to get E[ Z k E[Z E[Z 4k 4 where the absolute values iside the square root have bee droed because the exoets are eve. Now by assumtio so E[Z 4k 4 4k 4)!E[Z h 4k 6 E[ Z k We wat to satisfy the followig iequality by choice of h which meas h must satisfy 4k 4)!E[Z ) h 4k 6 4k 4)! E[Z h k 3 E[ Z k k )!E[Z h k 3 If we choose k )! 4k 4)! h k 3 h k 3. h max k 4k 4)! k )! ) k 3 h the the momet coditio will be satisfied for the odd exoets k. A uer boud for the term i brackets is, as show here. For k, the boud k)! k k!) holds ad ca be verified by iductio o k. This imlies ) 4k 4)! k 3 ) k 3. 4) k )! k Now, the term i aretheses o the right had side of 4) is always less tha for k. The fial ste is to show that lim k k ) k 3 =, which is verified by otig that lim log k k ) k 3 = lim log ) ) log k ) = 0. k k 3

20 0 Thus, the momet coditio is satisfied for odd momets with h = h. Also, if the momet coditio is satisfied for a give h, it is also satisfied for ay h h so holds for all itegers as claimed. E [ Z = E [ X E[X!varX)h) We will also eed results for how sums ad roducts of radom variables behave with resect to the momet coditio. For that, we have the followig two lemmas. Lemma Let Z = A + B be the sum of two zero-mea radom variables A ad B with variaces vara) = E[A ad varb) = E[B, ot both zero, ad such that E[AB 0. Suose both A ad B satisfy the momet coditio for a give iteger 3 with ositive costats h A ad h B resectively, ot both zero. That is The E [ A!varA)h A ad E [ B!varB)h B. E [ Z!varZ)h S where h S = / mi ) h A + h B ), ad mi is the miimum 3 for which the assumtios are satsified. Proof: First, defie V A = vara)/vara) + varb)), V B = varb)/vara) + varb)), H A = h A /h A + h B ), ad H B = h B /h A + h B ). Use Mikowski s Iequality to write E[ A + B [E[ A / + E[ B / [ ) vara)h ) A + varb)h B! =!vara) + varb)) [ ) V A h A + =!vara) + varb)) h A + h B )!vara + B) h A + h B ) V B h B ) [ ) V A H A + [ ) V A H A + V B H B V B H B ), ) where the last ste follows from the assumtio that E[AB 0, imlyig vara)+varb) vara+ B). Showig that [ ) V A H A + V B H B ) C,