Signal Reconstruction from Noisy Random Projections


 Isaac Franklin
 1 years ago
 Views:
Transcription
1 Sigal Recostructio from Noisy Radom Projectios Jarvis Haut ad Robert Nowak Deartmet of Electrical ad Comuter Egieerig Uiversity of WiscosiMadiso March, 005; Revised February, 006 Abstract Recet results show that a relatively small umber of radom rojectios of a sigal ca cotai most of its saliet iformatio. It follows that if a sigal is comressible i some orthoormal basis, the a very accurate recostructio ca be obtaied from radom rojectios. We exted this tye of result to show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. We also roose a ractical iterative algorithm for sigal recostructio, ad briefly discuss otetial alicatios to codig, A/D coversio, ad remote wireless sesig. Idex Terms samlig, sigal recostructio, radom rojectios, deoisig, wireless sesor etworks I. INTRODUCTION Recet theory iforms us that, with high robability, a relatively small umber of radom rojectios of a sigal ca cotai most of its relevat iformatio. For examle, the groudbreakig work i [ has show that k radom Fourier rojectios cotai eough iformatio to recostruct iecewise smooth sigals at a distortio level early equivalet to that attaiable from k otimally selected observatios. Similar results hold for radom Gaussia ad Rademacher rojectios i.e., rojectios cosistig of ideedet ad idetically distributed Gaussia or Rademacher radom variables) [, [3. The results reseted i these works ca be roughly summarized as follows. Assume that a sigal f R is comressible i some orthoormal basis i the followig sese. Let f m) deote the best mterm This research was suorted i art by the NSF uder grats CCR ad CCR03557, ad by the Office of Naval Research, grat N
2 aroximatio of f i terms of this basis ad suose that the average squared error obeys f f m) = f i f m) i ) C A m α for some α 0 ad some costat C A i= > 0. The arameter α govers the degree to which f is comressible with resect to the basis. I a oiseless settig, it ca be show that a aroximatio of such a sigal ca be recovered from k radom rojectios with a average squared error that is uer bouded by a costat times k/ log ) α, early as good as the best kterm aroximatio error. This aer takes this lie of iquiry a ste further by cosiderig the erformace of samlig via radom rojectios i oisy coditios. We show that if the rojectios are cotamiated with zeromea Gaussia oise, the comressible sigals ca be recostructed with a exected average squared error that is uer bouded by a costat times k/ log ) α α+. For truly sarse sigals with oly a small umber of ozero terms) a stroger result is obtaied; the exected average squared recostructio error is uer bouded by a costat times k/ log ). These bouds demostrate a remarkable caability of samlig via radom rojectios accurate recostructios ca be obtaied eve whe the sigal dimesio greatly exceeds the umber of samles k ad the samles themselves are cotamiated with sigificat levels of oise. This effect is highlighted by the followig eedle i a haystack roblem. Suose the sigal f is a vector of legth with oe ozero etry of amlitude. If we samle the vector at k radom locatios aki to covetioal samlig schemes), the the robability of comletely missig the ozero etry is /) k, which is very close to whe k is sigificatly smaller tha. This imlies that the exected average squared error may be almost, or larger if oise is reset. O the other had, by samlig with radomized rojectios our results guaratee that the exected average squared error will be o larger tha a costat times k/ log ), which ca be arbitrarily close to 0 eve whe k, rovided k > log. A closely related roblem is the recostructio of sigals with sarse Fourier sectra from a relatively small umber of ouiform time samles e.g., radom samles i time) [4 [7. Most of this work cocers oiseless situatios, but [5 addresses the roblem of recostructio from oisecorruted samles. Aother area of work related to our results cocers the recostructio of sigals with fiite degrees of freedom usig a small umber of otraditioal samles [8, [9. A secial istace of this setu is the case of sigals that are sarse i time the dual of the sectrally sarse case). Recostructio from oisecorruted samles is the focus of [9. I a sese, the samlig ad recostructio roblems addressed i the aers above are secial cases of the class of roblems cosidered here, where we allow sigals that
3 3 are sarse i ay domai. Agai, this more uiversal ersective is recisely the focus of [, [3, which cosider sigal recostructio from oiseless radom rojectios. A iterestig lie of similar work cocers the related roblem of sigal recostructio from radom rojectios corruted by a ukow but bouded erturbatio [0, [. Here we cosider ubouded, Gaussia oise cotamiatio i the samlig rocess. Fially, while this aer was uder review a related ivestigatio was reorted i [ ertaiig to the statistical estimatio of sarse sigals from uderdetermied ad oisy observatios. That work develos quatitave bouds for sarse sigal recostructio similar to ours, but is based o a uiform ucertaity ricile rather tha radomized desigs as here. The aer is orgaized as follows. I Sectio II we state the basic roblem ad mai theoretical results of the aer. I Sectio III we derive bouds o the accuracy of sigal recostructios from oisy radom rojectios. I Sectio IV we secialize the bouds to cases i which the uderlyig sigal is comressible i terms of a certai orthoormal basis. I Sectio V we roose a simle iterative algorithm for sigal recostructio. I Sectio VI we discuss alicatios to ecodig, A/D coversio, ad wireless sesig, ad we make cocludig remarks i Sectio VII. Detailed derivatios are relegated to the Aedix. II. MAIN RESULTS Cosider a vector f = [f f... f T R ad assume that i= f i ) f B for a kow costat B > 0. The assumtio simly imlies that the average er elemet eergy is bouded by a costat. This is a fairly weak restrictio, sice it ermits a very large class of sigals, icludig sigals with eak magitudes as large as O ). Now suose that we are able to make k measuremets of f i the form of oisy, radom rojectios. Secifically, let Φ = {φ i,j } be a k array of bouded, i.i.d. zeromea radom variables of variace E[φ i,j = /. Samles take the form y j = φ i,j fi + w j, j =,..., k ) i= where w = {w j } are i.i.d. zeromea radom variables, ideedet of {φ i,j }, with variace σ. The goal is to recover a estimate of f from these observatios. Defie the risk of a cadidate recostructio f to be Rf) = f f + σ where the orm is the Euclidea distace. Next assume that both {φ i,j } ad {y j } are available. The we
4 4 ca comute the emirical risk Rf) = k ) k y j φ i,j f i. j= i= It is easy to verify that E[ Rf) = Rf) usig the facts that {φ i,j } ad {w j } are ideedet radom variables ad E[φ i,j = /. Thus, Rf) is a ubiased estimator of Rf). We will use the emirical risk to obtai a estimator f of f, ad boud the resultig error E[ f f. The estimator is based o a comlexityregularized emirical risk miimizatio, ad we use the CraigBerstei cocetratio iequality to cotrol the estimatio error of the recostructio rocess. That iequality etails the verificatio of certai momet coditios, which deed o the ature of Φ ad w. Therefore, i this aer we focus o ormalized) Rademacher rojectios, i which case each φ i,j is ±/ with equal robability, ad assume that w is a sequece of zeromea Gaussia oises. Geeralizatios to other radom rojectios ad oise models may be ossible followig our aroach; this would oly require oe to verify the momet coditios required by the CraigBerstei iequality. Suose that we have a coutable collectio F of cadidate recostructio fuctios ad a oegative umber cf) assiged to each f F such that f F cf). Furthermore, assume that each f F satisfies f B. Select a recostructio accordig to the comlexityregularized emirical risk miimizatio f k { = arg mi Rf) + f F } cf) log where ɛ > 0 is a costat that deeds o B ad σ. The we have the followig oracle iequality. Theorem Let ɛ = /B + σ) ), the [ E f k f { f f C mi f F + } cf) log + 4, where the costat C is give by with S = B/σ, the sigaltooise ratio. C = 9S )S + 3 S )S + A imortat oit regardig the costats above is that they deed oly o σ ad B, the oise ower ad the average sigal ower, resectively. If f is comressible with resect to a certai orthoormal basis, the we ca obtai exlicit bouds o the error i terms of the umber of radom rojectios k ad the degree to which f is comressible.
5 5 Let f m) deote the best mterm aroximatio of f i the basis. That is, if f has a reresetatio f = i= θ iψ i i the basis {ψ i }, the f m) = m i= θ i)ψ i), where coefficiets ad basis fuctios are reordered such that θ ) θ ) θ ). Assume that the average squared error f f m) / i= f i f m) i ) satisfies for some α 0 ad some costat C A f f m) C A m α > 0. Powerlaw decays like this arise quite commoly i alicatios. For examle, smooth ad iecewise smooth sigals ad sigals of bouded variatio exhibit this sort of behavior [, [3. It is also uecessary to restrict oe s attetio orthoormal basis exasios. Much more geeral aroximatio strategies ca be accomodated [3, but to kee the resetatio as simle as ossible we will ot delve further ito such extesios. Now let us take F to be a suitably quatized collectio of fuctios, rereseted i terms of the basis {ψ i } the costructio of F is discussed i Sectio IV). We have the followig error boud. Theorem If cf) = log) {# ozero coefficiets of f} the there exists a costat C = C B, σ, C A ) > 0 such that [ E f k f ) k α/α+) C C, log where C is as give i Theorem. Note that the exoet α/α + ) is the usual exoet goverig the rate of covergece i oarametric fuctio estimatio. A stroger result is obtaied if the sigal is sarse, as stated i the followig Corollary. Corollary Suose that f has at most m ozero coefficiets. The there exists a costat C = C B, σ) > 0 such that [ E f k f C C ) k, m log where C is as give i Theorem.
6 6 Similar results hold if the sigal is additioally cotamiated with oise rior to the radom rojectio rocess. Corollary Suose observatios take the form y j = φ i,j fi + η i ) + w j, i= j =,..., k where {η i } are i.i.d. zeromea Gaussia radom variables with variace σ s that are ideedet of {φ i,j } ad {w j }. The Theorems ad ad Corollary hold with slightly differet costats C, C, C, ad ɛ. It is imortat to oit out that all the results above hold for arbitrary sigal legth, ad the costats do ot deed o the size of the roblem. The fact that the rate deeds oly logarithmically o is sigificat ad illustrates the scalability of this aroach. Oe ca iterret these bouds as good idicators of the excetioal erformace of radom rojectio samlig i large regimes. The deedece o k is show to be olyomial. I aalogy with oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). To drive this oit home, let us agai cosider the eedle i a haystack roblem, this time i a bit more detail. Suose the sigal f is a vector of legth with oe ozero etry of amlitude such that f / =. First, cosider radom satial oit samlig where observatios are oisefree i.e., each samle is of the form y j = f t j ), where t j is selected uiformly at radom from the set {,..., }). The squared recostructio error is 0 if the sike is located ad otherwise, ad the robability of ot fidig the sike i k trials is /) k, givig a average squared error of /) k + k/) 0 = /) k. If is large, we ca aroximate this by /) k e k/, which is very close to whe k is sigificatly smaller tha. O the other had, radomized Rademacher rojectios corruted with oise) yield a average squared recostructio error boud of C k/ log ), as give above i Corollary. This boud may be arbibrarily close to 0 eve whe k, rovided k > log. This shows that eve give the advatage of beig oiseless, the recostructio error from satial oit samlig may be far greater tha that resultig from radom rojectios. III. ORACLE INEQUALITY I this sectio we rove Theorem. For ease of otatio, we adot the shorthad otatio φ j = [φ,j φ,j... φ,j T for the vector corresodig to the j th rojectio. The emirical risk of a vector f
7 7 ca ow be writte as Rf) = k k y j φ T j f). j= We will boud rf, f ) Rf) Rf ), the excess risk betwee a cadidate recostructio f ad the actual fuctio f, usig the comlexityregularizatio method itroduced i [3. Note that rf, f ) = E [ y j φ T j f y j φ T j f) + φ T j f) φ T j f ) = E [ φ T j f φ T j f ) = f f. Defie the emirical excess risk rf, f ) Rf) Rf ). The rf, f ) = k = k k [ yj φ T j f ) y j φ T j f) j= k j= U j where U j = [y j φ T j f ) y j φ T j f) are i.i.d. Notice that rf, f ) rf, f ) = k k j= U j E [U j ). We will make use of the CraigBerstei iequality [4, which states that P k U j E [U j ) t ɛ k var k k + k j= j) U e t ζ) j= for 0 < ɛh ζ < ad t > 0, rovided the variables U j satisfy the momet coditio E [ U j E[U j k k! varu j) h k for some h > 0 ad all itegers k. If we cosider vectors f ad estimates f where f B ad f B, Rademacher rojectios, ad Gaussia oises with variace σ, the the momet coditio is satisfied with h = 6B +8 Bσ, as show i the Aedix. Alterative forms of radom rojectios ad oises ca also be hadled usig the aroach outlied ext, rovided the momet coditios are satisfied. To use the CraigBerstei iequality we also eed a boud o the variace of U j itself. Defiig g = f f, we have ) g varu j ) = E[φ T g) 4 + 4σ g. As show i the Aedix, for itegers k ) g k ) g E[φ T g) k k k)!!,
8 8 where k)!! )3)... k ). Thus we ca boud the variace of U j by ) varu j ) g g + 4σ. Sice g satisfies g 4B ad rf, f ) = f f / = g /, the boud becomes varu j ) 8B + 4σ ) rf, f ). So, we ca relace the term i the CraigBerstei iequality that deeds o the variace by k var k U j = k varu j ) 8B + 4σ ) rf, f ). k k j= For a give fuctio f, we have P rf, f ) rf, f ) > t 8B + + 4σ ) ) ɛrf, f ) < e t ζ) or, by lettig δ = e t P j= rf, f ) rf, f ) > log δ ) 8B + 4σ ) ) ɛrf, f ) + < δ. ζ) Now assig to each f F a oegative ealty term cf) such that the ealties satisfy the Kraft Iequality [5 cf) f F ad let δf) = cf) δ. The by alyig the uio boud we have for all f F ad for all δ > 0 rf, f ) rf, f ) cf) log + log ) δ 8B + 4σ ) ɛrf, f ) + ζ) with robability at least δ. Now set ζ = ɛh, defie 8B + 4σ ) ɛ a, ζ) ad choose ɛ < 0B + 8 Bσ + σ. Notice that a < ad ζ < by choice of ɛ. The a)rf, f ) rf, f ) + cf) log + log ) δ holds with robability at least δ for all f F ad ay δ > 0.
9 9 For the give traiig samles, we ca miimize the uer boud by choosig { } f k = arg mi rf, f cf) log ) + f F which is equivalet to { f k = arg mi Rf) + f F f k arg mi f F } cf) log sice we ca igore Rf ) whe erformig the otimizatio. If we defie { } Rf) + cf) log the with robability at least δ a)r f k, f ) r f k, f ) + c f k ) log + log ) δ rf k, f ) + cf k ) log + log ) δ sice f k miimizes the comlexityregularized emirical risk criterio. Usig the CraigBerstei iequality agai to boud rf k, f ) rf k, f ) with the same variace boud as before) we get that with robability at least δ rf k, f ) rfk, f ) a rfk, f ) + log ) δ. 3) We wat both ) ad 3) to hold simultaeously, so we use the uio boud to obtai ) r f + a k, f ) rfk a, f ) + cf k ) log + log )) δ a holdig with robability at least δ. Let δ = e t a)/ to obtai P r f k, f ) + a a Itegratig this relatio gives [ E r f k, f ) ) rfk, f ) cf k ) log ) a) t e t a)/. ) + a a rfk, f ) + cf k ) log + 4. a) )
10 0 Now, sice a is ositive, [ E f k f [ = E r f k, f ) where C = + a)/ a). = + a a + a a + a a ) + a mi a f F { f f + = C mi f F ) rfk, f ) + cf k ) log + 4 a) ) rfk, f ) + + a) cf k ) log + 4 a) ) { Rfk ) Rf ) + cf k ) log + 4 { Rf) Rf ) + Tyical values of C ca be determied by aroximatig the costat ɛ < 0B + 8 Bσ + σ. cf) log + 4 } } cf) log + 4 }, Uer boudig the deomiator guaratees that the coditio is satisfied, so let ɛ = / B + σ) ). Now 8B + 4σ ) ɛ a = = ζ) If we deote the sigal to oise ratio by S = B /σ the 4B + σ 5B )Bσ + σ. 4S + a = 5S )S + for which the extremes are a mi = / ad a max = 4/5, givig costats C i the rage of [3/9, 9. IV. ERROR BOUNDS FOR COMPRESSIBLE SIGNALS I this sectio we rove Theorem ad Corollary. Suose that f is comressible i a certai orthoormal basis {ψ i } i=. Secifically, let f m) deote the best mterm aroximatio of f i terms of {ψ i }, ad assume that the error of the aroximatio obeys for some α 0 ad a costat C A > 0. f f m) C A m α Let us use the basis {ψ i } for the recostructio rocess. Ay vector f F ca be exressed i terms of the basis {ψ i } as f = i= θ iψ i, where θ = {θ i } are the coefficiets of f i this basis. Let T deote the trasform that mas coefficiets to fuctios, so that f = T θ ad defie Θ = {θ : T θ B, θ i quatized to levels} to be the set of cadidate solutios i the basis {ψ i }. The ealty term
11 cf) writte i terms of the basis {ψ i } is cf) = cθ) = + ) log) i= I θ i 0 = + ) log θ 0. It is easily verified that f F cf) by otig that each θ Θ ca be uiquely ecoded via a refix code cosistig of + ) log bits er ozero coefficiet log bits for the locatios ad log bits for the quatized values) i which case the codelegths cf) must satisfy the Kraft iequality [6. The oracle iequality [ E f k f { f f C mi + f F } cf) log + 4 ca be writte i terms of the ew class of cadidate recostructios as [ E f k f { θ θ } cθ) log + 4 C mi + θ Θ where f = T θ. For each iteger m, let θ m) deote the coefficiets corresodig to the best mterm aroximatio of f ad let θ m) q deote the earest elemet i Θ. The maximum ossible dyamic rage for the coefficiet magitudes, ± B, is quatized to levels, givig θ q m) θ m) 4B / = C Q /. Thus, θ m) q θ = θ m) q θ m) + θ m) θ θ m) q θ m) + θ m) q θ m) θ m) θ + θ m) θ C Q CA C Q + m α + C A m α. Now isert θ q m) i lace of θ i the oracle boud, ad ote that cθ q m) ) = + )m log to obtai [ E f { } k f C Q m α CA C Q C mi + θ Θ / + C A m α + )m log log Choosig large eough makes the first two terms egligible. Balacig the third ad fourth terms gives ) ) + ) log α+ k α+ m = ɛc A log so ad sice ) α ) α + ) log C A m α α+ k α+ = C A, ɛc A log k < log k ) α α+
12 whe k > ad > e, [ E f k f ) α k α+ C C, log as claimed i the Theorem, where C = { C A + ) log ɛc A } α ɛ ) α Suose ow that f has oly m ozero coefficiets. I this case, θ m) q θ C Q sice θ m) θ = 0. Now the ealty term domiates i the oracle boud ad [ E f k f ) k C C, m log where { } + ) log + 4 C =. ɛ V. OPTIMIZATION SCHEME Although our otimizatio is ocovex, it does ermit a simle, iterative otimizatio strategy that roduces a sequece of recostructios for which the corresodig sequece of comlexityregularized emirical risk values is odecreasig. This algorithm, which is described below, has demostrated itself to be quite effective i similar deoisig ad recostructio roblems [7 [9. A ossible alterative strategy might etail covexifyig the roblem by relacig the l 0 ealty with a l ealty. Recet results show that ofte the solutio to this covex roblem coicides with or aroximates the solutio to the origial ocovex roblem [0. Let us assume that we wish to recostruct our sigal i terms of the basis {ψ i }. Usig the defiitios itroduced i the revious sectio, the recostructio is equivalet to f k = T θ k where f k = arg mi f F θ k = arg mi θ Θ { Rf) + { RT θ) + Thus, the otimizatio roblem ca the be writte as { y P T θ + θ k = arg mi θ Θ } cf) log } cθ) log } log) log) θ 0 ɛ
13 3 where P = Φ T, the trasose of the k rojectio matrix Φ, y is a colum vector of the k observatios, ad θ 0 = i= I {θ i 0}. To solve this, we use a iterative boudotimizatio rocedure, as roosed i [7 [9. This rocedure etails a twoste iterative rocess that begis with a iitializatio θ 0) ad comutes:. ϕ t) = θ t) + λ P T )T y P T θ t) ). θt+) i = ϕ t) i if ϕ t) log) log) i λɛ 0 otherwise where λ is the largest eigevalue of P P. This rocedure is desirable sice the secod ste, i which the comlexity term lays its role, ivolves a simle coordiatewise thresholdig oeratio. It is easy to verify that the iteratios roduce a mootoically oicreasig sequece of comlexityregularized emirical risk values [9. Thus, this rocedure rovides a simle iteratio that teds to miimize the origial objective fuctio, ad aears to give good results i ractice [7. The iteratios ca be termiated whe the etries uiformly satisfy θ t+) i θ t) i δ, for a small ositive tolerace δ. The comutatioal comlexity of the above rocedure is quite aealig. Each iteratio requires oly Ok) oeratios, assumig that the trasform T ca be comuted i O) oeratios. For examle, the discrete wavelet of Fourier trasforms ca be comuted i O) ad O log ) oeratios, resectively. Multilicatio by P is the most itesive oeratio, requirig Ok) oeratios. The thresholdig ste is carried out ideedetly i each coordiate, ad this ste requires O) oeratios as well. Of course, the umber of iteratios required is roblem deedet ad difficult to redict, but i our exeriece i this alicatio ad others [7, [9 algorithms of this sort ted to coverge i a reasoably small umber of iteratios, eve i very high dimesioal cases. Oe oit worthy of metio relates to the factor /ɛ = B +σ) i the ealty. As is ofte the case with coservative bouds of this tye, the theoretical ealty is larger tha what is eeded i ractice to achieve good results. Also, a otetial hurdle to calibratig the algorithm is that it deeds o kowledge of B ad σ, either of which may be kow a riori. Strictly seakig, these values do ot eed to be kow ideedetly but rather we eed oly estimate B +σ). To that ed, otice that each observatio is a radom variable with variace equal to f / + σ. Let B = f /, which is the miimum B satisfyig the stated boud f B. The the variace of each observatio is B + σ. Further, it is easy to verify that B +σ ) B +σ). So, a scheme could be develoed whereby the samle variace is used as a surrogate for the ukow quatities i the form i which they aear i the arameter ɛ.
14 4 This would etail usig aother cocetratio iequality to cotrol the error betwee the samle variace ad its mea value, ad roagatig this additioal error through the derivatio of the oracle iequality. While this is relatively straightforward, we omit a comlete derivatio here. To illustrate the erformace of the algorithm above, i Figure we cosider three stadard test sigals, each of legth = Rademacher rojectio samles cotamiated with additive white Gaussia oise) are take for the Blocks, Bums, ad Doler test sigals. The algorithm described above is emloyed for recostructio, with oe slight modificatio. Sice the theoretical ealty ca be a bit too coservative i ractice, the threshold used i this examle is /3 of the theoretical value i.e., a threshold of log) log)/λɛ)/3 was used). The SNR, defied as SNR = 0 log 0 B σ where B = f /, is db for each test sigal. To covey a sese of the oise level, colum a) of Figure shows the origial sigals cotamiated with the same level of oise i.e., the sigal resultig from covetioal oit samles cotamiated with oise of the same ower). Colum b) shows the recostructios obtaied from 600 rojectios; recostructios from 00 rojectios are show i colum c). The Blocks sigal to row) was recostructed usig the Haar wavelet basis Daubechies), wellsuited to the iecewise costat ature of the sigal. The Bums ad Doler sigals middle ad bottom row, resectively) were recostructed usig the Daubechies6 wavelet basis. Of course, the selectio of the best recostructio basis is a searate matter that is beyod the scoe of this aer. VI. APPLICATIONS Oe immediate alicatio of the results ad methods above is to sigal codig ad A/D coversio. I the oiseless settig, several authors have suggested the use of radom rojectio samlig for such uroses [ [3. Our results idicate how such schemes might erform i the resece of oise. Suose that we have a array of sesors, each makig a oisy measuremet. The oise could be due to the sesors themselves or evirometal factors. The goal of ecodig ad A/D coversio is to rereset the sesor readigs i a comressed form, suitable for digital storage or trasmissio. Our results suggest that k radom Rademacher rojectios of the sesor readigs ca be used for this urose, ad the error bouds suggest guidelies for how may rojectios might be required for a certai level of recisio. Our theory ad method ca also be alied to wireless sesig as follows. Cosider the roblem of sesig a distributed field e.g., temerature, light, chemical) usig a collectio of wireless sesors distributed uiformly over a regio of iterest. Such systems are ofte referred to as sesor etworks. The goal is to obtai a accurate, highresolutio recostructio of the field at a remote destiatio. Oe
15 5 a) b) c) Fig.. Simulatio examles usig Blocks, Bums, ad Doler test sigals of legth Colum a) shows the origial sigals with a equivalet level of additive erixel oise. Colums b) ad c) show recostructios from 600 ad 00 rojectios, resectively. aroach to this roblem is to require each sesor to digitally trasmit its measuremet to the destiatio, where field recostructio is the erformed. Alteratively, the sesors might collaboratively rocess their measuremets to recostruct the field themselves ad the trasmit the result to the destiatio i.e., the odes collaborate to comress their date rior to trasmissio). Both aroaches ose sigificat demads o commuicatio resources ad ifrastructure, ad it has recetly bee suggested that ocollaborative aalog commuicatio schemes offer a more resourceefficiet alterative [ [3. Assume that the sesor data is to be trasmitted to the destiatio over a additive white Gaussia oise chael. Suose the destiatio broadcasts erhas digitally) a radom seed to the sesors.
16 6 Each ode modifies this seed i a uique way kow to oly itself ad the destiatio e.g., this seed could be multilied by the ode s address or geograhic ositio). Each ode geerates a seudoradom Rademacher sequece, which ca also be costructed at the destiatio. The the odes trasmit the radom rojectios to the destiatio hasecoheretly i.e., beamformig). This is accomlished by requirig each ode to simly multily its readig by a elemet of its radom sequece i each rojectio/commuicatio ste ad trasmit the result to the destiatio via amlitude modulatio. If the trasmissios from all sesors ca be sychroized so that they all arrive i hase at the destiatio, the the averagig iheret i the multile access chael comutes the desired ier roduct. After receivig k rojectios, the destiatio ca emloy the recostructio algorithm above usig a basis of choice e.g., wavelet). The commuicatios rocedure is comletely oadative ad otetially very simle to imlemet. The collective fuctioig of the wireless sesors i this rocess is more aki to a esemble of hasecoheret emitters tha it is to covetioal etworkig oeratios. Therefore, we refer the term sesor esemble istead of sesor etwork i this cotext. A remarkable asect of the sesor esemble aroach is that the ower required to achieve a target distortio level ca be very miimal. Let σ s ad σ c deote the oise variace due to sesig ad commuicatio, resectively. Thus, each rojectio received at the destiatio is corruted by a oise of total ower σ s + σ c. The sesig oise variace is assumed to be a costat ad the additioal variace due to the commuicatio chael is assumed to scale like the iverse of the total received ower σ c P where P is the trasmit ower er sesor. Note that although the total trasmit ower is P, the received ower is a factor of larger as a result of the ower amlificatio effect of the hasecoheret trasmissios [3. I order to achieve rates of distortio decay that we claim, it is sufficiet that the variace due to the commuicatio chael behaves like a costat. Therefore, we require oly that P. This results i a rather surrisig coclusio. Ideal recostructio is ossible at the destiatio with total trasmit ower P tedig to zero as the desity of sesors icreases. If covetioal satial oit samles were take istead e.g., if a sigle sesor is selected at radom i each ste ad trasmits its measuremet to the destiatio), the the ower required er samle would be a costat, sice oly oe sesor would be ivolved i such a trasmissio. Thus, it aears that radom rojectio samlig could be much more desirable i wireless sesig alicatios.
17 7 VII. CONCLUSIONS AND FUTURE WORK We have show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. The squared error bouds for comressible sigals are Ok/ log ) α α+ ), which is withi a logarithmic factor of the usual oarametric estimatio rate, ad Ok/ log ) ) for sarse sigals. We also roosed a ractical iterative algorithm for sigal recostructio. Oe of the most romisig otetial alicatios of our theory ad method is to wireless sesig, wherei oe realizes a large trasmissio ower gai by radom rojectio samlig as oosed to covetioal satial oit samlig. The role of the oise variace i the rates we reseted is worthy of further attetio. As the oise variace teds to zero, oe ituitively exects to attai the fast rates that are kow to be achievable i the oiseless settig. Our theory is based i the oisy regime ad it does ot directly imly the reviously established bouds i the oiseless settig. Simly ut, our aalysis assumes a oise variace strictly greater tha zero. Let us commet briefly o the tightess of the uer bouds give by our theory. I aalogy with classical oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). Moreover, i the secial case of sarse sigals with m ozero terms, we obtai a error boud of m/k igorig costat ad logarithmic factors). Stadard arametric statistical aalysis suggests that oe should ot exect a rate of better tha m/k degreesoffreedom/ samlesize) i such cases, which agai suorts our ituitio regardig the tightess of the bouds i terms of the covergece rate). However, to our kowledge exlicit miimax lower bouds have ot bee established i the cotext of this roblem, ad the determiatio of such bouds is oe of our future research directios. Although we cosidered oly the case of Gaussia oise i the observatio model ), the same results could be achieved for ay zeromea, symmetrically distributed oise that is ideedet of the rojectio vector elemets ad satisfies E[w k j k)!!varw j )) k, a result that follows immediately usig the lemmas reseted i the Aedix. Aother extesio would be the cosideratio of other radom rojectios istead of the Rademacher rojectios cosidered here. Most of our basic aroach would go through i such cases; oe would oly eed to verify the momet coditios of the CraigBerstei iequality for articular cases.
18 8 VIII. ACKNOWLEDGEMENTS The authors would like to thak Rui Castro for his assistace i all stages of the work reseted here. A. The CraigBerstei Momet Coditio APPENDIX The cetral requiremet of the CraigBerstei iequality is the satisfactio of the momet coditio E [ X E[X!varX)h for itegers with some ositive costat h that does ot deed o, a coditio that is automatically verified for = with ay fiite value of h. For higher owers this coditio ca be very difficult to verify for several reasos, ot the least of which is the absolute value reset i the momets. Previous work that made use of the CraigBerstei iequality assumed that the observatios were bouded, forcig a ossibly urealistic case of bouded oise [3. This assumtio is ot sufficiet for the rates of covergece we claim. Ideed, the aïve boud o the observatios is y i B yieldig a costat h that would grow roortioally to. With that motivatio, we develo a framework uder which a boudig costat h ca be determied more directly. First, observe that the momet coditio is usually easier to verify for the eve owers because the absolute value eed ot be dealt with directly. This is sufficiet to guaratee the momet coditio is satisfied for all itegers, as roved i the followig lemma. Lemma Suose the CraigBerstei momet coditio holds for all eve itegers greater tha or equal to, that is E [ X E[X k k)!varx)hk, k sice the k = case is satisfied trivially for ay h. The the coditio holds also for the odd absolute momets with h = h. Thus A ad B, E [ X E[X k k )!varx) h k 3, k E [ X E[X!varX)h),. Proof: For ease of otatio, let Z = X E[X. Hölder s Iequality states, for ay radom variables E[ AB E[ A / E[ B q /q
19 9 where <, q < ad / + /q =. Take A = Z, B = Z k, ad = q = to get E[ Z k E[Z E[Z 4k 4 where the absolute values iside the square root have bee droed because the exoets are eve. Now by assumtio so E[Z 4k 4 4k 4)!E[Z h 4k 6 E[ Z k We wat to satisfy the followig iequality by choice of h which meas h must satisfy 4k 4)!E[Z ) h 4k 6 4k 4)! E[Z h k 3 E[ Z k k )!E[Z h k 3 If we choose k )! 4k 4)! h k 3 h k 3. h max k 4k 4)! k )! ) k 3 h the the momet coditio will be satisfied for the odd exoets k. A uer boud for the term i brackets is, as show here. For k, the boud k)! k k!) holds ad ca be verified by iductio o k. This imlies ) 4k 4)! k 3 ) k 3. 4) k )! k Now, the term i aretheses o the right had side of 4) is always less tha for k. The fial ste is to show that lim k k ) k 3 =, which is verified by otig that lim log k k ) k 3 = lim log ) ) log k ) = 0. k k 3
20 0 Thus, the momet coditio is satisfied for odd momets with h = h. Also, if the momet coditio is satisfied for a give h, it is also satisfied for ay h h so holds for all itegers as claimed. E [ Z = E [ X E[X!varX)h) We will also eed results for how sums ad roducts of radom variables behave with resect to the momet coditio. For that, we have the followig two lemmas. Lemma Let Z = A + B be the sum of two zeromea radom variables A ad B with variaces vara) = E[A ad varb) = E[B, ot both zero, ad such that E[AB 0. Suose both A ad B satisfy the momet coditio for a give iteger 3 with ositive costats h A ad h B resectively, ot both zero. That is The E [ A!varA)h A ad E [ B!varB)h B. E [ Z!varZ)h S where h S = / mi ) h A + h B ), ad mi is the miimum 3 for which the assumtios are satsified. Proof: First, defie V A = vara)/vara) + varb)), V B = varb)/vara) + varb)), H A = h A /h A + h B ), ad H B = h B /h A + h B ). Use Mikowski s Iequality to write E[ A + B [E[ A / + E[ B / [ ) vara)h ) A + varb)h B! =!vara) + varb)) [ ) V A h A + =!vara) + varb)) h A + h B )!vara + B) h A + h B ) V B h B ) [ ) V A H A + [ ) V A H A + V B H B V B H B ), ) where the last ste follows from the assumtio that E[AB 0, imlyig vara)+varb) vara+ B). Showig that [ ) V A H A + V B H B ) C,
Stéphane Boucheron 1, Olivier Bousquet 2 and Gábor Lugosi 3
ESAIM: Probability ad Statistics URL: http://wwwemathfr/ps/ Will be set by the publisher THEORY OF CLASSIFICATION: A SURVEY OF SOME RECENT ADVANCES Stéphae Bouchero 1, Olivier Bousquet 2 ad Gábor Lugosi
More informationSOME GEOMETRY IN HIGHDIMENSIONAL SPACES
SOME GEOMETRY IN HIGHDIMENSIONAL SPACES MATH 57A. Itroductio Our geometric ituitio is derived from threedimesioal space. Three coordiates suffice. May objects of iterest i aalysis, however, require far
More informationConsistency of Random Forests and Other Averaging Classifiers
Joural of Machie Learig Research 9 (2008) 20152033 Submitted 1/08; Revised 5/08; Published 9/08 Cosistecy of Radom Forests ad Other Averagig Classifiers Gérard Biau LSTA & LPMA Uiversité Pierre et Marie
More informationHOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1
1 HOW MANY TIMES SHOULD YOU SHUFFLE A DECK OF CARDS? 1 Brad Ma Departmet of Mathematics Harvard Uiversity ABSTRACT I this paper a mathematical model of card shufflig is costructed, ad used to determie
More informationWhich Extreme Values Are Really Extreme?
Which Extreme Values Are Really Extreme? JESÚS GONZALO Uiversidad Carlos III de Madrid JOSÉ OLMO Uiversidad Carlos III de Madrid abstract We defie the extreme values of ay radom sample of size from a distributio
More informationHow Has the Literature on Gini s Index Evolved in the Past 80 Years?
How Has the Literature o Gii s Idex Evolved i the Past 80 Years? Kua Xu Departmet of Ecoomics Dalhousie Uiversity Halifax, Nova Scotia Caada B3H 3J5 Jauary 2004 The author started this survey paper whe
More informationType Less, Find More: Fast Autocompletion Search with a Succinct Index
Type Less, Fid More: Fast Autocompletio Search with a Succict Idex Holger Bast MaxPlackIstitut für Iformatik Saarbrücke, Germay bast@mpiif.mpg.de Igmar Weber MaxPlackIstitut für Iformatik Saarbrücke,
More informationEverything You Always Wanted to Know about Copula Modeling but Were Afraid to Ask
Everythig You Always Wated to Kow about Copula Modelig but Were Afraid to Ask Christia Geest ad AeCatherie Favre 2 Abstract: This paper presets a itroductio to iferece for copula models, based o rak methods.
More informationSystemic Risk and Stability in Financial Networks
America Ecoomic Review 2015, 105(2): 564 608 http://dx.doi.org/10.1257/aer.20130456 Systemic Risk ad Stability i Fiacial Networks By Daro Acemoglu, Asuma Ozdaglar, ad Alireza TahbazSalehi * This paper
More informationNo Eigenvalues Outside the Support of the Limiting Spectral Distribution of Large Dimensional Sample Covariance Matrices
No igevalues Outside the Support of the Limitig Spectral Distributio of Large Dimesioal Sample Covariace Matrices By Z.D. Bai ad Jack W. Silverstei 2 Natioal Uiversity of Sigapore ad North Carolia State
More informationThe Unicorn, The Normal Curve, and Other Improbable Creatures
Psychological Bulleti 1989, Vol. 105. No.1, 156166 The Uicor, The Normal Curve, ad Other Improbable Creatures Theodore Micceri 1 Departmet of Educatioal Leadership Uiversity of South Florida A ivestigatio
More informationCrowds: Anonymity for Web Transactions
Crowds: Aoymity for Web Trasactios Michael K. Reiter ad Aviel D. Rubi AT&T Labs Research I this paper we itroduce a system called Crowds for protectig users aoymity o the worldwideweb. Crowds, amed for
More informationJ. J. Kennedy, 1 N. A. Rayner, 1 R. O. Smith, 2 D. E. Parker, 1 and M. Saunby 1. 1. Introduction
Reassessig biases ad other ucertaities i seasurface temperature observatios measured i situ sice 85, part : measuremet ad samplig ucertaities J. J. Keedy, N. A. Rayer, R. O. Smith, D. E. Parker, ad M.
More informationON THE EVOLUTION OF RANDOM GRAPHS by P. ERDŐS and A. RÉNYI. Introduction
ON THE EVOLUTION OF RANDOM GRAPHS by P. ERDŐS ad A. RÉNYI Itroductio Dedicated to Professor P. Turá at his 50th birthday. Our aim is to study the probable structure of a radom graph r N which has give
More informationDryad: Distributed DataParallel Programs from Sequential Building Blocks
Dryad: Distributed DataParallel Programs from Sequetial uildig locks Michael Isard Microsoft esearch, Silico Valley drew irrell Microsoft esearch, Silico Valley Mihai udiu Microsoft esearch, Silico Valley
More information4. Trees. 4.1 Basics. Definition: A graph having no cycles is said to be acyclic. A forest is an acyclic graph.
4. Trees Oe of the importat classes of graphs is the trees. The importace of trees is evidet from their applicatios i various areas, especially theoretical computer sciece ad molecular evolutio. 4.1 Basics
More informationBOUNDED GAPS BETWEEN PRIMES
BOUNDED GAPS BETWEEN PRIMES ANDREW GRANVILLE Abstract. Recetly, Yitag Zhag proved the existece of a fiite boud B such that there are ifiitely may pairs p, p of cosecutive primes for which p p B. This ca
More informationWhich Codes Have CycleFree Tanner Graphs?
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 45, NO. 6, SEPTEMBER 1999 173 Which Coes Have CycleFree Taer Graphs? Tuvi Etzio, Seior Member, IEEE, Ari Trachteberg, Stuet Member, IEEE, a Alexaer Vary,
More informationA Kernel TwoSample Test
Joural of Machie Learig Research 3 0) 73773 Subitted 4/08; Revised /; Published 3/ Arthur Gretto MPI for Itelliget Systes Speastrasse 38 7076 Tübige, Geray A Kerel TwoSaple Test Karste M. Borgwardt Machie
More informationTesting for Welfare Comparisons when Populations Differ in Size
Cahier de recherche/workig Paper 039 Testig for Welfare Comparisos whe Populatios Differ i Size JeaYves Duclos Agès Zabsoré Septembre/September 200 Duclos: Départemet d écoomique, PEP ad CIRPÉE, Uiversité
More informationTurning Brownfields into Greenspaces: Examining Incentives and Barriers to Revitalization
Turig Browfields ito Greespaces: Examiig Icetives ad Barriers to Revitalizatio Juha Siikamäki Resources for the Future Kris Werstedt Virgiia Tech Uiversity Abstract This study employs iterviews, documet
More informationTeaching Bayesian Reasoning in Less Than Two Hours
Joural of Experimetal Psychology: Geeral 21, Vol., No. 3, 4 Copyright 21 by the America Psychological Associatio, Ic. 963445/1/S5. DOI: 1.7//963445..3. Teachig Bayesia Reasoig i Less Tha Two Hours Peter
More informationSpinout Companies. A Researcher s Guide
Spiout Compaies A Researcher s Guide Cotets Itroductio 2 Sectio 1 Why create a spiout compay? 4 Sectio 2 Itellectual Property 10 Sectio 3 Compay Structure 15 Sectio 4 Shareholders ad Directors 19 Sectio
More informationare new doctors safe to practise?
Be prepared: are ew doctors safe to practise? Cotets What we foud 02 Why we ve writte this report 04 What is preparedess ad how ca it be measured? 06 How well prepared are medical graduates? 08 How has
More informationThe Arithmetic of Investment Expenses
Fiacial Aalysts Joural Volume 69 Number 2 2013 CFA Istitute The Arithmetic of Ivestmet Expeses William F. Sharpe Recet regulatory chages have brought a reewed focus o the impact of ivestmet expeses o ivestors
More informationLeadership Can Be Learned, But How Is It Measured?
Maagemet Scieces for Health NO. 8 (2008) O C C A S I O N A L PA P E R S Leadership Ca Be Leared, But How Is It Measured? How does leadership developmet cotribute to measurable chages i orgaizatioal performace,
More informationAdverse Health Care Events Reporting System: What have we learned?
Adverse Health Care Evets Reportig System: What have we leared? 5year REVIEW Jauary 2009 For More Iformatio: Miesota Departmet of Health Divisio of Health Policy P.O. Box 64882 85 East Seveth Place, Suite
More informationA New TwIST: TwoStep Iterative Shrinkage/Thresholding Algorithms for Image Restoration
SUBMITTED FOR PUBLICATION; 2007. 1 A Ne TIST: ToSte Iterative Shrinkage/Thresholding Algorithms for Image Restoration José M. BioucasDias, Member, IEEE, and Mário A. T. Figueiredo, Senior Member, IEEE
More informationCatalogue no. 62557XPB Your Guide to the Consumer Price Index
Catalogue o. 62557XPB Your Guide to the Cosumer Price Idex (Texte fraçais au verso) Statistics Caada Statistique Caada Data i may forms Statistics Caada dissemiates data i a variety of forms. I additio
More informationThe Review of Economic Studies Ltd.
The Review of Ecoomic Studies Ltd. Walras' Tâtoemet i the Theory of Exchage Author(s): H. Uzawa Source: The Review of Ecoomic Studies, Vol. 27, No. 3 (Ju., 1960), pp. 182194 Published by: The Review of
More information