Signal Reconstruction from Noisy Random Projections

Size: px
Start display at page:

Download "Signal Reconstruction from Noisy Random Projections"

Transcription

1 Sigal Recostructio from Noisy Radom Projectios Jarvis Haut ad Robert Nowak Deartmet of Electrical ad Comuter Egieerig Uiversity of Wiscosi-Madiso March, 005; Revised February, 006 Abstract Recet results show that a relatively small umber of radom rojectios of a sigal ca cotai most of its saliet iformatio. It follows that if a sigal is comressible i some orthoormal basis, the a very accurate recostructio ca be obtaied from radom rojectios. We exted this tye of result to show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. We also roose a ractical iterative algorithm for sigal recostructio, ad briefly discuss otetial alicatios to codig, A/D coversio, ad remote wireless sesig. Idex Terms samlig, sigal recostructio, radom rojectios, deoisig, wireless sesor etworks I. INTRODUCTION Recet theory iforms us that, with high robability, a relatively small umber of radom rojectios of a sigal ca cotai most of its relevat iformatio. For examle, the groudbreakig work i [ has show that k radom Fourier rojectios cotai eough iformatio to recostruct iecewise smooth sigals at a distortio level early equivalet to that attaiable from k otimally selected observatios. Similar results hold for radom Gaussia ad Rademacher rojectios i.e., rojectios cosistig of ideedet ad idetically distributed Gaussia or Rademacher radom variables) [, [3. The results reseted i these works ca be roughly summarized as follows. Assume that a sigal f R is comressible i some orthoormal basis i the followig sese. Let f m) deote the best m-term This research was suorted i art by the NSF uder grats CCR ad CCR-03557, ad by the Office of Naval Research, grat N

2 aroximatio of f i terms of this basis ad suose that the average squared error obeys f f m) = f i f m) i ) C A m α for some α 0 ad some costat C A i= > 0. The arameter α govers the degree to which f is comressible with resect to the basis. I a oiseless settig, it ca be show that a aroximatio of such a sigal ca be recovered from k radom rojectios with a average squared error that is uer bouded by a costat times k/ log ) α, early as good as the best k-term aroximatio error. This aer takes this lie of iquiry a ste further by cosiderig the erformace of samlig via radom rojectios i oisy coditios. We show that if the rojectios are cotamiated with zero-mea Gaussia oise, the comressible sigals ca be recostructed with a exected average squared error that is uer bouded by a costat times k/ log ) α α+. For truly sarse sigals with oly a small umber of o-zero terms) a stroger result is obtaied; the exected average squared recostructio error is uer bouded by a costat times k/ log ). These bouds demostrate a remarkable caability of samlig via radom rojectios accurate recostructios ca be obtaied eve whe the sigal dimesio greatly exceeds the umber of samles k ad the samles themselves are cotamiated with sigificat levels of oise. This effect is highlighted by the followig eedle i a haystack roblem. Suose the sigal f is a vector of legth with oe ozero etry of amlitude. If we samle the vector at k radom locatios aki to covetioal samlig schemes), the the robability of comletely missig the o-zero etry is /) k, which is very close to whe k is sigificatly smaller tha. This imlies that the exected average squared error may be almost, or larger if oise is reset. O the other had, by samlig with radomized rojectios our results guaratee that the exected average squared error will be o larger tha a costat times k/ log ), which ca be arbitrarily close to 0 eve whe k, rovided k > log. A closely related roblem is the recostructio of sigals with sarse Fourier sectra from a relatively small umber of o-uiform time samles e.g., radom samles i time) [4 [7. Most of this work cocers oiseless situatios, but [5 addresses the roblem of recostructio from oise-corruted samles. Aother area of work related to our results cocers the recostructio of sigals with fiite degrees of freedom usig a small umber of o-traditioal samles [8, [9. A secial istace of this setu is the case of sigals that are sarse i time the dual of the sectrally sarse case). Recostructio from oise-corruted samles is the focus of [9. I a sese, the samlig ad recostructio roblems addressed i the aers above are secial cases of the class of roblems cosidered here, where we allow sigals that

3 3 are sarse i ay domai. Agai, this more uiversal ersective is recisely the focus of [, [3, which cosider sigal recostructio from oiseless radom rojectios. A iterestig lie of similar work cocers the related roblem of sigal recostructio from radom rojectios corruted by a ukow but bouded erturbatio [0, [. Here we cosider ubouded, Gaussia oise cotamiatio i the samlig rocess. Fially, while this aer was uder review a related ivestigatio was reorted i [ ertaiig to the statistical estimatio of sarse sigals from uderdetermied ad oisy observatios. That work develos quatitave bouds for sarse sigal recostructio similar to ours, but is based o a uiform ucertaity ricile rather tha radomized desigs as here. The aer is orgaized as follows. I Sectio II we state the basic roblem ad mai theoretical results of the aer. I Sectio III we derive bouds o the accuracy of sigal recostructios from oisy radom rojectios. I Sectio IV we secialize the bouds to cases i which the uderlyig sigal is comressible i terms of a certai orthoormal basis. I Sectio V we roose a simle iterative algorithm for sigal recostructio. I Sectio VI we discuss alicatios to ecodig, A/D coversio, ad wireless sesig, ad we make cocludig remarks i Sectio VII. Detailed derivatios are relegated to the Aedix. II. MAIN RESULTS Cosider a vector f = [f f... f T R ad assume that i= f i ) f B for a kow costat B > 0. The assumtio simly imlies that the average er elemet eergy is bouded by a costat. This is a fairly weak restrictio, sice it ermits a very large class of sigals, icludig sigals with eak magitudes as large as O ). Now suose that we are able to make k measuremets of f i the form of oisy, radom rojectios. Secifically, let Φ = {φ i,j } be a k array of bouded, i.i.d. zero-mea radom variables of variace E[φ i,j = /. Samles take the form y j = φ i,j fi + w j, j =,..., k ) i= where w = {w j } are i.i.d. zero-mea radom variables, ideedet of {φ i,j }, with variace σ. The goal is to recover a estimate of f from these observatios. Defie the risk of a cadidate recostructio f to be Rf) = f f + σ where the orm is the Euclidea distace. Next assume that both {φ i,j } ad {y j } are available. The we

4 4 ca comute the emirical risk Rf) = k ) k y j φ i,j f i. j= i= It is easy to verify that E[ Rf) = Rf) usig the facts that {φ i,j } ad {w j } are ideedet radom variables ad E[φ i,j = /. Thus, Rf) is a ubiased estimator of Rf). We will use the emirical risk to obtai a estimator f of f, ad boud the resultig error E[ f f. The estimator is based o a comlexity-regularized emirical risk miimizatio, ad we use the Craig-Berstei cocetratio iequality to cotrol the estimatio error of the recostructio rocess. That iequality etails the verificatio of certai momet coditios, which deed o the ature of Φ ad w. Therefore, i this aer we focus o ormalized) Rademacher rojectios, i which case each φ i,j is ±/ with equal robability, ad assume that w is a sequece of zero-mea Gaussia oises. Geeralizatios to other radom rojectios ad oise models may be ossible followig our aroach; this would oly require oe to verify the momet coditios required by the Craig-Berstei iequality. Suose that we have a coutable collectio F of cadidate recostructio fuctios ad a o-egative umber cf) assiged to each f F such that f F cf). Furthermore, assume that each f F satisfies f B. Select a recostructio accordig to the comlexity-regularized emirical risk miimizatio f k { = arg mi Rf) + f F } cf) log where ɛ > 0 is a costat that deeds o B ad σ. The we have the followig oracle iequality. Theorem Let ɛ = /B + σ) ), the [ E f k f { f f C mi f F + } cf) log + 4, where the costat C is give by with S = B/σ, the sigal-to-oise ratio. C = 9S )S + 3 S )S + A imortat oit regardig the costats above is that they deed oly o σ ad B, the oise ower ad the average sigal ower, resectively. If f is comressible with resect to a certai orthoormal basis, the we ca obtai exlicit bouds o the error i terms of the umber of radom rojectios k ad the degree to which f is comressible.

5 5 Let f m) deote the best m-term aroximatio of f i the basis. That is, if f has a reresetatio f = i= θ iψ i i the basis {ψ i }, the f m) = m i= θ i)ψ i), where coefficiets ad basis fuctios are re-ordered such that θ ) θ ) θ ). Assume that the average squared error f f m) / i= f i f m) i ) satisfies for some α 0 ad some costat C A f f m) C A m α > 0. Power-law decays like this arise quite commoly i alicatios. For examle, smooth ad iecewise smooth sigals ad sigals of bouded variatio exhibit this sort of behavior [, [3. It is also uecessary to restrict oe s attetio orthoormal basis exasios. Much more geeral aroximatio strategies ca be accomodated [3, but to kee the resetatio as simle as ossible we will ot delve further ito such extesios. Now let us take F to be a suitably quatized collectio of fuctios, rereseted i terms of the basis {ψ i } the costructio of F is discussed i Sectio IV). We have the followig error boud. Theorem If cf) = log) {# o-zero coefficiets of f} the there exists a costat C = C B, σ, C A ) > 0 such that [ E f k f ) k α/α+) C C, log where C is as give i Theorem. Note that the exoet α/α + ) is the usual exoet goverig the rate of covergece i oarametric fuctio estimatio. A stroger result is obtaied if the sigal is sarse, as stated i the followig Corollary. Corollary Suose that f has at most m ozero coefficiets. The there exists a costat C = C B, σ) > 0 such that [ E f k f C C ) k, m log where C is as give i Theorem.

6 6 Similar results hold if the sigal is additioally cotamiated with oise rior to the radom rojectio rocess. Corollary Suose observatios take the form y j = φ i,j fi + η i ) + w j, i= j =,..., k where {η i } are i.i.d. zero-mea Gaussia radom variables with variace σ s that are ideedet of {φ i,j } ad {w j }. The Theorems ad ad Corollary hold with slightly differet costats C, C, C, ad ɛ. It is imortat to oit out that all the results above hold for arbitrary sigal legth, ad the costats do ot deed o the size of the roblem. The fact that the rate deeds oly logarithmically o is sigificat ad illustrates the scalability of this aroach. Oe ca iterret these bouds as good idicators of the excetioal erformace of radom rojectio samlig i large- regimes. The deedece o k is show to be olyomial. I aalogy with oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). To drive this oit home, let us agai cosider the eedle i a haystack roblem, this time i a bit more detail. Suose the sigal f is a vector of legth with oe ozero etry of amlitude such that f / =. First, cosider radom satial oit samlig where observatios are oisefree i.e., each samle is of the form y j = f t j ), where t j is selected uiformly at radom from the set {,..., }). The squared recostructio error is 0 if the sike is located ad otherwise, ad the robability of ot fidig the sike i k trials is /) k, givig a average squared error of /) k + k/) 0 = /) k. If is large, we ca aroximate this by /) k e k/, which is very close to whe k is sigificatly smaller tha. O the other had, radomized Rademacher rojectios corruted with oise) yield a average squared recostructio error boud of C k/ log ), as give above i Corollary. This boud may be arbibrarily close to 0 eve whe k, rovided k > log. This shows that eve give the advatage of beig oiseless, the recostructio error from satial oit samlig may be far greater tha that resultig from radom rojectios. III. ORACLE INEQUALITY I this sectio we rove Theorem. For ease of otatio, we adot the shorthad otatio φ j = [φ,j φ,j... φ,j T for the vector corresodig to the j th rojectio. The emirical risk of a vector f

7 7 ca ow be writte as Rf) = k k y j φ T j f). j= We will boud rf, f ) Rf) Rf ), the excess risk betwee a cadidate recostructio f ad the actual fuctio f, usig the comlexity-regularizatio method itroduced i [3. Note that rf, f ) = E [ y j φ T j f y j φ T j f) + φ T j f) φ T j f ) = E [ φ T j f φ T j f ) = f f. Defie the emirical excess risk rf, f ) Rf) Rf ). The rf, f ) = k = k k [ yj φ T j f ) y j φ T j f) j= k j= U j where U j = [y j φ T j f ) y j φ T j f) are i.i.d. Notice that rf, f ) rf, f ) = k k j= U j E [U j ). We will make use of the Craig-Berstei iequality [4, which states that P k U j E [U j ) t ɛ k var k k + k j= j) U e t ζ) j= for 0 < ɛh ζ < ad t > 0, rovided the variables U j satisfy the momet coditio E [ U j E[U j k k! varu j) h k for some h > 0 ad all itegers k. If we cosider vectors f ad estimates f where f B ad f B, Rademacher rojectios, ad Gaussia oises with variace σ, the the momet coditio is satisfied with h = 6B +8 Bσ, as show i the Aedix. Alterative forms of radom rojectios ad oises ca also be hadled usig the aroach outlied ext, rovided the momet coditios are satisfied. To use the Craig-Berstei iequality we also eed a boud o the variace of U j itself. Defiig g = f f, we have ) g varu j ) = E[φ T g) 4 + 4σ g. As show i the Aedix, for itegers k ) g k ) g E[φ T g) k k k)!!,

8 8 where k)!! )3)... k ). Thus we ca boud the variace of U j by ) varu j ) g g + 4σ. Sice g satisfies g 4B ad rf, f ) = f f / = g /, the boud becomes varu j ) 8B + 4σ ) rf, f ). So, we ca relace the term i the Craig-Berstei iequality that deeds o the variace by k var k U j = k varu j ) 8B + 4σ ) rf, f ). k k j= For a give fuctio f, we have P rf, f ) rf, f ) > t 8B + + 4σ ) ) ɛrf, f ) < e t ζ) or, by lettig δ = e t P j= rf, f ) rf, f ) > log δ ) 8B + 4σ ) ) ɛrf, f ) + < δ. ζ) Now assig to each f F a o-egative ealty term cf) such that the ealties satisfy the Kraft Iequality [5 cf) f F ad let δf) = cf) δ. The by alyig the uio boud we have for all f F ad for all δ > 0 rf, f ) rf, f ) cf) log + log ) δ 8B + 4σ ) ɛrf, f ) + ζ) with robability at least δ. Now set ζ = ɛh, defie 8B + 4σ ) ɛ a, ζ) ad choose ɛ < 0B + 8 Bσ + σ. Notice that a < ad ζ < by choice of ɛ. The a)rf, f ) rf, f ) + cf) log + log ) δ holds with robability at least δ for all f F ad ay δ > 0.

9 9 For the give traiig samles, we ca miimize the uer boud by choosig { } f k = arg mi rf, f cf) log ) + f F which is equivalet to { f k = arg mi Rf) + f F f k arg mi f F } cf) log sice we ca igore Rf ) whe erformig the otimizatio. If we defie { } Rf) + cf) log the with robability at least δ a)r f k, f ) r f k, f ) + c f k ) log + log ) δ rf k, f ) + cf k ) log + log ) δ sice f k miimizes the comlexity-regularized emirical risk criterio. Usig the Craig-Berstei iequality agai to boud rf k, f ) rf k, f ) with the same variace boud as before) we get that with robability at least δ rf k, f ) rfk, f ) a rfk, f ) + log ) δ. 3) We wat both ) ad 3) to hold simultaeously, so we use the uio boud to obtai ) r f + a k, f ) rfk a, f ) + cf k ) log + log )) δ a holdig with robability at least δ. Let δ = e t a)/ to obtai P r f k, f ) + a a Itegratig this relatio gives [ E r f k, f ) ) rfk, f ) cf k ) log ) a) t e t a)/. ) + a a rfk, f ) + cf k ) log + 4. a) )

10 0 Now, sice a is ositive, [ E f k f [ = E r f k, f ) where C = + a)/ a). = + a a + a a + a a ) + a mi a f F { f f + = C mi f F ) rfk, f ) + cf k ) log + 4 a) ) rfk, f ) + + a) cf k ) log + 4 a) ) { Rfk ) Rf ) + cf k ) log + 4 { Rf) Rf ) + Tyical values of C ca be determied by aroximatig the costat ɛ < 0B + 8 Bσ + σ. cf) log + 4 } } cf) log + 4 }, Uer boudig the deomiator guaratees that the coditio is satisfied, so let ɛ = / B + σ) ). Now 8B + 4σ ) ɛ a = = ζ) If we deote the sigal to oise ratio by S = B /σ the 4B + σ 5B )Bσ + σ. 4S + a = 5S )S + for which the extremes are a mi = / ad a max = 4/5, givig costats C i the rage of [3/9, 9. IV. ERROR BOUNDS FOR COMPRESSIBLE SIGNALS I this sectio we rove Theorem ad Corollary. Suose that f is comressible i a certai orthoormal basis {ψ i } i=. Secifically, let f m) deote the best m-term aroximatio of f i terms of {ψ i }, ad assume that the error of the aroximatio obeys for some α 0 ad a costat C A > 0. f f m) C A m α Let us use the basis {ψ i } for the recostructio rocess. Ay vector f F ca be exressed i terms of the basis {ψ i } as f = i= θ iψ i, where θ = {θ i } are the coefficiets of f i this basis. Let T deote the trasform that mas coefficiets to fuctios, so that f = T θ ad defie Θ = {θ : T θ B, θ i quatized to levels} to be the set of cadidate solutios i the basis {ψ i }. The ealty term

11 cf) writte i terms of the basis {ψ i } is cf) = cθ) = + ) log) i= I θ i 0 = + ) log θ 0. It is easily verified that f F cf) by otig that each θ Θ ca be uiquely ecoded via a refix code cosistig of + ) log bits er o-zero coefficiet log bits for the locatios ad log bits for the quatized values) i which case the codelegths cf) must satisfy the Kraft iequality [6. The oracle iequality [ E f k f { f f C mi + f F } cf) log + 4 ca be writte i terms of the ew class of cadidate recostructios as [ E f k f { θ θ } cθ) log + 4 C mi + θ Θ where f = T θ. For each iteger m, let θ m) deote the coefficiets corresodig to the best m-term aroximatio of f ad let θ m) q deote the earest elemet i Θ. The maximum ossible dyamic rage for the coefficiet magitudes, ± B, is quatized to levels, givig θ q m) θ m) 4B / = C Q /. Thus, θ m) q θ = θ m) q θ m) + θ m) θ θ m) q θ m) + θ m) q θ m) θ m) θ + θ m) θ C Q CA C Q + m α + C A m α. Now isert θ q m) i lace of θ i the oracle boud, ad ote that cθ q m) ) = + )m log to obtai [ E f { } k f C Q m α CA C Q C mi + θ Θ / + C A m α + )m log log Choosig large eough makes the first two terms egligible. Balacig the third ad fourth terms gives ) ) + ) log α+ k α+ m = ɛc A log so ad sice ) α ) α + ) log C A m α α+ k α+ = C A, ɛc A log k < log k ) α α+

12 whe k > ad > e, [ E f k f ) α k α+ C C, log as claimed i the Theorem, where C = { C A + ) log ɛc A } α ɛ ) α Suose ow that f has oly m ozero coefficiets. I this case, θ m) q θ C Q sice θ m) θ = 0. Now the ealty term domiates i the oracle boud ad [ E f k f ) k C C, m log where { } + ) log + 4 C =. ɛ V. OPTIMIZATION SCHEME Although our otimizatio is o-covex, it does ermit a simle, iterative otimizatio strategy that roduces a sequece of recostructios for which the corresodig sequece of comlexity-regularized emirical risk values is o-decreasig. This algorithm, which is described below, has demostrated itself to be quite effective i similar deoisig ad recostructio roblems [7 [9. A ossible alterative strategy might etail covexifyig the roblem by relacig the l 0 ealty with a l ealty. Recet results show that ofte the solutio to this covex roblem coicides with or aroximates the solutio to the origial o-covex roblem [0. Let us assume that we wish to recostruct our sigal i terms of the basis {ψ i }. Usig the defiitios itroduced i the revious sectio, the recostructio is equivalet to f k = T θ k where f k = arg mi f F θ k = arg mi θ Θ { Rf) + { RT θ) + Thus, the otimizatio roblem ca the be writte as { y P T θ + θ k = arg mi θ Θ } cf) log } cθ) log } log) log) θ 0 ɛ

13 3 where P = Φ T, the trasose of the k rojectio matrix Φ, y is a colum vector of the k observatios, ad θ 0 = i= I {θ i 0}. To solve this, we use a iterative boud-otimizatio rocedure, as roosed i [7 [9. This rocedure etails a two-ste iterative rocess that begis with a iitializatio θ 0) ad comutes:. ϕ t) = θ t) + λ P T )T y P T θ t) ). θt+) i = ϕ t) i if ϕ t) log) log) i λɛ 0 otherwise where λ is the largest eigevalue of P P. This rocedure is desirable sice the secod ste, i which the comlexity term lays its role, ivolves a simle coordiate-wise thresholdig oeratio. It is easy to verify that the iteratios roduce a mootoically o-icreasig sequece of comlexity-regularized emirical risk values [9. Thus, this rocedure rovides a simle iteratio that teds to miimize the origial objective fuctio, ad aears to give good results i ractice [7. The iteratios ca be termiated whe the etries uiformly satisfy θ t+) i θ t) i δ, for a small ositive tolerace δ. The comutatioal comlexity of the above rocedure is quite aealig. Each iteratio requires oly Ok) oeratios, assumig that the trasform T ca be comuted i O) oeratios. For examle, the discrete wavelet of Fourier trasforms ca be comuted i O) ad O log ) oeratios, resectively. Multilicatio by P is the most itesive oeratio, requirig Ok) oeratios. The thresholdig ste is carried out ideedetly i each coordiate, ad this ste requires O) oeratios as well. Of course, the umber of iteratios required is roblem deedet ad difficult to redict, but i our exeriece i this alicatio ad others [7, [9 algorithms of this sort ted to coverge i a reasoably small umber of iteratios, eve i very high dimesioal cases. Oe oit worthy of metio relates to the factor /ɛ = B +σ) i the ealty. As is ofte the case with coservative bouds of this tye, the theoretical ealty is larger tha what is eeded i ractice to achieve good results. Also, a otetial hurdle to calibratig the algorithm is that it deeds o kowledge of B ad σ, either of which may be kow a riori. Strictly seakig, these values do ot eed to be kow ideedetly but rather we eed oly estimate B +σ). To that ed, otice that each observatio is a radom variable with variace equal to f / + σ. Let B = f /, which is the miimum B satisfyig the stated boud f B. The the variace of each observatio is B + σ. Further, it is easy to verify that B +σ ) B +σ). So, a scheme could be develoed whereby the samle variace is used as a surrogate for the ukow quatities i the form i which they aear i the arameter ɛ.

14 4 This would etail usig aother cocetratio iequality to cotrol the error betwee the samle variace ad its mea value, ad roagatig this additioal error through the derivatio of the oracle iequality. While this is relatively straightforward, we omit a comlete derivatio here. To illustrate the erformace of the algorithm above, i Figure we cosider three stadard test sigals, each of legth = Rademacher rojectio samles cotamiated with additive white Gaussia oise) are take for the Blocks, Bums, ad Doler test sigals. The algorithm described above is emloyed for recostructio, with oe slight modificatio. Sice the theoretical ealty ca be a bit too coservative i ractice, the threshold used i this examle is /3 of the theoretical value i.e., a threshold of log) log)/λɛ)/3 was used). The SNR, defied as SNR = 0 log 0 B σ where B = f /, is db for each test sigal. To covey a sese of the oise level, colum a) of Figure shows the origial sigals cotamiated with the same level of oise i.e., the sigal resultig from covetioal oit samles cotamiated with oise of the same ower). Colum b) shows the recostructios obtaied from 600 rojectios; recostructios from 00 rojectios are show i colum c). The Blocks sigal to row) was recostructed usig the Haar wavelet basis Daubechies-), well-suited to the iecewise costat ature of the sigal. The Bums ad Doler sigals middle ad bottom row, resectively) were recostructed usig the Daubechies-6 wavelet basis. Of course, the selectio of the best recostructio basis is a searate matter that is beyod the scoe of this aer. VI. APPLICATIONS Oe immediate alicatio of the results ad methods above is to sigal codig ad A/D coversio. I the oiseless settig, several authors have suggested the use of radom rojectio samlig for such uroses [ [3. Our results idicate how such schemes might erform i the resece of oise. Suose that we have a array of sesors, each makig a oisy measuremet. The oise could be due to the sesors themselves or evirometal factors. The goal of ecodig ad A/D coversio is to rereset the sesor readigs i a comressed form, suitable for digital storage or trasmissio. Our results suggest that k radom Rademacher rojectios of the sesor readigs ca be used for this urose, ad the error bouds suggest guidelies for how may rojectios might be required for a certai level of recisio. Our theory ad method ca also be alied to wireless sesig as follows. Cosider the roblem of sesig a distributed field e.g., temerature, light, chemical) usig a collectio of wireless sesors distributed uiformly over a regio of iterest. Such systems are ofte referred to as sesor etworks. The goal is to obtai a accurate, high-resolutio recostructio of the field at a remote destiatio. Oe

15 5 a) b) c) Fig.. Simulatio examles usig Blocks, Bums, ad Doler test sigals of legth Colum a) shows the origial sigals with a equivalet level of additive er-ixel oise. Colums b) ad c) show recostructios from 600 ad 00 rojectios, resectively. aroach to this roblem is to require each sesor to digitally trasmit its measuremet to the destiatio, where field recostructio is the erformed. Alteratively, the sesors might collaboratively rocess their measuremets to recostruct the field themselves ad the trasmit the result to the destiatio i.e., the odes collaborate to comress their date rior to trasmissio). Both aroaches ose sigificat demads o commuicatio resources ad ifrastructure, ad it has recetly bee suggested that o-collaborative aalog commuicatio schemes offer a more resource-efficiet alterative [ [3. Assume that the sesor data is to be trasmitted to the destiatio over a additive white Gaussia oise chael. Suose the destiatio broadcasts erhas digitally) a radom seed to the sesors.

16 6 Each ode modifies this seed i a uique way kow to oly itself ad the destiatio e.g., this seed could be multilied by the ode s address or geograhic ositio). Each ode geerates a seudoradom Rademacher sequece, which ca also be costructed at the destiatio. The the odes trasmit the radom rojectios to the destiatio hase-coheretly i.e., beamformig). This is accomlished by requirig each ode to simly multily its readig by a elemet of its radom sequece i each rojectio/commuicatio ste ad trasmit the result to the destiatio via amlitude modulatio. If the trasmissios from all sesors ca be sychroized so that they all arrive i hase at the destiatio, the the averagig iheret i the multile access chael comutes the desired ier roduct. After receivig k rojectios, the destiatio ca emloy the recostructio algorithm above usig a basis of choice e.g., wavelet). The commuicatios rocedure is comletely o-adative ad otetially very simle to imlemet. The collective fuctioig of the wireless sesors i this rocess is more aki to a esemble of hase-coheret emitters tha it is to covetioal etworkig oeratios. Therefore, we refer the term sesor esemble istead of sesor etwork i this cotext. A remarkable asect of the sesor esemble aroach is that the ower required to achieve a target distortio level ca be very miimal. Let σ s ad σ c deote the oise variace due to sesig ad commuicatio, resectively. Thus, each rojectio received at the destiatio is corruted by a oise of total ower σ s + σ c. The sesig oise variace is assumed to be a costat ad the additioal variace due to the commuicatio chael is assumed to scale like the iverse of the total received ower σ c P where P is the trasmit ower er sesor. Note that although the total trasmit ower is P, the received ower is a factor of larger as a result of the ower amlificatio effect of the hase-coheret trasmissios [3. I order to achieve rates of distortio decay that we claim, it is sufficiet that the variace due to the commuicatio chael behaves like a costat. Therefore, we require oly that P. This results i a rather surrisig coclusio. Ideal recostructio is ossible at the destiatio with total trasmit ower P tedig to zero as the desity of sesors icreases. If covetioal satial oit samles were take istead e.g., if a sigle sesor is selected at radom i each ste ad trasmits its measuremet to the destiatio), the the ower required er samle would be a costat, sice oly oe sesor would be ivolved i such a trasmissio. Thus, it aears that radom rojectio samlig could be much more desirable i wireless sesig alicatios.

17 7 VII. CONCLUSIONS AND FUTURE WORK We have show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. The squared error bouds for comressible sigals are Ok/ log ) α α+ ), which is withi a logarithmic factor of the usual oarametric estimatio rate, ad Ok/ log ) ) for sarse sigals. We also roosed a ractical iterative algorithm for sigal recostructio. Oe of the most romisig otetial alicatios of our theory ad method is to wireless sesig, wherei oe realizes a large trasmissio ower gai by radom rojectio samlig as oosed to covetioal satial oit samlig. The role of the oise variace i the rates we reseted is worthy of further attetio. As the oise variace teds to zero, oe ituitively exects to attai the fast rates that are kow to be achievable i the oiseless settig. Our theory is based i the oisy regime ad it does ot directly imly the reviously established bouds i the oiseless settig. Simly ut, our aalysis assumes a oise variace strictly greater tha zero. Let us commet briefly o the tightess of the uer bouds give by our theory. I aalogy with classical oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). Moreover, i the secial case of sarse sigals with m o-zero terms, we obtai a error boud of m/k igorig costat ad logarithmic factors). Stadard arametric statistical aalysis suggests that oe should ot exect a rate of better tha m/k degrees-offreedom/ samle-size) i such cases, which agai suorts our ituitio regardig the tightess of the bouds i terms of the covergece rate). However, to our kowledge exlicit miimax lower bouds have ot bee established i the cotext of this roblem, ad the determiatio of such bouds is oe of our future research directios. Although we cosidered oly the case of Gaussia oise i the observatio model ), the same results could be achieved for ay zero-mea, symmetrically distributed oise that is ideedet of the rojectio vector elemets ad satisfies E[w k j k)!!varw j )) k, a result that follows immediately usig the lemmas reseted i the Aedix. Aother extesio would be the cosideratio of other radom rojectios istead of the Rademacher rojectios cosidered here. Most of our basic aroach would go through i such cases; oe would oly eed to verify the momet coditios of the Craig-Berstei iequality for articular cases.

18 8 VIII. ACKNOWLEDGEMENTS The authors would like to thak Rui Castro for his assistace i all stages of the work reseted here. A. The Craig-Berstei Momet Coditio APPENDIX The cetral requiremet of the Craig-Berstei iequality is the satisfactio of the momet coditio E [ X E[X!varX)h for itegers with some ositive costat h that does ot deed o, a coditio that is automatically verified for = with ay fiite value of h. For higher owers this coditio ca be very difficult to verify for several reasos, ot the least of which is the absolute value reset i the momets. Previous work that made use of the Craig-Berstei iequality assumed that the observatios were bouded, forcig a ossibly urealistic case of bouded oise [3. This assumtio is ot sufficiet for the rates of covergece we claim. Ideed, the aïve boud o the observatios is y i B yieldig a costat h that would grow roortioally to. With that motivatio, we develo a framework uder which a boudig costat h ca be determied more directly. First, observe that the momet coditio is usually easier to verify for the eve owers because the absolute value eed ot be dealt with directly. This is sufficiet to guaratee the momet coditio is satisfied for all itegers, as roved i the followig lemma. Lemma Suose the Craig-Berstei momet coditio holds for all eve itegers greater tha or equal to, that is E [ X E[X k k)!varx)hk, k sice the k = case is satisfied trivially for ay h. The the coditio holds also for the odd absolute momets with h = h. Thus A ad B, E [ X E[X k k )!varx) h k 3, k E [ X E[X!varX)h),. Proof: For ease of otatio, let Z = X E[X. Hölder s Iequality states, for ay radom variables E[ AB E[ A / E[ B q /q

19 9 where <, q < ad / + /q =. Take A = Z, B = Z k, ad = q = to get E[ Z k E[Z E[Z 4k 4 where the absolute values iside the square root have bee droed because the exoets are eve. Now by assumtio so E[Z 4k 4 4k 4)!E[Z h 4k 6 E[ Z k We wat to satisfy the followig iequality by choice of h which meas h must satisfy 4k 4)!E[Z ) h 4k 6 4k 4)! E[Z h k 3 E[ Z k k )!E[Z h k 3 If we choose k )! 4k 4)! h k 3 h k 3. h max k 4k 4)! k )! ) k 3 h the the momet coditio will be satisfied for the odd exoets k. A uer boud for the term i brackets is, as show here. For k, the boud k)! k k!) holds ad ca be verified by iductio o k. This imlies ) 4k 4)! k 3 ) k 3. 4) k )! k Now, the term i aretheses o the right had side of 4) is always less tha for k. The fial ste is to show that lim k k ) k 3 =, which is verified by otig that lim log k k ) k 3 = lim log ) ) log k ) = 0. k k 3

20 0 Thus, the momet coditio is satisfied for odd momets with h = h. Also, if the momet coditio is satisfied for a give h, it is also satisfied for ay h h so holds for all itegers as claimed. E [ Z = E [ X E[X!varX)h) We will also eed results for how sums ad roducts of radom variables behave with resect to the momet coditio. For that, we have the followig two lemmas. Lemma Let Z = A + B be the sum of two zero-mea radom variables A ad B with variaces vara) = E[A ad varb) = E[B, ot both zero, ad such that E[AB 0. Suose both A ad B satisfy the momet coditio for a give iteger 3 with ositive costats h A ad h B resectively, ot both zero. That is The E [ A!varA)h A ad E [ B!varB)h B. E [ Z!varZ)h S where h S = / mi ) h A + h B ), ad mi is the miimum 3 for which the assumtios are satsified. Proof: First, defie V A = vara)/vara) + varb)), V B = varb)/vara) + varb)), H A = h A /h A + h B ), ad H B = h B /h A + h B ). Use Mikowski s Iequality to write E[ A + B [E[ A / + E[ B / [ ) vara)h ) A + varb)h B! =!vara) + varb)) [ ) V A h A + =!vara) + varb)) h A + h B )!vara + B) h A + h B ) V B h B ) [ ) V A H A + [ ) V A H A + V B H B V B H B ), ) where the last ste follows from the assumtio that E[AB 0, imlyig vara)+varb) vara+ B). Showig that [ ) V A H A + V B H B ) C,

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

3 Energy. 3.3. Non-Flow Energy Equation (NFEE) Internal Energy. MECH 225 Engineering Science 2

3 Energy. 3.3. Non-Flow Energy Equation (NFEE) Internal Energy. MECH 225 Engineering Science 2 MECH 5 Egieerig Sciece 3 Eergy 3.3. No-Flow Eergy Equatio (NFEE) You may have oticed that the term system kees croig u. It is ecessary, therefore, that before we start ay aalysis we defie the system that

More information

Department of Computer Science, University of Otago

Department of Computer Science, University of Otago Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

More information

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

Soving Recurrence Relations

Soving Recurrence Relations Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

Unit 8: Inference for Proportions. Chapters 8 & 9 in IPS

Unit 8: Inference for Proportions. Chapters 8 & 9 in IPS Uit 8: Iferece for Proortios Chaters 8 & 9 i IPS Lecture Outlie Iferece for a Proortio (oe samle) Iferece for Two Proortios (two samles) Cotigecy Tables ad the χ test Iferece for Proortios IPS, Chater

More information

RF Engineering Continuing Education Introduction to Traffic Planning

RF Engineering Continuing Education Introduction to Traffic Planning RF Egieerig otiuig Educatio Itroductio to Traffic Plaig Queuig Systems Figure. shows a schematic reresetatio of a queuig system. This reresetatio is a mathematical abstractio suitable for may differet

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

Regression with a Binary Dependent Variable (SW Ch. 11)

Regression with a Binary Dependent Variable (SW Ch. 11) Regressio with a Biary Deedet Variable (SW Ch. 11) So far the deedet variable (Y) has bee cotiuous: district-wide average test score traffic fatality rate But we might wat to uderstad the effect of X o

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

Asymptotic Growth of Functions

Asymptotic Growth of Functions CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

CHAPTER 3 DIGITAL CODING OF SIGNALS

CHAPTER 3 DIGITAL CODING OF SIGNALS CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity

More information

Capacity of Wireless Networks with Heterogeneous Traffic

Capacity of Wireless Networks with Heterogeneous Traffic Capacity of Wireless Networks with Heterogeeous Traffic Migyue Ji, Zheg Wag, Hamid R. Sadjadpour, J.J. Garcia-Lua-Aceves Departmet of Electrical Egieerig ad Computer Egieerig Uiversity of Califoria, Sata

More information

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations

CS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad

More information

Sequences and Series

Sequences and Series CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their

More information

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n

THE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample

More information

Modified Line Search Method for Global Optimization

Modified Line Search Method for Global Optimization Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

More information

Divide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015

Divide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015 CS125 Lecture 4 Fall 2015 Divide ad Coquer We have see oe geeral paradigm for fidig algorithms: the greedy approach. We ow cosider aother geeral paradigm, kow as divide ad coquer. We have already see a

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

Normal Distribution.

Normal Distribution. Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

On the L p -conjecture for locally compact groups

On the L p -conjecture for locally compact groups Arch. Math. 89 (2007), 237 242 c 2007 Birkhäuser Verlag Basel/Switzerlad 0003/889X/030237-6, ublished olie 2007-08-0 DOI 0.007/s0003-007-993-x Archiv der Mathematik O the L -cojecture for locally comact

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem

Lecture 4: Cauchy sequences, Bolzano-Weierstrass, and the Squeeze theorem Lecture 4: Cauchy sequeces, Bolzao-Weierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits

More information

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

More information

Irreducible polynomials with consecutive zero coefficients

Irreducible polynomials with consecutive zero coefficients Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem

More information

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process

An example of non-quenched convergence in the conditional central limit theorem for partial sums of a linear process A example of o-queched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which

More information

Class Meeting # 16: The Fourier Transform on R n

Class Meeting # 16: The Fourier Transform on R n MATH 18.152 COUSE NOTES - CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,

More information

1 The Gaussian channel

1 The Gaussian channel ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise.

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

5 Boolean Decision Trees (February 11)

5 Boolean Decision Trees (February 11) 5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

THE HEIGHT OF q-binary SEARCH TREES

THE HEIGHT OF q-binary SEARCH TREES THE HEIGHT OF q-binary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average

More information

Major Coefficients Recovery: a Compressed Data Gathering Scheme for Wireless Sensor Network

Major Coefficients Recovery: a Compressed Data Gathering Scheme for Wireless Sensor Network This full text paper was peer reviewed at the directio of IEEE Commuicatios Society subject matter experts for publicatio i the IEEE Globecom proceedigs. Major Coefficiets Recovery: a Compressed Data Gatherig

More information

Systems Design Project: Indoor Location of Wireless Devices

Systems Design Project: Indoor Location of Wireless Devices Systems Desig Project: Idoor Locatio of Wireless Devices Prepared By: Bria Murphy Seior Systems Sciece ad Egieerig Washigto Uiversity i St. Louis Phoe: (805) 698-5295 Email: bcm1@cec.wustl.edu Supervised

More information

On the Throughput Scaling of Cognitive Radio Ad Hoc Networks

On the Throughput Scaling of Cognitive Radio Ad Hoc Networks O the Throughut Scalig of Cogitive Radio Ad Hoc Networks Chegzhi Li ad Huaiyu Dai Deartmet of Electrical ad Comuter Egieerig North Carolia State Uiversity, Raleigh, NC email: {cli3, hdai}@csu.edu Abstract

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean

Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean 1 Social Studies 201 October 13, 2004 Note: The examples i these otes may be differet tha used i class. However, the examples are similar ad the methods used are idetical to what was preseted i class.

More information

Sequences II. Chapter 3. 3.1 Convergent Sequences

Sequences II. Chapter 3. 3.1 Convergent Sequences Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,

More information

Universal coding for classes of sources

Universal coding for classes of sources Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)

BASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1) BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

Infinite Sequences and Series

Infinite Sequences and Series CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then

4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or p-series (the Compariso Test), but of

More information

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx

SAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval

More information

THE ABRACADABRA PROBLEM

THE ABRACADABRA PROBLEM THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected

More information

4.1 Sigma Notation and Riemann Sums

4.1 Sigma Notation and Riemann Sums 0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

More information

Gregory Carey, 1998 Linear Transformations & Composites - 1. Linear Transformations and Linear Composites

Gregory Carey, 1998 Linear Transformations & Composites - 1. Linear Transformations and Linear Composites Gregory Carey, 1998 Liear Trasformatios & Composites - 1 Liear Trasformatios ad Liear Composites I Liear Trasformatios of Variables Meas ad Stadard Deviatios of Liear Trasformatios A liear trasformatio

More information

Estimating Probability Distributions by Observing Betting Practices

Estimating Probability Distributions by Observing Betting Practices 5th Iteratioal Symposium o Imprecise Probability: Theories ad Applicatios, Prague, Czech Republic, 007 Estimatig Probability Distributios by Observig Bettig Practices Dr C Lych Natioal Uiversity of Irelad,

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

More information

Research Article Sign Data Derivative Recovery

Research Article Sign Data Derivative Recovery Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov

More information

1. MATHEMATICAL INDUCTION

1. MATHEMATICAL INDUCTION 1. MATHEMATICAL INDUCTION EXAMPLE 1: Prove that for ay iteger 1. Proof: 1 + 2 + 3 +... + ( + 1 2 (1.1 STEP 1: For 1 (1.1 is true, sice 1 1(1 + 1. 2 STEP 2: Suppose (1.1 is true for some k 1, that is 1

More information

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

Determining the sample size

Determining the sample size Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors

More information

The analysis of the Cournot oligopoly model considering the subjective motive in the strategy selection

The analysis of the Cournot oligopoly model considering the subjective motive in the strategy selection The aalysis of the Courot oligopoly model cosiderig the subjective motive i the strategy selectio Shigehito Furuyama Teruhisa Nakai Departmet of Systems Maagemet Egieerig Faculty of Egieerig Kasai Uiversity

More information

HIGH-DIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH NONCONVEXITY

HIGH-DIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH NONCONVEXITY The Aals of Statistics 2012, Vol. 40, No. 3, 1637 1664 DOI: 10.1214/12-AOS1018 Istitute of Mathematical Statistics, 2012 HIGH-DIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH

More information

Chapter 5: Inner Product Spaces

Chapter 5: Inner Product Spaces Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples

More information

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).

Example 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here). BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook - Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly

More information

Exploratory Data Analysis

Exploratory Data Analysis 1 Exploratory Data Aalysis Exploratory data aalysis is ofte the rst step i a statistical aalysis, for it helps uderstadig the mai features of the particular sample that a aalyst is usig. Itelliget descriptios

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13 EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may

More information

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

THE LEAST COMMON MULTIPLE OF A QUADRATIC SEQUENCE

THE LEAST COMMON MULTIPLE OF A QUADRATIC SEQUENCE THE LEAST COMMON MULTIPLE OF A QUADRATIC SEQUENCE JAVIER CILLERUELO Abstract. We obtai, for ay irreducible quadratic olyomial f(x = ax 2 + bx + c, the asymtotic estimate log l.c.m. {f(1,..., f(} log. Whe

More information

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution

University of California, Los Angeles Department of Statistics. Distributions related to the normal distribution Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chi-square (χ ) distributio.

More information

High-dimensional support union recovery in multivariate regression

High-dimensional support union recovery in multivariate regression High-dimesioal support uio recovery i multivariate regressio Guillaume Oboziski Departmet of Statistics UC Berkeley gobo@stat.berkeley.edu Marti J. Waiwright Departmet of Statistics Dept. of Electrical

More information

Overview of some probability distributions.

Overview of some probability distributions. Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

1. C. The formula for the confidence interval for a population mean is: x t, which was

1. C. The formula for the confidence interval for a population mean is: x t, which was s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : p-value

More information

Escola Federal de Engenharia de Itajubá

Escola Federal de Engenharia de Itajubá Escola Federal de Egeharia de Itajubá Departameto de Egeharia Mecâica Pós-Graduação em Egeharia Mecâica MPF04 ANÁLISE DE SINAIS E AQUISÇÃO DE DADOS SINAIS E SISTEMAS Trabalho 02 (MATLAB) Prof. Dr. José

More information

Factors of sums of powers of binomial coefficients

Factors of sums of powers of binomial coefficients ACTA ARITHMETICA LXXXVI.1 (1998) Factors of sums of powers of biomial coefficiets by Neil J. Cali (Clemso, S.C.) Dedicated to the memory of Paul Erdős 1. Itroductio. It is well ow that if ( ) a f,a = the

More information

Notes on exponential generating functions and structures.

Notes on exponential generating functions and structures. Notes o expoetial geeratig fuctios ad structures. 1. The cocept of a structure. Cosider the followig coutig problems: (1) to fid for each the umber of partitios of a -elemet set, (2) to fid for each the

More information

Acta Acad. Paed. Agriensis, Sectio Mathematicae 29 (2002) 77 87. ALMOST SURE FUNCTIONAL LIMIT THEOREMS IN L p( ]0, 1[ ), WHERE 1 p <

Acta Acad. Paed. Agriensis, Sectio Mathematicae 29 (2002) 77 87. ALMOST SURE FUNCTIONAL LIMIT THEOREMS IN L p( ]0, 1[ ), WHERE 1 p < Acta Acad. Paed. Agriesis, Sectio Mathematicae 29 22) 77 87 ALMOST SUR FUNCTIONAL LIMIT THORMS IN L ], [ ), WHR < József Túri Nyíregyháza, Hugary) Dedicated to the memory of Professor Péter Kiss Abstract.

More information

where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return

where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return EVALUATING ALTERNATIVE CAPITAL INVESTMENT PROGRAMS By Ke D. Duft, Extesio Ecoomist I the March 98 issue of this publicatio we reviewed the procedure by which a capital ivestmet project was assessed. The

More information

Convention Paper 6764

Convention Paper 6764 Audio Egieerig Society Covetio Paper 6764 Preseted at the 10th Covetio 006 May 0 3 Paris, Frace This covetio paper has bee reproduced from the author's advace mauscript, without editig, correctios, or

More information

Lecture 4: Cheeger s Inequality

Lecture 4: Cheeger s Inequality Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a d-regular

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

CS103X: Discrete Structures Homework 4 Solutions

CS103X: Discrete Structures Homework 4 Solutions CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible six-figure salaries i whole dollar amouts are there that cotai at least

More information

Unit 20 Hypotheses Testing

Unit 20 Hypotheses Testing Uit 2 Hypotheses Testig Objectives: To uderstad how to formulate a ull hypothesis ad a alterative hypothesis about a populatio proportio, ad how to choose a sigificace level To uderstad how to collect

More information

A Mixed-Integer Optimization Model for Compressor Selection in Natural Gas Pipeline Network System Operations

A Mixed-Integer Optimization Model for Compressor Selection in Natural Gas Pipeline Network System Operations Joural of virometal Iformatics 3 () 33-4 (2004) 04JI00025 726-235/684-8799 2004 ISIS www.iseis.org/jei A Mixed-Iteger Otimizatio Model for Comressor Selectio i Natural as Pielie Network System Oeratios

More information

EDEXCEL NATIONAL CERTIFICATE/DIPLOMA. PRINCIPLES AND APPLICATIONS of THERMODYNAMICS NQF LEVEL 3 OUTCOME 1 -EXPANSION AND COMPRESSION OF GASES

EDEXCEL NATIONAL CERTIFICATE/DIPLOMA. PRINCIPLES AND APPLICATIONS of THERMODYNAMICS NQF LEVEL 3 OUTCOME 1 -EXPANSION AND COMPRESSION OF GASES EDEXCEL NAIONAL CERIFICAE/DIPLOMA PRINCIPLES AND APPLICAIONS of HERMODYNAMICS NQF LEEL 3 OUCOME -EXPANSION AND COMPRESSION OF GASES CONEN Be able to aly thermodyamic riciles to the exasio ad comressio

More information

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the

More information

A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length

A Faster Clause-Shortening Algorithm for SAT with No Restriction on Clause Length Joural o Satisfiability, Boolea Modelig ad Computatio 1 2005) 49-60 A Faster Clause-Shorteig Algorithm for SAT with No Restrictio o Clause Legth Evgey Datsi Alexader Wolpert Departmet of Computer Sciece

More information

Confidence Intervals for Two Proportions

Confidence Intervals for Two Proportions PASS Samle Size Software Chater 6 Cofidece Itervals for Two Proortios Itroductio This routie calculates the grou samle sizes ecessary to achieve a secified iterval width of the differece, ratio, or odds

More information

A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design

A Combined Continuous/Binary Genetic Algorithm for Microstrip Antenna Design A Combied Cotiuous/Biary Geetic Algorithm for Microstrip Atea Desig Rady L. Haupt The Pesylvaia State Uiversity Applied Research Laboratory P. O. Box 30 State College, PA 16804-0030 haupt@ieee.org Abstract:

More information

A Recursive Formula for Moments of a Binomial Distribution

A Recursive Formula for Moments of a Binomial Distribution A Recursive Formula for Momets of a Biomial Distributio Árpád Béyi beyi@mathumassedu, Uiversity of Massachusetts, Amherst, MA 01003 ad Saverio M Maago smmaago@psavymil Naval Postgraduate School, Moterey,

More information

Statistical inference: example 1. Inferential Statistics

Statistical inference: example 1. Inferential Statistics Statistical iferece: example 1 Iferetial Statistics POPULATION SAMPLE A clothig store chai regularly buys from a supplier large quatities of a certai piece of clothig. Each item ca be classified either

More information

The Stable Marriage Problem

The Stable Marriage Problem The Stable Marriage Problem William Hut Lae Departmet of Computer Sciece ad Electrical Egieerig, West Virgiia Uiversity, Morgatow, WV William.Hut@mail.wvu.edu 1 Itroductio Imagie you are a matchmaker,

More information

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Solutions 9 Spring 2006 Exam format UC Bereley Departmet of Electrical Egieerig ad Computer Sciece EE 6: Probablity ad Radom Processes Solutios 9 Sprig 006 The secod midterm will be held o Wedesday May 7; CHECK the fial exam

More information

Interference Alignment and the Generalized Degrees of Freedom of the X Channel

Interference Alignment and the Generalized Degrees of Freedom of the X Channel Iterferece Aligmet ad the Geeralized Degrees of Freedom of the X Chael Chiachi Huag, Viveck R. Cadambe, Syed A. Jafar Electrical Egieerig ad Computer Sciece Uiversity of Califoria Irvie Irvie, Califoria,

More information