Signal Reconstruction from Noisy Random Projections


 Isaac Franklin
 2 years ago
 Views:
Transcription
1 Sigal Recostructio from Noisy Radom Projectios Jarvis Haut ad Robert Nowak Deartmet of Electrical ad Comuter Egieerig Uiversity of WiscosiMadiso March, 005; Revised February, 006 Abstract Recet results show that a relatively small umber of radom rojectios of a sigal ca cotai most of its saliet iformatio. It follows that if a sigal is comressible i some orthoormal basis, the a very accurate recostructio ca be obtaied from radom rojectios. We exted this tye of result to show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. We also roose a ractical iterative algorithm for sigal recostructio, ad briefly discuss otetial alicatios to codig, A/D coversio, ad remote wireless sesig. Idex Terms samlig, sigal recostructio, radom rojectios, deoisig, wireless sesor etworks I. INTRODUCTION Recet theory iforms us that, with high robability, a relatively small umber of radom rojectios of a sigal ca cotai most of its relevat iformatio. For examle, the groudbreakig work i [ has show that k radom Fourier rojectios cotai eough iformatio to recostruct iecewise smooth sigals at a distortio level early equivalet to that attaiable from k otimally selected observatios. Similar results hold for radom Gaussia ad Rademacher rojectios i.e., rojectios cosistig of ideedet ad idetically distributed Gaussia or Rademacher radom variables) [, [3. The results reseted i these works ca be roughly summarized as follows. Assume that a sigal f R is comressible i some orthoormal basis i the followig sese. Let f m) deote the best mterm This research was suorted i art by the NSF uder grats CCR ad CCR03557, ad by the Office of Naval Research, grat N
2 aroximatio of f i terms of this basis ad suose that the average squared error obeys f f m) = f i f m) i ) C A m α for some α 0 ad some costat C A i= > 0. The arameter α govers the degree to which f is comressible with resect to the basis. I a oiseless settig, it ca be show that a aroximatio of such a sigal ca be recovered from k radom rojectios with a average squared error that is uer bouded by a costat times k/ log ) α, early as good as the best kterm aroximatio error. This aer takes this lie of iquiry a ste further by cosiderig the erformace of samlig via radom rojectios i oisy coditios. We show that if the rojectios are cotamiated with zeromea Gaussia oise, the comressible sigals ca be recostructed with a exected average squared error that is uer bouded by a costat times k/ log ) α α+. For truly sarse sigals with oly a small umber of ozero terms) a stroger result is obtaied; the exected average squared recostructio error is uer bouded by a costat times k/ log ). These bouds demostrate a remarkable caability of samlig via radom rojectios accurate recostructios ca be obtaied eve whe the sigal dimesio greatly exceeds the umber of samles k ad the samles themselves are cotamiated with sigificat levels of oise. This effect is highlighted by the followig eedle i a haystack roblem. Suose the sigal f is a vector of legth with oe ozero etry of amlitude. If we samle the vector at k radom locatios aki to covetioal samlig schemes), the the robability of comletely missig the ozero etry is /) k, which is very close to whe k is sigificatly smaller tha. This imlies that the exected average squared error may be almost, or larger if oise is reset. O the other had, by samlig with radomized rojectios our results guaratee that the exected average squared error will be o larger tha a costat times k/ log ), which ca be arbitrarily close to 0 eve whe k, rovided k > log. A closely related roblem is the recostructio of sigals with sarse Fourier sectra from a relatively small umber of ouiform time samles e.g., radom samles i time) [4 [7. Most of this work cocers oiseless situatios, but [5 addresses the roblem of recostructio from oisecorruted samles. Aother area of work related to our results cocers the recostructio of sigals with fiite degrees of freedom usig a small umber of otraditioal samles [8, [9. A secial istace of this setu is the case of sigals that are sarse i time the dual of the sectrally sarse case). Recostructio from oisecorruted samles is the focus of [9. I a sese, the samlig ad recostructio roblems addressed i the aers above are secial cases of the class of roblems cosidered here, where we allow sigals that
3 3 are sarse i ay domai. Agai, this more uiversal ersective is recisely the focus of [, [3, which cosider sigal recostructio from oiseless radom rojectios. A iterestig lie of similar work cocers the related roblem of sigal recostructio from radom rojectios corruted by a ukow but bouded erturbatio [0, [. Here we cosider ubouded, Gaussia oise cotamiatio i the samlig rocess. Fially, while this aer was uder review a related ivestigatio was reorted i [ ertaiig to the statistical estimatio of sarse sigals from uderdetermied ad oisy observatios. That work develos quatitave bouds for sarse sigal recostructio similar to ours, but is based o a uiform ucertaity ricile rather tha radomized desigs as here. The aer is orgaized as follows. I Sectio II we state the basic roblem ad mai theoretical results of the aer. I Sectio III we derive bouds o the accuracy of sigal recostructios from oisy radom rojectios. I Sectio IV we secialize the bouds to cases i which the uderlyig sigal is comressible i terms of a certai orthoormal basis. I Sectio V we roose a simle iterative algorithm for sigal recostructio. I Sectio VI we discuss alicatios to ecodig, A/D coversio, ad wireless sesig, ad we make cocludig remarks i Sectio VII. Detailed derivatios are relegated to the Aedix. II. MAIN RESULTS Cosider a vector f = [f f... f T R ad assume that i= f i ) f B for a kow costat B > 0. The assumtio simly imlies that the average er elemet eergy is bouded by a costat. This is a fairly weak restrictio, sice it ermits a very large class of sigals, icludig sigals with eak magitudes as large as O ). Now suose that we are able to make k measuremets of f i the form of oisy, radom rojectios. Secifically, let Φ = {φ i,j } be a k array of bouded, i.i.d. zeromea radom variables of variace E[φ i,j = /. Samles take the form y j = φ i,j fi + w j, j =,..., k ) i= where w = {w j } are i.i.d. zeromea radom variables, ideedet of {φ i,j }, with variace σ. The goal is to recover a estimate of f from these observatios. Defie the risk of a cadidate recostructio f to be Rf) = f f + σ where the orm is the Euclidea distace. Next assume that both {φ i,j } ad {y j } are available. The we
4 4 ca comute the emirical risk Rf) = k ) k y j φ i,j f i. j= i= It is easy to verify that E[ Rf) = Rf) usig the facts that {φ i,j } ad {w j } are ideedet radom variables ad E[φ i,j = /. Thus, Rf) is a ubiased estimator of Rf). We will use the emirical risk to obtai a estimator f of f, ad boud the resultig error E[ f f. The estimator is based o a comlexityregularized emirical risk miimizatio, ad we use the CraigBerstei cocetratio iequality to cotrol the estimatio error of the recostructio rocess. That iequality etails the verificatio of certai momet coditios, which deed o the ature of Φ ad w. Therefore, i this aer we focus o ormalized) Rademacher rojectios, i which case each φ i,j is ±/ with equal robability, ad assume that w is a sequece of zeromea Gaussia oises. Geeralizatios to other radom rojectios ad oise models may be ossible followig our aroach; this would oly require oe to verify the momet coditios required by the CraigBerstei iequality. Suose that we have a coutable collectio F of cadidate recostructio fuctios ad a oegative umber cf) assiged to each f F such that f F cf). Furthermore, assume that each f F satisfies f B. Select a recostructio accordig to the comlexityregularized emirical risk miimizatio f k { = arg mi Rf) + f F } cf) log where ɛ > 0 is a costat that deeds o B ad σ. The we have the followig oracle iequality. Theorem Let ɛ = /B + σ) ), the [ E f k f { f f C mi f F + } cf) log + 4, where the costat C is give by with S = B/σ, the sigaltooise ratio. C = 9S )S + 3 S )S + A imortat oit regardig the costats above is that they deed oly o σ ad B, the oise ower ad the average sigal ower, resectively. If f is comressible with resect to a certai orthoormal basis, the we ca obtai exlicit bouds o the error i terms of the umber of radom rojectios k ad the degree to which f is comressible.
5 5 Let f m) deote the best mterm aroximatio of f i the basis. That is, if f has a reresetatio f = i= θ iψ i i the basis {ψ i }, the f m) = m i= θ i)ψ i), where coefficiets ad basis fuctios are reordered such that θ ) θ ) θ ). Assume that the average squared error f f m) / i= f i f m) i ) satisfies for some α 0 ad some costat C A f f m) C A m α > 0. Powerlaw decays like this arise quite commoly i alicatios. For examle, smooth ad iecewise smooth sigals ad sigals of bouded variatio exhibit this sort of behavior [, [3. It is also uecessary to restrict oe s attetio orthoormal basis exasios. Much more geeral aroximatio strategies ca be accomodated [3, but to kee the resetatio as simle as ossible we will ot delve further ito such extesios. Now let us take F to be a suitably quatized collectio of fuctios, rereseted i terms of the basis {ψ i } the costructio of F is discussed i Sectio IV). We have the followig error boud. Theorem If cf) = log) {# ozero coefficiets of f} the there exists a costat C = C B, σ, C A ) > 0 such that [ E f k f ) k α/α+) C C, log where C is as give i Theorem. Note that the exoet α/α + ) is the usual exoet goverig the rate of covergece i oarametric fuctio estimatio. A stroger result is obtaied if the sigal is sarse, as stated i the followig Corollary. Corollary Suose that f has at most m ozero coefficiets. The there exists a costat C = C B, σ) > 0 such that [ E f k f C C ) k, m log where C is as give i Theorem.
6 6 Similar results hold if the sigal is additioally cotamiated with oise rior to the radom rojectio rocess. Corollary Suose observatios take the form y j = φ i,j fi + η i ) + w j, i= j =,..., k where {η i } are i.i.d. zeromea Gaussia radom variables with variace σ s that are ideedet of {φ i,j } ad {w j }. The Theorems ad ad Corollary hold with slightly differet costats C, C, C, ad ɛ. It is imortat to oit out that all the results above hold for arbitrary sigal legth, ad the costats do ot deed o the size of the roblem. The fact that the rate deeds oly logarithmically o is sigificat ad illustrates the scalability of this aroach. Oe ca iterret these bouds as good idicators of the excetioal erformace of radom rojectio samlig i large regimes. The deedece o k is show to be olyomial. I aalogy with oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). To drive this oit home, let us agai cosider the eedle i a haystack roblem, this time i a bit more detail. Suose the sigal f is a vector of legth with oe ozero etry of amlitude such that f / =. First, cosider radom satial oit samlig where observatios are oisefree i.e., each samle is of the form y j = f t j ), where t j is selected uiformly at radom from the set {,..., }). The squared recostructio error is 0 if the sike is located ad otherwise, ad the robability of ot fidig the sike i k trials is /) k, givig a average squared error of /) k + k/) 0 = /) k. If is large, we ca aroximate this by /) k e k/, which is very close to whe k is sigificatly smaller tha. O the other had, radomized Rademacher rojectios corruted with oise) yield a average squared recostructio error boud of C k/ log ), as give above i Corollary. This boud may be arbibrarily close to 0 eve whe k, rovided k > log. This shows that eve give the advatage of beig oiseless, the recostructio error from satial oit samlig may be far greater tha that resultig from radom rojectios. III. ORACLE INEQUALITY I this sectio we rove Theorem. For ease of otatio, we adot the shorthad otatio φ j = [φ,j φ,j... φ,j T for the vector corresodig to the j th rojectio. The emirical risk of a vector f
7 7 ca ow be writte as Rf) = k k y j φ T j f). j= We will boud rf, f ) Rf) Rf ), the excess risk betwee a cadidate recostructio f ad the actual fuctio f, usig the comlexityregularizatio method itroduced i [3. Note that rf, f ) = E [ y j φ T j f y j φ T j f) + φ T j f) φ T j f ) = E [ φ T j f φ T j f ) = f f. Defie the emirical excess risk rf, f ) Rf) Rf ). The rf, f ) = k = k k [ yj φ T j f ) y j φ T j f) j= k j= U j where U j = [y j φ T j f ) y j φ T j f) are i.i.d. Notice that rf, f ) rf, f ) = k k j= U j E [U j ). We will make use of the CraigBerstei iequality [4, which states that P k U j E [U j ) t ɛ k var k k + k j= j) U e t ζ) j= for 0 < ɛh ζ < ad t > 0, rovided the variables U j satisfy the momet coditio E [ U j E[U j k k! varu j) h k for some h > 0 ad all itegers k. If we cosider vectors f ad estimates f where f B ad f B, Rademacher rojectios, ad Gaussia oises with variace σ, the the momet coditio is satisfied with h = 6B +8 Bσ, as show i the Aedix. Alterative forms of radom rojectios ad oises ca also be hadled usig the aroach outlied ext, rovided the momet coditios are satisfied. To use the CraigBerstei iequality we also eed a boud o the variace of U j itself. Defiig g = f f, we have ) g varu j ) = E[φ T g) 4 + 4σ g. As show i the Aedix, for itegers k ) g k ) g E[φ T g) k k k)!!,
8 8 where k)!! )3)... k ). Thus we ca boud the variace of U j by ) varu j ) g g + 4σ. Sice g satisfies g 4B ad rf, f ) = f f / = g /, the boud becomes varu j ) 8B + 4σ ) rf, f ). So, we ca relace the term i the CraigBerstei iequality that deeds o the variace by k var k U j = k varu j ) 8B + 4σ ) rf, f ). k k j= For a give fuctio f, we have P rf, f ) rf, f ) > t 8B + + 4σ ) ) ɛrf, f ) < e t ζ) or, by lettig δ = e t P j= rf, f ) rf, f ) > log δ ) 8B + 4σ ) ) ɛrf, f ) + < δ. ζ) Now assig to each f F a oegative ealty term cf) such that the ealties satisfy the Kraft Iequality [5 cf) f F ad let δf) = cf) δ. The by alyig the uio boud we have for all f F ad for all δ > 0 rf, f ) rf, f ) cf) log + log ) δ 8B + 4σ ) ɛrf, f ) + ζ) with robability at least δ. Now set ζ = ɛh, defie 8B + 4σ ) ɛ a, ζ) ad choose ɛ < 0B + 8 Bσ + σ. Notice that a < ad ζ < by choice of ɛ. The a)rf, f ) rf, f ) + cf) log + log ) δ holds with robability at least δ for all f F ad ay δ > 0.
9 9 For the give traiig samles, we ca miimize the uer boud by choosig { } f k = arg mi rf, f cf) log ) + f F which is equivalet to { f k = arg mi Rf) + f F f k arg mi f F } cf) log sice we ca igore Rf ) whe erformig the otimizatio. If we defie { } Rf) + cf) log the with robability at least δ a)r f k, f ) r f k, f ) + c f k ) log + log ) δ rf k, f ) + cf k ) log + log ) δ sice f k miimizes the comlexityregularized emirical risk criterio. Usig the CraigBerstei iequality agai to boud rf k, f ) rf k, f ) with the same variace boud as before) we get that with robability at least δ rf k, f ) rfk, f ) a rfk, f ) + log ) δ. 3) We wat both ) ad 3) to hold simultaeously, so we use the uio boud to obtai ) r f + a k, f ) rfk a, f ) + cf k ) log + log )) δ a holdig with robability at least δ. Let δ = e t a)/ to obtai P r f k, f ) + a a Itegratig this relatio gives [ E r f k, f ) ) rfk, f ) cf k ) log ) a) t e t a)/. ) + a a rfk, f ) + cf k ) log + 4. a) )
10 0 Now, sice a is ositive, [ E f k f [ = E r f k, f ) where C = + a)/ a). = + a a + a a + a a ) + a mi a f F { f f + = C mi f F ) rfk, f ) + cf k ) log + 4 a) ) rfk, f ) + + a) cf k ) log + 4 a) ) { Rfk ) Rf ) + cf k ) log + 4 { Rf) Rf ) + Tyical values of C ca be determied by aroximatig the costat ɛ < 0B + 8 Bσ + σ. cf) log + 4 } } cf) log + 4 }, Uer boudig the deomiator guaratees that the coditio is satisfied, so let ɛ = / B + σ) ). Now 8B + 4σ ) ɛ a = = ζ) If we deote the sigal to oise ratio by S = B /σ the 4B + σ 5B )Bσ + σ. 4S + a = 5S )S + for which the extremes are a mi = / ad a max = 4/5, givig costats C i the rage of [3/9, 9. IV. ERROR BOUNDS FOR COMPRESSIBLE SIGNALS I this sectio we rove Theorem ad Corollary. Suose that f is comressible i a certai orthoormal basis {ψ i } i=. Secifically, let f m) deote the best mterm aroximatio of f i terms of {ψ i }, ad assume that the error of the aroximatio obeys for some α 0 ad a costat C A > 0. f f m) C A m α Let us use the basis {ψ i } for the recostructio rocess. Ay vector f F ca be exressed i terms of the basis {ψ i } as f = i= θ iψ i, where θ = {θ i } are the coefficiets of f i this basis. Let T deote the trasform that mas coefficiets to fuctios, so that f = T θ ad defie Θ = {θ : T θ B, θ i quatized to levels} to be the set of cadidate solutios i the basis {ψ i }. The ealty term
11 cf) writte i terms of the basis {ψ i } is cf) = cθ) = + ) log) i= I θ i 0 = + ) log θ 0. It is easily verified that f F cf) by otig that each θ Θ ca be uiquely ecoded via a refix code cosistig of + ) log bits er ozero coefficiet log bits for the locatios ad log bits for the quatized values) i which case the codelegths cf) must satisfy the Kraft iequality [6. The oracle iequality [ E f k f { f f C mi + f F } cf) log + 4 ca be writte i terms of the ew class of cadidate recostructios as [ E f k f { θ θ } cθ) log + 4 C mi + θ Θ where f = T θ. For each iteger m, let θ m) deote the coefficiets corresodig to the best mterm aroximatio of f ad let θ m) q deote the earest elemet i Θ. The maximum ossible dyamic rage for the coefficiet magitudes, ± B, is quatized to levels, givig θ q m) θ m) 4B / = C Q /. Thus, θ m) q θ = θ m) q θ m) + θ m) θ θ m) q θ m) + θ m) q θ m) θ m) θ + θ m) θ C Q CA C Q + m α + C A m α. Now isert θ q m) i lace of θ i the oracle boud, ad ote that cθ q m) ) = + )m log to obtai [ E f { } k f C Q m α CA C Q C mi + θ Θ / + C A m α + )m log log Choosig large eough makes the first two terms egligible. Balacig the third ad fourth terms gives ) ) + ) log α+ k α+ m = ɛc A log so ad sice ) α ) α + ) log C A m α α+ k α+ = C A, ɛc A log k < log k ) α α+
12 whe k > ad > e, [ E f k f ) α k α+ C C, log as claimed i the Theorem, where C = { C A + ) log ɛc A } α ɛ ) α Suose ow that f has oly m ozero coefficiets. I this case, θ m) q θ C Q sice θ m) θ = 0. Now the ealty term domiates i the oracle boud ad [ E f k f ) k C C, m log where { } + ) log + 4 C =. ɛ V. OPTIMIZATION SCHEME Although our otimizatio is ocovex, it does ermit a simle, iterative otimizatio strategy that roduces a sequece of recostructios for which the corresodig sequece of comlexityregularized emirical risk values is odecreasig. This algorithm, which is described below, has demostrated itself to be quite effective i similar deoisig ad recostructio roblems [7 [9. A ossible alterative strategy might etail covexifyig the roblem by relacig the l 0 ealty with a l ealty. Recet results show that ofte the solutio to this covex roblem coicides with or aroximates the solutio to the origial ocovex roblem [0. Let us assume that we wish to recostruct our sigal i terms of the basis {ψ i }. Usig the defiitios itroduced i the revious sectio, the recostructio is equivalet to f k = T θ k where f k = arg mi f F θ k = arg mi θ Θ { Rf) + { RT θ) + Thus, the otimizatio roblem ca the be writte as { y P T θ + θ k = arg mi θ Θ } cf) log } cθ) log } log) log) θ 0 ɛ
13 3 where P = Φ T, the trasose of the k rojectio matrix Φ, y is a colum vector of the k observatios, ad θ 0 = i= I {θ i 0}. To solve this, we use a iterative boudotimizatio rocedure, as roosed i [7 [9. This rocedure etails a twoste iterative rocess that begis with a iitializatio θ 0) ad comutes:. ϕ t) = θ t) + λ P T )T y P T θ t) ). θt+) i = ϕ t) i if ϕ t) log) log) i λɛ 0 otherwise where λ is the largest eigevalue of P P. This rocedure is desirable sice the secod ste, i which the comlexity term lays its role, ivolves a simle coordiatewise thresholdig oeratio. It is easy to verify that the iteratios roduce a mootoically oicreasig sequece of comlexityregularized emirical risk values [9. Thus, this rocedure rovides a simle iteratio that teds to miimize the origial objective fuctio, ad aears to give good results i ractice [7. The iteratios ca be termiated whe the etries uiformly satisfy θ t+) i θ t) i δ, for a small ositive tolerace δ. The comutatioal comlexity of the above rocedure is quite aealig. Each iteratio requires oly Ok) oeratios, assumig that the trasform T ca be comuted i O) oeratios. For examle, the discrete wavelet of Fourier trasforms ca be comuted i O) ad O log ) oeratios, resectively. Multilicatio by P is the most itesive oeratio, requirig Ok) oeratios. The thresholdig ste is carried out ideedetly i each coordiate, ad this ste requires O) oeratios as well. Of course, the umber of iteratios required is roblem deedet ad difficult to redict, but i our exeriece i this alicatio ad others [7, [9 algorithms of this sort ted to coverge i a reasoably small umber of iteratios, eve i very high dimesioal cases. Oe oit worthy of metio relates to the factor /ɛ = B +σ) i the ealty. As is ofte the case with coservative bouds of this tye, the theoretical ealty is larger tha what is eeded i ractice to achieve good results. Also, a otetial hurdle to calibratig the algorithm is that it deeds o kowledge of B ad σ, either of which may be kow a riori. Strictly seakig, these values do ot eed to be kow ideedetly but rather we eed oly estimate B +σ). To that ed, otice that each observatio is a radom variable with variace equal to f / + σ. Let B = f /, which is the miimum B satisfyig the stated boud f B. The the variace of each observatio is B + σ. Further, it is easy to verify that B +σ ) B +σ). So, a scheme could be develoed whereby the samle variace is used as a surrogate for the ukow quatities i the form i which they aear i the arameter ɛ.
14 4 This would etail usig aother cocetratio iequality to cotrol the error betwee the samle variace ad its mea value, ad roagatig this additioal error through the derivatio of the oracle iequality. While this is relatively straightforward, we omit a comlete derivatio here. To illustrate the erformace of the algorithm above, i Figure we cosider three stadard test sigals, each of legth = Rademacher rojectio samles cotamiated with additive white Gaussia oise) are take for the Blocks, Bums, ad Doler test sigals. The algorithm described above is emloyed for recostructio, with oe slight modificatio. Sice the theoretical ealty ca be a bit too coservative i ractice, the threshold used i this examle is /3 of the theoretical value i.e., a threshold of log) log)/λɛ)/3 was used). The SNR, defied as SNR = 0 log 0 B σ where B = f /, is db for each test sigal. To covey a sese of the oise level, colum a) of Figure shows the origial sigals cotamiated with the same level of oise i.e., the sigal resultig from covetioal oit samles cotamiated with oise of the same ower). Colum b) shows the recostructios obtaied from 600 rojectios; recostructios from 00 rojectios are show i colum c). The Blocks sigal to row) was recostructed usig the Haar wavelet basis Daubechies), wellsuited to the iecewise costat ature of the sigal. The Bums ad Doler sigals middle ad bottom row, resectively) were recostructed usig the Daubechies6 wavelet basis. Of course, the selectio of the best recostructio basis is a searate matter that is beyod the scoe of this aer. VI. APPLICATIONS Oe immediate alicatio of the results ad methods above is to sigal codig ad A/D coversio. I the oiseless settig, several authors have suggested the use of radom rojectio samlig for such uroses [ [3. Our results idicate how such schemes might erform i the resece of oise. Suose that we have a array of sesors, each makig a oisy measuremet. The oise could be due to the sesors themselves or evirometal factors. The goal of ecodig ad A/D coversio is to rereset the sesor readigs i a comressed form, suitable for digital storage or trasmissio. Our results suggest that k radom Rademacher rojectios of the sesor readigs ca be used for this urose, ad the error bouds suggest guidelies for how may rojectios might be required for a certai level of recisio. Our theory ad method ca also be alied to wireless sesig as follows. Cosider the roblem of sesig a distributed field e.g., temerature, light, chemical) usig a collectio of wireless sesors distributed uiformly over a regio of iterest. Such systems are ofte referred to as sesor etworks. The goal is to obtai a accurate, highresolutio recostructio of the field at a remote destiatio. Oe
15 5 a) b) c) Fig.. Simulatio examles usig Blocks, Bums, ad Doler test sigals of legth Colum a) shows the origial sigals with a equivalet level of additive erixel oise. Colums b) ad c) show recostructios from 600 ad 00 rojectios, resectively. aroach to this roblem is to require each sesor to digitally trasmit its measuremet to the destiatio, where field recostructio is the erformed. Alteratively, the sesors might collaboratively rocess their measuremets to recostruct the field themselves ad the trasmit the result to the destiatio i.e., the odes collaborate to comress their date rior to trasmissio). Both aroaches ose sigificat demads o commuicatio resources ad ifrastructure, ad it has recetly bee suggested that ocollaborative aalog commuicatio schemes offer a more resourceefficiet alterative [ [3. Assume that the sesor data is to be trasmitted to the destiatio over a additive white Gaussia oise chael. Suose the destiatio broadcasts erhas digitally) a radom seed to the sesors.
16 6 Each ode modifies this seed i a uique way kow to oly itself ad the destiatio e.g., this seed could be multilied by the ode s address or geograhic ositio). Each ode geerates a seudoradom Rademacher sequece, which ca also be costructed at the destiatio. The the odes trasmit the radom rojectios to the destiatio hasecoheretly i.e., beamformig). This is accomlished by requirig each ode to simly multily its readig by a elemet of its radom sequece i each rojectio/commuicatio ste ad trasmit the result to the destiatio via amlitude modulatio. If the trasmissios from all sesors ca be sychroized so that they all arrive i hase at the destiatio, the the averagig iheret i the multile access chael comutes the desired ier roduct. After receivig k rojectios, the destiatio ca emloy the recostructio algorithm above usig a basis of choice e.g., wavelet). The commuicatios rocedure is comletely oadative ad otetially very simle to imlemet. The collective fuctioig of the wireless sesors i this rocess is more aki to a esemble of hasecoheret emitters tha it is to covetioal etworkig oeratios. Therefore, we refer the term sesor esemble istead of sesor etwork i this cotext. A remarkable asect of the sesor esemble aroach is that the ower required to achieve a target distortio level ca be very miimal. Let σ s ad σ c deote the oise variace due to sesig ad commuicatio, resectively. Thus, each rojectio received at the destiatio is corruted by a oise of total ower σ s + σ c. The sesig oise variace is assumed to be a costat ad the additioal variace due to the commuicatio chael is assumed to scale like the iverse of the total received ower σ c P where P is the trasmit ower er sesor. Note that although the total trasmit ower is P, the received ower is a factor of larger as a result of the ower amlificatio effect of the hasecoheret trasmissios [3. I order to achieve rates of distortio decay that we claim, it is sufficiet that the variace due to the commuicatio chael behaves like a costat. Therefore, we require oly that P. This results i a rather surrisig coclusio. Ideal recostructio is ossible at the destiatio with total trasmit ower P tedig to zero as the desity of sesors icreases. If covetioal satial oit samles were take istead e.g., if a sigle sesor is selected at radom i each ste ad trasmits its measuremet to the destiatio), the the ower required er samle would be a costat, sice oly oe sesor would be ivolved i such a trasmissio. Thus, it aears that radom rojectio samlig could be much more desirable i wireless sesig alicatios.
17 7 VII. CONCLUSIONS AND FUTURE WORK We have show that comressible sigals ca be accurately recovered from radom rojectios cotamiated with oise. The squared error bouds for comressible sigals are Ok/ log ) α α+ ), which is withi a logarithmic factor of the usual oarametric estimatio rate, ad Ok/ log ) ) for sarse sigals. We also roosed a ractical iterative algorithm for sigal recostructio. Oe of the most romisig otetial alicatios of our theory ad method is to wireless sesig, wherei oe realizes a large trasmissio ower gai by radom rojectio samlig as oosed to covetioal satial oit samlig. The role of the oise variace i the rates we reseted is worthy of further attetio. As the oise variace teds to zero, oe ituitively exects to attai the fast rates that are kow to be achievable i the oiseless settig. Our theory is based i the oisy regime ad it does ot directly imly the reviously established bouds i the oiseless settig. Simly ut, our aalysis assumes a oise variace strictly greater tha zero. Let us commet briefly o the tightess of the uer bouds give by our theory. I aalogy with classical oarametric estimatio theory e.g., estimatig smooth fuctios from radom oit samles), the olyomial rate i k is recisely what oe exects i geeral, ad thus we believe the uer bouds are tight u to costat ad logarithmic factors). Moreover, i the secial case of sarse sigals with m ozero terms, we obtai a error boud of m/k igorig costat ad logarithmic factors). Stadard arametric statistical aalysis suggests that oe should ot exect a rate of better tha m/k degreesoffreedom/ samlesize) i such cases, which agai suorts our ituitio regardig the tightess of the bouds i terms of the covergece rate). However, to our kowledge exlicit miimax lower bouds have ot bee established i the cotext of this roblem, ad the determiatio of such bouds is oe of our future research directios. Although we cosidered oly the case of Gaussia oise i the observatio model ), the same results could be achieved for ay zeromea, symmetrically distributed oise that is ideedet of the rojectio vector elemets ad satisfies E[w k j k)!!varw j )) k, a result that follows immediately usig the lemmas reseted i the Aedix. Aother extesio would be the cosideratio of other radom rojectios istead of the Rademacher rojectios cosidered here. Most of our basic aroach would go through i such cases; oe would oly eed to verify the momet coditios of the CraigBerstei iequality for articular cases.
18 8 VIII. ACKNOWLEDGEMENTS The authors would like to thak Rui Castro for his assistace i all stages of the work reseted here. A. The CraigBerstei Momet Coditio APPENDIX The cetral requiremet of the CraigBerstei iequality is the satisfactio of the momet coditio E [ X E[X!varX)h for itegers with some ositive costat h that does ot deed o, a coditio that is automatically verified for = with ay fiite value of h. For higher owers this coditio ca be very difficult to verify for several reasos, ot the least of which is the absolute value reset i the momets. Previous work that made use of the CraigBerstei iequality assumed that the observatios were bouded, forcig a ossibly urealistic case of bouded oise [3. This assumtio is ot sufficiet for the rates of covergece we claim. Ideed, the aïve boud o the observatios is y i B yieldig a costat h that would grow roortioally to. With that motivatio, we develo a framework uder which a boudig costat h ca be determied more directly. First, observe that the momet coditio is usually easier to verify for the eve owers because the absolute value eed ot be dealt with directly. This is sufficiet to guaratee the momet coditio is satisfied for all itegers, as roved i the followig lemma. Lemma Suose the CraigBerstei momet coditio holds for all eve itegers greater tha or equal to, that is E [ X E[X k k)!varx)hk, k sice the k = case is satisfied trivially for ay h. The the coditio holds also for the odd absolute momets with h = h. Thus A ad B, E [ X E[X k k )!varx) h k 3, k E [ X E[X!varX)h),. Proof: For ease of otatio, let Z = X E[X. Hölder s Iequality states, for ay radom variables E[ AB E[ A / E[ B q /q
19 9 where <, q < ad / + /q =. Take A = Z, B = Z k, ad = q = to get E[ Z k E[Z E[Z 4k 4 where the absolute values iside the square root have bee droed because the exoets are eve. Now by assumtio so E[Z 4k 4 4k 4)!E[Z h 4k 6 E[ Z k We wat to satisfy the followig iequality by choice of h which meas h must satisfy 4k 4)!E[Z ) h 4k 6 4k 4)! E[Z h k 3 E[ Z k k )!E[Z h k 3 If we choose k )! 4k 4)! h k 3 h k 3. h max k 4k 4)! k )! ) k 3 h the the momet coditio will be satisfied for the odd exoets k. A uer boud for the term i brackets is, as show here. For k, the boud k)! k k!) holds ad ca be verified by iductio o k. This imlies ) 4k 4)! k 3 ) k 3. 4) k )! k Now, the term i aretheses o the right had side of 4) is always less tha for k. The fial ste is to show that lim k k ) k 3 =, which is verified by otig that lim log k k ) k 3 = lim log ) ) log k ) = 0. k k 3
20 0 Thus, the momet coditio is satisfied for odd momets with h = h. Also, if the momet coditio is satisfied for a give h, it is also satisfied for ay h h so holds for all itegers as claimed. E [ Z = E [ X E[X!varX)h) We will also eed results for how sums ad roducts of radom variables behave with resect to the momet coditio. For that, we have the followig two lemmas. Lemma Let Z = A + B be the sum of two zeromea radom variables A ad B with variaces vara) = E[A ad varb) = E[B, ot both zero, ad such that E[AB 0. Suose both A ad B satisfy the momet coditio for a give iteger 3 with ositive costats h A ad h B resectively, ot both zero. That is The E [ A!varA)h A ad E [ B!varB)h B. E [ Z!varZ)h S where h S = / mi ) h A + h B ), ad mi is the miimum 3 for which the assumtios are satsified. Proof: First, defie V A = vara)/vara) + varb)), V B = varb)/vara) + varb)), H A = h A /h A + h B ), ad H B = h B /h A + h B ). Use Mikowski s Iequality to write E[ A + B [E[ A / + E[ B / [ ) vara)h ) A + varb)h B! =!vara) + varb)) [ ) V A h A + =!vara) + varb)) h A + h B )!vara + B) h A + h B ) V B h B ) [ ) V A H A + [ ) V A H A + V B H B V B H B ), ) where the last ste follows from the assumtio that E[AB 0, imlyig vara)+varb) vara+ B). Showig that [ ) V A H A + V B H B ) C,
Chapter 7 Methods of Finding Estimators
Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of
More information3 Energy. 3.3. NonFlow Energy Equation (NFEE) Internal Energy. MECH 225 Engineering Science 2
MECH 5 Egieerig Sciece 3 Eergy 3.3. NoFlow Eergy Equatio (NFEE) You may have oticed that the term system kees croig u. It is ecessary, therefore, that before we start ay aalysis we defie the system that
More informationDepartment of Computer Science, University of Otago
Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS200609 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly
More informationI. Chisquared Distributions
1 M 358K Supplemet to Chapter 23: CHISQUARED DISTRIBUTIONS, TDISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad tdistributios, we first eed to look at aother family of distributios, the chisquared distributios.
More informationProperties of MLE: consistency, asymptotic normality. Fisher information.
Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout
More informationModule 4: Mathematical Induction
Module 4: Mathematical Iductio Theme 1: Priciple of Mathematical Iductio Mathematical iductio is used to prove statemets about atural umbers. As studets may remember, we ca write such a statemet as a predicate
More informationSoving Recurrence Relations
Sovig Recurrece Relatios Part 1. Homogeeous liear 2d degree relatios with costat coefficiets. Cosider the recurrece relatio ( ) T () + at ( 1) + bt ( 2) = 0 This is called a homogeeous liear 2d degree
More informationIn nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008
I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces
More informationChapter 6: Variance, the law of large numbers and the MonteCarlo method
Chapter 6: Variace, the law of large umbers ad the MoteCarlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value
More informationUnit 8: Inference for Proportions. Chapters 8 & 9 in IPS
Uit 8: Iferece for Proortios Chaters 8 & 9 i IPS Lecture Outlie Iferece for a Proortio (oe samle) Iferece for Two Proortios (two samles) Cotigecy Tables ad the χ test Iferece for Proortios IPS, Chater
More informationRF Engineering Continuing Education Introduction to Traffic Planning
RF Egieerig otiuig Educatio Itroductio to Traffic Plaig Queuig Systems Figure. shows a schematic reresetatio of a queuig system. This reresetatio is a mathematical abstractio suitable for may differet
More informationOutput Analysis (2, Chapters 10 &11 Law)
B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should
More information7. Sample Covariance and Correlation
1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y
More informationRegression with a Binary Dependent Variable (SW Ch. 11)
Regressio with a Biary Deedet Variable (SW Ch. 11) So far the deedet variable (Y) has bee cotiuous: districtwide average test score traffic fatality rate But we might wat to uderstad the effect of X o
More informationAsymptotic Growth of Functions
CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll
More informationCHAPTER 3 DIGITAL CODING OF SIGNALS
CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity
More information3. Covariance and Correlation
Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics
More informationCS103A Handout 23 Winter 2002 February 22, 2002 Solving Recurrence Relations
CS3A Hadout 3 Witer 00 February, 00 Solvig Recurrece Relatios Itroductio A wide variety of recurrece problems occur i models. Some of these recurrece relatios ca be solved usig iteratio or some other ad
More informationTHE REGRESSION MODEL IN MATRIX FORM. For simple linear regression, meaning one predictor, the model is. for i = 1, 2, 3,, n
We will cosider the liear regressio model i matrix form. For simple liear regressio, meaig oe predictor, the model is i = + x i + ε i for i =,,,, This model icludes the assumptio that the ε i s are a sample
More informationSequences and Series
CHAPTER 9 Sequeces ad Series 9.. Covergece: Defiitio ad Examples Sequeces The purpose of this chapter is to itroduce a particular way of geeratig algorithms for fidig the values of fuctios defied by their
More informationCapacity of Wireless Networks with Heterogeneous Traffic
Capacity of Wireless Networks with Heterogeeous Traffic Migyue Ji, Zheg Wag, Hamid R. Sadjadpour, J.J. GarciaLuaAceves Departmet of Electrical Egieerig ad Computer Egieerig Uiversity of Califoria, Sata
More informationNPTEL STRUCTURAL RELIABILITY
NPTEL Course O STRUCTURAL RELIABILITY Module # 0 Lecture 1 Course Format: Web Istructor: Dr. Aruasis Chakraborty Departmet of Civil Egieerig Idia Istitute of Techology Guwahati 1. Lecture 01: Basic Statistics
More informationModified Line Search Method for Global Optimization
Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o
More informationNormal Distribution.
Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by
More informationDivide and Conquer. Maximum/minimum. Integer Multiplication. CS125 Lecture 4 Fall 2015
CS125 Lecture 4 Fall 2015 Divide ad Coquer We have see oe geeral paradigm for fidig algorithms: the greedy approach. We ow cosider aother geeral paradigm, kow as divide ad coquer. We have already see a
More information1 Computing the Standard Deviation of Sample Means
Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.
More informationChapter 7  Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:
Chapter 7  Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries
More informationOn the L p conjecture for locally compact groups
Arch. Math. 89 (2007), 237 242 c 2007 Birkhäuser Verlag Basel/Switzerlad 0003/889X/0302376, ublished olie 2007080 DOI 0.007/s0003007993x Archiv der Mathematik O the L cojecture for locally comact
More informationThe second difference is the sequence of differences of the first difference sequence, 2
Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for
More informationA probabilistic proof of a binomial identity
A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two
More informationAn example of nonquenched convergence in the conditional central limit theorem for partial sums of a linear process
A example of oqueched covergece i the coditioal cetral limit theorem for partial sums of a liear process Dalibor Volý ad Michael Woodroofe Abstract A causal liear processes X,X 0,X is costructed for which
More informationLecture 4: Cauchy sequences, BolzanoWeierstrass, and the Squeeze theorem
Lecture 4: Cauchy sequeces, BolzaoWeierstrass, ad the Squeeze theorem The purpose of this lecture is more modest tha the previous oes. It is to state certai coditios uder which we are guarateed that limits
More informationTaking DCOP to the Real World: Efficient Complete Solutions for Distributed MultiEvent Scheduling
Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed MultiEvet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria
More informationIrreducible polynomials with consecutive zero coefficients
Irreducible polyomials with cosecutive zero coefficiets Theodoulos Garefalakis Departmet of Mathematics, Uiversity of Crete, 71409 Heraklio, Greece Abstract Let q be a prime power. We cosider the problem
More information5 Boolean Decision Trees (February 11)
5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected
More informationClass Meeting # 16: The Fourier Transform on R n
MATH 18.152 COUSE NOTES  CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,
More information1 The Gaussian channel
ECE 77 Lecture 0 The Gaussia chael Objective: I this lecture we will lear about commuicatio over a chael of practical iterest, i which the trasmitted sigal is subjected to additive white Gaussia oise.
More informationNonlife insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring
Nolife isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy
More informationConfidence Intervals for One Mean
Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a
More informationPROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUSMALUS SYSTEM
PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUSMALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics
More informationMajor Coefficients Recovery: a Compressed Data Gathering Scheme for Wireless Sensor Network
This full text paper was peer reviewed at the directio of IEEE Commuicatios Society subject matter experts for publicatio i the IEEE Globecom proceedigs. Major Coefficiets Recovery: a Compressed Data Gatherig
More information5: Introduction to Estimation
5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample
More informationTHE HEIGHT OF qbinary SEARCH TREES
THE HEIGHT OF qbinary SEARCH TREES MICHAEL DRMOTA AND HELMUT PRODINGER Abstract. q biary search trees are obtaied from words, equipped with the geometric distributio istead of permutatios. The average
More informationSystems Design Project: Indoor Location of Wireless Devices
Systems Desig Project: Idoor Locatio of Wireless Devices Prepared By: Bria Murphy Seior Systems Sciece ad Egieerig Washigto Uiversity i St. Louis Phoe: (805) 6985295 Email: bcm1@cec.wustl.edu Supervised
More informationOn the Throughput Scaling of Cognitive Radio Ad Hoc Networks
O the Throughut Scalig of Cogitive Radio Ad Hoc Networks Chegzhi Li ad Huaiyu Dai Deartmet of Electrical ad Comuter Egieerig North Carolia State Uiversity, Raleigh, NC email: {cli3, hdai}@csu.edu Abstract
More informationConvexity, Inequalities, and Norms
Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for
More informationDefinition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean
1 Social Studies 201 October 13, 2004 Note: The examples i these otes may be differet tha used i class. However, the examples are similar ad the methods used are idetical to what was preseted i class.
More informationSequences II. Chapter 3. 3.1 Convergent Sequences
Chapter 3 Sequeces II 3. Coverget Sequeces Plot a graph of the sequece a ) = 2, 3 2, 4 3, 5 + 4,...,,... To what limit do you thik this sequece teds? What ca you say about the sequece a )? For ǫ = 0.,
More informationLecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)
18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the BruMikowski iequality for boxes. Today we ll go over the
More informationUniversal coding for classes of sources
Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric
More informationBASIC STATISTICS. f(x 1,x 2,..., x n )=f(x 1 )f(x 2 ) f(x n )= f(x i ) (1)
BASIC STATISTICS. SAMPLES, RANDOM SAMPLING AND SAMPLE STATISTICS.. Radom Sample. The radom variables X,X 2,..., X are called a radom sample of size from the populatio f(x if X,X 2,..., X are mutually idepedet
More informationIncremental calculation of weighted mean and variance
Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically
More informationMaximum Likelihood Estimators.
Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio
More informationInfinite Sequences and Series
CHAPTER 4 Ifiite Sequeces ad Series 4.1. Sequeces A sequece is a ifiite ordered list of umbers, for example the sequece of odd positive itegers: 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29...
More information4 n. n 1. You shold think of the Ratio Test as a generalization of the Geometric Series Test. For example, if a n ar n is a geometric sequence then
SECTION 2.6 THE RATIO TEST 79 2.6. THE RATIO TEST We ow kow how to hadle series which we ca itegrate (the Itegral Test), ad series which are similar to geometric or pseries (the Compariso Test), but of
More informationSAMPLE QUESTIONS FOR FINAL EXAM. (1) (2) (3) (4) Find the following using the definition of the Riemann integral: (2x + 1)dx
SAMPLE QUESTIONS FOR FINAL EXAM REAL ANALYSIS I FALL 006 3 4 Fid the followig usig the defiitio of the Riema itegral: a 0 x + dx 3 Cosider the partitio P x 0 3, x 3 +, x 3 +,......, x 3 3 + 3 of the iterval
More informationTHE ABRACADABRA PROBLEM
THE ABRACADABRA PROBLEM FRANCESCO CARAVENNA Abstract. We preset a detailed solutio of Exercise E0.6 i [Wil9]: i a radom sequece of letters, draw idepedetly ad uiformly from the Eglish alphabet, the expected
More information4.1 Sigma Notation and Riemann Sums
0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas
More informationGregory Carey, 1998 Linear Transformations & Composites  1. Linear Transformations and Linear Composites
Gregory Carey, 1998 Liear Trasformatios & Composites  1 Liear Trasformatios ad Liear Composites I Liear Trasformatios of Variables Meas ad Stadard Deviatios of Liear Trasformatios A liear trasformatio
More informationEstimating Probability Distributions by Observing Betting Practices
5th Iteratioal Symposium o Imprecise Probability: Theories ad Applicatios, Prague, Czech Republic, 007 Estimatig Probability Distributios by Observig Bettig Practices Dr C Lych Natioal Uiversity of Irelad,
More informationMARTINGALES AND A BASIC APPLICATION
MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measuretheoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this
More informationWeek 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable
Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5
More informationResearch Article Sign Data Derivative Recovery
Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov
More informationExample 2 Find the square root of 0. The only square root of 0 is 0 (since 0 is not positive or negative, so those choices don t exist here).
BEGINNING ALGEBRA Roots ad Radicals (revised summer, 00 Olso) Packet to Supplemet the Curret Textbook  Part Review of Square Roots & Irratioals (This portio ca be ay time before Part ad should mostly
More information1. MATHEMATICAL INDUCTION
1. MATHEMATICAL INDUCTION EXAMPLE 1: Prove that for ay iteger 1. Proof: 1 + 2 + 3 +... + ( + 1 2 (1.1 STEP 1: For 1 (1.1 is true, sice 1 1(1 + 1. 2 STEP 2: Suppose (1.1 is true for some k 1, that is 1
More informationDifferential Quartet, A Novel Circuit Building Block for High Slew Rate Differential Amplification
4th WSEAS Iteratioal Coferece o ELECTRONICS, CONTROL ad SIGNAL PROCESSING, Miami, Florida, USA, 79 November, 5 (.666 Differetial Quartet, A Novel Circuit Buildig Block for High Slew Rate Differetial
More informationVladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT
Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee
More informationHIGHDIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH NONCONVEXITY
The Aals of Statistics 2012, Vol. 40, No. 3, 1637 1664 DOI: 10.1214/12AOS1018 Istitute of Mathematical Statistics, 2012 HIGHDIMENSIONAL REGRESSION WITH NOISY AND MISSING DATA: PROVABLE GUARANTEES WITH
More informationHypothesis testing. Null and alternative hypotheses
Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate
More informationDetermining the sample size
Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors
More informationCase Study. Normal and t Distributions. Density Plot. Normal Distributions
Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca
More informationDescriptive statistics deals with the description or simple analysis of population or sample data.
Descriptive statistics Some basic cocepts A populatio is a fiite or ifiite collectio of idividuals or objects. Ofte it is impossible or impractical to get data o all the members of the populatio ad a small
More informationChapter 5: Inner Product Spaces
Chapter 5: Ier Product Spaces Chapter 5: Ier Product Spaces SECION A Itroductio to Ier Product Spaces By the ed of this sectio you will be able to uderstad what is meat by a ier product space give examples
More informationThe analysis of the Cournot oligopoly model considering the subjective motive in the strategy selection
The aalysis of the Courot oligopoly model cosiderig the subjective motive i the strategy selectio Shigehito Furuyama Teruhisa Nakai Departmet of Systems Maagemet Egieerig Faculty of Egieerig Kasai Uiversity
More informationSECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES
SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,
More informationExploratory Data Analysis
1 Exploratory Data Aalysis Exploratory data aalysis is ofte the rst step i a statistical aalysis, for it helps uderstadig the mai features of the particular sample that a aalyst is usig. Itelliget descriptios
More informationUniversity of California, Los Angeles Department of Statistics. Distributions related to the normal distribution
Uiversity of Califoria, Los Ageles Departmet of Statistics Statistics 100B Istructor: Nicolas Christou Three importat distributios: Distributios related to the ormal distributio Chisquare (χ ) distributio.
More informationMeasures of Spread and Boxplots Discrete Math, Section 9.4
Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,
More informationDiscrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13
EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may
More informationTHE LEAST COMMON MULTIPLE OF A QUADRATIC SEQUENCE
THE LEAST COMMON MULTIPLE OF A QUADRATIC SEQUENCE JAVIER CILLERUELO Abstract. We obtai, for ay irreducible quadratic olyomial f(x = ax 2 + bx + c, the asymtotic estimate log l.c.m. {f(1,..., f(} log. Whe
More informationHighdimensional support union recovery in multivariate regression
Highdimesioal support uio recovery i multivariate regressio Guillaume Oboziski Departmet of Statistics UC Berkeley gobo@stat.berkeley.edu Marti J. Waiwright Departmet of Statistics Dept. of Electrical
More information{{1}, {2, 4}, {3}} {{1, 3, 4}, {2}} {{1}, {2}, {3, 4}} 5.4 Stirling Numbers
. Stirlig Numbers Whe coutig various types of fuctios from., we quicly discovered that eumeratig the umber of oto fuctios was a difficult problem. For a domai of five elemets ad a rage of four elemets,
More informationFactors of sums of powers of binomial coefficients
ACTA ARITHMETICA LXXXVI.1 (1998) Factors of sums of powers of biomial coefficiets by Neil J. Cali (Clemso, S.C.) Dedicated to the memory of Paul Erdős 1. Itroductio. It is well ow that if ( ) a f,a = the
More informationOverview of some probability distributions.
Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability
More information1. C. The formula for the confidence interval for a population mean is: x t, which was
s 1. C. The formula for the cofidece iterval for a populatio mea is: x t, which was based o the sample Mea. So, x is guarateed to be i the iterval you form.. D. Use the rule : pvalue
More informationWe have seen that the physically observable properties of a quantum system are represented
Chapter 14 Probability, Expectatio Value ad Ucertaity We have see that the physically observable properties of a quatum system are represeted by Hermitea operators (also referred to as observables ) such
More informationA Recursive Formula for Moments of a Binomial Distribution
A Recursive Formula for Momets of a Biomial Distributio Árpád Béyi beyi@mathumassedu, Uiversity of Massachusetts, Amherst, MA 01003 ad Saverio M Maago smmaago@psavymil Naval Postgraduate School, Moterey,
More informationThe Field Q of Rational Numbers
Chapter 3 The Field Q of Ratioal Numbers I this chapter we are goig to costruct the ratioal umber from the itegers. Historically, the positive ratioal umbers came first: the Babyloias, Egyptias ad Grees
More informationActa Acad. Paed. Agriensis, Sectio Mathematicae 29 (2002) 77 87. ALMOST SURE FUNCTIONAL LIMIT THEOREMS IN L p( ]0, 1[ ), WHERE 1 p <
Acta Acad. Paed. Agriesis, Sectio Mathematicae 29 22) 77 87 ALMOST SUR FUNCTIONAL LIMIT THORMS IN L ], [ ), WHR < József Túri Nyíregyháza, Hugary) Dedicated to the memory of Professor Péter Kiss Abstract.
More informationConfidence Intervals for Two Proportions
PASS Samle Size Software Chater 6 Cofidece Itervals for Two Proortios Itroductio This routie calculates the grou samle sizes ecessary to achieve a secified iterval width of the differece, ratio, or odds
More informationwhere: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return
EVALUATING ALTERNATIVE CAPITAL INVESTMENT PROGRAMS By Ke D. Duft, Extesio Ecoomist I the March 98 issue of this publicatio we reviewed the procedure by which a capital ivestmet project was assessed. The
More informationCenter, Spread, and Shape in Inference: Claims, Caveats, and Insights
Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the
More informationConvention Paper 6764
Audio Egieerig Society Covetio Paper 6764 Preseted at the 10th Covetio 006 May 0 3 Paris, Frace This covetio paper has bee reproduced from the author's advace mauscript, without editig, correctios, or
More informationEscola Federal de Engenharia de Itajubá
Escola Federal de Egeharia de Itajubá Departameto de Egeharia Mecâica PósGraduação em Egeharia Mecâica MPF04 ANÁLISE DE SINAIS E AQUISÇÃO DE DADOS SINAIS E SISTEMAS Trabalho 02 (MATLAB) Prof. Dr. José
More informationLecture 4: Cheeger s Inequality
Spectral Graph Theory ad Applicatios WS 0/0 Lecture 4: Cheeger s Iequality Lecturer: Thomas Sauerwald & He Su Statemet of Cheeger s Iequality I this lecture we assume for simplicity that G is a dregular
More information0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5
Sectio 13 KolmogorovSmirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.
More informationCS103X: Discrete Structures Homework 4 Solutions
CS103X: Discrete Structures Homewor 4 Solutios Due February 22, 2008 Exercise 1 10 poits. Silico Valley questios: a How may possible sixfigure salaries i whole dollar amouts are there that cotai at least
More informationA MixedInteger Optimization Model for Compressor Selection in Natural Gas Pipeline Network System Operations
Joural of virometal Iformatics 3 () 334 (2004) 04JI00025 726235/6848799 2004 ISIS www.iseis.org/jei A MixedIteger Otimizatio Model for Comressor Selectio i Natural as Pielie Network System Oeratios
More informationUnit 20 Hypotheses Testing
Uit 2 Hypotheses Testig Objectives: To uderstad how to formulate a ull hypothesis ad a alterative hypothesis about a populatio proportio, ad how to choose a sigificace level To uderstad how to collect
More information