16. Mean Square Estimation

Size: px

Start display at page:

Download "16. Mean Square Estimation"

Shannon Scott
8 years ago
Views:

1 6 Me Sque stmto Gve some fomto tht s elted to uow qutty of teest the poblem s to obt good estmte fo the uow tems of the obseved dt Suppose epeset sequece of dom vbles bout whom oe set of obsevtos e vlble d epesets uow dom vble he poblem s to obt good estmte fo tems of the obsevtos Let ˆ ϕ ϕ 6- epeset such estmte fo Note tht ϕ c be le o ole fucto of the obsevto Clely ˆ ϕ epesets the eo the bove estmte d the sque of 6-

epesets uow dom vble he poblem s to obt good estmte fo tems of the obsevtos Let ˆ ϕ ϕ 6- epeset such

2 the eo Sce s dom vble epesets the me sque eo Oe sttegy to obt good estmto would be to mme the me sque eo by vyg ove ll possble foms of ϕ d ths pocedue gves se to the Mmto of the Me Sque o MMS cteo fo estmto hus ude MMS cteothe estmto ϕ s chose such tht the me sque eo s t ts mmum Net we show tht the codtol me of gve s the best estmto the bove sese heoem: Ude MMS cteo the best estmto fo the uow tems of s gve by the codtol me of gves hus ˆ ϕ 6-3 Poof : Let ˆ ϕ epeset estmte of tems of he the eo d the me sque ˆ eo s gve by ˆ ϕ 6-4

eo s t ts mmum Net we show tht the codtol me of gve s the best estmto the bove sese heoem: Ude MMS cteo the best estmto fo the uow

3 Sce [ ] [ we c ewte 6-4 s ] 6-5 ϕ [ ϕ ] whee the e epectto s wth espect to d the oute oe s wth espect to hus [ ϕ ] ϕ f o obt the best estmto ϕ we eed to mme 6-6 wth espect to ϕ I 6-6 sce f ϕ d the vble ϕ ppes oly the tegd tem mmto of the me sque eo 6-6 wth espect to ϕ s equvlet to mmto of ϕ wth espect to ϕ d 6-6 3

mme 6-6 wth espect to ϕ I 6-6 sce f ϕ d the vble ϕ ppes oly the tegd tem mmto

4 Sce s fed t some vlue ϕ s o loge dom d hece mmto of ϕ s equvlet to hs gves ϕ ϕ 6-7 o ϕ But ϕ 6-8 ϕ ϕ 6-9 sce whe ϕ s fed umbe ϕ Usg 6-9 4

5 6-8 we get the desed estmto to be ˆ ϕ hus the codtol me of gve epesets the best estmto fo tht mmes the me sque eo he mmum vlue of the me sque eo s gve by 6- m [ ] v v 6-3 As emple suppose s the uow he the best MMS estmto s gve by ˆ Clely f the deed ˆ 3 s the best estmto fo 6-5

eo s gve by 6- m [ ] v v 6-3 As emple suppose s the uow he the best

6 tems of hus the best estmto c be ole Net we wll cosde less tvl emple mple : Let y < < y < f y othewse whee > s sutble omlto costt o deteme the best estmte fo tems of we eed y f f y dy y ydy f < < hus f y y y f y ; < < y < f / 6-3 Hece the best MMS estmto s gve by y 6

o deteme the best estmte fo tems of we eed y f f y dy y ydy f < <

7 7 Oce g the best estmto s ole I geel the best estmto s dffcult to evlute d hece et we wll eme the specl subclss of best le estmtos Best Le stmto I ths cse the estmto s le fucto of the obsevtos hus whee e uow quttes to be detemed he me sque eo s gve by ˆ 3 3 y dy y dy y dy y f y y ϕ 6-4 ˆ l ˆ 6-5 ˆ l

cse the estmto s le fucto of the obsevtos hus whee e uow quttes to be

8 ˆ l 6-6 d ude the MMS cteo should be chose so tht the me sque eo s t ts mmum possble vlue Let epeset tht mmum possble vlue he m 6-7 o mme 6-6 we c equte hs gves 6-8 But 6-9 8

9 6- Substtutg 6-9 to 6-8 we get o the best le estmto must stsfy 6- Notce tht 6- epesets the estmto eo d epesets the dt hus fom 6- the eo s othogol to the dt fo the best le estmto hs s the othogolty pcple I othe wods the le estmto 6-5 the uow costts must be selected such tht the eo 9

s othogol to the dt fo the best le estmto hs s the othogolty pcple I

10 s othogol to evey dt fo the best le estmto tht mmes the me sque eo Iteestgly geel fom of the othogolty pcple holds good the cse of ole estmtos lso Nole Othogolty Rule: Let h epeset y fuctol fom of the dt d the best estmto fo gve Wth e we shll show tht mplyg tht e hs follows sce eh eh h [ h h h ] h [ [ h ] h ] h 6-

Othogolty Rule: Let h epeset y fuctol fom of the dt d the best estmto fo gve

11 hus the ole veso of the othogolty ule the eo s othogol to y fuctol fom of the dt he othogolty pcple 6- c be used to obt the uows the le cse Fo emple suppose d we eed to estmte tems of lely hus Fom 6- the othogolty ule gves hus o d ˆ l

obt the uows the le cse Fo emple suppose d we eed to

12 6-3 c be solved to obt tems of the cosscoeltos he mmum vlue of the me sque eo 6-7 s gve by But usg 6- the secod tem 6-4 s eo sce the eo s othogol to the dt whee e chose to be optmum hus the mmum vlue of the me sque eo s gve by d 6-3 m m m m m l 6-4

eo sce the eo s othogol to the dt whee e chose to be optmum

13 whee e the optmum vlues fom 6- Sce the le estmte 6-5 s oly specl cse of the geel estmto ϕ 6- the best le estmto tht stsfes 6- cot be supeo to the best ole estmto Ofte the best le estmto wll be feo to the best estmto 6-3 hs ses the followg questo Ae thee stutos whch the best estmto 6-3 lso tus out to be le? I those stutos t s eough to use 6- d obt the best le estmtos sce they lso epeset the best globl estmtos Such s the cse f d e dstbuted s jotly Guss We summe ths the et theoem d pove tht esult heoem: If d e jotly Guss eo 6-5 3

the best estmto 6-3 lso tus out to be le?

14 me dom vbles the the best estmte fo tems of s lwys le Poof : Let ˆ ϕ 6-6 epeset the best possbly ole estmte of d l ˆ 6-7 the best le estmte of he fom 6- l s othogol to the dt hus Also fom

15 Usg we get 6-3 Fom 6-3 we obt tht d e eo me ucoelted dom vbles fo But tself epesets Guss dom vble sce fom 6-8 t epesets le combto of set of jotly Guss dom vbles hus d e jotly Guss d ucoelted dom vbles As esult d e depedet dom vbles hus fom the depedece 6-3 But fom 6-3 d hece fom Substtutg 6-8 to 6-33 we get 5

dom vbles hus d e jotly Guss d ucoelted dom vbles As esult d e depedet dom vbles

16 o Fom 6-6 ϕ epesets the best possble estmto d fom 6-8 epesets the best le estmto hus the best le estmto s lso the best possble ovell estmto the Guss cse Net we tu ou tteto to pedcto poblems usg le estmtos Le Pedcto Suppose e ow d s uow hus d ths epesets oe-step pedcto poblem If the uow s the t epesets -step hed pedcto poblem Retug bc to the oe-step pedcto let ˆ epeset the best le pedcto he l

estmtos Le Pedcto Suppose e ow d s uow hus d ths epesets oe-step pedcto poblem If the uow s the t

17 7 whee the eo s othogol to the dt e Usg we get Suppose epesets the smple of wde sese sttoy 6-35 ˆ ˆ

18 8 stochstc pocess so tht hus 6-38 becomes pdg 6-4 fo we get the followg set of le equtos Smlly usg 6-5 the mmum me sque eo s gve by R t

19 9 he equtos 6-4 togethe wth 6-4 c be epeseted s Let

20 Notce tht s Hemt oeplt d postve defte Usg 6-44 the uows 6-43 c be epeseted s Let of colum Lst 6-45

21 he fom 6-45 hus >

22 d q 6-49 epesets the best le pedcto coeffcets d they c be evluted fom the lst colum of 6-45 Usg these he best oe-step hed pedcto 6-35 te the fom d fom 6-48 the mmum me sque eo s gve by the ety of Fom 6-36 sce the oe-step le pedcto eo 6-49 ˆ

23 3 we c epeset 6-5 fomlly s follows hus let them fom the bove fgue we lso hve the epesetto he flte epesets AR flte d ths shows tht le pedcto leds to uto egessve AR model A 6-5 A A H 6-53

24 4 he polyoml c be smplfed usg o see ths we ewte s o smplfy 6-54 we c me use of the followg mt detty A A ] [ ] [ A 6-54 CA B D C A I AB I D C B A 6-55

25 5 g detemts we get I ptcul f we get Usg wth B CA D A D C B A 6-56 D C B A A B CA 6-57 ] [ B A C

26 6 we get Refeg bc to 6-43 usg Cme s ule to solve fo we get A 6-58

27 7 o hus the polyoml 6-58 educes to he polyoml 6-53 c be ltetvely epeseted s 6-6 d fct epesets stble > A A ~ AR A H

28 AR flte of ode whose put eo sgl s whte ose of costt spectl heght equl to d output s / It c be show tht A hs ll ts eos > povded > thus estblshg stblty Le pedcto o Fom 6-59 the me sque eo usg smples s gve by > 6-6 Suppose oe moe smple fom the pst s vlble to evlute e e vlble Poceedg s bove the ew coeffcets d the me sque eo c be detemed Fom

29 Usg othe mt detty t s esy to show tht Sce > we must hve s o fo evey > s < Fom 6-63 we hve o s 6-63 s s < 6-64 sce s hus the me sque eo deceses s moe < d moe smples e used fom the pst the le pedcto I geel fom 6-64 the me sque eos fo the oe-step pedcto fom mootoc ocesg sequece 9

30 whose lmtg vlue Clely coespods to the educble eo le pedcto usg the ete pst smples d t s elted to the powe spectum of the udelyg pocess though the elto whee S ω epesets the powe spectum of Fo y fte powe pocess we hve d sce > hus 6-65 π ep l S ω dω π π 6-66 π π S ω dω < S ω l S ω S ω π π π l S ω d ω S ω d ω < π

31 Moeove f the powe spectum s stctly postve t evey Fequecy e the fom 6-66 S ω > - π < ω < π 6-68 d hece π π l S ω dω > π ep l S ω dω > e π π e Fo pocesses tht stsfy the stct postvty codto 6-68 lmost eveywhee the tevl π π the fl mmum me sque eo s stctly postve see 6-7 e Such pocesses e ot completely pedctble eve usg the ete set of pst smples o they e heetly stochstc 3

32 sce the et output cots fomto tht s ot coted the pst smples Such pocesses e ow s egul stochstc pocesses d the powe spectum s stctly postve S ω ω π π Powe Spectum of egul stochstc Pocess Covesely f pocess hs the followg powe spectum S ω π ω ω ω π such tht S ω ω the fom 6-7 < ω < ω 3

33 Such pocesses e completely pedctble fom the pst dt smples I ptcul cos ω t φ 6-7 s completely pedctble fom ts pst smples sce of le spectum S ω S ω cossts ω ω ω 6-7 s shpe detemstc stochstc pocess 33

Sequences and Series

Sequences and Series Secto 9. Sequeces d Seres You c thk of sequece s fucto whose dom s the set of postve tegers. f ( ), f (), f (),... f ( ),... Defto of Sequece A fte sequece s fucto whose dom s the set of postve tegers.