Chaptr 4 Adaptiv Filtr hory ad Applicatios frcs: B.Widro ad M..Hoff, Adaptiv sitchig circuits, Proc. Of WSCON Cov. c., part 4, pp.96-4, 96 B.Widro ad S.D.Stars, Adaptiv Sigal Procssig, Prtic-Hall, 985 O.Macchi, Adaptiv Procssig: h Last Ma Squars Approach ith Applicatios i rasmissio, Wily, 995 P.M.Clarso, Optimal ad Adaptiv Sigal Procssig, CC Prss, 993 S.Hayi, Adaptiv Filtr hory, Prtic-Hall, D.F.Marshall, W.K.Jis ad J.J.Murphy, "h us of orthogoal trasforms for improvig prformac of adaptiv filtrs", I ras. Circuits & Systms, vol.36, April 989, pp.474-483
Adaptiv Sigal Procssig is cocrd ith th dsig, aalysis, ad implmtatio of systms hos structur chags i rspos to th icomig data. Applicatio aras ar similar to thos of optimal sigal procssig but o th viromt is chagig, th sigals ar ostatioary ad/or th paramtrs to b stimatd ar tim-varyig. For ampl, cho cacllatio for Had-Fr lphos h spch cho is a ostatioary sigal qualizatio of Data Commuicatio Chals h chal impuls rspos is chagig, particularly i mobil commuicatios im-varyig Systm Idtificatio th systm trasfr fuctio to b stimatd is o-statioary i som cotrol applicatios
Adaptiv Filtr Dvlopmt Yar Applicatio Dvloprs 959 96 Adaptiv pattr rcogitio systm Adaptiv avform rcogitio Widro t al Jacoatz 965 Adaptiv qualizr for tlpho chal Lucy 967 Adaptiv ata systm Widro t al 97 Liar prdictio for spch aalysis Prst umrous applicatios, structurs, algorithms Atal 3
Adaptiv Filtr Dfiitio A adaptiv filtr is a tim-variat filtr hos cofficits ar adjustd i a ay to optimiz a cost fuctio or to satisfy som prdtrmid optimizatio critrio. Charactristics of adaptiv filtrs: hy ca automatically adapt slf-optimiz i th fac of chagig viromts ad chagig systm rquirmts hy ca b traid to prform spcific filtrig ad dcisio-maig tass accordig to som updatig quatios traiig ruls Why adaptiv? It ca automatically oprat i chagig viromts.g. sigal dtctio i irlss chal ostatioary sigal/ois coditios.g. LPC of a spch sigal tim-varyig paramtr stimatio.g. positio tracig of a movig sourc 4
Bloc diagram of a typical adaptiv filtr is sho blo: Adaptiv Filtr y - Σ d {h} Adaptiv Algorithm : iput sigal y : filtrd output d : dsird rspos h : impuls rspos of adaptiv filtr N- h cost fuctio may b { } or Σ FI or II adaptiv filtr filtr ca b ralizd i various structurs adaptiv algorithm dpds o th optimizatio critrio 5
Basic Classs of Adaptiv Filtrig Applicatios.Prdictio : sigal codig, liar prdictio codig, spctral aalysis 6
.Idtificatio : adaptiv cotrol, layrd arth modlig, vibratio studis of mchaical systm 7
3.Ivrs Filtrig : adaptiv qualizatio for commuicatio chal, dcovolutio 8
4.Itrfrc Caclig : adaptiv ois caclig, cho cacllatio 9
Dsig Cosidratios. Cost Fuctio choic of cost fuctios dpds o th approach usd ad th applicatio of itrst som commoly usd cost fuctios ar ma squar rror MS critrio : miimizs { } hr dots pctatio opratio, d y is th stimatio rror, d is th dsird rspos ad y is th actual filtr output potially ightd last squars critrio : miimizs λ N N hr N is th total umbr of sampls ad λ dots th potially ightig factor hos valu is positiv clos to.
. Algorithm dpds o th cost fuctio usd covrgc of th algorithm : Will th cofficits of th adaptiv filtr covrg to th dsird valus? Is th algorithm stabl? Global covrgc or local covrgc? rat of covrgc : his corrspods to th tim rquird for th algorithm to covrg to th optimum last squars/wir solutio. misadjustmt : css ma squar rror MS ovr th miimum MS producd by th Wir filtr, mathmatically it is dfid as lim { } εmi M 4. ε his is a prformac masur for algorithms that us th miimum MS critrio mi
tracig capability : his rfrs to th ability of th algorithm to trac statistical variatios i a ostatioary viromt. computatioal rquirmt : umbr of opratios, mmory siz, ivstmt rquird to program th algorithm o a computr. robustss : his rfrs to th ability of th algorithm to oprat satisfactorily ith ill-coditiod data,.g. vry oisy viromt, chag i sigal ad/or ois modls 3. Structur structur ad algorithm ar itr-rlatd, choic of structurs is basd o quatizatio rrors, as of implmtatio, computatioal complity, tc. four commoly usd structurs ar dirct form, cascad form, paralll form, ad lattic structur. Advatags of lattic structurs iclud simpl tst for filtr stability, modular structur ad lo ssitivity to quatizatio ffcts.
.g., p z z D p z z D p z z C p z z C A z A z z B z H Q. Ca you s a advatag of usig cascad or paralll form? 3
Commoly Usd Mthods for Miimizig MS For simplicity, it is assumd that th adaptiv filtr is of causal FI typ ad is implmtd i dirct form. hrfor, its systm bloc diagram is z - z -... z - L- L-... y - d 4
h rror sigal at tim is giv by y d 4. hr W i y L i i, L L L L W ] [ ] [ L L call that miimizig th ill giv th Wir solutio i optimal filtrig, it is dsird that } { d MMS W W lim 4.3 I adaptiv filtrig, th Wir solutio is foud through a itrativ procdur, W W W 4.4 hr W is a icrmtig vctor. 5
o commo gradit sarchig approachs for obtaiig th Wir filtr ar. Nto Mthod µ } { W W 4.5 hr µ is calld th stp siz. It is a positiv umbr that cotrols th covrgc rat ad stability of th algorithm. h adaptiv algorithm bcoms MMS d d W W W W W W W W µ µ µ µ µ µ } { 4.6 6
Solvig th quatio, hav W W MMS µ W W MMS 4.7 hr W is th iitial valu of W. o sur th choic of µ should b lim W W 4.8 MMS < µ < < µ < 4.9 I particular, h µ. 5, hav W W.5 W W MMS MMS W MMS 4. h ights jump form ay iitial W to th optimum sttig W MMS i a sigl stp. 7
A ampl of th Nto mthod ith µ. 5 ad ights is illustratd blo. 8
. Stpst Dsct Mthod hus W W W µ { } µ W { W µ W W } d 4. 4. I I µ µ W µ W W W MMS MMS W MMS hr I is th L L idtity matri. Dot W hav V W 4.3 W MMS V I µ V 4.4 9
Usig th fact that is symmtric ad ral, it ca b sho that Q Λ Q Q Λ Q 4.5 hr th modal matri Q is orthoormal. h colums of Q, hich ar th L igvctors of that Q Q. Whil, ar mutually orthogoal ad ormalizd. Notic Λ is th so-calld spctral matri ad all its lmts ar zro cpt for th mai diagoal, hos lmts ar th st of igvalus of, λ, λ, L, λ L. It has th form Λ λ M λ L O L M λ L 4.6
It ca b provd that th igvalus of ar all ral ad gratr or qual to zro. Usig ths rsults ad lt W hav h solutio is Q U U I Q I V Q U 4.7 µ µ Q I Q µ Q I µλ U Q U Q U I µλ hr U is th iitial valu of U algorithm is stabl ad covrgt if Q U 4.8 U U 4.9 I µλ lim. hus th stpst dsct
or lim µλ M hich implis lim µλ L L O hr λ ma is th largst igvalu of. lim M µλ ma L 4. µλma < < µ < 4. λ If this coditio is satisfid, it follos that lim U lim W W lim Q MMS V lim Q W W MMS 4.
A illustratio of th stpst dsct mthod ith to ights ad µ.3 is giv as blo. 3
mars: Stpst dsct mthod is simplr tha th Nto mthod sic o matri ivrsio is rquird. h covrgc rat of Nto mthod is much fastr tha that of th stpst dsct mthod. Wh th prformac surfac is uimodal, W ca b arbitrarily chos. If it is multimodal, good iitial valus of W is cssary i ordr for global miimizatio. Hovr, both mthods rquir act valus of ad d hich ar ot commoly availabl i practical applicatios. 4
Widro s Last Ma Squar LMS Algorithm A. Optimizatio Critrio o miimiz th ma squar rror { } B. Adaptatio Procdur It is a approimatio of th stpst dsct mthod hr th pctatio oprator is igord, i.., { } W is rplacd by W 5
h LMS algorithm is thrfor: [ ], W B A B A W W d W W W W W W µ µ µ µ or,,,, µ L i i i i L 4.3 6
C. Advatags lo computatioal complity simpl to implmt allo ral-tim opratio dos ot d statistics of sigals, i.., ad d D. Prformac Surfac h ma squar rror fuctio or prformac surfac is idtical to that i th Wir filtrig: W W W W { } ε mi MMS MMS 4.4 hr W is th adaptiv filtr cofficit vctor at tim. 7
. Prformac Aalysis o importat prformac masurs i LMS algorithms ar rat of covrgc & misadjustmt rlats to stady stat filtr ight variac.. Covrgc Aalysis For as of aalysis, it is assumd that W is idpdt of. aig pctatio o both sids of th LMS algorithm, hav { W } { W } µ { } { W } { W } µ µ { d W } d µ I µ { W } µ W MMS { W } 4.5 hich is vry similar to th adaptiv quatio 4. i th stpst dsct mthod. 8
Folloig th prvious drivatio, W ill covrg to th Wir filtr ights i th ma ss if lim µλ L lim µλ M M O L lim µλ L µλ <, i,, L i, < µ < λma 4.6 Dfi gomtric ratio of th pth trm as r µλ, p,, L L 4.7 p p, It is obsrvd that ach trm i th mai diagoal forms a gomtric sris {, r, r, L, r, r, r, L}. p p p p p L 9
potial fuctio ca b fittd to approimat ach gomtric sris: { } r p p r p p τ p τ p 4.8 hr τ p is calld th pth tim costat. For slo adaptatio, i.., µλ p <<, τ p is approimatd as τ p τ p l µλ µλ p p µλ p µλ µλ p 3 p 3 L µλ p 4.9 Notic that th smallr th tim costat th fastr th covrgc rat. Morovr, th ovrall covrgc is limitd by th slost mod of covrgc hich i turs stms from th smallst igvalu of, λ mi. 3
hat is, τ ma 4.3 µλ mi I gral, th rat of covrgc dpds o to factors: th stp siz µ : th largr th µ, th fastr th covrgc rat th igvalu sprad of, χ : th smallr χ, th fastr th covrgc rat. χ is dfid as λma χ 4.3 λ mi Notic that χ <. It is orthy to ot that although χ caot b chagd, th rat of covrgc ill b icrasd if trasform to aothr squc, say, y, such that χ is clos to. yy 3
ampl 4. A Illustratio of igvalu sprad for LMS algorithm is sho as follos. Uo Systm Σ h z i i Σ z i i -i -i d y d h h - q y - d y d - u u- Σ Σ - q 3
; fil am is s.m clar all N; % umbr of sampl is p.; % ois por is. sp ; % sigal por is hich implis SN db h[ ]; % uo impuls rspos sqrtsp.*rad,n; d cov,h; d d:n sqrtp.*rad,n; ; % iitial filtr ights ar ; mu.5; % stp siz is fid at.5 y *; % itratio at d - y; % sparat bcaus is ot dfid *mu**; ; 33
for :N % th LMS algorithm y * *-; d - y; *mu**; *mu**-; d :N; subplot,, plot, % plot filtr ight stimat vrsus tim ais[.] subplot,, plot, ais[.] figur subplot,, :N; smilogy,.*; % plot squar rror vrsus tim 34
35
36
Not that both filtr ights covrg at similar spd bcaus th igvalus of th ar idtical: call For hit procss ith uity por, hav {. } {. } As a rsult, χ 37
; fil am is s.m clar all N; p.; sp ; h[ ]; u sqrtsp/.*rad,n; u:n u:n; % is o a MA procss ith por d cov,h; d d:n sqrtp.*rad,n; ; ; mu.5; y *; d - y; *mu**; ; 38
for :N y * *-; d - y; *mu**; *mu**-; d :N; subplot,, plot, ais[.] subplot,, plot, ais[.] figur subplot,, :N; smilogy,.*; 39
4
4
Not that th covrgc spd of is slor tha that of Ivstigatig th : As a rsult, {. } { u { u { u u u { u u } u.5.5 {. }.5 } u.5 u u.5 } } λ mi. 5 ad λ ma. 5 χ 3 MALAB commad: ig 4
; fil am is s.m clar all N; p.; sp ; h[ ]; u sqrtsp/5.*rad,n4; u:n u:n u3:n u4:n3 u5:n4; d cov,h; d d:n sqrtp.*rad,n; % is 5th ordr MA procss ; ; mu.5; y *; d - y; *mu**; ; 43
for :N y * *-; d - y; *mu**; *mu**-; d :N; subplot,, plot, ais[.5] subplot,, plot, ais[.5] figur subplot,, :N; smilogy,.*; 44
45
46
W s that th covrgc spds of both ights ar vry slo, although that of is fastr. Ivstigatig th : As a rsult, {. } { u.8 u }... {. }} { u { u u.. } { u u.8 3 u 3}.8 { u 4} 4} λ mi. ad λ ma. 8 χ 9 47
. Misadjustmt Upo covrgc, if qual to lim W W MMS, th th miimum MS ill b { d } d W MMS ε mi 4.3 Hovr, this ill ot occur i practic du to radom ois i th ight vctor W. Notic that hav lim { W } W MMS but ot lim W W MMS. h MS of th LMS algorithm is computd as { } ε ε mi mi d W { W W } MMS W W MMS { W W W W } MMS MMS 4.33 48
h scod trm of th right had sid at MS ad it is giv by is o as th css css MS lim lim lim µε { W W W W } { V } V { Λ U } U L λ mi i i µε MMS mi tr [ ] MMS 4.34 hr [ ] tr is th trac of hich is qual to th sum of all lmts of th pricipl diagoal: tr [ ] L L { } 4.35 49
As a rsult, th misadjustmt M is giv by M lim µε mi µ L { L ε ε { } ε mi { mi } } hich is proportioal to th stp siz, filtr lgth ad sigal por. mars: mi 4.36.hr is a tradoff bt fast covrgc rat ad small ma squar rror or misadjustmt. Wh µ icrass, both th covrgc rat ad M icras; if µ dcrass, both th covrgc rat ad M dcras. 5
.h boud for µ is < µ < 4.37 λ I practic, th sigal por of ca grally b stimatd mor asily tha th igvalu of. W also ot that λ ma L λ i i tr ma [ ] L { } A mor rstrictiv boud for µ hich is much asir to apply thus is < µ < L { } 4.38 4.39 Morovr, istad of a fid valu of µ, ca ma it tim-varyig as µ. A dsig ida of a good µ is larg valu iitially, to sur fast iital covrgc rat µ small valu fially, to sur small misadjustmt upo covrgc 5
LMS Variats. Normalizd LMS NLMS algorithm th product vctor is modifid ith rspct to th squard uclida orm of th tap-iput vctor : µ W W 4.4 c hr c is a small positiv costat to avoid divisio by zro. ca also b cosidrd as a LMS algorithm ith a tim-varyig stp siz: µ µ 4.4 c substitutig c, it ca b sho that th NLMS algorithm covrgs if < µ <.5 slctio of stp siz i th NLMS is much asir tha that of LMS algorithm 5
. Sig algorithms pilot LMS or sigd rror or sigd algorithm: W W µ sg[ ] 4.4 clippd LMS or sigd rgrssor: W W µ sg[ ] 4.43 zro-forcig LMS or sig-sig: W W µ sg[ ]sg[ ] 4.44 thir computatioal complity is simplr tha th LMS algorithm but thy ar rlativly difficult to aalyz 53
3. Lay LMS algorithm th LMS updat is modifid by th prsc of a costat laag factor γ: hr < γ <. W γ W µ 4.45 oprats h has zro igvalus. 4. Last ma fourth algorithm istad of miimizig { }, { 4 } is miimizd basd o LMS approach: 3 W W 4µ 4.46 ca outprform LMS algorithm i o-gaussia sigal ad ois coditios 54
Applicatio ampls ampl 4.. Liar Prdictio Suppos a sigal is a scod-ordr autorgrssiv A procss that satisfis th folloig diffrc quatio:.558.8 v hr v is a hit ois procss such that vv m { v v m} σv, m, othris W at to us a to-cofficit LMS filtr to prdict by ˆ i i i 55
Upo covrgc, dsir { }.558 ad { }.8 z - z - d - 56
h rror fuctio or prdictio rror is giv by i d i i hus th LMS algorithm for this problm is µ µ µ ad µ µ h computatioal rquirmt for ach samplig itrval is multiplicatios : 5 additio/subtractio : 4 57
o valus of µ,. ad.4, ar ivstigatd: Covrgc charactristics for th LMS prdictor ith µ. 58
Covrgc charactristics for th LMS prdictor ith µ. 4 59
Obsrvatios:.Wh µ., had a fast covrgc rat th paramtrs covrgd to th dsird valus i approimatly itratios but larg fluctuatio istd i ad..wh µ. 4, small fluctuatio i ad but th filtr cofficits did ot covrg to th dsird valus of.558 ad -.8 rspctivly aftr th 3th itratio. 3.h larig bhaviours of ad agrd ith thos of { } ad { }. Notic that { } ad { } ca b drivd by taig pctatio o th LMS algorithm. 6
ampl 4.3. Systm Idtificatio Giv th iput sigal ad output sigal d, ca stimat th impuls rspos of th systm or plat usig th LMS algorithm. Suppos th trasfr fuctio of th plat is h hich is a causal FI i i i z uo systm, th d ca b rprstd as d h i i i 6
Assumig that th ordr th trasfr fuctio is uo ad us a - cofficit LMS filtr to modl th systm fuctio as follos, Plat Σ h z i i -i Σ z i i -i Σ d y - h rror fuctio is computd as d i d y d i i 6
hus th LMS algorithm for this problm is µ µ µ ad µ µ h larig bhaviours of th filtr ights ad ca b obtaid by taig pctatio o th LMS algorithm. o simplify th aalysis, assum that is a statioary hit ois procss such that σ othris,, } { m m m 63
Assum th filtr ights ar idpdt of ad apply pctatio o th first updatig rul givs { } { } } { } { } { } { } { } { } { } { } { } { } { i i i i h h h h h h h i i h y d σ µ σ µ µ µ µ µ µ µ µ µ µ 64
} { } { } { } { } { } { } { } { h h h h σ µ µσ σ µ µσ σ µ µσ σ µ µσ LLLLLLLLLLLLLLLLLLLL Multiplyig th scod quatio by o both sids, th third quatio by, tc., ad summig all th rsultat quatios, hav µσ µσ { } { } { } { } { } { } { } { } h h h h h µσ µσ µσ µσ µσ σ µ µσ µσ µσ σ µ µσ L 65
Hc providd that µσ lim { } < < µσ < < µ < σ Similarly, ca sho that th pctd valu of { } is h providd that } { } h { µσ h µσ < < µσ < < µ < σ It is orthy to ot that th choic of th iitial filtr ights { } ad { } do ot affct th covrgc of th LMS algorithm bcaus th prformac surfac is uimodal. 66
Discussio: Sic th LMS filtr cosists of to ights but th actual trasfr fuctio compriss thr cofficits. h plat caot b actly modld i this cas. his rfrs to udr-modlig. If us a 3-ight LMS filtr ith trasfr fuctio i i z, th th plat ca b modld actly. If us i mor tha 3 cofficits i th LMS filtr, still stimat th trasfr fuctio accuratly. Hovr, i this cas, th misadjustmt ill b icrasd ith th filtr lgth usd. Notic that ca also us th Wir filtr to fid th impuls rspos of th plat if th sigal statistics,,, d ad d ar availabl. Hovr, do ot hav ad although d σ ad ar o. hrfor, th LMS adaptiv filtr ca b cosidrd as a adaptiv ralizatio of th Wir filtr ad it is usd h th sigal statistics ar ot compltly o. d 67
ampl 4.4 3. Itrfrc Cacllatio Giv a rcivd sigal r hich cosist of a sourc sigal s ad a siusoidal itrfrc ith o frqucy. h tas is to tract s from r. Notic that th amplitud ad phas of th siusoid is uo. A ll-o applicatio is to rmov 5/6 Hz por li itrfrc i th rcordig of th lctrocardiogram CG. Sourc Sigal Siusoidal Itrfrc rs Acosω φ frc Sigal siω b - Σ 9 Phas-Shift cosω b - 68
h itrfrc cacllatio systm cosists of a phas-shiftr ad a to-ight adaptiv filtr. By proprly adjustig th ights, th rfrc avform ca b chagd i magitud ad phas i ay ay to modl th itrfrig siusoid. h filtrd output is of th form 9 cos si b b r ω ω h LMS algorithm is si b b b b b ω µ µ µ ad cos b b b b ω µ µ 69
aig th pctd valu of b, hav { } { } si } { si } { cos si } { si si si cos } { } si ] cos si si si cos cos {[ } { } si ] cos si cos {[ } { } si { } { } { φ µ µ µ φ µ ω φ µ ω φ µ ω φ µ ω ω ω φ ω φ ω µ ω ω ω φ ω µ ω µ A b b A b b A b b A b A b b b A A b b b A s b b b 7
Folloig th drivatio i ampl 4.3, providd that < µ < 4, th larig curv of { b } ca b obtaid as { b } Asi φ Similarly, { b } is calculatd as { b } Asi φ µ Wh { b } Acos φ, hav { b } Acos φ µ ad lim { b } A si φ lim { b } A cos φ 7
h filtrd output is th approimatd as r Asi φsi ω Acos φ cos ω s hich mas that s ca b rcovrd accuratly upo covrgc. Suppos { b } { b }, µ. ad at to fid th umbr of itratios rquird for { b } to rach 9% of its stady stat valu. Lt th rquird umbr of itratios b ad it ca b calculatd from { b }.9Acos φ.. log. 9. log.99 Hc 3 itratios ar rquird. Acos φ µ Acos φ 7
If us Wir filtr ith filtr ights ad, th ma squar rror fuctio ca b computd as b b } { cos si } { A s b A b A b b φ φ h Wir cofficits ar foud by diffrtiatig ith rspct to b ad b ad th st th rsultat prssio to zros. W hav } { si ~ si } { φ φ A b A b b ad cos ~ cos } { φ φ A b A b b 73
ampl 4.5 4. im Dlay stimatio stimatio of th tim dlay bt to masurd sigals is a problm hich occurs i a divrs rag of applicatios icludig radar, soar, gophysics ad biomdical sigal aalysis. A simpl modl for th rcivd sigals is r r s αs D hr s is th sigal of itrst hil ad ar additiv oiss. h α is th attuatio ad D is th tim dlay to b dtrmid. I gral, D is ot a itgral multipl of th samplig priod. Suppos th samplig priod is scod ad s is badlimitd bt -.5 Hz ad.5 Hz - π rad/s ad π rad/s. W ca driv th systm hich ca produc a dlay of D as follos. 74
aig th Fourir trasform of D s s D yilds ˆ ω ω ω S S D j his mas that a systm of trasfr fuctio D j ω ca grat a dlay of D for s. Usig th ivrs DF formula of I.9, th impuls rspos of jωd is calculatd as sic D d d h D j j D j ω π ω π π π ω π π ω ω hr v v v π π si sic 75
As a rsult, s D ca b rprstd as s D s h for sufficitly larg P. i P ip s s isic i isic i D D his mas that ca us a o-casual FI filtr to modl th tim dlay ad it has th form: W z P ip It ca b sho that i β sic i D for i P, P, L, P usig th miimum ma squar rror approach. h tim dlay ca b stimatd from } usig th folloig itrpolatio: { i Dˆ arg ma t P ip i z i sic i i t 76
r P Σ Wz i z i-p i -i - Σ r P r r i ip i 77
h LMS algorithm for th tim dlay stimatio problm is thus P P P j j r j j j j j j,,,, L µ µ µ h tim dlay stimat at tim is: sic arg ma ˆ t i D i P P i t 78
potially Wightd cursiv Last-Squars A. Optimizatio Critrio o miimiz th ightd sum of squars hr λ is a ightig factor such that < λ. J λ l for ach tim Wh λ, th optimizatio critrio is idtical to that of last squarig filtrig ad this valu of λ should ot b usd i a chagig viromt bcaus all squard rrors currt valu ad past valus hav th sam ightig factor of. o smooth out th ffct of th old sampls, λ should b chos lss tha for opratig i ostatioary coditios. B. Drivatio Assum FI filtr for simplicity. Folloig th drivatio of th last squars filtr, diffrtiat J ith rspct to th filtr ight vctor at tim, i.., W, ad th st th L rsultat quatios to zro. l l 79
By so doig, hav G W 4.47 hr l l l l λ λ l l l l d G L l L l l l l ] [ L Notic that ad G ca b computd rcursivly from l l l l λ λ 4.48 d G d l l d G l l λ λ 4.49 8
Usig th ll-o matri ivrsio lmma: If C C B A 4.5 hr A ad B ar N N matri ad C is a vctor of lgth N, th B C C B C C B B A 4.5 hus ca b ritt as λ λ 4.5 8
h filtr ight W is calculatd as [ ] W d W d W d d W d G d G d G G W λ λ λ λ λ λ λ λ λ λ λ λ λ λ 8
As a rsult, th potially ightd rcursiv last squars LS algorithm is summarizd as follos,. Iitializ W ad. For,, L, comput d W 4.53 α 4.54 λ W W α 4.55 λ [ α ] 4.56 83
mars:.wh λ, th algorithm rducs to th stadard LS algorithm that miimizs l. l.for ostatioary data,.95 < λ <. 9995 has b suggstd. 3.Simpl choics of W ad σ is a small positiv costat. C. Compariso ith th LMS algorithm. Computatioal Complity ar ad σ I, rspctivly, hr LS is mor computatioally psiv tha th LMS. Assum thr ar L filtr taps, LMS rquirs 4 L additios ad 4 L 3 multiplicatios pr updat hil th potially ightd LS ds a total of 3L L additios/subtractios ad 4L 4L multiplicatios/divisios. 84
. at of Covrgc LS provids a fastr covrgc spd tha th LMS bcaus LS is a approimatio of th Nto mthod hil LMS is a approimatio of th stpst dsct mthod. th pr-multiplicatio of i th LS algorithm mas th rsultat igvalu sprad bcoms uity. Improvmt of LMS algorithm ith th us of Orthogoal rasform A. Motivatio Wh th iput sigal is hit, th igvalu sprad has a miimum valu of. I this cas, th LMS algorithm ca provid optimum rat of covrgc. Hovr, may practical sigals ar ohit, ho ca improv th rat of covrgc usig th LMS algorithm? 85
B. Ida o trasform th iput to aothr sigal v so that th modifid igvalu sprad is. o stps ar ivolvd:. rasform to v usig a N N orthogoal trasform so that hr vv σ M σ L V L O σ N σ M N V L v ] [ v v vn [ L N N ] N 86
{ } { } vv V V. Modify th igvalus of vv so that th rsultat matri has idtical igvalus: σ σ σ σ ormalizatoi por ' L O M M L vv vv 87
Bloc diagram of th trasform domai adaptiv filtr 88
C. Algorithm h modifid LMS algorithm is giv by V W W Λ µ hr σ σ σ σ Λ / / / / N N L O M M L y d W V W y N N W ] [ L 89
Writig i scalar form, hav i i µ v σ i i, i,, L, N Sic σ i is th por of v i ad it is ot o a priori ad should b stimatd. A commo stimatio procdur for { v } is i hr σ i ασi v < α < i I practic, α should b chos clos to, say, α. 9. 9
Usig a -cofficit adaptiv filtr as a ampl: A -D rror surfac ithout trasform 9
rror surfac ith discrt cosi trasform DC 9
rror surfac ith trasform ad por ormalizatio 93
mars:. h lgths of th pricipl as of th hyprllipss ar proportioal to th igvalus of.. Without por ormalizatio, o covrgc rat improvmt of usig trasform ca b achivd. 3. h bst choic for should b Karhu-Lov KL trasform hich is sigal dpdt. his trasform ca ma vv to a diagoal matri but th sigal statistics ar rquird for its computatio. 4. Cosidratios i choosig a trasform: fast algorithm ists? compl or ral trasform? lmts of th trasform ar all por of? 5. ampls of orthogoal trasforms ar discrt si trasform DS, discrt Fourir trasform DF, discrt cosi trasform DC, Walsh-Hadamard trasform WH, discrt Hartly trasform DH ad por-of- PO trasform. 94
Improvmt of LMS algorithm usig Nto's mthod Sic th igvalu sprad of Nto basd approach is, ca combi th LMS algorithm ad Nto's mthod to form th "LMS/Nto" algorithm as follos, µ W W W mars: W µ. h computatioal complity of th LMS/Nto algorithm is smallr tha th LS algorithm but gratr tha th LMS algorithm.. Wh is ot availabl, it ca b stimatd as follos, ˆ l, αˆ l, l, l,,, L L hr ˆ l, rprsts th stimat of l at tim ad < α < 95
Possibl sarch Dirctios for Adaptiv Sigal Procssig. Adaptiv modlig of o-liar systms For ampl, scod-ordr Voltrra systm is a simpl o-liar systm. h output y is rlatd to th iput by L L y j j j, j j j j j L j Aothr rlatd rsarch dirctio is to aalyz o-liar adaptiv filtrs, for ampl, ural tors, hich ar grally mor difficult to aalyz its prformac.. N optimizatio critrio for No-Gaussia sigals/oiss For ampl, LMF algorithm miimizs { 4 }. I fact, a class of stpst dscd algorithms ca b gralizd by th last-ma-p LMP orm. h cost fuctio to b miimizd is giv by J { p } 96
Som rmars: Wh p, it bcoms last-ma-dviatio LMD, h p, it is last-ma-squar LMS ad if p4, it bcoms th last-ma-fourth LMF. h LMS is optimum for Gaussia oiss ad it may ot b tru for oiss of othr probability dsity fuctios PDFs. For ampl, if th ois is impulsiv such as a α-stabl procss ith α <, LMD prforms bttr tha LMS; if th ois is of uiform distributio or if it is a siusoidal sigal, th LMF outprforms LMS. hrfor, th optimum p dpds o th sigal/ois modls. h paramtr p ca b ay ral umbr but it ill b difficult to aalyz, particularly for o-itgr p. Combiatio of diffrt orms ca b usd to achiv bttr prformac. Som suggsts mid orm critrio,.g. a { } b { 4 } 97
Mdia opratio ca b mployd i th LMP algorithm for opratig i th prsc of impulsiv ois. For ampl, th mdia LMS blogs to th family of ordr-statistics-last-ma-squar OSLMS adaptiv filtr algorithms. 3. Adaptiv algorithms ith fast covrgc rat ad small computatio For ampl, dsig of optimal stp siz i LMS algorithms 4. Adaptiv II filtrs Adaptiv II filtrs hav advatags ovr adaptiv FI filtrs: It gralizs FI filtr ad it ca modl II systm mor accuratly Lss filtr cofficits ar grally rquird Hovr, dvlopmt of adaptiv II filtrs ar grally mor difficult tha th FI filtrs bcaus h prformac surfac is multimodal th algorithm may loc at a udsird local miimum It may lad to biasd solutio It ca b ustabl 98
5. Usuprvisd adaptiv sigal procssig blid sigal procssig What hav discussd prviously rfrs to suprvisd adaptiv sigal procssig hr thr is alays a dsird sigal or rfrc sigal or traiig sigal. I som applicatios, such sigals ar ot availabl. o importat applicatio aras of usuprvisd adaptiv sigal procssig ar: Blid sourc sparatio.g. spar idtificatio i th oisy viromt of a coctail party.g. sparatio of sigals ovrlappd i tim ad frqucy i irlss commuicatios Blid dcovolutio ivrs of covolutio.g. rstoratio of a sourc sigal aftr propagatig through a uo irlss chal 6. N applicatios For ampl, cho cacllatio for had-fr tlpho systms ad sigal stimatio i irlss chals usig spac-tim procssig. 99
Qustios for Discussio. h LMS algorithm is giv by 4.3: µ i, i,,, L i i L hr d y y L i i i W, Basd o th ida of LMS algorithm, driv th adaptiv algorithm that miimizs { }. v Hit: sg v v hr sg v if v > ad sg v othris
. For adaptiv II filtrig, thr ar basically to approachs, amly, output-rror ad quatio-rror. Lt th uo II systm b i i M i j j N j z a z b z A z B z H Usig miimizig ma squar rror as th prformac critrio, th output-rror schm is a dirct approach hich miimizs } { hr y d ith
ˆ ˆ ˆ ˆ ˆ ˆ i y a j b y z a z b z A z B z z Y i M i j N j i i M i j j N j Hovr, as i Q. of Chaptr 3, this approach has to problms, amly, stability ad multimodal prformac surfac. O th othr had, th quatio-rror approach is alays stabl ad has a uimodal surfac. Its systm bloc diagram is sho i th t pag. Ca you s th mai problm of it ad suggst a solutio? Hit: Assum is hit ad ami } {
s Uo systm Hz d Σ r Bz Σ Az 3