Yair Weiss. Suppose we are given a set of datapoints that were generated by multiple processes,

Transcription

1 Moton Segmentaton usng EM - a short tutoral Yar Wess MIT E-, Cambrdge, MA 9, USA ywess@psyche.mt.edu The expectaton-maxmzaton algorthm (EM) s used for many estmaton problems n statstcs. Here we gve a short tutoral on how to program a segmentaton algorthm usng EM. Those nterested n the theory or n more advanced versons of the algorthm should consult the references at the end. Suppose we are gven a set of dataponts that were generated by multple processes, for example two lnes. We need to estmate two thngs: () the parameters (slope and ntercept) of the two lnes and () the assgnment of each datapont to the process that generated t. The ntuton behnd EM s that each of these steps s easy assumng the other one s solved. That s, assumng we know the assgnment of each datapont, then we can estmate the parameters of each lne by takng nto consderaton only those ponts assgned to t. Lkewse, f we know the parameters of the lnes we can assgn each pont to the lne that ts t best. Ths gves the basc structure of an EM algorthm: start wth random parameter values for the two models. Iterate untl parameter values converge: { E step: assgn ponts to the model that ts t best. { M step: update the parameters of the models usng only ponts assgned to t. In fact both steps are slghtly more complcated, due to the assgnment beng contnuous rather than bnary valued. The followng sectons go nto more detal regardng the E step and the M step. The Expectaton (E) step In the E step we compute for each datapont twoweghts w ();w () (the soft assgnment of the pont to models and respectvely ). Agan, we assume that the parameters of the processes are known. Thus for each datapont we can calculate two resduals r ();r () - the derence between the observaton at pont and the predctons of each model. In the case of lne ttng the resdual s smply gven by: r () =a x +b y () Throughout ths tutoral we assume the number of models s known and s equal to two. A method for automatcally estmatng the number of models s presented n (Wess and Adelson, 996)

2 and smlarly for r (). For example, suppose we havetwo lne models: () y = x + and () y =x. Suppose the th datapont sx=;y =:. Then the resdual for lne s r()=(+ :) =:9 and for lne we get r() =( : )=:. Intutvely we would expect ths datapont to be assgned to model because the resdual s smaller. Indeed the formula for the weghts s consstent wth the ntuton: w () = w () = e r ()= e r ()= + e r ()= () e r ()= e r ()= + e r ()= () There are a few thngs to note about ths formula. Frst, note that w () and w () sum to one. Ths s because these weghts are actually probabltes - the formula s smply derved from Bayes rule. Second, note that there s a "free parameter" here. Roughly speakng, ths parameter corresponds to the amount of resdual expected n your data (e.g. the nose level). Fnally, note that f r () smuch smaller than r () then w () = and w () =. For ths reason equatons - are sometmes known as the "softmn" equatons. They are a way of generalzng the concept of a mnmum of two numbers nto a smooth functon. More detals on ths can be found n the references. So to summarze, the E step calculates two weghts for every datapont by rst calculatng resduals (usng the parameters of each model) and then runnng the resduals through a softmn functon as dened above. The Maxmzaton (M) step In the M step we assume the weghts are gven,.e. for each datapont we know w () and w (). To estmate the parameters of each process we just use weghted least squares. An example of least squares estmaton s when we want to t a lne to dataponts. It s easy to show that the parameters of the lne a; b are a soluton to the followng equaton:!" " x x x a b = x y y Now nweghted least squares we are also gven w for each pont and the equaton smply become: w x!" " w x a = w x y (4) w x w b w y So n the M step we solve the above equaton twce. Frst wth w = w () for the parameters of lne and then wth w = w () for the parameters of lne. In general, n the M step we solve twoweghted least squares - one for each model, wth the weghts gven by the results of the E step. So what about moton segmentaton? To apply the EM algorthm for moton segmentaton we need to make the followng decsons:

3 t = t= t= Fgure : An example of the EM algorthm for ttng two lnes to data. The algorthm starts at random condtons and converges n three teratons. The top panels show the lne ts at every teraton and the bottom panels show the weghts. (data was generated by settng y = x+ for jx :5j < :5 and y = x otherwse. The parameter =:).

4 What s the class of possble moton models we are consderng? What are the parameters that need to be estmated for each model? (e.g. translatonal models or ane models) What s the nature of the data that the models need to t? (e.g. raw pxel data or results of optcal ow). Here we wll descrbe the smplest verson of an EM based moton segmentaton algorthm: the models wll be assumed to be translatonal and the mage data wll be results of optcal ow. More complex versons are descrbed n the references. We are gven the results of optcal ow v x (; j) and v y (; j). We assume ths ow was generated by two global translaton models. Each translaton model s characterzed by two numbers: (u; v) whch gve the horzontal and vertcal translaton. The problem s to estmate the parameters (u ;v );(u ;v )aswell as assgnng each pxel to the model that generated t. The E step s entrely analogous to the one descrbed for lne ttng. We estmate two weghts for every pxel w (; j) and w (; j). Ths s done by assumng (u ;v );(u ;v ) are known and calculatng resduals: r (; j) = (u v x (; j)) +(v v y (; j)) (5) r (; j) = (u v x (; j)) +(v v y (; j)) (6) The resduals are converted nto weghts by passng them through the softmn functon: w (; j) = e r (;j)= e r (;j)= + e r (;j)= (7) e r (;j)= w (; j) = (8) e r (;j)= + e r (;j)= The M step s also analogous to the one used for lne ttng. In ths case, the parameters (u ;v ) satsfy:!" " w ;j (; j) u = w (; j)v w ;j (; j) ;j x (; j) (9) v w ;j (; j)v y (; j) And the equatons for (u ;v ) are the same wth w (; j) replaced everywhere by w (; j). To summarze, the moton segmentaton starts by choosng an ntal random value for the translaton parameters, and teratng the E and M steps detaled above untl the translaton parameters converge. In the smple case descrbed here, snce the EM algorthm s usng the results of an optcal ow algorthm, t makes sense to get the nal segmentaton by gong back to the raw pxel data. That s, after the algorthm has converged and two translaton models have been found, we let the models \compete" to explan the pxel data. Ths s done by: warpng frame to frame wth the translaton of model. Subtractng the warped mage from the true frame. Ths gves the predcton error of model. repeatng the prevous step wth the translaton of model and obtanng the predcton error of model. assgnng each pxel to the model whose predcton error s lowest at that pxel. 4

5 4 Concluson Ths tutoral has dscussed how to use the EM algorthm for moton segmentaton. Although wehave gven an nformal exposton, t should be noted that the actual algorthm s derved from a rgorous statstcal estmaton framework and proofs about convergence and rate of convergence can be found n the lterature. The combnaton of the algorthm's prncpled dervaton and ntutvely decoupled steps s probably responsble for ts contnued success n many estmaton problems. References Ayer, S. and Sawhney, H. S. (995). Layered representaton of moton vdeo usng robust maxmum lkelhood estmaton of mxture models and mdl encodng. In roc. Int'l Conf. Comput. Vson, pages 777{784. Dempster, A.., Lard, N. M., and Rubn, D. B. (977). Maxmum lkelhood from ncomplete data va the EM algorthm. J. R. Statst. Soc. B, 9:{8. Jepson, A. and Black, M. J. (99). Mxture models for optcal ow computaton. In roc. IEEE Conf. Comput. Vson attern Recog., pages 76{76, New York. Wess, Y. and Adelson, E. H. (994). erceptually organzed EM: a framework for moton segmentaton that combnes nformaton about form and moton. Techncal Report 5, MIT Meda Lab, erceptual Computng Secton. Wess, Y. and Adelson, E. H. (996). A uned mxture framework for moton segmentaton: ncorporatng spatal coherence and estmatng the number of models. In roc. IEEE Conf. Comput. Vson attern Recog., pages {6. 5