Shadow Elimination Method for Video Surveillance

Transcription

1 Vol 3, No 7 Modern Appled Scence Shadow Elmnaton Method for Vdeo Survellance Huaqang Lu Department of Physcs Lny Normal Unversty Lny , Chna E-mal: huaqang1019@163com Feng Guo College of Engneerng Lny Normal Unversty Lny , Chna E-mal: guofeng_g@lytueducn Abstract Wth regard to the weaness and shortage of tradtonal movng object segmentaton method, ths paper presents an effectve segmentaton method for movng objects n vdeo survellance The dfference mage of color dstance whch s between current mage and the reference bacground mage n RGB color space s frst obtaned Accordng to the mono-modal feature of hstogram of the dfference mage, an adaptve clusterng segmentaton method based on hstogram s proposed The morphology flterng s employed to remove the nose exstng n the segmented bnary mage An updatng scheme for bacground mage s ntroduced to follow the varaton of llumnaton condtons and changes n envronmental condtons In order to remove unwanted shadows of movng regons, an effcent mult-object shadows dstngushng and elmnatng method for survellance scene was presented n ths paper Expermental results show that the proposed method s smple and effectve for movng object segmentaton and elmnatng shadows Keywords: Image processng, Segmentaton, Hstogram, Shadow elmnaton 1 Introducton Segmentaton and extracton of movng objects are basc tass n many applcatons such as vdeo survellance (Lpton A, 1998),vdeo retreval (NHoose, 1991), ntellgent transportaton, and so on In the past, there have been researchers nvestgatng nds of methods for segmentng movng objects n real tme to acheve these vson-based applcatons For example, dfference mage method was proposed n (Jng Zhong, 2003), and the optcal flow technque was ntroduced n (Barron J, 1994) The mpact of nose, many lght source, shadows (YMWu, 2002), transparency and shelter reduces the relablty and accuracy of the algorthm whch s based on optcal flow technology for movng object detecton methods Despte these dffcultes, computatonal complexty and tme-consumng further complcate the movng object detecton n real-tme montorng Image dfference method was dvded nto the bacground dfference method and the nter-frame dfference method The former algorthm s smple but t lacs a reasonable method for the bacground update, whch changes wth llumnaton and other factors The latter usng the adjacent frame dfference can extract nformaton of movng objects, but objects of sudden stoppng cannot be detected, and ths method can not solve the problem of bacground exposed and the overlappng of movng objects n the adjacent frames In addton, the current dfference method, whch ddn t mae full use of rch color nformaton, s generally lmted to two mages of the brghtness, but color nformaton s ndspensable n practcal applcaton In ths paper, a novel approach s proposed for separatng movng objects from current frame through hstogram, whch could be obtaned from results of bacground subtracton n color space In order to change n tme wth the llumnaton of bacground scene, the quc update strategy of reference bacground mage s gven Due to the smplcty of the proposed method, the movng objects can be extracted accurately and qucly 78

2 Modern Appled Scence July, Movng Objects Segmentaton Based on Hstogram 21 Determnng Color Dfference Model Tradtonal method of bacground subtracton was fulflled wth bacground mages and the current mage n gray space Because of lttle nformaton avalable n gray space, the gray value of the target and bacground are very close Fracture and large hole whch are not conducve to further processng would appear n dfference mage Through observaton and analyss of many mages, the same movng object and bacground n gray space are the same Generally, they would not be dentcal n color space Thus the color mage dfference model s chosen n ths paper RGB color model, whch was been wdely used n computer graphcs and magng, has an mportant poston n the computer feld Therefore, RGB color model was chosen, and transform formula s as follows: color dfference model s shown as follows: dd = a* d + d + d, (1) d = R (, j) R (, j), (2) 1 b d = G (, j) G (, j), (3) 2 b d = B (, j) B (, j), (4) 3 b where Y (, j ), U (, j) and V (, j) are three values of the pxel n the bacground respectvely, and Y (, j ), U (, j ) b b b and V (, j ) are three values of the pxel n the current frame The a s an experenced value, whch s obtaned from a large number of experments, and 06 s the value of a n the paper Because of full use of nformaton n the color dfference model, the method greatly mproves the ntegrty of the target regon 22 Segmentaton Algorthm Based on Hstogram Dfferental mage segmentaton algorthms manly nclude automatc threshold approach based on class varance, eucldean dstance among all nstances of dfferent class, the hstogram segmentaton method and the clusterng segmentaton method based on the Gaussan model However, the above methods are all not applcable, f color dstance s used as the dfference result n the color space Snce automatc threshold segmentaton method was most sutable n the bmodal hstogram of the mage, and the hstogram from color dstance as shown n Fg1 s sngle-pea form, Gaussan model-based approach s to assume that pxel values of dfference mage obey the Gaussan dstrbuton, n whch there must be the margn of plus or mnus Accordng to (1), the dstance value s postve, and negatve value does not exst Thus new segmentaton method should be explored In Fg1, horzontal axs represents the gray value of the mage Vertcal axs represents the frequency of occurrence of each gray value As can be seen from Fg1, the hstogram shows the characterstcs of a sngle and hghly concentrated pea Sngle-pea represents the bacground regon The color dfference s not exactly zero, but an nterval greater than zero The upper lmt of nterval s decded by the wdth of sngle-pea Based on these features, a segmentaton algorthm s presented as follows: 1) Fnd the maxmum pea value max f of the dfference mage hstogram and the gray value mc, whch s clusterng centre of the bacground regon and correspondng to the pea 2) See the wdth pw of pea Experments ndcate that pea wdth correspondng to 004 * max f heght can acheve more satsfactory segmentaton results, and pw could be obtaned by the statstcs of hstogram 3) Fnd clusterng radus r r = pw mc 4) In accordance wth the establshed clusterng centre and clusterng radus, clusterng starts on the dfference mage Suppose any pxel gray value s f and f mc r, the pxel belongs to bacground; otherwse, t belongs to movng object Obvously the clusterng centre and the clusterng radus depend on the sngle-pea shape of the hstogram and change wth t, so t has strong adaptve ablty 23 Morphologcal Method Dealng wth Bnary Image Wnd and other factors wll lead to changes of elements, such as trees, grass n bacground, so dfference mage contans lots of nose The dfference mage after bnarzaton stll has a lot of useless nose spots n the bacground and objects In ths paper, Mathematcal Morphology (Zhu,Wegang, 2002) s used to carry out post-processng of the segmented bnary mage Basc morphologcal operatons nclude expanson, corroson, openng, closng, etc Combned openng and closng can acheve morphologcal nose flter, whch can effectvely remove nose spots Snce the scale of morphology flter s fxed, the target regon may also have empty holes after segmentng Area surrounded method s appled to fll the hole of object n order to provde complete movng templates, beneftng to the bacground update and so on 79

3 Vol 3, No 7 Modern Appled Scence 24 Updatng Bacground Strategy In vdeo survellance, because of changes of outdoor lght and clmatc condton, bacground mage must have ts correspondng update strategy If no movng object s detected at a fxed tme nterval, the current frame mage substtutes for the bacground mage; otherwse, the bacground should be updated after segmentaton of each movng object The updated regon s obtaned from movng templates ahead The bacground of current frame can be adapted as follows: 80 B (, j) K I (, j) f pxel (, j) not belongs to moton regon K 1 =, (5) B (, j) otherwse 1 where B (, j ), I (, j ) and B (, j ) are pont pxels of the Kth bacground, ( K 1) th frame and the ( K 1) th K 1 K 1 bacground respectvely and j are the coordnates 3 Shadow elmnaton method for movng object At outdoor survellance scenes, because of the nfluence of sunshne, vdeo mages contan the shadow of movng object, and redundant shadow s ncluded n the movng object regon after segmentng, whch causes the falure of object recognton (Hseh,Junwe, 2003) and (Huang,Yongl, 2002) proposed methods to detect and elmnate the shadows, but n (Junwe Hseh, 2003), the method was too complcated and only appled to movng object whch be human, thus t s dffcult to ft real-tme montorng scenes The method, as n (Huang,Yongl, 2002), manly fts deal lghtng condtons ndoors In order to overcome these problems, a smple and effectve mult-objectve shadow dstngushng and elmnatng method s proposed n ths paper When the object segmented contans some non-connected objects, dfferent regons should be mared and the shadow should be removed one by one We only deal wth the stuaton for a sngle object and objects connected Steps are as follows: 1) For the detected target regon, the number of object and the general locaton of object top are determned n ths regon wth vertcal hstogram projecton method If there s only one object, step 3 should be drectly turned to; 2) If the regon contans many objects, the method based on the quadratc upper contour curve s used to determne the rough border of each object; 3) Determne the drecton of shadow objects; 4) Determne the approxmate area of shadow, and then clusterng method s appled to determne precse shadow regon and remove t 31 Determnng Method for the Number and the Top Locaton of Object By observng the vertcal projecton hstogram for the bnary mage of the segmented movng object regon, t s found that the horzontal poston of the object top concdes wth the pea horzontal poston of ts vertcal projecton hstogram, as s shown n Fg2 Ths artcle uses the vertcal projecton hstogram of the segmented movng object regon to determne the top locaton and number of object If t s assumed that the movng object regon derved from segmented mage s Rxy, (, ) specfc steps are as follows: 1) Calculate the vertcal projecton hstogram of the object regon H ( x ) R H R = ur( x, y), (6) y 1(, x y) R where ur ( x, y) = 0 ( x, y) R 2) Fnd the smoothed vertcal projecton hstogram SH R ( X ) l SH R( X ) = ( H R( x + t))/(2l + 1), (7) t= l where l = 4 3) Fnd pulse functon Px ( ) 1 f SH R TS Px ( ) =, (8) 0 else where Ts = (1/ 3) max( SHR( X)) 4) Determne the number of object and the general locaton of object top A certan wdth pulse shows the exstence of an object Based on experence, the smallest pulse wdth correspondng

4 Modern Appled Scence July, 2009 wth the object n ths paper s 6 Thus, the number of receved postve pulse wdth functon Px ( ) whch s greater than 6, s the total number of object, recorded as N h The centre of each pulse s the abscssa of the object top, recorded as H () 32 Determnng the boundares of each object In order to determne the boundary of each object, the quadratc upper contour curve CR ( x ) of the object regon was ntroduced, as s shown n Fg3 The object regon s recorded as Rxy (, ) The quadratc upper contour curve of the object area s defned as: for the bnary mage ncludng Rxy (, ), the gray value of the pont changes for the second tme along the y-axs from bottom to top, whch forms curve As s shown n Fg3, t can be seen that the place of the quadratc upper contour curve changes rapdly n vertcal drecton, whch corresponds wth the border of object The outlne of the shadow nformaton can be fully retaned n the quadratc upper contour curve The shadow s generally close to the ground In addton, the upper horzontal extended part of objects can be removed by the quadratc upper contour curve In Fg3, the outlne nformaton of outstretched arm does not ext The method of determnng object border s llustrated n Fg4 and steps are as follows: 1) Obtan HCR ( x ) by movng CR down CR yb f CR 0 HCR =, (9) 0 else where y B s the ordnate of the bottom pont n Rxy (, ) 2) Smooth HCR to obtan SHCR ( x ) l SHCR = ( HCR)/(2 l + t), (10) t= l where l = 2 The smoothed result s shown n Fg4(b) 3) Determne dfference curve DSHCR of SHCR ( x ) DSHCR = SHCR( x + 1) SHCR( x 1) (11) The dfference curve s shown n Fg4(c) 4) Determne pulse functon Pc ( x ) 1 f DSHCR TD Pc =, (12) 0 else where TD = (1/ 3) max( DSHCR( X)) 5) Determne the canddate for locaton of the object border Abscssa wth greater value of the pont on the quadratc upper contour curve s probably the border locaton of object To remove nose, the lne segment whch s probably the border s recorded only when the value of the pulse functon Pc s 1 and the number of lned ponts s not less than 4, and the mddle value of the lne segment s selected as abscssa of the canddate for border of object, recorded as seg(), = 1,2, Nb, where Nb s the total number of the canddate for border locaton 6) Determne the object border accordng to the canddate for locaton The object top, H () of object has been determned In Fg4 (c), the left or the rght canddate for border, whch s nearest to H () at the horzontal drecton, s the border of object B () s set as the border of the th object, and ts left and rght borders are as follows: B () l= segm ( ), (13) B () r= segn ( ), (14) where seg( m) and seg( n ) are the canddates for border locatons whch are nearest to H () at the left and on the rght respectvely 33 Determnng the shadow drecton Generally, the shadow s on the left or the rght sde of the object and near the ground The method to determne the shadow drecton s dscussed under these crcumstances Fg2 and Fg3 are typcal examples for the shadow on the rght sde of the object Accordng to the dstance between the rght-border of the most rght sde of the object and the most rght x coordnates of the whole regon Rxy (, ), the shadow drecton can be judged That s, f the dstance s longer than the threshold, the shadow should be on the rght The same method can be adopted to judge the shadow at the left, but the fact that the shadow s close to the ground should be taen nto consderaton Therefore, the quadratc upper contour curve CR ( X) s used to determne the most left or the most rght locaton of the object regon n ths paper The most left and rght ponts of CR ( X ) are P r and 81

5 Vol 3, No 7 Modern Appled Scence Pl respectvely and the method to determne the shadow drecton s as follows: 1) Determne whether the ordnates of P r, P l, the most rght and left of CR ( x ), are closed to ground or not They are expressed by the nequalty as follows: Pl y Thl, (15) Pr y Thr, (16) where the threshold Thl andt hr are one-thrd of the heghts of the most left and rght objects respectvely If (15) or (16) s satsfed, the next step s turned to; otherwse, the ordnate of Pr or P l s not closed to ground That s, the shadow does not nclude Pr and P l P r and P l should move gradually towards the centre of the object to fnd the ponts whch can satsfy the condton n (15) or (16) The earlest pont to be found s taen as the new P r or P l 2) Obtan the shadow drecton wth followng steps: Pr x B( Nh) r T0 and B(1) l Pl x T0, (17) where T0 s one-sxth of the object wdth, B(1) l and BN ( h ) r are the left border of the leftmost and the rght border of the rghtmost respectvely If (17) s satsfed, there s no shadow; otherwse, the shadow can be judged accordng to the followng nequalty Pr x B( Nh) r > B(1) l Pl x, (18) If (18) s satsfed, the shadow s on the rght; otherwse, on the left 34 Movng Object Shadow Elmnaton The rough shadow regon s determned, and then clusterng s used to fnd precse shadow and then to remove t Specfc steps are as follows: 1) If the object border and shadow drecton have been determned through above method, the rough regon of shadow can be ganed When there are multple objects, the regon between the two objects and close to the ground s regarded as shadow The method to decde the shadow of the most left or rght sde s as follows: If the shadow s on the rght, the regon from the rght border of the most rght object to P r can be dentfed as the rough shadow Smlarly, the shadow at the left can be determned Wth ths method the rght border and left border of rough shadow regon recorded as S can be determned, whch are recorded as Srand Sl respectvely However, the regon not close to ground s probably ncluded n S, such as outstretched arm as shown n Fg3 These regons need to be further judged wth clusterng That s, only the connected regon close to ground s mared as a rough shadow Dfferent connected regon s mared wth dfferent tags, thus seres of rough shadow can be obtaned 2) Determne the precse shadow regon and remove t The rough shadow s dentfed, and then the gray of shadow regon s generally unform The gray dfference between the object and the shadow s large Clusterng s used to further defne the precse shadow and remove t In ths paper, 8-neghborhood gray clusterng method s used to do t The mean, clusterng threshold and the ntal cluster seed of the rough shadow regon are calculated by the followng equatons where IS ( xys, ) gray value of the pxel ( x, y) n The ntal seed C locates n the centre of 82 S, and T I x y u 2 = (1/ 3) max ( (, ) ), (19) ( xy, ) S S u s the mean of S S at the horzontal drecton: (( S l+ Sr) / 2, CR( S l+ Sr) / 2) (20) The clusterng starts from the seed C, and the pont P n S s examned n turn If at least one pont n the 8-neghborhood of P has been mared as a rough shadow regon, standard devaton of P s calculated by the followng equaton: 2 δ P = ( I (, ) ) P xy u, (21) where I P ( xy, ) s the gray value of P If δ P < T, P must be shadow pont; otherwse, the pont need not be mared The pont P s checed constantly untl no new pont s mared At last all the mared shadow ponts n Rxy (, ) are removed 4 Expermental results The algorthms dscussed above were tested on personal computer wth an INTEL Pentum CPU usng Mcrosoft Vsual C++60 In ths paper, all mages used were obtaned by a general vdeo camera and the sze of the mage s In order to analyse the effectveness of the proposed method, many experments under dfferent

6 Modern Appled Scence July, 2009 condtons were performed Fg5 and Fg6 were typcal expermental results The bacground frame and the current frame are shown n Fg5(a) and Fg5(b) respectvely The segmentaton and morphologcal flterng results of the algorthms are shown n Fg5(c) Besdes, Fg5(d) s removng shadow result At last, an ntegrated movng object can be seen n Fg5(d) The segmentaton and removng shadow methods of multple movng objects and connected shadows wth objects are gven n Fg6 Expermental results ndcate that the proposed method s smple and effectve for movng object segmentaton and elmnatng shadows The expermental results also shows that the method can segment movng object accurately and remove 98% of the shadow of movng mages, and the removng shadow method requres certan dfference of gray value at the juncton between the shadow and the object 5 Concluson In ths paper, methods of movng objects segmentaton and shadow elmnaton based on hstogram were proposed to detect movng objects effectvely The method used RGB color model by usng (1) to obtan dfference mage of color dstance Then a hstogram could be obtaned by statstcs of the dfference mage The nformaton extracted by algorthm n ths paper was used for segmentng movng targets In partcular, the bacground subtracton was appled to detect mage moton, and the algorthm correctly dstngushed the changed areas n the scene from the bacground Expermental results ndcated that the proposed algorthm s smple and effectve n segmentng movng objects Snce the bacground update was performed only n the changed areas where the movng objects occurred too frequently, the computatonal load s reduced sgnfcantly Detected movng objects nclude many shadows, whch affects the accuracy of detecton and further processng The method of removng the shadows whch mproves the accuracy of target detecton was proposed n the paper Moreover, the proposed methods are based on general scenes, so t s sutable for other survellance sequence References Barron J, Fleet D & Beauchemn S (1994) Performance of optcal flow technques Internatonal Journal of computer Vson 12(1), pp 2-77 Huang,Yongl, Cao,Danhua, & Wu,Yubn (2002) Segmentaton of Moton Human Body Image n Real-Tme Montor System Opto-electronc Engneerng 1, pp 9-72 Jng,Zhong & Stan Sclaroff (2003) Segmentng Foreground Objects from a Dynamc Textured Bacground va a Robust Kalman Flter Proceedngs of the Nnth IEEE InternatonalConference on Computer Vson (ICCV 2003) vol2 Hseh,Junwe, Hu,Wenfong, Chang,Chajung & Chen,Yungsheng (2003) Shadow elmnaton for effectve movng object detecton by Gaussan shadow modelng Image and Vson Computng 21, pp Lpton A, Fujyosh H & Patl R (1998) Movng target classfcaton and tracng from real-tme vdeo Proc IEEE worshop on Applcatons of computer vson Prnceton, NJ pp 8-14 NHoose (1991) Computer Image Processng n Traffc Engneerng UK:Tauton Research Studes Press NParagos & RDerche (2002) Geodesc actve contours and level sets for the detecton and tracng of movng objects IEEE Transactons on Pattern Analyss and Machne Intellgenc 22, pp YMWu, XQYe & WKGu (2002) A Shadow Handler n Traffc Montorng System Proceedngs of IEEE Vehcular Technology Conference vol1, pp Zhu,Wegang, Hou,Guojang & Ja,Xng (2002) A study of locatng vehcle lcense plate based on color feature and mathematcal morphology Sgnal Processng 1, pp

7 Vol 3, No 7 Modern Appled Scence Fgure 3 Object regon and ts quadratc upper contour curve 84

8 Modern Appled Scence July, 2009 Fgure 4 Determnng the object border Fgure 5 Movng object segmentaton and removng sngle shadow result Fgure 6 Movng object segmentaton and removng multple shadows result 85