A model based method of pedestrian abnormal behavior detection in traffic scene

Transcription

1 A model based method of pedestran abnormal behavor detecton n traffc scene Jang Qanyn 1,2,3, L Guomng 1,2,3, Yu Jnwe 1,2, L Xyng 1,2,3 1. esearch Center of Intellgent Transportaton System, School of Engneerng, Sun Yat-sen Unversty, Guangzhou, Chna 2. Guangdong Provncal Key Laboratory of Intellgent Transportaton System Guangzhou, Chna 3. Key Laboratory of Vdeo and Image Intellgent Analyss and Applcaton Technology of MPS, P.. Chna, Guangzhou, Chna qyjang19@163.com, @qq.com, stslxy@mal.sysu.edu.cn Abstract In order to reduce traffc accdents caused by the pedestran, fve nds of dangerous pedestran abnormal behavors are studed n the paper. A behavor model between the pedestran trajectory and the road s bult to descrbe the fve nds of dangerous pedestran abnormal behavors: crossng road border, llegal stay, crossng the road, movng along the curb, enterng road area. The method contans pedestran detecton, shadow elmnaton, pedestran recognton, pedestran tracng and abnormal behavor detecton. Bacground subtracton method s used to detect movng targets. After shadow elmnaton, pedestrans are dstngushed from vehcles accordng to the rato. Then, pedestran trajectores are gotten by pedestran tracng. Fnally, based on the relaton between trajectory and road, the model of fve nds of pedestran abnormal behavors s establshed, and abnormal behavors are detected accordng ths model. Experments show that the method can dstngush and detect the pedestran abnormal behavorseffectvelynshorttme,andtssutabletousenreal tme traffc montorng. Keywords traffc survellance vdeo; pedestran detecton; pedestran tracng; pedestran abnormal behavor model; pedestran abnormal behavor detecton I. INTODUCTION Traffc accdent problem need to be pad great attenton to n the transport system. The pedestran abnormal behavor s the man cause of the frequent traffc accdents. Therefore, pedestran abnormal behavor detecton attracts wdespread concerns n scholars who launch specal dscussons and researches on ths ssue. In the pedestran behavor detecton researches, the current methods are mostly focused on the followng two categores: the method based on template matchng and the method based on state space transfer. In the researches about the method based on template, movng target moton sequences are analyzed, many moton feature, such as velocty and drecton of movement, are calculated to match the events model. Masoud [1] used nfnte mpulse response flters to descrbe the behavors of pedestrans' goals and projected them onto a feature space, and then measured the smlarty n the vdeo mages between pedestrans and reference templates wth the Hausdorff dstance. Yn Yong [2] used approxmate motons perodc characterstc of human actons to realze the recognton of human behavors. Specfcally, frstly, the bacground dfference method based on Gaussan mxture model was used to detect human targets; then the approxmate perod of human moton was gotten by the changes of the human body shape, the whole body movement sequences were decomposed nto a seres of approxmate moton cycle unts, and the transformaton characterstcs of all movement cycle unts were extracted; fnally, the dynamc tme warpng method was used to determne the dfferent human acton categores. Dong-Gyu Lee [3] proposed a crowd behavor representaton method usng adaptve optcal flow flterng for abnormal behavor detecton and localzaton. Zhcheng Wang and Jun Zhang [4] desgned a fuzzy functon based on body s contour angle movement and proposed Fuzzy Assocatve Memory (FAM) to nfer abnormal behavor. The centrod and fuzzy feature were used to recognze abnormal behavor. The method based on template matchng s easly affected by nose, and more dffcult to buld a unfed template. In the researches about the method based on state space transfer, complex human behavor are decompose to seres smple acton or movement and recognzed wthout model preset. Bregler [5] put forward a comprehensve networ based on the human body dynamcs to recognze the movements of the body. Zhou Ybo and He Xaoha [6] detect human body usng the contour nformaton and extract the posture features by the contour fttng. A behavor classfer based on the normal behavor template s establshed to determne whether a human behavor s normal or not. Zhang Ljun and Wu Xaojuan [7] selected several specal meanng landmar ponts n the montorng mages, used these ponts to decompose complex human behavors nto a seres of smple combnatons of human actons, and created the HMMS model for smple human actons. And then, they used Level - Buldng algorthm wth an approprate threshold model and behavor grammar to dentfy the complex human behavors. Besdes, Bouttefroy [8] ntroduced a method for abnormal behavor detecton by usng contextual nformaton n Marov random felds. Modelng the local densty of object feature vector, they used a Gaussan Marov random feld mxture for abnormal behavor detecton. Shen-Bn Hsu [9] pad attenton to fallng and slppng detecton for pedestrans. They used the detecton algorthm n multple vew-angles, and then constructed the manfold walng models from the tranng samples and fallng or slppng behavor could be detected. The complexty of the method based on the state space transfer s relatvely hgh, and the calculaton process s tme consumng /15/$ IEEE

2 After analyzng the dangerous pedestran behavor n traffc, ths paper manly studes the detecton of fve nds of pedestran abnormal behavors whch are easy to cause traffc accdent n traffc survellance vdeo: pedestran crossng road border, llegal stay, crossng the road, movng along the curb and enterng road area. The aspect rato characterstc s used to dstngush between pedestrans and vehcles n pedestran recognton. The tradtonal tracng method based on features s used n pedestran tracng. In the pedestran behavors' understandng and descrpton, a new pedestran behavor detecton method s proposed based on the behavor model. The model descrbes dfferent characterstcs of the relatons between fve nds of pedestran behavors and road structures were analyzed, ncludng trajectory, dstance of trajectory ponts, angle, and dstance between trajectory and road. II. PEDESTIAN ECOGNITION AND TACKING A. Pedestran Detecton and ecognton Bacground subtracton method s an effectve method for movng target detecton whle the camera s statc. The flowchart of pedestran detecton and recognton s shown n Fg. 1. Where B-1(,j)s the gray value of the old bacground mage n the coordnates (,j), and W(,j) s the gray value of the bacground subtracton bnary mage n the coordnates (,j). The movng targets gotten by bacground subtracton method usually contans shadow whch mpacts on detecton of targets characterstcs such as shapes, contours, edges, thus pedestran detecton processng wll be nfluenced negatvely. It s necessary to elmnate the shadow. Generally, bacground regon covered by shadow causes brghtness change but not hue. Dvdng the bacground subtracton mage by bacground mage shows that the rato of the gray value of shadow regon to the gray value of bacground regon s approxmately fxed but the rato of the gray value of target regon to the gray value of bacground regon changes relatvely larger. Thus, t can be easly dstngushed between shadow regon and target regon. The shadow elmnaton procedures are shown below: 1) Dvde the bacground subtracton mage by bacground mage so that a resultng mage can be obtaned. 2) By usng three pars of gradent operators shown n Fg. 2, calculate the gradent value of each pxel n the resultng mage: g = G1 + G2 (3) Consequently, horzontal gradent value, vertcal gradent value and dagonal gradent value are obtaned. If one of the three drecton gradent value s less than a gven threshold, the pxel belongs to the shadow regon. (a) the frst par (b) the second par (c) the thrd par Fg. 2. Three pars of gradent operators Fg. 1. Pedestran detecton and dentfcaton flowchart The ntal bacground can be gotten by usng medan flterng method: B(,j) =Medan (I(,j)) (150) (1) Where,I(,j) s the gray value of the frstframe mage n the coordnates (,j). In vew of the slow change of the bacground caused by llumnaton, the Surendra method s used to update the bacground [10] n real tme. The core dea of Surendra algorthm s the non-movng areas' bacgrounds of each frame s updated n real tme but not the movng areas' bacgrounds. The prncple can be represented by (2): B (, j)! B 1(, j) ( W (, j) 1) #(1 ) I (, j) B 1(, j) ( W (, j) 0) (2) (a)orgnal target Fg. 3. Shadow elmnaton results (b)orgnal detecton (c) shadow removal (d)rght detecton The results of shadow elmnaton are shown n Fg. 3. Fg. 3(c) shows the result of shadow elmnatng of Fg. 3(a) and t can be seen that the shadow s elmnated effectvely. Fg. 3(b) s the pedestran detecton mage wthout shadow elmnatng. Fg. 3(d) s the pedestran detecton mage performed shadow elmnatng on. The shadow s elmnated n the mage, whch can mprove the pedestran detecton effect. In normal traffc stuatons, the movng targets gotten by bacground subtracton method nclude pedestrans and vehcles, whch accounts for the vast majorty are vehcles. In order to facltate subsequent processng, the movng targets are recognzed to dstngush between pedestrans and vehcles.

3 By comparng pedestrans and vehcles, the aspect rato of them can be seen as a sgnfcant dfference n the shape features, whch are shown n Fg. 4. Target s aspect rato s the rato of ts external rectangle s heght to weght. (a)walng pedestran (b) runnng pedestran (c) car Fg. 4. Targets and ther external rectangle (d) bus In vew of the dfferent expermental vdeos n whch walng pedestrans, runnng pedestrans, cars(small vehcles) and buses(bg vehcles) are respectvely calculated the sequence values of aspect rato, whch are represented as and 4. What TABLE I shows s the numercal dstrbuton of the aspect rato of walng pedestrans and runnng pedestrans. What TABLE shows s the numercal dstrbuton of the aspect rato of small vehcles and large vehcles. TABLE I. Frame Number THE ASPECT ATIO OF WALKING PEDESTIANS AND UNNING PEDESTIANS TABLE II. Frame Number THE ASPECT ATIO OF SMALL VEHICLES AND LAGE VEHICLES Showng the numercal dstrbuton of the aspect rato of walng pedestrans and runnng pedestrans, TABLE I llustrates the aspect rato of walng pedestrans 1 ranges from 2.13 to 3.57, and the rato of runnng pedestrans 2 ranges from 1.93 to Showng the numercal dstrbuton of the aspect rato of small vehcles and large vehcles, TABLE llustrates the aspect rato of small vehcles 3 from 0.74 to 0.88, the rato of large vehcles 4 ranges from 1.08 to So, pedestran and vehcle can be dstngushed by a threshold of aspect rato. B. Pedestran Tracng In order to get the pedestran trajectory, pedestran tracng s requred. Pedestran tracng s manly dvded nto four steps: Frst, loo for pedestran and determne whether the target has been traced. Due to there s no rule where pedestran target wll appear, the detecton area s set as all vdeo mage area. As far as possble to fnd all pedestrans, and to determne whether the appearance of new, all connected doman are scanned from the upper left to the lower rght, and recognzed usng pedestran recognton method. Then the centrod dstance between new target and traced targets s calculated to determne whether a new pedestran s appeared, and record the poston. Second, predct pedestran poston n next frame usng lnear predcton model. Accordng to the pedestran trajectory s contnuty of tme and space and the others characterstcs, for example, slow movement n mage area, a three-pont lnear predcton method s used to estmate the poston of the pedestran n next frame. The lnear predcton model can be descrbed by the followng equaton: ' f 2 f f ( 2,3,, N) (4) 1 1 Where, f +1 represents the target's predctve poston n the +1 th frame, f'-1 represents the target's real poston n the-1 th frame, and f' represents the target's real poston n the th frame; represents predctve model s error parameter, t can be obtaned as:! 0 ( 2) % & d (5) % 3 ( 3, 4,, N) %# 2 d f f ( 3, 4,, N) (6) ' Where, d represents the subtracton between predctve poston and real poston n the th frame target mage. Thrd, match targets usng the cost functon and record trajectory. After the target s poston s obtaned, the cost functon [11] s ntroduced to complete the target matchng. Consderng the targetn the th frame, calculate cost functon value between all canddate targets n the +1 th frame. Cost functon can be represented by the followng equaton: V (, j) D (, j)! H (, j) A (, j) (7) D (, j) ( X X ) ( Y Y ) (8) j j 1 1 j H (, j) G G 1 (9) A (, j) S S (10) j 1 Where, X Y G and S respectvely represent centrod abscssa, centrod ordnate, the average gray value and the area of target n the th frame; D(,j), H(,j) and A(,j) represent the centrod dstance, average gray subtracton and area subtracton between the targetn the th frame and the target j n the +1 th frame;! are emprcal value.

4 Choose target j whch maes cost functon value mnmum from all canddate targets n the +1 th frame predctve area as subsequent target of target n+1 th frame mage, and record the poston of the target j, whch s the coordnate of the center of target j s external rectangle. At last, clear the target nformaton of the pedestran who s left or lost to avod tracng nformaton redundancy. III. PEDESTIAN ABNOMAL BEHAVIO DETECTION A. Pedestran abnormal behavor analyss Ths paper manly dscusses fve nds of abnormal behavors of pedestran, ncludng pedestran crossng road border, llegal stay, crossng the road, movng along the curb, enterng road area. Fg. 5 s a schematc dagram of these fve abnormal behavors where the area wthn the two blac straghts represents the road area. Lne A shows the trajectory of the pedestran crossng the road border. Pont a1 s the startng pont for trajectory of pedestran, and a2 s the current pont for trajectory of pedestran. Lne B shows the trajectory of the pedestrans stayng llegally. Pont b1 s the startng pont for trajectory of pedestran, and b2 s the stop pont for trajectory of pedestran. Lne C shows the trajectory of the pedestran across the road. Pont c1 s the startng pont for trajectory of pedestran, and c2 s the current pont for trajectory of pedestran. Lne D shows the trajectory of pedestran who moves along the curb. Pont d1 s the startng pont for trajectory of pedestran, and the d2 s the current pont for trajectory of pedestran. Lne E shows the trajectory of the pedestran gong nto the road area. Ponte1 s the startng pont for trajectory of pedestran, and the e2 s the current pont for trajectory of pedestran. Fg. 5. Fve nds of abnormal behavor movement dagram (a) ntersecton relaton (b) wthn relaton Fg. 6. Poston relatonshp of pedestran trajectory and road structure Accordng to Fg. 5, there are dfferent characterstcs and relatonshp between the trajectory and road structure of each abnormal behavor, aand they are reflected manly from angle and poston. Thus we can now that the poston and angle relatonshp of pedestran trajectoryy and road structure (border of road), as shown n Fg. 6 and Fg. 7. In the Fg. 6,represents road andprepresents pedestran trajectory. Fg. 6(a) shows that pedestran trajectory ntersects road structure, for example, pedestran crossng road border. Fg. 6(b) shows the poston relatonshp that pedestran trajectory s wthn road structure, such as llegal stay, crossng the road, movng along the curb and enterng road area. Fg. 7. Angle relatonshp of pedestran trajectory and road structure In the Fg. 7, "1" means pedestran trajectory does not form angle wth road structure, for example the trajectory of llegal stay, because of ts statc pedestran trajectory. "2" means pedestran trajectory forms the nearly vertcal angle wth road structure, for example the trajectory of crossng the road. "3" means pedestran trajectory s nearly parallel to road structure, for example the trajectory of movng along curb. "4" means pedestran trajectory forms the others angle relatonshp wth road structure, such as the trajectory of crossng road border and enterng road area, whose angle relatonshp between pedestran trajectory and road structure s not regular. Accordng to the poston and angle relatonshp between pedestran trajectory and road structure, fve nds of pedestran abnormal behavors can be dstngushed. B. Pedestran abnormal behavor modelng There s much nose or some lost on pedestran trajectory obtaned by pedestran tracng because of shelter, error tracng and other reasons. So t s necessary to process the orgnal trajectory [12]. In ths paper, the average flterng method s used to smooth the orgnal pedestran trajectory, as shown n (11), and ts prncple s that an average value s obtaned every 15 frames as pedestran trajectory pont coordnate after average flterng. 15n 14 pn [ ] (& p [ ]) / 15 ( n 0,1 N) (11) 15n 1: No angle value 2: Nearly vertcal 3: Nearly parallel 4: Other angle values Where, p[] represents the orgnal pedestran trajectory pont whch s the center coordnate value of the th pedestran s external rectangle; p[ n] represents the smoothed trajectory pont; n s the number of the pedestran trajectory ponts after average flterng, and there are total N+1 numbers; N s zero or postve nteger. Four pedestran behavor model parameters are defned to descrbe the relatonshp between pedestran trajectory and road structure:

5 1) Trajectory coordnates chan: P ( p[0], p[1],, pn [ ],, p[ N ]) ; 2) Trajectory path chan: D dstance( P) ( d, d,, d,, d ) ; 1 2 Where d represents the dstance of the adjacent two trajectory ponts after average flterng, and the unt s pxel d (( p [ ]. xp[ j]. x) ( p [ ]. yp[ j]. y) ) ( 1, N; j 1) ld 2 3) Angle chan between trajectory and road structure: A angle( P, ) ( a, a,, a,, a ) ; 1 2 N M (12) Where, P represents trajectory coordnate chan and represents the border of road structure. Use lnear fttng method to ft the curve paths nto multple lne segments, and a represents the angle between the th lne segment and road structure, whose value ranges from 0 to ( n radan. a arctan ( ) / (1 ) ( 1,2, M) (13) Where represents the slope of the th lne segment; represents the slope of road structure; M s the number of lne segments by lnear fttng. 4) Dstance chan between trajectory and road structure: LD dstance( P, ) ( ld, ld,, ld,, ld ) 0 1 Where, P represents trajectory coordnate chan and represents the border of road structure; ld represents the dstance between the trajectory pont and the border of road, the unt s pxel; A B C are the parameters for the lnear equaton of road structure border whose lnear equaton expresson s y Ax By C. 2 2 ld ( Ap [ ]. x Bp [ ]. y C) / A B ( 0,1, N) (14) The relatonshp between the trajectory of fve nds of pedestran abnormal behavors and road structure can be quantfed by usng the mathematcal model of fve nds of pedestran abnormal behavors, as shown n TABLE. TABLE III. THE MATHEMATICAL MODEL OF PEDESTIAN ABNOMAL BEHAVIO Pedestrans Abnormal Behavor crossng road border llegal stay crossng the road movng along the curb enterng road area The Mathematcal Model of Pedestran Abnormal Behavor d, 1 ld, 1 Pɳ d, d 2, 0&&P) N a (/2 &&P) ld, ld 2, < Td* &&P) P) * Td s an emprcal dstance parameter. C. Abnormal behavor detecton algorthms TABLE shows that there are obvous dfferences between dfferent nds of pedestran abnormal behavors. Accordng to the mathematcal model of pedestran behavor, Fg. 8 s flow chart of abnormal behavor detecton. Fg. 8. Detecton flow chart of fve nds of pedestran abnormal behavor IV. ESULTS AND ANALYSIS Several survellance vdeos gotten from real Guangzhou traffc montorng system and test vdeos shot n test road are used to detecton pedestran abnormal behavor. All vdeos are 25 fps and frame resoluton s 704*576 pxel and. The program developed by Vsual C++ and OpenCV Beta 4.0. Detecton results are shown n Fg. 9, where border of road are mared by a par of red lnes, and pedestrans are mared by ther external rectangle. Fg. 9(a) shows the detecton result of pedestrans crossng the road border, and ts alarm type s "crossng the road border". Fg. 9(b) shows the detecton result of llegal stay, and ts alarm type s "llegal stay". Fg. 9(c) shows the detecton result of crossng the road, and ts alarm type s "crossng the road". Fg. 9(d) shows the detecton result of movng along the curb, and ts alarm type s "movng along the curb". Fg. 9(e) shows the detecton result of enterng road area, and ts alarm type s "enterng road area". All pedestran tracng are real-tme and the alarm wll ssue after pedestran tracng successful. Correct rate of abnormal behavor detecton # s defned by the rato of the number of alarm(wn) and the number of abnormal behavor (AN) n expermental vdeo: # WN / AN (15) The expermental results are dsplay n TABLE IV. The detecton accuracy of fve nds of pedestran abnormal behavor are more than 85%, and can be detected effectvely. The man reasons affectng the expermental results are: (1) when few pedestrans wal together, t s dffcult to detect each one; (2) If the dstance threshold s set large, t s easy to confuse the llegal stay wth crossng the road whose speed s slow; (3) t s easy to lost tracng for suddenly acceleraton

6 runnng pedestrans; (4) the bnary threshold s vary n dfferent vdeos. abnormal behavors n the traffc scenes, other pedestran abnormal behavors' mathematcal models can be establshed by the method also. (a)pedestran crossng the road border (No. 1) (b)illegal stay (No. 2) V. CONCLUSION In the paper, a behavor mathematcal model s establshed for the common fve nds of pedestran abnormal behavors n the traffc scenes, and behavor detecton s also realzed. The expermental detecton results ndcate the effectveness of the proposed method. There are some problems need to be dealt wth n future research, such as adheson and occluson among multple pedestrans, tracng problem when pedestran s covered by vehcle and detecton of pedestran abnormal behavor at nght. VI. ACKNOWLEDGEMENT Ths research was supported by Guangdong Provncal Department of Scence and Technology (Project No.: 2013B ). (c) Crossng the road (No.5) (e) Enterng road area(no.1 and No.2) (d) Movng along the curb (No.3) Fg. 9. Some detecton results of fve nds of pedestran abnormal behavors TABLE IV. THE DETECTION ACCUACY OF PEDESTIAN ABNOMAL BEHAVIO IN EXPEIMENTAL VIDEOS Pedestrans Abnormal Behavor Number of alarm Number of abnormal behavor n vdeos Detecton accuracy crossng road border % llegal stay % crossng the road % movng along the curb % enterng road area % Indcated by the experment, the model of pedestran abnormal behavor dscussed n ths paper s smple and tme savng, and provdes a unfed representaton method and detecton standard for each nd of abnormal behavor. Although ths paper only consders the common fve nds of EFEENCES [1] MASOUD O, PAPANIKOLOPOULOS N. A method for human acton recognton. Image and Vson Computng, 2003, 21(8): [2] Yn Yong, Wang Jandong, and Jn Xangang. Human Abnormal actvty recognton for approxmate perod moton. Computer Engneerng and Applcatons, 2010, 46(26): [3] Dong-Gyu Lee, Heung-Il Su and Seong-Whan Lee. Modelng crowd motons for abnormal actvty detecton. 11th IEEE Internatonal Conference on Advanced Vdeo and Sgnal Based Survellance (AVSS), pp , [4] Zhcheng Wang, and Jun Zhang. Detectng Pedestran abnormal behavor based on fuzzy assocatve memory. Fourth Internatonal Conference on Natural Computaton, vol. 6, pp , [5] BEGLE C. Learnng and recognzng human dynamcs n vdeo sequences. Proc of IEEE Conference on Computer Vson and Pattern ecognton. 1997: [6] Zhou Ybo, He Xaoha, Zhang Shengjun, and Qng Lnbo. A new detecton algorthm of abnormal behavor.computer Engneerng and Applcaton, 2012,48(3): [7] Zhang Ljun, Wu Xaojuan, Sheng Zan, and Q Le. Behabor recognton method n complex envronment usng HMM. Computer Engneerng, 2008, 34(7): [8] P. L. M. Bouttefroy, A. Beghdad, A. Bouzerdoum, ands. L. Phung. Marov random felds for abnormal behavor detecton on hghways. Vsual Informaton Processng(EUVIP), nd European Worshop on, pp , July, [9] Shen-Bn Hsu, Chn-Chuan Han, Cheng-Ta Hseh, and Kuo-Chn Fan. Fallng and slppng detecton for pedestrans usng a manfold learnng approach. Proceedngs of the 2013 Internatonal Conference on Machne Learnng and Cyberntcs, July, [10] Wang Zhengqn, and Lu Fuqang. Comparson of adaptve bacground extracton algorthm of Object Scene. Computer Engneerng, 2008, 34(23): [11] L X, Cao Guangy, and Fu Xaowe. The pedestran tracng countng method based on real-tme mage sequence. Computer Smulaton, 2005,22(2): [12] Zhao Youtng, L Xyng, and Luo Donghua. Based on vdeo vehcle trajectory model of traffc ncdent automatc detecton method research. Journal of Sun Yansen Unversty, 2011,50(4):56-60.