Human Crowd Behavior Analysis Based On Graph Modeling and Matching In Synoptic Video

Transcription

1 ISSN (Onlne) : ISSN (Prn) : Inernaonal Journal of Innovave Research n Scence, Engneerng and Technology Volume 3, Specal Issue 3, March Inernaonal Conference on Innovaons n Engneerng and Technology (ICIET 14) On 21 s & 22 nd March Organzed by K.L.N. College of Engneerng, Madura, Taml Nadu, Inda Human Crowd Behavor Analyss Based On Graph Modelng and Machng In Synopc Vdeo B.Yogameena #1,K.Sndhu Prya *2 #1 Deparmen of ECE, Thagarajar College of engneerng, Madura, Tamlnadu, Inda. *2 Deparmen of ECE, Thagarajar College of engneerng, Madura, Tamlnadu, Inda. ABSTRACT Huge vdeo daase capured by usng varous resources lke survellance cameras, webcams.ec. I s very me consumng o wach whole vdeo manually. In hs paper, Vdeo synopss s used o represen a shor vdeo whle preservng he essenal acves for a long vdeo. In he exsng mehodology, usually a sngle movng objec s spled no a few small peces n a connuous acvy. For ha Gaussan mxure model s used o deec compac foreground agans her shadows. Bu n hgh-densy crowds background subracon fals due o occluson. Afer deecng he occluson, background s subraced. Trackng s done by Cenrod of he slhouee. For vdeo synopss more fluen ubes are generaed by concaenaon. Then each solaed regon s consdered as a verex and a human crowd s hus modeled by a graph. Afer modelng he graph by usng Delaunay rangulaon and Convex-Hull, graph machng algorhs developed o deec he problem of behavor analyss of human crowds. Expermenal resuls obaned by usng exensve daase show ha he proposed algorhs effecve n deecng occluson and anomalous evens usng vdeo synopss. KEYWORDS Vdeo Synopss, Fluen Tube, Trangulaon, Occluson, Graph Modelng I.INTRODUCTION Delaunay people n a small spaal regon or neracng wh each oher. The movng dynamcs of a crowd conan rch nformaon abou human acves. If he moon rajecores of crowds are ou of predcons, usually means ha some abnormal evens have occurred. The evens nduced by he neracons whn crowds are also exremely mporan snce he abnormal behavors of crowds would usually resul n some massve damages In he leraure, Andrade e al. [1] used opcal flows o descrbe crowd and o exrac feaures from opcal flows n an unsupervsed manner. Cheryada and Radke [2] presen a survey of algorhms of compuer vson ha deal wh crowds of people and revew model-based crowd analyss algorhms, n whch some ype of human model s appled o rackng or segmenaon. In crowded scenaros o deec ypcal moon paerns. Ryan e al. [3] proposed a scene ndependen approach ha can coun he number of people n crowded scenaros whou any ranng. A global scalng facor s used o compensae for camera angle and dsance, n each scene based on a reference person. Chan e al. [4] developed a prvacy-preservng sysem o esmae he nhomogeneous crowd s arbues. The approach does no depend on feaure rackng or objec deecon. Jang e al. [5] employed he conexual anomaly concep for crowd analyss. An unsupervsed approach s used o follow n her sysem, ha auomacally denfy mporan conexual nformaon from he crowd vdeo and o deec he blobs correspondng o conexually anomalous behavors. Mehran e al. [6] used socal force model To preven crmnal behavors, now a day s vdeo survellance sysems have become more and more popular n many publc places lke arpors, ran saons, bus sand, ec. People are usually he man objecs of neres n survellance asks. In publc places, people usually move owards some orenaon(s) and hus form some crowd(s). A crowd s defned as wo or more Copyrgh o IJIRSET

2 Human Crowd Behavor Analyss Based On Graph Modellng and Machng n Synop... o deec and localze unusual evens n a crowd of people. To deec occluson, Jang e al. [7] employ neger lnear programmng, however he of arges number needs o be known. To overcome hs lmaon, Berclaz e al. [8] nroduce snk locaons and vrual source o nae and ermnae rajecores. Bnary opmzaon problems are a common ra of hese works ha are usually solved o global opmaly by relaxng hem o lnear programs (LPs). Mos of he cameras work 24 hours a day. I s very me-consumng o wach whole vdeo manually and dffcul o fnd neresng acves n long vdeos. Vdeo synopss ams a producng a bref descrpon of he orgnal vdeo whle exracng key acves, whch makes a full vew of he vdeo conen quckly [9]. Vdeo synopss provdes effecve summary of vdeos n qucker me. Hence he conrbuon of he vdeo synopss s o adap abnormal behavor analyss. II.METHODOLOGY Frs, he npu vdeo s aken and has been checked for any occluson s presen. Because, n hgh-densy crowds background subracon fals due o occluson. Afer occluson deecon, background s esmaed usng GMM and hen foreground s deeced usng background subracon. To reduce he lengh of he long vdeo, vdeo synopss s used. Then he graph s o be modeled for he human crowd. For modelng he graph, Delaunay Trangulaon s used o connec all he verex of he conneced componen. Then he even can be deeced by comparng he graph n subsequen frames. Vdeo npu Occluson Deecon Background Subracon Vdeo Synopss Graph Modelng Graph Machng hs problem and comparng he varous capables, whch cause occluson problen some frame work exsng vdeo rackng sysems would be much easer. There are hree dfferen ypes of occluson lke: (1) usually n dense crowded scenes, arges frequenly occlude each oher causng ner-objec occluson (2) a arge may move behnd sac objecs lke rees, rock or road sgns, ha are all examples of common scene occluders; (3) dependng on he ype of objec, exensve arculaons, orenaon or deformaons changes whch leads o self occluson. All hree ypes of occluson reduce or compleely suppress evdence of he mage for a arge s presence. In hs paper, he focus s on he ner-objec challenge occlusons. Accordng o he suaons where dynamc arges occlude each oher, he man ask s o overcome he dffcules ha arse from he complex dependence beween a arge s vsbly and of several oher arges rajecores ha could poenally block he of sgh lne. Global arge vsbly s proposed o overcome hs problem. The oal area of mage s hus gven as Ro(x) dx. To compue he relave area of arge whch s occluded by arge j, 1 O X dx O x O j x dx (1) we assume here ha arge j s n arge fron, hen we wll address he general case below. By usng he relave area we defne he arge vsbly as gven n Eq. (2), hen he vsbly s no dfferenable wh respec o he objec posons of X or X j, whch ncludes gradenbased opmzaon mehods before processng Here we propose o use a Gaussan ndcaor funcon N (x) := N(x; c ;c j ) o address he ssue. Lke before, we compue he overlap area by negrang he wo producs of he ndcaor funcons, where as Gaussans Z j = N X. N j X dx (3) Besdes dfferenably, he Gaussans choce allows hs negral o be deermned n closed form. Z j = N(c ; c j ;C j ) wh C j = c + c j. we denoe usng he normalzed Gaussan Classfcaon Normal/Abnormal Fg 1 shows Flow dagram for human crowd classfcaon Normal/Abnormal A. Deecon of Occluded People Trackng mulple objecs n crowded scenes necessarly leads o he occluson problem. Dscussng V j = exp 1 2 c c j T C j 1 c c j (4) whch s dfferenable wh respec o c and c j and has he desred propery ha V j = 1 when c = c j. Due o he Gaussans symmerc propery, s consdered ha V j = V j. Copyrgh o IJIRSET

3 B. Background Subracon To perform foreground exracon, Gaussan mxure models s used o represen he background [10]. The probably ha an observed pxel, x = (x, y), wll have an nensy value R a me, s esmaed by K Gaussan dsrbuons as follows, (5) P R = Human Crowd Behavor Analyss Based On Graph Modellng and Machng n Synop... K ω l, l=1 e 1 2π 1/2 2 R μ T 1 l l R μ l Where ω l, and μ l are he wegh and he mean of he l h dsrbuon, where l s he Gaussan s covarance marx ha s assumed o be dagonal. To updae hs model, each pxel value n a new vdeo frame s processed o deermne f hey maches any of he exsng Gaussan dsrbuons a ha pxel s locaon. If a mach s confrmed for he l h dsrbuon a me, hen a Boolean varable, M l,, s se. A each me nsan, each dsrbuon for he weghng facor s updaed accordng o ω l, = 1 α ω l, 1 + α M l, (6) where α s he learnng rae ha conrols he speed of learnng. The acual Gaussans a each pxel locaon are updaed as follows μ l, = 1 β μ l, 1 + βr (7) In addon o GMM-based background subracon, o solae foreground blobs we use hree-frame emporal dfferencng. Specfcally, frame dfferencng s performed by usng (8) Δ = I x I 1 x I x I +1 x By usng he resuls of Eq. (9) are hreshold o remove false posve pxels o oban accurae foreground regons. Specfcally, we use an adapve hreshold funcon medan( )o deermne a movng arge s pxels Q (x, y), Q x, y = Δ x, y I x, y medan Δ (9) To oban he fnal foreground regons, he background subracon and emporal dfferencng resuls are combned by applyng a bnary OR operaon o he foreground map derved by background subracon. Ths combnaon ensures ha movng foreground objecs can be denfed by usng he resul. Copyrgh o IJIRSET C. Tube Generaon By usng GMM mehod, more compac foreground regon can be deeced o buld fluen ubes for vdeo synopc. If one movng objec s segmened no few small peces, and hen ube wll be dsconneced, leads o sudden nerrupon of objecs n he vdeo synopc, herefore, cenrod based racker algorhm [11] s used o concaenae ube fragmens and generae fluen ubes. Where x and y are he hdden sae and observaon he daa a me and gven varous observaon y 0: ={y 0,...y } o flered esmaon of x s p(x y 0: ) p x y 0: p y x (10) p x x 1 p x 1 y 0: 1 dx 1 Where p x x 1 s he ranson model deermnng how objecs can move beween frames and p y x s he observaon model o represen he lkelhood of objecs. A se of M weghed parcles x =1 M s used o deermne he dsrbuon of flerng p (x y 0: ) (11) p x y 0: M =1 w δ x x The parcles x =1 M whch are aken from p x x 1 and he wegh w s aken from he daa lkelhoodp y x. The pror dsrbuon for he process of rackng n an mage pach s based on he HSV hsogram. The observaon model s (12) p y x e λd2 h 0 h x The racker s used o concaenae he dsconneced fragmens, for a se of nal rajecory fragmens. III. MODELING HUMAN CROWDS Each solaed regon s consdered as a verex and a human crowd s hus modeled by a graph. Afer modelng he graph, graph machng algorhs developed [12]. D. Delaunay Trangulaon Each conneced componen s represened by s cenrod and hen a se of verces of a human crowd P. I can be denoed as P = {P1, P2,..., Pn P = (X, Y )}, where (X, Y ) s he coordnae of he h conneced componen. To consruc a graph for a human crowd, wo properes of graph heory s used. The frs s he

4 unqueness of he graph. I means one-o-one mappng ha he same se of verces would correspond o a sngle graph. Accordng o hs propery, Delaunay rangulaon s used o sysemacally connec he verces of he conneced componen snce he unon of all smplces n he rangulaon s he convex hull of he verces. The second propery s ha he consruced graph should be he larges convex hull coverng all he verces. Accordng o hs propery, convex hull s used o cover Fg. 2. (a) Black dos denoe an ndvdual se, (c) graph s denoed by a Trangles se. (d) Alernaons person.(b)larges convex hull coverng he ouer verex afer he process of rangulaon wh an exra verex added. he rangulaon. Wh such propery, he opology varaon of he graph would explcly show he global changes n he graph. Therefore, a Delaunay rangulaon for a se P n he plane s a rangulaon, say DT(P), such ha no pon n P s nsde he crcumcrcle of any rangle n DT(P) [12]. When an addonal verex s seen n he graph, alernaon of he rangulaon would be lmed around he exra verex n he local area raher han re-rangulaon hrough all verces. The rangulaon process s shown n Fg.2. IV. GRAPH-BASED EVENT DETECTION Then abnormal evens are deeced by he varaon of graph opology by usng adjacency marx, sub-graph analyss, descrpon of shape deformaon, and global graph machng. A. Adjacency Marx of a Graph If node and node j are conneced, A G (,j) or A G (j,) s nonzero; oherwse, A G (,j) and A G (j,) are zero Adjacency marx s used o denfy he connecvy of he graph. The marx A G (,j)s he adjacency marx of a graph G and s a symmerc (0,1)-marx wh zeros on he man dagonal. When he graph s conneced he adjacency marx of a graph s ndecomposable precsely. Delaunay rangulaon s consruced by connecng he graph G(V, E) ha none of V s dsconneced. Human Crowd Behavor Analyss Based On Graph Modellng and Machng n Synop... (a) (b) (c) (d) Copyrgh o IJIRSET B. Sub graph Analyss Whou generaly loss, le G and H are he wo consruced graphs wh he relaons E(H) E(G) and V(H) V(G). H s hen a sub graph of G. If G and H sasfy he condons E(H) E(G)and V(H) V(G), H s named a spannng sub graph of G. The sub graph nduced by nonempy se S of verces n G s ha sub graph H wh verex-se S whose edge se conss of hose edges of G ha jon wo verces n S; s denoed by G(S). A sub graph H of G s nduced f H =<V (H)>. 1) Corollary: If H s an nduced sub graph of G, f μ 1 μ 2... μ m are he egenvalues of he marx relaed o he graph H and f λ 1 λ 2... λ n are he egenvalues of G, hen λ +n m μ λ, for =1,2,...,m. Sub graph analyss s mporan snce he graph a me wh slgh changes of one or few verces presen addonally or absen would sll be a sub graph of graph a me 1. Based on he heorems menoned above, he smlary beween consecuve graphs can be formulaed as he dfference beween he egenvalues of adjacency marx A. If he wo se of egenvalues from consecuve graphs are of he same number and he same values, hese wo graphs are sad o be graph somorphsm. If consecuve graphs sasfy he corollary, hese wo graphs are regarded as smlar bu no exacly he same. Snce G and G 1 do no sasfy he relaon E(G E(G 1 ) he corollary canno be used o deermne f hey are of he sub graph relaonshp. C. Descrpon of Shape Deformaon A shape deformaon of graph s employed o descrbe he graph varaon of local changes by usng rangle machng mehod [13]. 1) Trangle Machng Mehod: Afer processng he Delaunay rangulaon, o oban a rangle se of G (V, E), where= {δ 1, δ 2,..., δ k,δ, = (v 1, v 2, v 3 ), = 1 k}, k s he number of rangles of G (V, E) and v j V (G ), j = 1 3 are he hree verces of he rangle. Le -1 and be he rangle se of graph G -1 (V, E) and G (V, E), respecvely. The cenrod of he rangle -1 and s wren n he form of M 1 = v v v 3, 3 = 1,2,3,, p,, k 1 (13) M j = v j1 + v j2 + v j3, j 3 = 1,2,3,, q,, k (14) The rangle-par δ p 1 and δ q ha s of he shores dsance can hus be defned as

5 Human Crowd Behavor Analyss Based On Graph Modellng and Machng n Synop... δ p 1, δ q = argmn d M 1, M j (15) Therefore, o fnd he smlary beween δ p 1 and δ q s = 1 e 1 p 12 eq12 S p,q mn e 1 p 12,eq12 1 e 1 p 31 eq31 m n e 1 p 31,eq31 e 1 p12, e 1 1 p23, e p31 and 1 e 1 p 23 eq 23 e q12 mn e 1 p 23,eq23, e q23 (16), e q31 are he Where edges se for he rangles δ 1p and δ q, respecvely. By usng hs equaon f we ge he resul as 1 hen we denfes ha here s no changes n he rangles. If we ge he value less he 1 hen here are some changes n he rangles s represened by green rangles. 3) Global Graph Machng: The conour of a human crowd s represened by he convex hull and hus he convex hull varaon n shape would reflec he alernaons n ouer area of he human crowd,.e., he global changes. Momen nvarans have been used o mage paern recognon n a varey of applcaons due o s nvaran feaures on mage lke scalng, ranslaon and roaon. m p,q = x y I x, y x p y q (19) Where I (x, y) s pxel nensy n coordnae (x, y). Where p s he x-order and q s he y-order, where he power o whch he correspondng componen s aken n he sum as deermned as follows, Wh Hu s momens, o compare wo areas enclosng by convex hulls and o fnd wheher hey are smlar. To make hs effecve process, Eq. (22) we have o compue and compare her momens accordng o a creron ha we provde. 7 =1 H 1 H H 1 M H 1, H = 7 (20) M H 1, H o compare wo areas enclosng by convex hulls. Fg.3 Mss mached rangles n he graph a me -1 and s represened by green rangles. (21) H 1 = sgn h H 1. log h H 1 H = sgn h H. log h H 2) Algnmen of Mss-Mached Trangles: To algn he rangles ha are no mached beween frame pars, o make proper algnmen he cos of he algnmen of mssmached rangles s o be mnmzed. Le he mssmached rangles se a me -1 be -1, where Ψ -1 = {φ 1 1, φ 2 1,., φ α 1 φ 1 = (θ 1, θ 2, θ 3 )}, Ψ -1-1, where α s he number of mss-mached rangles, and (θ 1, θ 2, θ 3 ) s he nernal angles se of he rangleφ 1. Smlarly, he mss-mached rangles se a me s Ψ, where Ψ = {φ 1, φ 2,, φ β φ = (θ 1, θ 2, θ 3 )}, Ψ and where β s he number of rangles. The opmal machng for cos C s expressed as he machng beween subses ζ 1 and ζ : ζ 1, ζ = argmn C (17) Therefore, he verces order can be deermned by he x and y coordnaes, and hus he respecve cos C X and C Y can hen be obaned by C = 0.5 C X C Y (18) Copyrgh o IJIRSET ) Abnormal Even Deecon: To deec he abnormal even, a mulcue measure s used, say P E, whch s he combnaon of he properes menoned above s defned as P E = max 1 P s, P n ξ P m + 1 ξ P (22) Where ξ s a pre-defned wegh for balancng he global momen cos P m and he local cos of rangle machng s expressed as P. Where Ps s he probably of he sub graph relaonshp beween wo graphs and s defned as R P s = (23 ) mn n 1, n Where R s he number of egenvalues of G (V, E) and G 1 (V, E). P n s a probably whch s used o measure he number nodes ha are mss-algned and s defned by

6 Human Crowd Behavor Analyss Based On Graph Modellng and Machng n Synop... Q P n = 1 max k 1, k (24) where Q represens he number of mached rangles for graphs G (V, E) and G 1 (V, E). The graph machng cos by global conour s measured by P = C C max C mn (25) Fnally, a emporal sldng wndow wh wdh Ω s se o 5 emprcally. An abnormal even can be deeced f he accumulaed value P ET s larger. T P ET = P E, T 5 (26) =T 4 Fg.4, For occluson deecon, we use he some daase o show he resul 1) Inpu Frames (UMN Daase): The orgnal vdeo s gven as npu ha conans 100 frames and are shown n fg.5. Fg.5, Inpu Frames (UMN Daase) 2) Background Subraced Frames: Background Subraced frame are shown n Fg.6. If P ET 5: s abnormal even oherwse normal even V.RESULTS AND DISCUSSION The effcency of he algorhm proposed has been evaluaed usng Ma lab For even deecon, he UMN daase[14] s aken o deermne he even deecon process. In hs paper, he proposed mehod processes a vdeo of abou 25 frames per second for color mages a sze of 320 x 240. The resul of occluson s shown n fgure. The resul of occluson deecon s shown n fg.4. Fg.6, Background Subraced Frames 3) Normal acvy frames: Afer exracng he foreground mages, he Graph modeled frames and normal acvy s deeced usng a hreshold value of and are shown n Fg.7 Se of Daa s Resul of occluson Fg.7, Normal acvy frames 4) Abnormal acvy frames: Afer exracng he foreground mages, he Graph s modeled and abnormal acvy s deeced usng a hreshold value of and are shown n Fg.8. 5) Performance Analyss: Copyrgh o IJIRSET Fg.8, Abnormal acvy frames Sensvy Se s calculaed usng Se = TP/TP+FN

7 Human Crowd Behavor Analyss Based On Graph Modellng and Machng n Synop... Accuracy Ac s calculaed usng Ac = (TP+TN)/ (TP+TN+FP+FN) Error Rae Er s calculaed usng Er = (FN+FP)/ (TP+TN+FP+FN) Usng hese equaons algorhm effcency s calculaed. Table.1 shows esmaed values of he performance analyss parameers. Table.2 shows Summarzaon of he daase. Delaunay rangulaon and Convex-Hull, graph machng algorhs developed o deec he problem of behavor analyss of human crowds. Expermenal resuls obaned by usng UMN daases show ha he proposed algorhs effecve n deecng occluson and anomalous evens n qucker me for unconrolled envronmen of survellance vdeos. The Fuure work s o avod collson due o vdeo synopss among he movng objec. Because, hs collson amongs foreground blobs merges o each oher and s dffcul o analyze he behavor. Table.1 Accuracy Analyss Daase Sensvy Accuracy Error Rae DS1/pes % 89.40% 5.7% DS2/UMN/ % 89.51% 4.6% DS3/UMN/ % 90.45% 5.7% DS5/ UCSD/Peds % 90.58% 5.9% Table.2 Summarzaon of he Daase Daase Sequence Lengh Frame Sze DS1/pes *576 pxels DS2/UMN/ *158 pxels DS3/UMN/ *240 pxels DS5/ UCSD/Peds2 200(aken a shor clp) 360*240 pxels VI.CONCLUSION Even deeced Abnormal Even Normal Even Abnormal Even Normal Even In hs paper, vdeo synopss s used o represen a shor vdeo whle preservng he essenal acves for a long vdeo. In he exsng mehodology, usually a sngle movng objec s spled no a few small peces n a connuous acvy. For ha Gaussan Mxure Model s used o deec compac foreground agans her shadows. Bu n hgh-densy crowds background subracon fals due o occluson. Afer deecng he occluson, background s subraced. A cenrod based racker s used o concaenae wo rajecores f hey belong o he same foreground acvy. For vdeo synopss more fluen ubes are generaed. Then each solaed regon s consdered as a verex and a human crowd s hus modeled by a graph. Afer modelng he graph by usng Copyrgh o IJIRSET REFERENCES [1] E. Andrade and R. Fsher, Hdden markov models for opcal flow analyss n crowds, n Proc. 18h In. Conf. Paern Recogn., vol. 1. Sep. 2006, pp Y. [2] N. Ihaddadene and C. Djeraba, Real-me crowd moon analyss, n Proc. In. Conf. Paern Recogn., Dec. 2008, pp [3] D. Ryan, S. Denman, C. Fookes, and S. Srdharan, Scene nvaranncrowd counng for real-me survellance, n Proc. IEEE 2nd In. Conf.Sgnal Process. Commun. Sys., Dec. 2008, pp [4] A. B. Chan, Z. Sheng, J. Lang, and N. Vasconcelos, Prvacy preservng crowd monorng: Counng people whou people models or rackng, n Proc. Compu. Vs. Paern Recogn., Jun. 2008, pp. 17. [5] F. Jang, Y. Wu, and A. K. Kasaggelos, Deecng conexual anomales of crowd moon n survellance vdeo, n Proc. IEEE In. Conf. Image Process., Nov. 2009, pp [6] R. Mehran, A. Oyama, and M. Shah, Abnormal crowd behavor deecon usng socal force model, n Proc. IEEE In. Conf. Compu. Vs. Paern Recong., Mam, FL, USA, Jun. 2009, pp [7] H. Jang, S. Fels, and J. J. Lle. A lnear programmng approach for mulple objec rackng. In CVPR, [8]J. Berclaz, F. Fleure, and P. Fua. Mulple objec rackng usng flow lnear programmng. In Wner-PETS, [9] Prch, A. Rav-Acha, and S. Peleg, Nonchronologcal vdeo synopss and ndexng, IEEE Transacons on Paern Analyss and Machne Inellgence, vol. 30, no.11, pp , [10] C. Sauffer and W. E. L. Grmson, Adapve background mxure modelsfor real-me rackng, n Proc. IEEE In. Conf. Compu. Vs. Paern Recogn., vol. 19. Jun. 1999, pp [11] P. P erez, C. Hue, J. Vermaak, and M. Gangne, Color based probablsc rackng, Compuer Vson ECCV 2002, pp , [12]A. Okabe, B. Boos, and K. Sughara. Spaal Tessellaons: Conceps and Applcaons of Vorono Dagrams. New York, USA: Wley, 2000 [13] Duan-Yu Chen and Po-Chung Huang, Vsual-Based Human Crowds Behavor Analyss Based on Graph Modelng and Machng, In IEEE Sensors Journal, Vol. 13, No. 6, June 2013 [14]Unversy of Mnnesoa Crowd Acvy Daase, (2010)[Onlne].Avalable: hp://mha.cs.umn.edu/moves/crowd- Acvy-All.av