Who are you with and Where are you going?

Size: px
Start display at page:

Download "Who are you with and Where are you going?"

Transcription

1 Who are you wth and Where are you gong? Kota Yamaguch Alexander C. Berg Lus E. Ortz Tamara L. Berg Stony Brook Unversty Stony Brook Unversty, NY 11794, USA {kyamagu, aberg, leortz, Abstract We propose an agent-based behavoral model of pedestrans to mprove trackng performance n realstc scenaros. In ths model, we vew pedestrans as decson-makng agents who consder a plethora of personal, socal, and envronmental factors to decde where to go next. We formulate predcton of pedestran behavor as an energy mnmzaton on ths model. Two of our man contrbutons are smple, yet effectve estmates of pedestran destnaton and socal relatonshps (groups). Our fnal contrbuton s to ncorporate these hdden propertes nto an energy formulaton that results n accurate behavoral predcton. We evaluate both our estmates of destnaton and groupng, as well as our accuracy at predcton and trackng aganst state of the art behavoral model and show mprovements, especally n the challengng observatonal stuaton of nfrequent appearance observatons somethng that mght occur n thousands of webcams avalable on the Internet. 1. Introducton Despte many recent advances n trackng algorthms, effectve trackng n realstc scenaros s stll qute challengng. One common, yet less well studed scenaro, s survellance of scenes wth nfrequent appearance observatons such as the sporadc frames one would get from the thousands of webcams streamng pctures from around the globe. In ths case, the vdeo stream conssts of mages that are low resoluton, low frame rate (sometmes every few seconds), and dsplay uncontrolled lghtng condtons. Addtonal confuson can result from occluson between multple targets due to crowdng. Havng a strong pror on what we observe wll be essental for successful trackng n these challengng stuatons. In ths paper, we look at low frame rate and crowded trackng scenaros wth a focus on the behavoral model of pedestrans. Ths focus helps us both predct where people wll go, and who they are wth, and leads to mproved trackng results. Pedestrans exhbt complex behavor from varous socal and envronmental factors. For nstance, a pedestran has hs or her own destnaton n mnd a comfortable walkng speed, and plans a moton path that avods other pedestrans and scene obstacles. Our goal n ths paper s to buld a behavoral model that takes nto account these hgher level decsons and whch can easly be plugged nto exstng appearance-based algorthms. Wth ths n mnd, we model ndvdual pedestrans as agents who make decsons about velocty n the next tme step, gven factors from the scene (e.g. other pedestrans to avod or walk wth, or obstacles). We frame ths decson process as mnmzaton of an energy functon that encodes physcal condton, personal motvaton, and socal nteracton. One aspect of our approach s that we explctly address the problem of estmatng hdden factors that mght effect a pedestran s decson makng. One factor s the desred groupng behavor who a pedestran s tryng to walk wth. Another s the pedestran s desred destnaton n the scene. Nether of these factors s usually known n the survellance settng. We estmate these hdden personal propertes by vewng them as a classfcaton problem, predctable from the trajectory of a pedestran n the scene. In a survellance scenaro, t s reasonable to assume that there s a set of a few destnatons n the scene, such as the entrance of a buldng. Ths naturally lmts the pattern of trajectores n the scene. Also, people undergong socal nteractons tend to show a unque behavoral pattern compared wth ndvduals movng alone. We defne a feature representaton of trajectores on top of our velocty observatons, and predct both of these hdden personal varables usng effcent classfcaton approaches. The contrbutons of ths paper are: 1) producng an explct energy functon based behavoral model that encodes personal, socal, and envronmental decson factors, 2) data-drven estmaton of hdden personal propertes that affect the behavor of pedestrans, and 3) use of our proposed behavoral model for mproved trackng performance n low frame rate scenaros. We emphasze that our energy functon consders socal nteractons (groupng of pedestrans as they walk, talk and nteract), a factor whch has only 1345

2 recently been explored n [9]. Our approach to socal group estmaton s smpler, and more computatonally effcent, whle remanng effectve. Ths paper s organzed as follows. Secton 2 descrbes related work. Secton 3 descrbes our comprehensve behavoral model, followed by parameter learnng n Secton 4. Secton 5 detals the estmaton method of hdden personal propertes usng trajectory features. Secton 6 descrbes the quanttatve evaluaton of our behavoral model and property estmaton wth applcaton n trackng, and Secton 7 concludes ths paper. 2. Related work The pedestran behavor model has been extensvely studed n the felds where smulaton plays an mportant role, such as graphcs [5], and cvl engneerng [10] [11], or where accurate predcton s requred, such as robotcs [13, 6]. In most crowd smulaton contexts, the base model dates back to the classc socal force model [4], n whch the behavoral factors are assumed to gve an equaton that drves pedestrans n analogy to physcs. In computer vson, the attempt to detect abnormal events wth the socal force model s reported n [7]. [12], several socal factors are known to affect a person s behavor. Antonn s work [1] s one of the earlest n computer vson to take advantage of the rch behavoral nformaton n a trackng applcaton. The dscrete choce model n ther work assumes that ndvdual pedestrans make a choce from a dscrete set of velocty optons at each tme step based on socal and envronmental factors n the scene. The assumpton of dscretzed choce allows effcent predcton wth analytcal solutons despte the large number of factors consdered n the model [1, 10]. However, due to the nature of the dscretzaton, the behavoral predcton tends to show artfacts when metrc s contnuous. In contrast, contnuous models have been recently proposed by [8, 11]. An advantage of contnuous model s the flexblty of constructng complex models, however, prevous work focuses on ndvdual motvaton of the behavor [8, 11], and the only socal context s collson avodance. of socal nteracton n behavoral model. Socal nteracton n the pedestran group began to be studed only recently n computer vson [2, 3, 9]. A trackng applcaton s ncluded n [9]. There the problem s formulated as a smultaneous dscrete assgnment of hypothetcal tracks and estmaton of socal relatonshps based on observatons over a short tme frame usng a CRF. The CRF formulaton ndrectly encodes a behavoral model. Our focus s to buld an explct behavoral model whch can explot the rch behavoral context n socal nteractons, yet reman straghtforward and effcent enough to be plugged nto other trackng approaches as a module. 3. Behavoral model 3.1. An energy functon for pedestran navgaton Our behavoral model s based on an energy functon for each pedestran that expresses the desrablty of possble drectons of moton for the pedestran. The energy functon combnes terms for the varous factors that nfluence the pedestran s choce. These are explaned n ths secton. We optmze the parameters of ths model so that choosng the mnmum energy drecton accurately predcts the behavors of pedestrans n labeled tranng data, and then evaluate the performance of the model on prevously unseen test data. The fttng procedure s descrbed n Secton 3.2. At each tme step t, pedestran s represented by a state varable s (t) = (p (t), v (t), u (t), z (t), A (t) ), where p (t), v (t), are the poston, velocty, preferred speed and u (t) and z (t) chosen destnaton, respectvely, of pedestran at tme t, whle A s the set of pedestrans n the same socal group as pedestran, ncludng hmself. Note that u (t), z (t) and A are not observable and usually assumed statc,.e., u (t) = u, z (t) = z and A (t) = A are tme-nvarant 1. As n [8], our model assumes that each pedestran makes a decson on the velocty v (t+ t) based on varous envronmental and socal factors n the scene, and we model ths decson-makng process as the mnmzaton of an energy functon. Our energy functon E Θ, where Θ = {λ 0, λ 1, λ 2, λ 3, λ 4, σ d, σ w, β} denotes a set of parameters, s as follows and conssts of a lnear combnaton 2 of sx components: E Θ (v; s, s ) λ 0 E dampng (v; s )+ λ 1 E speed (v; s )+ λ 2 E drecton (v; s )+ λ 3 E attracton (v; s, s A )+ λ 4 E group (v; s, s A )+ E collson (v; s, s σ d, σ w, β), (1) where we defne s A to be a set of state varables of the pedestrans n s socal group A, and s to be the set of states of other pedestrans except. From now on, the tme step t s dropped from each varable for notatonal smplcty. The followng paragraphs provde a descrpton of each of the sx components of the energy functon E Θ. Dampng. The dampng term penalzes sudden changes n the choce of velocty, relatve to the current state: E dampng (v; s ) v v 2. (2) 1 Sec. 4 shows how we automatcally estmate these. 2 The coeffcents are relatve, so we fx the collson coeffcent to

3 Speed. Pedestrans have ther own preferred speed dependng on physcal state, culture or scene envronment. The speed term penalzes choosng a speed that devates from the (hdden) preferred speed u of the pedestran : E speed (v; s ) (u v ) 2. (3) Drecton. The drecton term concerns the choce of the correct drecton towards the goal. We model ths by usng the negatve cosne between the velocty choce v and the drecton to the destnaton z from the current locaton p : E drecton (v; s ) z p z p v v. (4) Attracton. People n the same group tend to stay close to each other whle movng together. To capture ths effect, we defne the attracton term as E attracton (v; s, s A ) ( j A {} v v v j v j ) ( pj p j v v where p j = p p j. The second factor penalzes choosng a forward drecton that s far from another pedestran j A {} n the group A of pedestran. The frst factor s a weght that flps ths attracton effect f person j s movng n a drecton opposte to. Groupng. People n the same group tend to walk at smlar speeds and drectons. The groupng term penalzes velocty choces that are dfferent from the average velocty of the group: ) (5) E group (v; s, s A ) v v A 2 (6) where v A 1 A j A v j. (7) Note that the socal group A always ncludes pedestran. If A s a sngleton, the groupng term has the same effect as the dampng term. Collson. Pedestrans try to avod collsons wth obstacles or other pedestrans. We use the model descrbed n [8] to capture ths effect: E collson (v; s, s σ d, σ w, β) ( ) j w(s, s j ) exp. (8) d2 (v,s,s j) 2σ 2 d Note that ths term requres three parameters σ d, σ w, and β. The factor w(s, s j ) s a weght coeffcent, whle the functon d(v, s, s j ) n the exponent s the expected mnmum dstance between pedestran and j under a constantvelocty assumpton [8]: w(s, s j ) exp ( p j 2 ) ( ( 1 2σw 2 1 p )) β j 2 p j v v (9) d(v, s, s j ) p j p j (v v j ) v v j 2 (v v j ) (10) The frst term n (9) assgns less nfluence to dstant pedestrans, whle the second term n (9) assgns less weght to pedestrans outsde the vew of pedestran Dynamcal model We now descrbe how to ft the parameters of the model. Recall our assumpton that the personal propertes of ndvdual pedestrans are statc,.e., u (t) u, z (t) z, and A (t) A. Wth ths assumpton, and our energy functon encodng pedestran velocty preferences (defned n the prevous subsecton), the state transton of pedestran from tme t to t + t s defned by the followng dynamcal system: p (t+ t) v (t+ t) = p (t) = argmn v + v (t+ t) t (11) E Θ (v; s (t), s (t) ). (12) We use a gradent descent algorthm to solve for the mnma of the energy functon. We learn the 8 parameters Θ = {λ 0, λ 1, λ 2, λ 3, λ 4, σ d, σ w, β} requred by our energy functon from prevously annotated data. In order to make behavoral predctons wth our model, we need to deal wth the fact that personal propertes u, z and A are unobservable and thus unavalable at predcton tme. To deal wth ths problem, we estmate the personal propertes from the past hstory of states, as descrbed n Secton 5. We learn optmal parameters Θ by fttng the energy functon to fully observed trajectores n the labeled tranng data. Durng tranng, whle predctng the behavor of an ndvdual pedestran, we fx the states of the other pedestrans to the ground truth. Let us denote the ground truth data by s. We defne the learnng problem as computng Θ argmn Θ t p (t+ t) p (t+ t) (s (t), s (t), Θ) 2. (13) Ths s a complex nonlnear optmzaton problem and computng a global optmum s hard. We use a varant of smplex algorthm, wth restarts, to solve for a local mnma. We use eth and hotel sequences from [8] as tranng data. The dataset ncludes a total of 750 pedestrans wth 15,452 observatons of postons under 2.5 frames per second. To estmate the personal propertes, we assume the preferred speed of a pedestran s the average speed over that person s trajectores. The destnaton s set to be one of 4-5 manually labeled postons outsde the scene accordng to the drecton and poston at the end of the trajectores. 1347

4 The socal group s also manually labeled. We model scene obstacles as vrtual pedestrans wth zero-velocty, and manually set these postons along the actual scene obstacles. We sub-sampled at most 12 consecutve tracks (4.8 seconds) every 4 frames (1.6 seconds) for each pedestran track from each dataset. Then we use these tracks n (13) to learn the parameters. 4. Estmaton of personal propertes Our model requres knowledge of the hdden personal propertes, preferred speed, u, destnaton, z, and socal groupng, A, for behavoral predcton. As descrbed n more detal n ths secton, we estmate these varables usng the trajectory s hstory nformaton avalable at predcton tme t Preferred speed We assume a mean speed of past N past steps as the preferred speed of the person. A smple, but neffectve alternatve s to assume a sngle global speed for all pedestrans. Accordng to pedestran speed statstcs n [10], a typcal person walks around 1.3 m/s. However, ths approach gnores ndvdual dfferences and seems too rough n complex scenes (e.g., sometmes a person slows down to look around, or walks together wth kds) Destnaton The key observaton here s that a scene contans only a few types of trajectores. For example, f a scene s a street layng from left to rght, we observe persons ether passng from left to rght or rght to left. In ths case, t s easy to see that a person walkng toward the rght sde also has hs destnaton n the rght sde of the scene. Ths smple assumpton mght not work f the scene s more complex and has more potental destnatons. But lookng at certan prevous steps n someone s past moton gves us a strong cue as to where hs destnaton s n the scene. We generalze ths observaton to the destnaton predcton problem. Gven a past trajectory r (t) = {s (t ) } t t, we want to predct a destnaton z {Z 1, Z 2, Z 3,..., Z K } of the pedestran. We ntroduce a trajectory feature functon f dest (r (t) ) and tran a K-class classfer C dest to predct the destnaton: ẑ (t) = C dest (f dest (r (t) )). (14) The feature representaton of the trajectory s a concatenaton of the normalzed hstograms of poston p, speed v and drecton arctan(v ). In our experments, poston, speed and drecton hstograms are dscretzed nto 7-by-7, 7 and 9 bns, respectvely. All the hstograms have equally spaced bns. We adopt lnear support vector machne (SVM) as a classfer for ths task. It s generally preferred to use as lttle trajectory hstory nformaton as possble to estmate the destnaton, especally when usng behavoral predcton n real tme applcatons. We evaluate the estmaton performance wth respect to number of past step used to compute features n the next secton Socal groups Pedestrans walkng n groups tend to behave dfferently from pedestran walkng alone. Pedestran n groups tend to walk at the same speed whle keepng certan dstance between each other. As attempted n [9], we also try to estmate socal groups n a scene, but usng a smple yet more effcent approach. The task s to decde whether a pedestran and another pedestran j are n the same group. More precsely, gven a par of past trajectores (r (t), r (t) j ), we want to assgn a bnary label y j {+1, 1} that ndcates whether they are n the same group (+1) or not ( 1). Ths s a bnary classfcaton problem over parwse trajectores. By defnng a feature functon f group (r (t) C group from tranng data:, r (t) j ), we can tran a classfer ŷ j = C group (f group (r (t), r (t) j )). (15) The predcted socal group s then gven by  = {j ŷ j = +1, j } {}. (16) We use the followng quanttes as features: 1. normalzed hstogram of dstance p p j, 2. normalzed hstogram of absolute dfference n speed v v j, 3. normalzed hstogram of absolute dfference n drecton arctan(v ) arctan(v j ), 4. normalzed hstogram of absolute dfference n velocty drecton and relatve poston arctan(p j p ) arctan(v ), and 5. tme-overlap rato T (t) T (t) = {t t t, s (t ) steps n whch pedestran appears. T (t) j / T (t) T (t) j, where },.e., a set of past tme As wth destnaton estmaton, we use an SVM classfer. In the next secton, we show the accuracy of predcton as a functon of the number of past steps used to produce the feature values. 5. Evaluaton 5.1. Datasets For evaluaton, we used the eth and hotel sequences from [8], and the zara01, zara02 and stu03 sequences 1348

5 Table 1. Total number of annotatons n datasets Dataset eth hotel zara01 zara02 stu03 Frames Pedestrans Groups Observatons Destnatons Obstacles Table 2. Destnaton predcton accuracy N past Dataset eth hotel zara zara stu Table 3. Socal group predcton precson and recall from [5]. These sequences have annotated postons. We manually added the annotatons of scene obstacles, destnatons and socal groups. Table 1 summarzes the number of annotatons n the dfferent datasets. All the sequences have 25 fps, and annotatons are gven every 0.4 seconds. We use all the sequences to evaluate our personalproperty estmator. For the experments on behavoral predcton and trackng, we use eth and hotel to learn parameters, and zara01, zara02 and stu03 to evaluate Personal-property estmaton To evaluate the performance of destnaton and group estmaton, we ran a 3-fold cross-valdaton on predcton accuracy. In ths experment, we do ths by subsamplng tracks {s (t ) } t Npast t t t for N past = {0, 1, 2, 4, 8, }, 3 every 4 frames for each person. The sampled trajectores are then unformly splt nto the 3 sets used for the 3-fold cross-valdaton. Table 2 shows the average accuracy of destnaton predcton whle Table 3 shows the average precson and recall of socal group predcton, both as a functon of the number of past steps used to compute trajectory features. The expermental results n Table 2 suggest that the dffculty of destnaton estmaton depends strongly on the type of scene. Typcally, confuson occurs when trajectores havng dfferent destnatons share a sub-trajectory. In fact, our estmaton s worse n the eth and hotel datasets than n the zara01 and zara02, because the ntal part of trajectores n the former datasets look very smlar, even f those trajectores later dverge as pedestrans move. The estmaton n stu03 dataset s worst because, n that dataset, many people standng at the same locaton, whch confuses our predctor. Note that n the annotaton we automatcally assgned the closest destnaton located outsde the scene to pedestran temporarly standng. Also, we can observe that the number of past steps used to compute trajectory features has almost no nfluence on predcton accuracy. Rather, t s remarkable that the estmaton usng only the current state s (t) already gves reasonable performance. Ths ndcates that the locaton and velocty of the pedestran n the current 3 By N past = we mean all past steps avalable. Precson Recall N past Dataset eth hotel zara zara stu eth hotel zara zara stu scene already provdes enough nformaton to guess where that person wll move n the future. Table 3 shows that the socal group can be estmated wth reasonably well, regardless of scene envronment. Also, n ths case, havng more past nformaton does ndeed mprove estmaton performance. Note that n ths experment we predct a label of a drectonal lnk label between two persons but do not consder the symmetrc and transtve propertes of socal relatons n groups. Imposng these propertes va addtonal constrants mght further mprove estmaton performance Behavoral predcton To evaluate the performance of behavoral predcton, we calculated the average dsplacement of the predcted poston of a sngle pedestran from the ground truth poston. As n parameter learnng, n ths experment, we also fx the states of other pedestrans to the ground truth. We evaluated the error n the zara01, zara02 and stu03 datasets usng the parameters learned from the eth and hotel datasets. Because the destnaton estmator requres scene specfc data, we performed 2-fold cross-valdaton by splttng each dataset nto a frst and a second half (correspondng to the ntal and last perod of tme n the vdeo). Smlarly to parameter learnng, we subsampled at most 12 consecutve tracks (4.8 seconds) every 4 frames (1.6 seconds) for each pedestran track, and computed predcton 1349

6 Table 4. Error n behavoral predcton (m) Dataset Method zara01 zara02 stu03 LIN LTA LTA+D ATTR ATTR+D ATTR+G ATTR+DG error usng the average of the objectve functon gven n (13). Tracks n the tranng set are then used to buld personal property estmators. We allow estmators to use at most 5 past steps of nformaton to compute features. In ths evaluaton, we compared the constant speed model (LIN); the collson avodance model of [8] wth ground truth destnaton (LTA) and wth predcted destnaton (LTA+D); and our model wth ground truth (ATTR), predcted destnaton (ATTR+D), predcted socal groups (ATTR+G) and predcted destnaton and socal groups combned (ATTR+DG). The preferred speed s set to ground truth n all cases. Table 4 summarzes the average dsplacement (n meters) from ground truth at each predcton. The result s the average of a 2-fold cross-valdaton. Both LTA and our model perform better than LIN n all cases, wth or wthout ground truth. We can also see that usng predcted destnaton and socal groups does not degrade the error sgnfcantly, and n fact, ther combnaton wth our behavoral model produces better results n the zara02 and the students03 datasets. Ths may seem to contradct our ntuton, because the model may be usng ncorrectly predcted destnatons. However, those datasets have many crowded scenes n whch often pedestrans stop walkng to chat or look around. In that case, predctng a destnaton outsde the scene apparently s unreasonable. We beleve our predctons dynamcally provded more reasonable destnatons for such trcky pedestrans and thus better descrbe the actual stuaton. Fgure 1 shows an example of the predcton over 12 steps n the zara01 dataset. A subject s movng towards the rghtsde of the scene wth another person, and s about to pass by another group movng n the opposte drecton. The LIN model loses ts track from the ground truth. Both LTA and our model track the ground-truth pedestran path more closely. However, LTA predcts a straght path towards the goal whle our model also predcts fluctuatons as a consequence of the socal nteracton Trackng We evaluated the effect of our behavoral model n a trackng applcaton. Havng n mnd a vdeo survellance Fgure 1. Example of behavoral predcton scenaro usng a low frame rate webcam, we compare the number of successful tracks acheved by the dfferent models under an observaton frequency of every 0.4 seconds (2.5 fps) and 1.6 seconds (0.625 fps), n the zara01, zara02 and students03 datasets, keepng behavoral predcton runnng every 0.4 seconds n both cases. To llustrate the postve effect of our behavoral pror n ths settng, we use a smple pxel correlaton for the appearance model. Our tracker s a smple combnaton of appearance and behavoral model: P (p) = P appearance (p) P behavor (p) ) ( (1 NCC(p))2 exp ( 2σa 2 exp ) p p(t) 2 2σb 2. where NCC s the normalzed cross correlaton of pxels, p (t) s the predcton from the behavoral model, and σ a and σ b are the varance parameter for each model. The predcted poston at tme t s thus gven by ˆp (t) = argmax p P (p). Under the less frequent mage observaton, we treat the behavoral predcton as the combned predcton when we do not have an appearance term. The vdeo sequences n the datasets have relatvely nosy background. We frst apply background subtracton as an mage preprocessng step before we compute pxel correlatons. We use a runnng average as background of the scene, and regarded a regon havng small absolute dfference from the model as a background. We start to accumulate background 5 steps before we start trackng n each sequence. We experment wth trackng n a subsequence of vdeos. As n the case of the prevous secton, we splt the dataset 1350

7 nto a frst half and a second half, tran a personal-property estmator n one fold, and test n the other. Snce our socal group estmator s compatble across datasets, n ths experment we use a sngle group estmator traned usng all three datasets. We start trackng every 16 frames and keep them runnng for at most 24 subsequent frames as long as the ground truth data exst for the scene. The expermental data contans 55, 64 and 34 subsequences for the zara01, zara02 and students03 datasets, respectvely, n total for the 2-fold experments. The tracker starts from a ground-truth state, wth at most 5 past steps avalable for personal-property predcton and background subtracton. Once the tracker starts, no future or ground truth nformaton s avalable. We compare the trackng performance between a lnear model wth full appearance observaton under 2.5 fps (LIN+FULL), wth less frequent appearance observaton under fps (LIN+LESS), LTA model wth full or less frequent appearance observaton (LTA+FULL, LTA+LESS), our model wth full or less frequent appearance observaton (ATTR+FULL or ATTR+LESS, respectvely), and for reference, a smple correlaton tracker wthout any behavoral pror under full mage observaton. In ths experment, we use predcted personal propertes for LTA and our model. To comply wth the trackng method n [8], the LTA model uses nearest-neghbor decsons to predct destnaton, usng current drecton of velocty and drecton to a set of destnatons. Our model uses the predcton method of secton 5. The evaluaton conssts of calculatng how many trackers stay wthn 0.5 meter from the ground-truth annotaton of the same person at the N = {8, 16, 24}-th step snce the ntal state. A track that s more than 0.5 meter away from ts correspondng track n the ground-truth data s regarded as lost, whle a track that s wthn 0.5 meter from ground-truth but whose closest track n the ground-truth s of dfferent person s consdered ID swtch. Fgure 2 compares the number of successful tracks between trackng methods. The performance of a tracker under full appearance observaton (2.5 fps) does not vary among behavoral models, and full appearance observaton always results n performance mprovement. However, under less frequent observaton, our method outperforms n zara01 and zara02 dataset. Table 5 summarzes the number of successful, ID-swtched, or lost tracks under less frequent appearance observaton. The stu03 result shows that the lnear model s the best among others. Ths s lkely the result of rregular type of pedestrans n the dataset: n those scenes, there are many people who walk around unpurposefully and stop to chat. Both LTA and our model assume that a pedestran s always walkng towards the goal and cannot correctly deal wth a standng person. Ths resulted n better performance for the lnear model. Fgure 3. Tracker example. The lne ndcates the dsplacement from the prevous frame. Under fps, t s hard to fnd a correspondence between frames wthout pror. Fgure 3 shows an example of a tracker from the zara01 dataset. Our behavoral model gves stronger preference to keepng the dstance between pedestrans n the same socal group constant. 6. Concluson and Future Work We propose an agent-based formulaton of pedestran behavor and a method to estmate hdden personal propertes. Our evaluaton of destnaton and socal group estmaton, together wth that of the behavoral predcton error, suggests that t s possble to get a reasonable estmate of unobservable personal nformaton from purely behavoral and envronmental data only. Our trackng experment shows that, for usual scenes where pedestrans do not exhbt sudden rregular motons, our behavoral model further mproves performance over smpler behavoral models under low frame rates. It would be nterestng to extend our behavoral model by usng an explct model of pedestran behavor that accounts for more that just a walkng state. Also, n our future work, we wll take nto account the nteracton between pedestran behavor and scene events or objects. References [1] G. Antonn, S. Martnez, M. Berlare, and J. Thran. Behavoral prors for detecton and trackng of pedestrans n vdeo sequences. Internatonal Journal of Computer Vson, 69: , /s [2] W. Cho, K. Shahd, and S. Savarese. What are they dong? : Collectve actvty classfcaton usng spato-temporal relatonshp among people. In Computer Vson Workshops (ICCV Workshops), 2009 IEEE 12th Internatonal Conference on, pages , [3] W. Ge, R. Collns, and B. Ruback. Automatcally detectng the small group structure of a crowd. In Applcatons of Computer Vson (WACV), 2009 Workshop on, pages 1 8. IEEE,

8 zara01 zara02 stu #TRK:387 CORR LIN+FULL LTA+FULL ATTR+FULL LIN+LESS LTA+LESS ATTR+LESS #TRK:685 CORR LIN+FULL LTA+FULL ATTR+FULL LIN+LESS LTA+LESS ATTR+LESS #TRK:1255 CORR LIN+FULL LTA+FULL ATTR+FULL LIN+LESS LTA+LESS ATTR+LESS Success count #TRK:228 #TRK:104 Success count #TRK:487 #TRK:290 Success count #TRK: 832 #TRK: Predcton step Predcton step Predcton step Fgure 2. Number of successful tracks over predcton steps that stayed wthn 0.5 meter from the truth. Under the full appearance observaton, the performance does not change among behavoral models. However, approprate behavoral pror helps mprovng trackng performance under lmted mage observaton n zara01 and zara02. Table 5. Trackng result zara01 zara02 stu03 Track Method N=8 N=16 N=24 N=8 N=16 N=24 N=8 N=16 N=24 LIN+LESS Success LTA+LESS ATTR+LESS LIN+LESS ID swtch LTA+LESS ATTR+LESS LIN+LESS Lost LTA+LESS ATTR+LESS [4] D. Helbng and P. Molnar. Socal force model for pedestran dynamcs. Phys. Rev. E, 51(5): , May [5] A. Lerner, Y. Chrysanthou, and D. Lschnsk. Crowds by example. EUROGRAPHICS, , 1349 [6] M. Luber, J. Stork, G. Tpald, and K. Arras. People trackng wth human moton predctons from socal forces. In Robotcs and Automaton (ICRA), 2010 IEEE Internatonal Conference on, pages IEEE, [7] R. Mehran, A. Oyama, and M. Shah. Abnormal crowd behavor detecton usng socal force model. IEEE Conference on Computer Vson and Pattern Recognton, pages , [8] S. Pellegrn, A. Ess, K. Schndler, and L. van Gool. You ll never walk alone: Modelng socal behavor for mult-target trackng. Internatonal Conference on Computer Vson (ICCV), , 1347, 1348, 1350, 1351 [9] S. Pellegrn, A. Ess, and L. van Gool. Improvng data assocaton by jont modelng of pedestran trajectores and groupngs. ECCV 2010, , 1348 [10] T. Robn, G. Antonn, M. Berlare, and J. Cruz. Specfcaton, estmaton and valdaton of a pedestran walkng behavor model. Transportaton Research Part B: Methodologcal, 43(1):36 56, , 1348 [11] P. Scovanner and M. Tappen. Learnng pedestran dynamcs from the real world. Internatonal Conference on Computer Vson (ICCV), pages , [12] H. Tmmermans. Pedestran Behavor: Data Collecton and Applcatons. Emerald Group Publshng Lmted, [13] B. Zebart, N. Ratlff, G. Gallagher, K. Peterson, J. A. Bagnell, M. Hebert, A. K. Dey, and S. Srnvasa. Plannngbased predcton for pedestrans. Proceedngs of the 2009 IEEE/RSJ Internatonal Conference on Intellgent Robots and Systems,

Boosting as a Regularized Path to a Maximum Margin Classifier

Boosting as a Regularized Path to a Maximum Margin Classifier Journal of Machne Learnng Research 5 (2004) 941 973 Submtted 5/03; Revsed 10/03; Publshed 8/04 Boostng as a Regularzed Path to a Maxmum Margn Classfer Saharon Rosset Data Analytcs Research Group IBM T.J.

More information

Do Firms Maximize? Evidence from Professional Football

Do Firms Maximize? Evidence from Professional Football Do Frms Maxmze? Evdence from Professonal Football Davd Romer Unversty of Calforna, Berkeley and Natonal Bureau of Economc Research Ths paper examnes a sngle, narrow decson the choce on fourth down n the

More information

As-Rigid-As-Possible Image Registration for Hand-drawn Cartoon Animations

As-Rigid-As-Possible Image Registration for Hand-drawn Cartoon Animations As-Rgd-As-Possble Image Regstraton for Hand-drawn Cartoon Anmatons Danel Sýkora Trnty College Dubln John Dnglana Trnty College Dubln Steven Collns Trnty College Dubln source target our approach [Papenberg

More information

Face Alignment through Subspace Constrained Mean-Shifts

Face Alignment through Subspace Constrained Mean-Shifts Face Algnment through Subspace Constraned Mean-Shfts Jason M. Saragh, Smon Lucey, Jeffrey F. Cohn The Robotcs Insttute, Carnege Mellon Unversty Pttsburgh, PA 15213, USA {jsaragh,slucey,jeffcohn}@cs.cmu.edu

More information

As-Rigid-As-Possible Shape Manipulation

As-Rigid-As-Possible Shape Manipulation As-Rgd-As-Possble Shape Manpulaton akeo Igarash 1, 3 omer Moscovch John F. Hughes 1 he Unversty of okyo Brown Unversty 3 PRESO, JS Abstract We present an nteractve system that lets a user move and deform

More information

(Almost) No Label No Cry

(Almost) No Label No Cry (Almost) No Label No Cry Gorgo Patrn,, Rchard Nock,, Paul Rvera,, Tbero Caetano,3,4 Australan Natonal Unversty, NICTA, Unversty of New South Wales 3, Ambata 4 Sydney, NSW, Australa {namesurname}@anueduau

More information

MANY of the problems that arise in early vision can be

MANY of the problems that arise in early vision can be IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 26, NO. 2, FEBRUARY 2004 147 What Energy Functons Can Be Mnmzed va Graph Cuts? Vladmr Kolmogorov, Member, IEEE, and Ramn Zabh, Member,

More information

Ensembling Neural Networks: Many Could Be Better Than All

Ensembling Neural Networks: Many Could Be Better Than All Artfcal Intellgence, 22, vol.37, no.-2, pp.239-263. @Elsever Ensemblng eural etworks: Many Could Be Better Than All Zh-Hua Zhou*, Janxn Wu, We Tang atonal Laboratory for ovel Software Technology, anng

More information

Assessing health efficiency across countries with a two-step and bootstrap analysis *

Assessing health efficiency across countries with a two-step and bootstrap analysis * Assessng health effcency across countres wth a two-step and bootstrap analyss * Antóno Afonso # $ and Mguel St. Aubyn # February 2007 Abstract We estmate a sem-parametrc model of health producton process

More information

can basic entrepreneurship transform the economic lives of the poor?

can basic entrepreneurship transform the economic lives of the poor? can basc entrepreneurshp transform the economc lves of the poor? Orana Bandera, Robn Burgess, Narayan Das, Selm Gulesc, Imran Rasul, Munsh Sulaman Aprl 2013 Abstract The world s poorest people lack captal

More information

TrueSkill Through Time: Revisiting the History of Chess

TrueSkill Through Time: Revisiting the History of Chess TrueSkll Through Tme: Revstng the Hstory of Chess Perre Dangauther INRIA Rhone Alpes Grenoble, France perre.dangauther@mag.fr Ralf Herbrch Mcrosoft Research Ltd. Cambrdge, UK rherb@mcrosoft.com Tom Mnka

More information

Complete Fairness in Secure Two-Party Computation

Complete Fairness in Secure Two-Party Computation Complete Farness n Secure Two-Party Computaton S. Dov Gordon Carmt Hazay Jonathan Katz Yehuda Lndell Abstract In the settng of secure two-party computaton, two mutually dstrustng partes wsh to compute

More information

Turbulence Models and Their Application to Complex Flows R. H. Nichols University of Alabama at Birmingham

Turbulence Models and Their Application to Complex Flows R. H. Nichols University of Alabama at Birmingham Turbulence Models and Ther Applcaton to Complex Flows R. H. Nchols Unversty of Alabama at Brmngham Revson 4.01 CONTENTS Page 1.0 Introducton 1.1 An Introducton to Turbulent Flow 1-1 1. Transton to Turbulent

More information

DISCUSSION PAPER. Should Urban Transit Subsidies Be Reduced? Ian W.H. Parry and Kenneth A. Small

DISCUSSION PAPER. Should Urban Transit Subsidies Be Reduced? Ian W.H. Parry and Kenneth A. Small DISCUSSION PAPER JULY 2007 RFF DP 07-38 Should Urban Transt Subsdes Be Reduced? Ian W.H. Parry and Kenneth A. Small 1616 P St. NW Washngton, DC 20036 202-328-5000 www.rff.org Should Urban Transt Subsdes

More information

DISCUSSION PAPER. Is There a Rationale for Output-Based Rebating of Environmental Levies? Alain L. Bernard, Carolyn Fischer, and Alan Fox

DISCUSSION PAPER. Is There a Rationale for Output-Based Rebating of Environmental Levies? Alain L. Bernard, Carolyn Fischer, and Alan Fox DISCUSSION PAPER October 00; revsed October 006 RFF DP 0-3 REV Is There a Ratonale for Output-Based Rebatng of Envronmental Leves? Alan L. Bernard, Carolyn Fscher, and Alan Fox 66 P St. NW Washngton, DC

More information

UPGRADE YOUR PHYSICS

UPGRADE YOUR PHYSICS Correctons March 7 UPGRADE YOUR PHYSICS NOTES FOR BRITISH SIXTH FORM STUDENTS WHO ARE PREPARING FOR THE INTERNATIONAL PHYSICS OLYMPIAD, OR WISH TO TAKE THEIR KNOWLEDGE OF PHYSICS BEYOND THE A-LEVEL SYLLABI.

More information

4.3.3 Some Studies in Machine Learning Using the Game of Checkers

4.3.3 Some Studies in Machine Learning Using the Game of Checkers 4.3.3 Some Studes n Machne Learnng Usng the Game of Checkers 535 Some Studes n Machne Learnng Usng the Game of Checkers Arthur L. Samuel Abstract: Two machne-learnng procedures have been nvestgated n some

More information

EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM 1

EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM 1 Int. J. Systems Sc., 1970, vol. 1, No. 2, 89-97 EVERY GOOD REGULATOR OF A SYSTEM MUST BE A MODEL OF THAT SYSTEM 1 Roger C. Conant Department of Informaton Engneerng, Unversty of Illnos, Box 4348, Chcago,

More information

The Relationship between Exchange Rates and Stock Prices: Studied in a Multivariate Model Desislava Dimitrova, The College of Wooster

The Relationship between Exchange Rates and Stock Prices: Studied in a Multivariate Model Desislava Dimitrova, The College of Wooster Issues n Poltcal Economy, Vol. 4, August 005 The Relatonshp between Exchange Rates and Stock Prces: Studed n a Multvarate Model Desslava Dmtrova, The College of Wooster In the perod November 00 to February

More information

Why Don t We See Poverty Convergence?

Why Don t We See Poverty Convergence? Why Don t We See Poverty Convergence? Martn Ravallon 1 Development Research Group, World Bank 1818 H Street NW, Washngton DC, 20433, USA Abstract: We see sgns of convergence n average lvng standards amongst

More information

Income per natural: Measuring development as if people mattered more than places

Income per natural: Measuring development as if people mattered more than places Income per natural: Measurng development as f people mattered more than places Mchael A. Clemens Center for Global Development Lant Prtchett Kennedy School of Government Harvard Unversty, and Center for

More information

The Developing World Is Poorer Than We Thought, But No Less Successful in the Fight against Poverty

The Developing World Is Poorer Than We Thought, But No Less Successful in the Fight against Poverty Publc Dsclosure Authorzed Pol c y Re s e a rc h Wo r k n g Pa p e r 4703 WPS4703 Publc Dsclosure Authorzed Publc Dsclosure Authorzed The Developng World Is Poorer Than We Thought, But No Less Successful

More information

Should marginal abatement costs differ across sectors? The effect of low-carbon capital accumulation

Should marginal abatement costs differ across sectors? The effect of low-carbon capital accumulation Should margnal abatement costs dffer across sectors? The effect of low-carbon captal accumulaton Adren Vogt-Schlb 1,, Guy Meuner 2, Stéphane Hallegatte 3 1 CIRED, Nogent-sur-Marne, France. 2 INRA UR133

More information

Ciphers with Arbitrary Finite Domains

Ciphers with Arbitrary Finite Domains Cphers wth Arbtrary Fnte Domans John Black 1 and Phllp Rogaway 2 1 Dept. of Computer Scence, Unversty of Nevada, Reno NV 89557, USA, jrb@cs.unr.edu, WWW home page: http://www.cs.unr.edu/~jrb 2 Dept. of

More information

The Global Macroeconomic Costs of Raising Bank Capital Adequacy Requirements

The Global Macroeconomic Costs of Raising Bank Capital Adequacy Requirements W/1/44 The Global Macroeconomc Costs of Rasng Bank Captal Adequacy Requrements Scott Roger and Francs Vtek 01 Internatonal Monetary Fund W/1/44 IMF Workng aper IMF Offces n Europe Monetary and Captal Markets

More information

Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C.

Finance and Economics Discussion Series Divisions of Research & Statistics and Monetary Affairs Federal Reserve Board, Washington, D.C. Fnance and Economcs Dscusson Seres Dvsons of Research & Statstcs and Monetary Affars Federal Reserve Board, Washngton, D.C. Banks as Patent Fxed Income Investors Samuel G. Hanson, Andre Shlefer, Jeremy

More information

From Computing with Numbers to Computing with Words From Manipulation of Measurements to Manipulation of Perceptions

From Computing with Numbers to Computing with Words From Manipulation of Measurements to Manipulation of Perceptions IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I: FUNDAMENTAL THEORY AND APPLICATIONS, VOL. 45, NO. 1, JANUARY 1999 105 From Computng wth Numbers to Computng wth Words From Manpulaton of Measurements to Manpulaton

More information

THE CATHOLIC EDUCATION SERIES MUSIC FIRST YEAR JUSTINE WARD AND ELIZABETH W. PERKINS THE CATHOLIC EDUCATION PRESS WASHINGTON, D. C.

THE CATHOLIC EDUCATION SERIES MUSIC FIRST YEAR JUSTINE WARD AND ELIZABETH W. PERKINS THE CATHOLIC EDUCATION PRESS WASHINGTON, D. C. THE CATHOLIC EDUCATION SERIES MUSIC FIRST YEAR By JUSTINE WARD AND ELIZABETH W. PERKINS THE CATHOLIC EDUCATION PRESS WASHINGTON, D. C. COPYRIGHT, 90, BY CATHOLIC EDUCATION PRESS WASHINGTON, D. C. NATIONAL

More information

What to Maximize if You Must

What to Maximize if You Must What to Maxmze f You Must Avad Hefetz Chrs Shannon Yoss Spegel Ths verson: July 2004 Abstract The assumpton that decson makers choose actons to maxmze ther preferences s a central tenet n economcs. Ths

More information

WHICH SECTORS MAKE THE POOR COUNTRIES SO UNPRODUCTIVE?

WHICH SECTORS MAKE THE POOR COUNTRIES SO UNPRODUCTIVE? MŰHELYTANULMÁNYOK DISCUSSION PAPERS MT DP. 2005/19 WHICH SECTORS MAKE THE POOR COUNTRIES SO UNPRODUCTIVE? BERTHOLD HERRENDORF ÁKOS VALENTINYI Magyar Tudományos Akadéma Közgazdaságtudomány Intézet Budapest

More information