Visual Tracking of Athletes in Volleyball Sport Videos

Transcription

1 Visual Tracking of Athletes in Volleyball Sport Videos H.Salehifar 1 and A.Bastanfard 2 1 Faculty of Electrical, Computer and IT, Islamic Azad University, Qazvin Branch, Qazvin, Iran 2 Computer Group, Faculty of Engineering, Islamic Azad University, Karaj Branch, Karaj, Iran Abstract - Locating, labeling, and tracking players have broad application, especially for a team analyzer, and in the broadcast sports videos industry. In this paper we have presented a framework based on Gaussian Mixture Model and Kalman Filter for detecting and segmenting players in volleyball games. In this method, the players is detected by Gaussian Mixture Model and using Kalman filter the next location of every player in next frames is predicted. The proposed approach has the robust ability to track moving objects in consecutive frames under some kinds of difficulties such as rapid appearance changes caused by image noise and occlusion. Keywords: Human Motion; Tracking; Volleyball Athletes 1 Introduction Object tracking has received tremendous attention in the video processing community due to its numerous potential applications in video surveillance, human activity analysis, traffic monitoring, and so on. Recently the focus of the community is on Multi-Target Tracking (MTT) that requires determining the number as well as the dynamics of targets. However, due to a combination of several factors, reliable target tracking remains a challenging domain of research. The underlying difficulties behind multi-target tracking are founded mostly upon the apparent similarity of targets and then multi-target occlusion. MTT for targets whose appearance is distinctive is comparatively easier since it can be solved reasonably well by using multiple Independent single-target trackers. However, MTT for targets whose appearance is similar such as pedestrians in crowded scenes is a much more difficult task. In addition, with this MTT must deal with multi-target occlusion, namely, the tracker must separate the targets and assign them correct labels. Computational complexity also plays an important role, as the tracking should be real time. All these issues make target tracking or multi-object tracking a sturdy task even now. One the remarkable application is analyzing the players activity in different sports. Locating, labeling, and tracking players have broad application, especially for a team analyzer, and in the broadcast sports videos industry. There are several issues which make this kind of video analyzing a challenging task. Players occlusion, similar player s appearance, various numbers of players, unexpected camera motion, video blur and noise are the samples of this difficulty. Many algorithms have been proposed to deal with the multiple target-tracking (MTT) problem in which Particle Filter [1], [2], Joint Probabilistic Data Association Filter (JPDAF) [3], Multiple Hypothesis Tracking (MHT) [4], MCMC data association [5], [6] and track linking [7], [8], [9] are used. Several researchers also study the specific problem of labeling and tracking of players in sports video [2][10][11]. In [10], a clustering based trajectory matching method is proposed to track players in a soccer video. In this work, labeling of individuals is achieved through a supervised classification method. Comaniciu et al [11] build a track graph, and take the tracking problem as an inference problem in a Bayesian network. In both works, a multi-camera system is used to get a fixed, high-resolution and wide-field view of a soccer game. These pre-defined settings ensured a reliable background subtraction system, which are not very practical approach. This paper presents our approach in volleyball for a vision-based tracking system which can be used in practice by trainers and athletes. In this paper we have presented a framework based on GMM 1 and Kalman Filter for detecting and segmenting players in volleyball games. This paper is organized as follows: proposed framework and tracking algorithm are described in section 2. Evaluation is shown in section 3, and conclusions are made in section 4. 2 Proposed framework Object tracking is the problem of estimating the positions and other relevant information of moving objects in image sequences. This section describes the tracking method used in this paper. The proposed approach is confection of GMM and Kalman Filter algorithms. At first, we explained the GMM and Kalman Filter. 2.1 Gaussian mixture model Before we start with tracking of moving objects, we need to extract moving objects from the background. Background subtraction is one of the most common approaches for detecting foreground objects from video sequences. Recently, some statistical methods are used to extract change regions from the background. The Gaussian mixture model is the most representative background model [12]. The value of a pixel at time t in RGB or some other color space is denoted by. Pixel-based background subtraction involves decision if the pixel belongs to background (BG) or some foreground object (FG). Bayesian decision R is made by: 1 Gaussian Mixture Model

2 In a general case we don't know anything about the foreground objects that can be seen nor when and how often they will be present. Therefore we set P(FG) = P(BG) and assume uniform distribution for the foreground object appearance. The pixel belongs to the background if: The squared distance from the m-th component is calculated as: D m 2 (X (t) )= δ m T δ m /σ m 2. If there are no 'close' components a new component is generated with π M+1 =α, µ M+1 = X (t) and σ M+1 =σ 0. Where σ 0 is some appropriate initial variance. If the maximum number of components is reached, the component is discard by the smallest π M. The presented algorithm presents an on-line clustering algorithm. Usually, the intruding foreground objects will be represented by some additional clusters with small weights π M. Therefore, the background model is approximated by the first B largest clusters: (3) Where is a threshold value. We will refer to as the background model. The background model is estimated from a training set denoted as. The estimated model is denoted by and depends on the training set as denoted explicitly. In practice, the illumination in the scene could change gradually or suddenly. New object could be brought into the scene or a present object removed from it. In order to adapt to changes we can update the training set by adding new samples and discarding the old ones. So is chosen reasonable time period T and at time t, T is: T = {X (t),, X (t-t) }. For each new sample, the training data set T is updated and is re-estimated. However, among the samples from the recent history there could be some values that belong to the foreground objects and this estimate is denoted as P (X, BG+FG). GMM is used with M components: If the components are sorted to have descending weights π M we have: Where is a measure of the maximum portion of the data that can belong to foreground objects without influencing the background model. Flowchart of GMM algorithm is illustrated in figure1. The output of this process is shown in figure2. Where µ 1,, µ 2 are the estimates of the means and σ 1,,σ 2 are the estimates of the variances that describe the Gaussian components. The covariance matrices are assumed to be diagonal and the identity matrix I has proper dimensions. The mixing weights denoted by π m are non-negative and add up to one. Given a new data sample X (t) at time the recursive update equations are [13]: Where δ m = X (t) - µ m. Instead of the time interval, T that was mentioned above, here constant α=1/t describes an exponentially decaying envelope that is used to limit the influence of the old data. For a new sample the ownership ο m (t) is set to 1 for the 'close' component with largest π m and the others are set to zero. The sample is defined 'close' to a component if the Mahalanobis distance from the component is for example less than three standard deviations. Figure1. Flowchart of GMM algorithm.

3 Besides, we represented the rectangular window using center coordinate. Because the moving state changed little in the neighboring consecutive frames, we thought of the system as linear Gaussian one and the state parameters of Kalman filter were object location, its velocity, and the width of the rectangle which represent the width of a Human. The statespace representation of the tracker used in the Kalman filter is given in Eq. (10) (a) Original frame where, and are the predicted coordinates of the object and and are the velocities in the respective direction, represents the width of the Human rectangle, Δt represents the time interval of state correction and is the white Gaussian noise with diagonal variance Q. The predicted coordinates and dimensions of the rectangle are used to locate the Human in the present frame. When the Humans are distinguished, the Kalman vector is updated using the measurement equation as shown in Eq. (11). (b) Foreground detection Figure2. The execution of the GMM algorithm. (a)original image (b) the foreground detection. 2.2 Kalman filter A Kalman filter is applied to estimate the state of a linear system where the state is assumed to be distributed by a Gaussian. Kalman filtering is composed of two steps, prediction and correction [14, 15]. The Kalman filter is a recursive estimator. This means that only the estimated state form the previous time step and the current measurement are needed to compute the estimate for the current state. In contrast to batch estimation techniques [16], no history of observations and/or estimates is required. Kalman filter consists of five equations and it can be divide them into two groups: the update equations and the correct equations. The update equations are responsible for projecting forward the current state and error covariance estimates to obtain the priori estimates for the next time step. The correct equations are responsible for the feedback, in other words, for incorporating a new measurement into the priori estimate to obtain an improved posteriori estimate. The state variable and observation of Kalman filter in this paper are object locations. Where and are the measured coordinates, the value is the measure width of Human at time t+1 and is white Gaussian noise with diagonal variance R. The position, velocity and acceleration are updated based on the values obtained in the present frame and the data from the previous frame. 2.3 Schematic description of the tracking algorithm After extract moving objects from background in each video frame using the GMM algorithm, the kalman-prediction function is used to predict the next position players in the next frame. Since each frame may identify new objects, we need to specify the previous objects and objects that are formed newly. This detection is done using the data-association function. New objects in the current frame using the Add New-Hypotheses function, are added And after removal of noise using the Kalman-Update function is transferred to the next frame. This loop repeated for each frame and thus tracking algorithm is complete. Overall procedure of the proposed method is illustrated in figure3.

4 Figure 4. The sample of images of PVD dataset. 2.4 Dataset Figure3.The proposed tracking algorithm scheme. The recent approach of researchers in the field of image processing is about video image and one of the most important usages of moving images processing is analyzing sport plays.to support all the motions and rules of the play, we need a dataset that is not specified to a special some action.. Cause of the lack a dataset in volleyball, we presented in previous paper [17] a complete view depended volleyball video dataset under the uncontrolled conditions. The proposed dataset includes the complete volleyball play video sequences. We tried to prepare data in uncontrolled conditions and different viewpoints when planning dataset. Since the ball is in various heights, we need an angle in appropriate height, which keep the ball and players image when playing. Therefore, the recording angles are selected witch make possible the best condition to cover players and ball. The data is prepared in various sport gym to guarantee the data comprehensively. The images of the dataset are prepared from official Iran volleyball league matches. The sample of images of dataset sequences are shown in figure4. 3 Evaluation This section show the results obtained by the proposed tracking algorithm. Our approach is implemented using Window Vista operation system, and Matlab R2007b are used. In addition, the image sequences consists of JPG images with 320x240 resolutions per frame. The system evaluation is measured by calculating the detection rate using temporal differencing [18] and the proposed tracking system. For tracking based on the temporal differencing, the difference between two corresponding pixels in successive frames is performed. If the difference is zero or vary small, the corresponding pixel does not belong to any moving object. Otherwise, the pixel is belonging to a moving object. The detection rate is calculated by dividing the true Humans detected by the proposed system by the valid moving Humans appears partially or completely over the video frames. The experiments show that results obtained by the proposed tracking system are better than that obtained using other methods for tracking group of players partially occluded with each others as described in table1. Table1. The performance evaluation for the proposed system with other method. Evaluation Valid moving players (frame 30-80) tracking based on the temporal differencing The proposed system Correct detection Missed detection Detection rate 49.33% 73.66%

5 4 Conclusion Multiple-target tracking is a very active field nowadays due to its wide practical applicability in video processing. While talking about Multiple-target tracking, multi-target occlusion is a common problem that needs to be addressed. In this paper an innovative methodology for multi-players tracking in volleyball play has been proposed based on GMM and Kalman filter. Main contribution to this work is to overcome the problem of partial occlusion. The missed detection by the proposed system referred some complex situations due to full-occluded objects with each other since the proposed system deals only on the partial occlusion. Another reason for the missed detection by the proposed system is the image resolution. In conclusion, this fast and efficient algorithm can detect players in analyzing a Volleyball game. Figure5 demonstrates the ability of the proposed method in tracking and labeling the athletes. In conclusion, this fast and efficient algorithm can detect players in analyzing a Volleyball game. Figure5. The players are successfully tracked by using the proposed system, red color rectangle show the location where the object detected.

6 5 References [1] J. Sullivan and S. Carlsson, Tracking and Labeling of Interacting MultipleTargets, In Proc. European Conf. on Computer Vision (ECCV), [2] T. Mauthner1 and C. Koch, Visual Tracking of Athletes in Beach Volleyball Using a Single Camera, International Journal of Computer Science in Sport,2008. [3] M. Isard and J. MacCormick, BraMBLe: A Bayesian Multiple-Blob Tracker, In Proc. IEEE Int l Conf. on Computer Vision (ICCV), [15] M. Grewal and A. Andrews, Kalman Filtering Theory and Practice using MATLAB, second edition, John Wiley & Sons, Inc., [16] J. Świątek, Parameter Estimation of Systems Described by the Relation with Noisy Observations, Journal of Universal Computer Science, vol. 13, no. 2, , [17] H.salehifar and A.bastanfard, A Complete View Depended Volleyball Video Dataset Under The Uncontrolled Conditions, accepted in [18] H.salehifar and A.bastanfard, A Fast Algorithm for Detecting, Labeling and Tracking Volleyball Players in Sport Videos, rd International Conference on Signal Acquisition and Processing (ICSAP 2011) [4] K. Okuma, A. Taleghani, N. de Freitas, J. J. Little, and D. G. Lowe, A Boosted Particle Filter: Multitarget Detection and Tracking, In Proc. European Conf. on Computer Vision (ECCV), [5] D. Reid, An Algorithm For Tracking Multiple Targets, IEEE Transaction on Automatic Control, vol. 24, no. 6, pp , December [6] S. Oh, S. Russell, and S. Sastry, Markov Chain Monte Carlo Data Association for Multiple-Target Tracking, University of California, Berkeley, Technical Report UCB//ERL M05/19, June [7] Q. Yu, G. Medioni, Map-Enhanced Detection and Tracking from a Moving Platform with Local and Global Data Association, IEEE Workshop on Motion and Video Computing (WMVC'07), [8] A.G.A.PereraC.Srinivas,A.Hoogs,G.Brooksby,Wensheng Hu, Multi-Object Tracking Through Simultaneous Long Occlusions and Split-Merge Conditions, IEEE Int l Conf. on Computer Vision and Pattern Recognition (CVPR), [9] C. Stauffer, Estimating Track Sources and Sinks, In Proc. IEEE Workshop on Event Mining in Video, [10] P. Nillius, J. Sullivan and S. Carlsson, Multi-Target Tracking Linking Identities using Bayesian Network Inference, In Proc. IEEE Int l Conf. on Computer Vision and Pattern Recognition (CVPR), [11] D. Comaniciu, P. Meer, Mean shift: A Robust Approach Toward Feature Space Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (5) (2002) [12] C. Stauffer and W. Grimson, Adaptive Background Mixture Models For Real-Time Tracking, In Proceedings CVPR,pp , [13] Z.Zivkovic and F.van der Heijden, Recursive Unsupervised Learning of Finite Mixture Models. IEEE Trans. on PAMI,vol.26., no.5, [14] B. Shalom and Y. Foreman, Tracking and Data Association. Academic Press Inc