Object Detection and Tracking using Statistical and Stochastic Techniques
- 1. Object Detection and Tracking using Statistical and Stochastic Techniques Dr. S.Vasuhi Department of Electronics Engineering Madras Institute of Technology, Anna University, Chennai, India. vasuhisrinivasan@yahoo.com 2/22/2024 1 ICIC - 2015
- 2. Introduction • The motion tracking is – The process of keeping tracks of moving objects – Problem of estimating the trajectory of an object – Determine its relative movement with respect to other objects. • The bounding boxes are placed around the detected blobs to show the tracking action. • The rectangular boxes surround the target that is being tracked. 2/22/2024 2 ICIC - 2015
- 3. Why Tracking • Visual surveillance • Security monitoring • Anomaly detection / Intruder detection • Motion Capture and Recognition • Traffic flow measurement • Accident detection on highways • Automotive safety and Intelligent control 2/22/2024 3 ICIC - 2015
- 4. Objective • To effectively detect and track the person in the occluded environment. • To make the system adaptive to uncontrolled changes like • Illumination • Colour • Outdoor weather changes • To track multiple people under cross over scenario with the single camera. 2/22/2024 4 2/22/2024 4 ICIC - 2015
- 5. Problems • Abrupt object and Camera Motion • Multi Camera and Multi Object • Computational Expensive • Challenges – Occlusion and – Target miss association – Lighting • Real Time • Multiple people in scene • Camera Modeling – Single fixed camera (Road traffic tracking system) – Multiple fixed cameras (Simple surveillance system) – Single moving camera (Animation) – Multiple moving camera (Robot navigation system) • Complex shape and Complex Motion 2/22/2024 5 ICIC - 2015
- 6. Existing methods - To detect the targets in Video i). Motion Segmentation. • Difference between the current image and the sequence of images. –Easy, Fast, but problem occurs when multiple targets and target stops. ii). Background modeling and Subtraction. Though the BG is static, there may be some BG variations. – Illumination changes – Pose – View point variations – Door opening and closing 2/22/2024 6 ICIC - 2015
- 7. Cont. • In more complex scenarios, the blob based analysis faces several problems, – A single blob may contain multiple humans due to • Physical proximity • The camera viewing angle. – A single object may be fragmented into several blobs due to • Low colour contrast. – A blob may contain pixels corresponding to the shadows or reflections caused by the moving objects as well as noise. 2/22/2024 7 ICIC - 2015
- 8. System Flow Diagram Image Acquisition Background Modeling and Foreground Extraction Feature Extraction Object Modelling Object Tracking 2/22/2024 8 ICIC - 2015
- 9. Background Modeling - Mixture of Gaussians • MoG is a probability density function of pixels X. • The intensity feature of each pixel is modeled by Mixture of M Gaussians. M P(X )= w N(X , μ , ) t t j,t j,t j,t j=1 where, M is the number of distributions. M w = 1 j,t j=1 is the weight parameter of the jth Gaussian component at time t (frame) 2/22/2024 9 N(X , μ , ) t j,t j,t is the Gaussian probability density function of jth component The MoG distinguish the pixel which was associated to the foreground and which was assigned to background. ICIC - 2015
- 10. Background Modeling - Mixture of Gaussians • Every pixel value in a frame is checked against the existing M Gaussian distribution until a match is found • Based on the matching BG is updated. • If a particular pixel matches none of the M distributions, then the least probable distribution is replaced. • For an incoming new frame at times t+1, a match test is performed for each pixel X μ σ <D j j where, D is a constant deviation threshold (~2.5). 2/22/2024 10 N t 1 T -1 - (X -μ) (X -μ) 1 t t 2 X ,μ, = e 1 2 D 2 2π ICIC - 2015
- 11. Cont. After performing the match test for the newly observed pixel, if a match is found with one of the Gaussians, the update is done for mean , variances and weight. μ =(1- )μ + X j,t j,t+1 t+1 2 2 T σ =(1- )σ + (X - μ )(X - μ ) t+1 j,t+1 t+1 j,t+1 j,t+1 j,t w = (1- )w + j,t j,t+1 2/22/2024 11 ICIC - 2015
- 12. Cont. • If the no match is found, then the Gaussian distribution with low probability is replaced with the new distribution. • The weight is updated as follows, w =(1- )wj,t j,t+1 The first B Gaussians distributions which exceed certain threshold Th are chosen as the background distribution. 2/22/2024 12 ICIC - 2015
- 13. Cont. For the foreground detection, the Gaussian distributions and the weights are normalized, to standard deviation ratio is given by w j j BG components which have the low variance and high weight will stay at the top of the distribution. The first B Gaussians distributions which exceed certain threshold T are chosen as the background distribution, b B= arg min( w >T) j,t b j=1 Where, b is the number of background components. The remaining distributions are considered to represent the foreground. 2/22/2024 13 ICIC - 2015
- 14. Feature Extraction • The Principal Component Analysis (PCA) features are obtained from the detected image. • The PCA is an analysis of n-dimensional data and it examines correspondence between different dimensions. • The mean and covariance of the detected image is computed. • From the covariance matrix, the Eigen values and Eigen vectors are calculated. 2/22/2024 ICIC - 2015 14
- 15. Object Modeling-Hidden Markov Model The detected foreground objects are processed with a PCA based feature extraction and 2D array of data is applied to p2DHMM for learning the structure of the human body. Each column of the image will be assigned to one of the super states and blocks (pixels) in the column will be assigned as states. • The parameters associated with HMM are • Number of states • Number of events • Initial-state probabilities • State-transition probabilities and • Discrete output probabilities. The set of states are denoted by St = St ,St ,St ........Stn 1 2 3 2/22/2024 15 ICIC - 2015
- 16. o= o o o .....o 1 2 3 k λ=(A ,B ,π) h h π p(o/ λ) Consider an observation sequence After modeling, HMM can be formalized via the description where, Ah = set of transition probabilities Bh = set of output probabilities = set of initial state probabilities of the observed sequence of the forward-backward procedure can be computed. The probability The probability of the occurrence of a pixel at time t depends only on occurrence of a pixel at time t-1. Cont. 2/22/2024 16 ICIC - 2015
- 17. • KF uses the output of the P2DHMM for tracking the detected person by the estimation of a bounding box trajectory indicating the location of the person within the video sequence. • From the output of the P2DHMM, a bounding box is constructed around the detected person. • The Centre of Gravity (CoG) of the person is calculated from the output of the P2DHMM using the Viterbi alignment. 2/22/2024 17 Tracking using KF in Occluded environment ICIC - 2015
- 18. x = Fx +Gp k k-1 k-1 The state equation The measurement equation Z H x v k k k The state transition matrix 1 0 T 0 0 0 0 1 0 T 0 0 0 0 1 0 0 0 F = 0 0 0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 1 The Measurement matrix 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 H The state vector The measurement matrix provides the information about the center of the detected object and size of the bounding box. p p v v M = T = x x y x y w h t p p T Z = x y w h Tracking using KF 2/22/2024 18 ICIC - 2015
- 19. • The image is acquired from video cameras. • Videos run at 30 fps speed and the frame size is 240*320 pixels. Image Acquisition 2/22/2024 19 ICIC - 2015
- 20. Tracking of single person 2/22/2024 20 Input Video Frame Tracking Output using Proposed System Tracking Output using GMM and KF Tracking Output using FD+KF ICIC - 2015
- 21. Tracking of Multiple Objects 2/22/2024 21 Input Video Frame Tracking Output using Proposed System Tracking Output using GMM and KF Tracking Output using FD+KF ICIC - 2015
- 22. Tracking Error Comparison 2/22/2024 22 ICIC - 2015
- 23. • The major contribution of this paper is – GMM for background modeling – 2DHMM for Object Modeling – KF for tracking and • The proposed methods effectively overcomes – Simultaneous tracking of multiple objects – Accurately track in the presence of clutter and swaying trees. 2/22/2024 23 Summary ICIC - 2015
- 24. References • Wei Qu, Dan Schonfeld, and Magdi Mohamed, “Distributed bayesian multiple-target tracking in crowded environments using multiple collaborative cameras”, Journal on Advances in Signal Processing, Hindawi Publishing Corporation, pp.253-263, 2007. • Ansuman, M., Tusar, K. M., Pankaj, K. S., Banshidhar, M. , “Human recognition system for outdoor videos using Hidden Markovmodel”, International Journal of Electronics and Communications, Elsevier, Vol. 68, No. 3, pp 227–236, 2014. • Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M. and Shafer, S. “Multi camera Multi- person Tracking for Easy Living”, in Proc. on Visual Surveillance, pp. 1- 8, 2000. • T. Darrell, D. Demirdjian, N. Checka, and P. Felzenszwalb, “Plan-view trajectory estimation with dense stereo background models,” in Proc. on Computer Vision, pp.132-140, 2001. • Harville, M. “Stereo person tracking with short and long term plan-view appearance models of shape and colour”, in IEEE Proc. on Advanced Video and Signal based Surveillance, pp.382- 295, 2005. • Zhao, T. Aggarwal, M., Kumar, R and Sawhney, H, “Real-time wide area multi-camera stereo tracking”, in IEEE Proc. on Computer Vision and Pattern Recognition, Vol.1, pp. 976–983, 2005. • Anurag Mittal and Larry S. Davis, “M2tracker: A multiview approach to segmenting and tracking people in a cluttered scene using region-based stereo”, in European Conference on Computer Vision, pp. 18–36, 2002. • T. Zhao, R. Nevatia, and F. Lv, “Segmentation and tracking of multiple humans in complex situations”, in Proc. on Computer Vision and Pattern Recognition, 2001. • Lopez, C., Canton F. and Casas, J.R. “Multiperson 3D tracking with particle filters on voxels,” in IEEE Proc. on Acoustics, Speech and Signal Processing, Vol. 1, pp. 913–916, 2007. 2/22/2024 24 ICIC - 2015
- 25. References • D. Comaniciu, V. Ramesh, and P. Meer, “Kernel-based object tracking”, IEEE Trans. Pattern Analysis Machine Intelligence, pp.564–575, 2003. • J.H.Piater and J.L.Crowley, “Multi-modal tracking of interacting targets using gaussian approximations”, IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, pp. 1 - 8, 2001. • Gong, Y and Letaief, K.B, “Space frequency time coded OFDM for broadband wireless communications” in IEEE Proc. on Global Telecommunications (GLOBECOM), San Antonio, pp.519- 523, 2001. • Yunqiang, C., Yong, R and Hunag,T “JPDAF based hmm for real- time contour tracking”, IEEE in Proc. on Computer Vision and Pattern Recognition, Vol.1, pp.232-245, 2001. • Osawa, T., Xiaojun W., Wakabayashi, K. and Takayuki, Y, “Human tracking by particle filtering using full 3D model of both target and environment,” in 18th International Conference on Pattern Recognition, 2006. • Saad M. K and Mubarak, S, “A multiview approach to tracking people in crowded scenes uses a planar homography constraint”, in European Conference on Computer Vision, pp.782-79, 2006. • Stauffer, Grimson, “Adaptive background mixture models for Real-time tracking”, in Proc. on IEEE CVPR, Vol.2, pp.246-252, 1999. • Z.Zivkovi, Heijden, “Efficient adaptive density estimation per image pixel for the task of background subtraction”, Pattern Recognition Letters, Vol. 27, No.7, pp. 773-780, 2006. • Chen, N. Pears, M. Freeman and J. Austin, “Background subtraction in video using recursive mixture models, spatio-temporal filtering and shadow removal”, in Proc. of 5th ISVC, in Lecture Notes in Computer Science, Vol. 5876, 2009, pp. 1141-1150, 2009. 2/22/2024 25 ICIC - 2015
- 26. References • Lindsay, Smith, “A tutorial on Principal Components Analysis”, February 26, 2002. • S.Kuo, O.E.Agazzi, “Keyword spotting in poorly printed documentsusing pseudo 2-D hidden Markov models”, IEEE Transactions on Pattern Analysis and Machine Intelligence 842–848, 1994. • S.Marchand-Maillet, “1D and Pseudo-2D Hidden Markov Models for Image Analysis- Applications and Results”, Technical Report MMWP-99xx, Department of Multimedia Communications. EURECOM, Institute, Sophia, Antipolis, 552- 559, 1999. • T. Zhao and R. Nevatia, “Tracking Multiple Humans in Complex Situations”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 9, pp.1208 – 1221, 2004. • Dalal, N., Triggs, B, “Histograms of oriented gradients for human detection” IEEE in Proc. on Computer Vision and Pattern Recognition, 2005, pp. 886-893. • Mirabi, M. and Javadi, S. “People Tracking in Outdoor Environment Using Kalman Filter”, in Proc. on Intelligent Systems Modelling and Simulation, 2012, pp. 303 – 307. • Ayache, N and Faugeras, O.D., “Building Registrating and Fusing Noisy Visual maps” Multisensor Integration and Fusion for Intelligent Machines and Systems”, book chapter edited by Ren C. Luo, Michael G. Kay, pp. 495 – 540, 1989. • L. Rabiner, “A tutorial on hidden markov models and selected applications in speech recognition”, IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Vol.7. • Vasuhi, S., V. Vaidehi, “Target Detection and Tracking for Video Surveillance”, WSEAS Transactions on Signal Processing, Vol. 10, pp. 179 - 188, 2014. 2/22/2024 26 ICIC - 2015
- 27. Thank You 2/22/2024 27 ICIC - 2015