ABSTRACT VIDEO INDEXING AND SUMMARIZATION USING MOTION ACTIVITY. by Kadir Askin Peker

Size: px

Start display at page:

Download "ABSTRACT VIDEO INDEXING AND SUMMARIZATION USING MOTION ACTIVITY. by Kadir Askin Peker"

Meagan Walton
8 years ago
Views:

1 ABSTRACT VIDEO INDEXING AND SUMMARIZATION USING MOTION ACTIVITY by Kadir Askin Peker In this dissertation, video-indexing techniques using low-level motion activity characteristics and their application to video summarization are presented. The MPEG-7 motion activity feature is defined as the subjective level of activity or motion in a video segment. First, a novel psychophysical and analytical framework for automatic measurement of motion activity in compliance with its subjective perception is developed. A psychophysically sound subjective ground truth for motion activity and a test-set of video clips is constructed for this purpose. A number of low-level, compressed domain motion vector based, known and novel descriptors are then described. It is shown that these descriptors successfully estimate the subjective level of motion activity of video clips. Furthermore, the individual strengths and limitations of the proposed descriptors are determined using a novel pairwise comparison framework. It is verified that the intensity of motion activity descriptor of the MPEG-7 standard is one of the best performers, while a novel descriptor proposed in this dissertation performs comparably or better. A new descriptor for the spatial distribution of motion activity in a scene is proposed. This descriptor is supplementary to the intensity of motion activity descriptor. The new descriptor is shown to have comparable query retrieval performance to the current spatial distribution of motion activity descriptor of the MPEG-7 standard. The insights obtained from the motion activity investigation are applied to video summarization. A novel approach to summarizing and skimming through video using motion activity is presented. The approach is based on allocation of playback

First, a novel psychophysical and analytical framework for automatic measurement of motion activity in compliance with its subjective perception is developed.

2 time to video segments proportional to the motion activity of the segments. Low activity segments are played faster than high activity segments in such a way that a constant level of activity is maintained throughout the video. Since motion activity is a low-complexity descriptor, the proposed summarization techniques are extremely fast. The summarization techniques are successfully used on surveillance video. The proposed techniques can also be used as a preprocessing stage for more complex summarization and content analysis techniques, thus providing significant cost gains.

video. Since motion activity is a low-complexity descriptor, the proposed summarization techniques are extremely fast.

3 VIDEO INDEXING AND SUMMARIZATION USING MOTION ACTIVITY by Kadir Askin Peker A Dissertation Submitted to the Faculty of New Jersey Institute of Technology in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Electrical Engineering Department of Electrical and Computer Engineering January 2001

Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

5 APPROVAL PAGE VIDEO INDEXING AND SUMMARIZATION USING MOTION ACTIVITY Kadir Askin Peker Dr. Ali N. Akansu, Dissertation Advisor Professor of Electrical and Computer Engineering, NJIT Dr. Ajay Divakaran, Dissertation Co-Advisor Principal Technical Staff, Mitsubishi Electric Research Labs, Murray Hill, NJ Dr. Nirwan Ansari, Committee Member Professor of Electrical and Computer Engineering, NJIT Dr. Richard Haddad, Committee Member Professor of Electrical and Computer Engineering, NJIT Dr. Yun-Qing Shi, Committee Member Associate Professor of Electrical and Computer Engineering, NJIT

Ajay Divakaran, Dissertation Co-Advisor Principal Technical Staff, Mitsubishi Electric Research Labs, Murray Hill, NJ Dr.

6 BIOGRAPHICAL SKETCH Author: Degree: Kadir Askin Peker Doctor of Philosophy in Electrical Engineering : January 2001 of Birth: April 2, 1972 Place of Birth: Burdur, Turkey Undergraduate and Graduate Education: Doctor of Philosophy in Electrical Engineering, New Jersey Institute of Technology, Newark, NJ, 2001 Master of Science in Electrical Engineering, Rutgers The State University of New Jersey, New Brunswick, NJ, 1996 Bachelor of Science in Electrical and Electronics Engineering, Bilkent University, Ankara, Turkey, 1993 Major: Electrical Engineering Presentations and Publications: Kadir A. Peker, A. Aydin Alatan, and Ali N. Akansu, "Low-level motion activity features for semantic characterization of video," Proc. of IEEE International Conference on Multimedia and Expo Ajay Divakaran, Kadir A. Peker, and Huifang Sun, "A region-based descriptor for spatial distribution of motion activity for compressed video," Proc. of ICIP 2000, pp Kadir A. Peker and Ajay Divakaran, "Automatic measurement of intensity of motion activity of video segments," to be presented at SPIE Storage and Retrieval for Media Databases Ajay Divakaran, Kadir A. Peker, and Huifang Sun, "Video summarization using motion descriptors," to be presented at SPIE Storage and Retrieval for Media Databases Ajay Divakaran, Kadir A. Peker, and Huifang Sun, "Video indexing using descriptors of spatial distribution of motion activity," submitted to IEEE Transactions on Circuits and Systems for Video Technology. Kadir A. Peker, Ajay Divakaran, and Thomas V. Papathomas, "Automatic measurement of intensity of motion activity of video segments," to be submitted to IEEE Transactions on Multimedia.

Jersey, New Brunswick, NJ, 1996 Bachelor of Science in Electrical and Electronics Engineering, Bilkent University, Ankara, Turkey, 1993 Major: Electrical Engineering Presentations and Publications:

7 Dedicated to the one who gives me comprehension, and to the teacher, then to my parents and family, to Hacer and Dilara, and to all the other precious people whose company I treasure

8 ACKNOWLEDGMENT I would like to express my sincere appreciation to Professor Ali N. Akansu for his support since the beginning, and for his patience. I would like to thank Dr. A. Aydin Alatan, who has a significant part in the making of this dissertation with his guidance and support. I am especially grateful to Dr. Ajay Divakaran for his supervision, guidance, friendship, and moral support throughout this research. I regard myself fortunate to have him as my co-supervisor. I would like to thank Prof. Thomas Papathomas as well, for his guidance on the subjective experiment and its analysis. Special thanks go to Professors Nirwan Ansari, Richard Haddad, and Yun Q. Shi, for serving as members of the committee. I would like to thank my friends and colleagues at New Jersey Center for Multimedia Research (NJCMR) and Mitsubishi Electric Research Labs (MERL), especially those who took part in the subjective experiment. Special thanks go to Burak, Sebnem, Zafer, Taha, Anil, Ramkumar, and Xaidong from NJCMR, and to Dr. Sun, Anthony, and Fatih from MERL. Finally, I would like to thank my parents who made contributions with their hearts, and to my wife who went through this all along with me.

I would like to thank Prof. Thomas Papathomas as well, for his guidance on the subjective experiment and its analysis. Special thanks go to Professors Nirwan Ansari, Richard Haddad, and Yun Q.

[ANNUAL REPORT] Middle East Technical University. İpek Düzgören

[ANNUAL REPORT] Middle East Technical University. İpek Düzgören 2013 Middle East Technical University İpek Düzgören [ANNUAL REPORT] Names, email addresses, and Member numbers of elected Officers, List of current Student Chapter Members, Details of Chapter activities