Video Copy Detection Using Inclined Video Tomography and Bag-of-Visual-Words
0 likes411 views
This document proposes a new approach to video copy detection using inclined video tomography and bag-of-visual-words. It extracts inclined tomography images from video clips at different angles based on motion, then characterizes the images using bag-of-visual-words. An experiment applies transformations to video clips and measures the precision of detecting the original clips using histograms of visual word weights between clips. Results show the approach can effectively detect transformed video copies while exploiting both spatial and temporal information.
Video Copy Detection Using Inclined Video Tomography and Bag-of-Visual-Words
- 1. Video Copy Detection Using Inclined Video Tomography and Bag-of-Visual-Words Hyun-seok Min, Se Min Kim, Wesley De Neve, and Yong Man Ro Image and Video Systems Lab Korea Advanced Institute of Science and Technology (KAIST) Daejeon, South Korea e-mail: ymro@ee.kaist.ac.kr website: http://ivylab.kaist.ac.kr I. INTRODUCTION III. VIDEO MATCHING USING HISTOGRAMS - The dissimilarity between two video clips Vq and Vr: - BoVW-based approaches can be effectively used for the detection of both image and video copies N p: the position of the video shot in the 1 - however, these approaches typically ignore the inherent temporal q D(V , V ) = min p N r ∑ q r Dshot (S i , S i + p ), reference video clip at which similarity measurement starts nature of video content i =1 q N r r L - Conventional video tomography extracts slices from a space-time cube q V = Si V = Sl i =1 l =1 that are parallel to the time axis - however, slices that are parallel to the time axis do not take advantage - The dissimilarity between two video shots Sq and Sr is measured by of spatial information making use of the cosine similarity: - This paper proposes to create a content-based video signature by means M of the following two sequential steps ∑ q aj r ×aj M: the number of visual words in the vocabulary 1) extraction of inclined tomography images from the video content q r j =1 - angle of inclination is dependent on the amount of motion Dshot (S , S ) = 1 - , aj: the weight of the jth visual word M M 2) characterization of the inclined tomography images by means of BoVW ∑ )∑ ) ( ( q 2 aj r 2 aj j =1 j =1 II. CREATION OF A VIDEO SIGNATURE BY MEANS OF IV. EXPERIMENTS INCLINED VIDEO TOMOGRAPHY AND BOVW 1. Experimental setup 1. Extraction of inclined tomography images - Use of TRECVID 2009 for creating NDVCs and reference video clips - To extract inclined tomography images from a video clip V, we first - Use of 100 query video clips by applying five transformations to 20 segment V into N space-time cubes such that V = <S1, S2, …, SN> video clips randomly selected from the reference video database - We subsequently segment each space-time cube into several space- - blurring: we blurred frames using a Gaussian kernel with a radius time sub-cubes of 15; - picture-in-picture: we inserted a picture with a size that is 30% of Fv, Fb : number of frames in a space- the size of the main frame; time cube and space-time sub-cube - change in brightness: we increased the brightness with 40%; Wv, Wb : width of a space-time cube - mirroring: we reversed frames from the left to the right; and space-time sub-cube - change in frame rate: we halved the frame rate. Hv, Hb : height of a space-time cube and space-time sub-cube 2. Experimental results 1.1 1.1 1 1 0.9 0.9 Fig. 1. Segmentation of a space-time cube into space-time sub-cubes. 0.8 0.8 0.7 0.7 Precision Recall 0.6 0.6 - The angle of inclination of the tomography image extracted reflects the 0.5 0.5 intensity of motion in the space-time sub-cube under consideration 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 blur pattern change in mirroring frame rate average blur pattern change in mirroring frame rate average insertion brightness change insertion brightness change Transformations Transformations Proposed video signature BoVW using SIFT Video tomography Proposed video signature BoVW using SIFT Video tomography Fig. 4. Comparison of the effectiveness of several video signatures. Fig. 2. Extraction of an inclined tomography image from a space-time sub-cube. L(x, y, t): the luminance value of a β θ= Wb × H b × Fb ∑L( x, y, f + 1) - L( x, y, f ) , pixel (x, y) of a particular frame at time t ( x, y , f ) β: a weight parameter 2. BoVW applied to inclined tomography images - each space-time cube Si can be represented as a vector Ai that summarizes how the space-time sub-cubes are distributed over the vocabulary of visual words used (a) (b) M: the number of visual words vj in the Fig. 5. Example images: (a) example key frame and (b) 16 inclined tomography images A i = ai,1 , ai ,2 ,...,ai ,M , vocabulary used extracted from the key frame shown in (a). ai,j: the weight of the jth visual word V. CONCLUSIONS - This paper introduced a novel video signature that takes advantage of both inclined video tomography and BoVW - The proposed video signature is able to capture both spatial and temporal information - the angle of inclination of the extracted tomography images is Fig. 3. Extraction of a histogram of visual words from an inclined tomography image. dependent on the amount of motion in the local volumes IEEE International Conference on Multimedia and Expo (ICME), July 2012, Melbourne (Australia)