PCA-SIFT: A More Distinctive Representation for Local Image Descriptors
- 1. PCA-SIFT: A More Distinctive Representation for Local Image Descriptors by Yan Ke and Rahul Sukthankar Presentation by Guy Tannenbaum
- 2. Introduction Local descriptors – computed efficiently, resistant to partial occlusion and changes in viewpoint. 2 independent aspects: Finding keypoints (in position and scale) Building a descriptor PCA-SIFT is a modification of SIFT, which changes how the keypoint descriptors are constructed.
- 3. Quick review of SIFT 2 parts of the algorithm: Finding keypoints Scale-space peak selection Keypoint localization Keypoint descriptor image gradients in local neighborhood of keypoint 4x4 array of histograms, each with 8 orientation bins (128 element vector)
- 4. PCA-SIFT: Basic idea Use PCA to efficiently represent the gradient patch around the keypoint.
- 5. PCA-SIFT: computing a projection matrix Select a representative set of pictures and detect all keypoints in these pictures For each keypoint: Extract an image patch around it with size 41 x 41 pixels Calculate horizontal and vertical gradients, resulting in a vector of size 39 x 39 x 2 = 3042 Put all these vectors into a k x 3042 matrix A where k is the number of keypoints detected Calculate the covariance matrix of A
- 6. PCA-SIFT: computing a projection matrix Compute the eigenvectors and eigenvalues of covA Select the first n eigenvectors; the projection matrix is a n x 3042 matrix composed of these eigenvectors n can either be a fixed value determined empirically or set dynamically based on the eigenvalues The projection matrix is only computed once and saved
- 7. Dimension reduction through PCA The image patches do not span the entire space of pixel values, and also not the smaller space of patches from natural images. They consist of the highly restricted set of patches that passed the first 3 stages of SIFT.
- 8. Constructing PCA-SIFT descriptor Input: location of keypoint, scale, orientation. Extract a 41 x 41 patch around the keypoint at the given scale, rotated to its orientation Calculate 39 x 39 horizontal and vertical gradients, resulting in a vector of size 3042 Multiply this vector using the precomputed n x 3042 projection matrix This results in a PCA-SIFT descriptor of size n
- 9. Results - Methodology Experimental Setup: Datasets contain images of some object, under different (synthetic or real) viewing conditions. Keypoints of all images in data set are found. All pairs of keypoint descriptors from different images are examined, those with Euclidean distance smaller than a threshold are considered a match.
- 10. Results - Methodology Evaluation Metric: Recall vs. 1-percision graph Recall = #correct-positives / #total-positives 1-precision = #false-positives / #total-matches ROC graphs plot positive detection rate vs. false detection rate Positive detection rate = #correct-positives / #total-positives False detection rate = #false-positives / #total-negatives Recall vs. 1-percision graphs are better suited than ROC graphs to evaluate performance on detection tasks because the number of negatives in the data set is not well defined.
- 11. Results – Controlled transformation
- 12. Results 2 – Grafitti dataset Low recall rate at high precision is acceptable for real-world applications. Recall of 5% at 1-percision of 20% - about 1000 keypoints in image, of which 50 are reliable matches. Sufficient for applications like image retrival.
- 13. Results3 – running time
- 14. Eigenspace construction PCA-SIFT’s performance is not sensitive to the images used in the creation of the eigenspace.
- 15. Effect of PCA dimension Optimal performance at n=36 Hypothesis – First several components of the PCA subspace are sufficient for encoding variations caused by keypoint identity, while the later components represent details that are not useful, of potentially detrimental, such as distortion from projective wrap.
- 16. Summary PCA-SIFT is an alternate representation for local image descriptors of the SIFT algorithm. More distinctive, and more compact leading to improvements in accuracy and running time.
- 17. Credits Based on the article PCA-SIFT: A More Distinctive Representation for Local Image Descriptors by Yan Ke and Rahul Sukthankar. www.danet.dk/sensor_fusion/SIFT features.ppt http://campar.in.tum.de/twiki/pub/Chair/TeachingOberSeminar/Slides_AndreasHaug.pdf