PCA-SIFT: A More Distinctive Representation for Local Image Descriptors

by Yan Ke and Rahul Sukthankar


Stable local feature detection and representation is a fundamental component of many image registration and object recognition algorithms.  Mikolajczyk and Schmid recently evaluated a variety of approaches and identified the SIFT algorithm as being the most resistant to common image deformations.  This paper examines (and improves upon) the local image descriptor used by SIFT.  Like SIFT, our descriptors encode the salient aspects of the image gradient in the feature point's neighborhood; however, instead of using SIFT's smoothed weighted histograms, we apply Principal Components Analysis (PCA) to the normalized gradient patch.  Our experiments demonstrate that the PCA-based local descriptors are more distinctive, more robust to image deformations, and more compact than the standard SIFT representation.  We also present results showing that using these descriptors in an image retrieval application results in increased accuracy and faster matching.

Y. Ke and R. Sukthankar, Computer Vision and Pattern Recognition, 2004. [PDF 670KB]

PCA-SIFT (calculates representation only) source code: pcasift-0.91nd.tar.gz (700KB). You'll need the netpbm development libraries to compile this code. If you want to train PCA on your own patches, here's the Matlab code for it. training-matlab.tgz

Keypoint detection as Linux binary and modified matching program as source code from David Lowe.  Works on PCA-SIFT keys and Lowe's SIFT keys.  Includes example images for matching.  mod_lowe_demoV2.tar.gz (430KB)

Source code for PCA-SIFT with integrated Difference of Gaussian (DoG) interest point detector can be obtained for research purposes. However, before you ask me for it, please note that the PCA-SIFT code supplied above is sufficient for most cases. If you are simply looking for the DoG interest point detector, there are many other sources online.

Dataset used in the experiments: testimages.tgz (550KB) for recall-precision curves and objects.tgz (9 MB) for image retrieval.

Parts based image retrieval

David Lowe's Keypoints

Page maintained by Yan Ke < y k e @ c m u . e d u >

April 2004