Correlation Filters for Object Alignment

Vishnu Naresh Boddeti, Takeo Kanade and B.V.K. Vijaya Kumar

People

Vishnu Naresh Boddeti (Contact)

Takeo Kanade

Vijayakumar Bhagavatula

Yan Li

Data

Cars in the Wild: Ground truth landmarks with labeled landmark indices for different viewpoints.

Code

Overview

Alignment of 3D objects from 2D images is one of the most important and well studied problems in computer vision. A typical object alignment system consists of a landmark appearance model which is used to obtain an initial shape and a shape model which refines this initial shape by correcting the initialization errors. Since errors in landmark initialization from the appearance model propagate through the shape model, it is critical to have a robust landmark appearance model. While there has been much progress in designing sophisticated and robust shape models, there has been relatively less progress in designing robust landmark detection models. In this paper we present an efficient and robust landmark detection model which is designed specifically to minimize localization errors thereby leading to state-of-the-art object alignment performance. We demonstrate the efficacy and speed of the proposed approach on the challenging task of multi-view car alignment.

Car Alignment

Car alignment example
A comparison between two different appearance models used with the same shape model for the task of car alignment. Top Row: car alignment with a random forest based landmark appearance model. Bottom Row: car alignment with the proposed landmark detector.

Appearance Model

Vector Correlation Filter
Vector Correlation Filter: The outputs of each feature channel are aggregated to compute the final correlation output which would have a sharp peak at the target location.
Vector Correlation Filter
The filters for the feature channels are jointly learned.

Shape Model

Bayesian Partial Shape Inference
We use the Bayesian Partial Shape Inference shape model to regularize the landmarks detected by the appearance model.

Results

Results
Comparison of the sorted RMSE for each pose for different appearance models along with example alignment results with the VCF appearance model corresponding to small, medium and large landmark RMSE.

Computation Complexity (in ms)

Pose RF VCF/SVM RANSAC BPSI Greedy BPSI
2 4000 200 700 90
3 3000 150 600 70
4 4000 200 700 90

References

Correlation Filters for Object Alignment [Supplementary Material]

Vishnu Naresh Boddeti, Takeo Kanade and B.V.K. Vijaya Kumar
CVPR 2013

Robustly Aligning a Shape Model and Its Application to Car Alignment of Unknown Pose

Yan Li, Leon Gu, Takeo Kanade
PAMI 2011

A Robust Shape Model for Multi-View Car Alignment

Yan Li, Leon Gu, and Takeo Kanade
CVPR 2009