All Pose Face Alignment Robust to Occlusion
Vishnu Naresh Boddeti and Takeo Kanade
Within a facial image, the task of accurately localizing the set of landmark points that define the shape of the face is called face alignment. This paper presents an approach to robustly align a facial shape to a face image of any unknown pose even in the presence of partial occlusions. Most face alignment approaches are vulnerable to partial occlusions of the face, since it is assumed, either explicitly or implicitly, that faces are not significantly occluded. To deal with the multi-modal nature of unknown facial pose and unknown occlusion, our approach explicitly searches over a discrete representation of pose and binary occlusion labels. Towards this end, to handle unknown pose, we first adopt a discrete shape representation as well as a discrete representation for the multi-modal nature of landmark detector responses arising from facial appearance variation with pose. Secondly, to handle partial occlusions, we propose an efficient randomized hypothesize-and-test search over sampled subsets of landmark detector response modes. This method of explicit search over discretized pose and randomized subset based shape fitting can effectively handle unknown pose even in the presence of partial occlusions and recover the correct facial shape. We evaluate the proposed method on a large number of face images, occluded and unoccluded, spanning a wide range of facial poses. Our results demonstrate that facial alignment is accurate and stable over a wide spectrum of occlusion and pose variations demonstrating excellent performance on many real-world face datasets.