Rectification Without Lines
Carl Doersch and Alexei Efros
Abstract: When inferring scene geometry from a
single image, one of the first steps is to estimate the vanishing points
of each surface in the image in order to recover its orientation.
If the surface contains detectable parallel lines
then this is easy. But what if there aren't any? Another option is to
make use of repeated patterns. Many surfaces, like the building above,
contain small elements, like the stones, that are approximately repeated
across the surface. If the surface is rectified (its vanishing line is
at infinity), then these elements will all be approximately the same size.
This intuition leads to a simple way to 'score' a homography: we transform
the image by that homography, extract
random patches, and correlate them with the rest of the transformed image. If the
homography comes close to rectifying the image, then there should be a
large number of strong peaks in the correlation map. Given the scoring
function, we can simply search over the space of homographies. We found
that gradient ascent works reasonably well on this scoring metric, even
though there are no
guarantees of convexity.
The algorithm qualitatively worked well on a handful of test
surface patches (for example, the one above, where the left is the
original surface and the right is the output of our algorithm). However,
we never found a good way to evaluate our algorithm in general,
since the sorts of
surfaces where this algorithm would be useful are somewhat rare.
pdf: under construction
