Dynamic Seethroughs:

Synthesizing Hidden Views of Moving Objects

Peter Barnum, Yaser Sheikh, Ankur Datta, and Takeo Kanade

A simulated time lapse of a seethrough of a person walking A simulated time lapse of a seethrough of  person making a snow angle

This project involves creating an illusion of seeing moving objects through occluding surfaces in a video. This illusion is achieved by transferring information from a camera viewing the occluded area. In typical view interpolation approaches for 3D scenes, some form of correspondence across views is required. For occluded areas, establishing direct correspondence is impossible as information is missing in one of the views. Instead, we use a 2D projective invariant to capture information about occluded objects (which may be moving). Since invariants are quantities that do not change across views, a visually compelling rendering of hidden areas is achieved without the need for explicit correspondences. A piece-wise planar model of the scene allows the entire rendering process to take place without any 3D reconstruction, while still producing visual parallax. Because of the simplicity and robustness of the 2D invariant, we are able to transfer both static backgrounds and moving objects in real time. A complete working system has been implemented that runs live at 5Hz. Applications for this technology include the ability to look through corners at tight intersections for automobile safety, concurrent visualization of a surveillance camera network, and monitoring systems for patients/elderly/children.


"Dynamic Seethroughs: Synthesizing Hidden Views of Moving Objects",
P.C. Barnum, Y. Sheikh, A. Datta, and T. Kanade
International Symposium on Mixed and Augmented Reality, October 2009
[PDF] [Slides + videos] [Slides only]

Media Coverage

"Augmented reality system lets you see through walls ",
New Scientist, October 2009
[Link to Article] [Link to YouTube video]



A short summary video, including a narration
(56mb, Divx compressed)


Japanese television commecial

(3mb, Windows Media File)

English translation:
Children: Let's go to see the future of cars!
Lady: I'll let you drive a car of the future.
Children: Can we see hidden objects?
Children: Yes, we can see the guy behind the wall.
Children: The car too. The motorcycle too.
Lady: We are studying how to generate seethrough images by transferring observed images.
Children: With this technology, no traffic accidents will happen.
Children: But due to this technology, we can't play hide-and-seek anymore.
Lady: (laughs) We also have other interesting ones.