Proxemics Recognition in Personal Photos

Yi Yang, Simon Baker, Anitha Kannan and Deva Ramanan

Abstract

Proxemics is the study of how people interact. We present a computational formulation of visual proxemics by attempting to label each pair of people in an image with a subset of physically based touch codes. A baseline approach would be to first perform pose estimation and then detect the touch codes based on the estimated joint locations. We found that this sequential approach does not perform well because pose estimation step is too unreliable for images of interacting people, due to difficulties with occlusion and limb ambiguities. Instead, we propose a direct approach where we build an articulated model tuned for each touch code. Each such model contains two people, connected in an appropriate manner for the touch code in question. We fit this model to the image and then base classification on the fitting error. Experiments show that this approach significantly outperforms the sequential baseline as well as other related approches.

Download Data & Codes

The latest copy of our code and dataset: DOWNLOAD

Please read the README file for proper installation.

Examples

Publications

If you use our software or our data sets, please cite:

Yi Yang, Simon Baker, Anitha Kannan, Deva Ramanan. "Recognizing Proxemics in Personal Photos". IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012. [Paper][Slides][Poster][Talk]

@conference{yang2012recognizing,
  title={Recognizing proxemics in personal photos},
  author={Yang, Y. and Baker, S. and Kannan, A. and Ramanan, D.},
  booktitle={2012 IEEE Conference on Computer Vision and Pattern Recognition},
  year={2012},
  organization={IEEE}
}

Acknowledgements & Funding

The research described was conducted while Yi Yang was an intern at Microsoft Research. Deva Ramanan was supported by NSF Grant 0954083, ONR-MURI Grant N00014-10-1-0933, and the Intel Science and Technology Center for Visual Computing.

License

Copyright © 2012 Yi Yang, Simon Baker, Anitha Kannan and Deva Ramanan

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONtrACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.