Abhinav Gupta
Assistant Professor
The Robotics Institute
School of Computer Science
Carnegie Mellon University (CMU)
Affiliate Appt: Machine Learning Department
Office:   EDSH 213
Phone:   412-268-2067 (email is the best option to reach me)
Email :   abhinavg [at] cs [dot] cmu [dot] edu



ME
News


About Me
I am an assistant professor at Carnegie Mellon University. Prior to this, I was a post-doctoral fellow here working with Alyosha Efros and Martial Hebert. Before coming to Pittsburgh, I was working with Larry Davis at UMD and Jianbo Shi at UPenn. My PhD thesis was on "Beyond Nouns and Verbs".


Research Interests


My research interest include:

  • How do we represent the visual world ? My research focuses on developing representation and reasoning approaches for deeper understanding of the scene. I am interested in formulating the scene understanding problem in terms of the underlying 3D scene and develop reasoning approaches based on physical, functional and causal relationships between the different elements in the scene. The key idea is to have a qualitative representation and yet have a meaningful grounding in the physical scene.
  • What is the link between Language and Vision ? What role does language play in visual learning? I am interested in exploring how declarative information and other linguistic information can be harnessed to efficiently learn how the world works (structural information). I am also interesting in exploring how we can obtain such linguistic information.
  • How are actions and objects related to each other? I have been focusing on studying how do humans interact with their environment and how does their perception of visual world depends on these interactions and their abilities. Building upon Gibson's idea of affordances, we have recently proposed the concept of human centric scene understanding.


Postdocs and Students
Graduated PhD Students
Former MS Students and Collaborators


Courses
Press Coverage
  • Self-Supervised Grasping and Curious Robot
  • Never Ending Image Learner (NEIL)
  • .........

  • What makes Paris look like Paris?
  • Data Driven Visual Similarity
  •            

  • Blocks World Revisited
  • Storylines
Selected Projects
(Please see Publications for a complete list)
(Please see Downloads for code and datasets)

curious


Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park and Abhinav Gupta. The Curious Robot: Learning Visual Representations via Physical Interactions. ECCV 2016. (Spotlight)

pdf project page



voxels


R. Girdhar, D. Fouhey, M. Rodriguez, A. Gupta. Learning a Predictable and Generative Vector Representation for Objects. ECCV 2016. (Spotlight)

pdf project page



SSGAN


Xiaolong Wang and Abhinav Gupta. Generative Image Modeling using Style and Structure Adversarial Networks. ECCV 2016.

pdf project page



priming


Abhinav Shrivastava, Abhinav Gupta. Contextual Priming and Feedback for Faster R-CNN. ECCV 2016.

pdf project page



HiH


Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ivan Laptev, Ali Farhadi, Abhinav Gupta. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. ECCV 2016.

pdf project page



priming


J Walker, C Doersch, A Gupta, M Hebert. An Uncertain Future: Forecasting from Static Images Using Variational Autoencoders. ECCV 2016.

pdf project page



priming


Roozbeh Mottaghi, Mohammad Rastegari, Abhinav Gupta, Ali Farhadi. "What happens if..." Learning to Predict the Effect of Forces in Images. ECCV 2016.

pdf project page



priming


Gunnar A. Sigurdsson, Xinlei Chen, Abhinav Gupta, "Learning Visual Storylines with Skipping Recurrent Neural Networks", ECCV 2016.

pdf project page



baxter_grip


Lerrel Pinto and Abhinav Gupta. Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours. ICRA 2016. (Best Student Paper Award)

pdf project page



voxels


D. Fouhey, A. Gupta, A. Zisserman. 3D Shape Attributes. CVPR 2016. (Oral)

pdf project page



ohem


Abhinav Shrivastava, Abhinav Gupta, Ross Girshick. Training Region-based Object Detectors with Online Hard Example Mining. CVPR 2016 (Oral)

pdf project page



actions


Xiaolong Wang, Ali Farhadi, and Abhinav Gupta. Actions ~ Transformations. CVPR 2016.

pdf project page



actions


Aayush Bansal, Bryan Russell, Abhinav Gupta. Marr Revisited: 2D-3D Alignment via Surface Normal Prediction. CVPR 2016.

pdf project page



ohem


Ishan Misra*, Abhinav Shrivastava*, Abhinav Gupta, Martial Hebert. Cross-stitch Networks for Multi-task Learning. CVPR 2016

pdf project page



esvm


Carl Doersch, Abhinav Gupta, Alexei Efros. Unsupervised Visual Representation Learning by Context Prediction. ICCV 2015. (Oral)

pdf project page



esvm


Xiaolong Wang, Abhinav Gupta. Unsupervised Learning of Visual Representations using Videos. ICCV 2015. (Models and Code available for download!)

pdf project page



esvm


Xinlei Chen, Abhinav Gupta. Webly Supervised Learning of Convolutional Networks. ICCV 2015. (Oral)

pdf project page



esvm


David Fouhey, Abhinav Gupta, Martial Hebert. Single Image 3D without a Single 3D Image. ICCV 2015.

pdf project page



esvm


Jacob Walker, Abhinav Gupta, Martial Hebert. Dense Optical Flow Prediction from a Static Image. ICCV 2015.

pdf project page



esvm


Xiaolong Wang, David Fouhey, Abhinav Gupta. Designing Deep Networks for Surface Normal Estimation. CVPR 2015. (Models and Code available on request!)

pdf project page



esvm


Xinlei Chen, Alan Ritter, Abhinav Gupta, Tom Mitchell. Sense Discovery via Co-Clustering on Images and Text. CVPR 2015.

pdf project page



esvm


David F. Fouhey, Abhinav Gupta, Martial Hebert. Unfolding an Indoor Origami World. ECCV 2014. (Oral)

pdf project page



esvm


Carl Doersch, Abhinav Gupta, and Alexei A. Efros. Context as Supervisory Signal: Discovering Objects with Predictable Context. ECCV 2014.

pdf project page



esvm


Jacob Walker, Abhinav Gupta, and Martial Hebert.Patch to the Future: Unsupervised Visual Prediction. CVPR 2014. (Oral)

pdf project page



esvm


Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta. Enriching Visual Knowledge Bases via Object Discovery and Segmentation. In CVPR 2014.

pdf project page



esvm


Xinlei Chen, Abhinav Shrivastava and Abhinav Gupta. NEIL: Extracting Visual Knowledge from Web Data. In ICCV 2013. (Oral)

pdf project page



esvm


David Fouhey, Abhinav Gupta and Martial Hebert. Data-Driven 3D Primitives for Single Image Understanding. In ICCV 2013

pdf project page



esvm


Abhinav Shrivastava and Abhinav Gupta. Building Part-based Object Detectors via 3D Geometry. In ICCV 2013

pdf project page



esvm


Carl Doersch, Abhinav Gupta and Alexei Efros. Mid-level Visual Element Discovery as Discriminative Mode Seeking. In NIPS 2013

pdf project page



esvm


Arpit Jain, Abhinav Gupta, Mikel Rodriguez and Larry S. Davis. Representing Videos using Mid-level Discriminative Patches. In CVPR 2013.

pdf project page



esvm


David Fouhey, Vincent Delaitre, Abhinav Gupta, Alexei A. Efros, Ivan Laptev, Josef Sivic. People Watching: Human Actions as a Cue for Single View Geometry. In ECCV 2012.(Oral)

pdf project page



ssl


Abhinav Shrivastava, Saurabh Singh, Abhinav Gupta.Constrained Semi-Supervised Learning Using Attributes and Comparative Attributes. In ECCV 2012.(Oral)

pdf project page



esvm


Saurabh Singh, Abhinav Gupta, Alexei A. Efros. Unsupervised Discovery of Mid-Level Discriminative Patches. In ECCV 2012.

pdf project page



esvm


Vincent Delaitre, David Fouhey, Ivan Laptev, Josef Sivic Abhinav Gupta, Alexei A. Efros. Scene Semantics from Long-term Observation of People. In ECCV 2012.

pdf project page



esvm


Carl Doersch, Saurabh Singh, Abhinav Gupta, Josef Sivic, Alexei A. Efros. What makes Paris look like Paris? In SIGGRAPH 2012. (Oral)

pdf project page



esvm Abhinav Shrivastava, Tomasz Malisiewicz, Abhinav Gupta, Alexei A. Efros, Data-driven Visual Similarity for Cross-domain Image Matching, In SIGGRAPH Asia 2011

pdf project page



esvm
Tomasz Malisiewicz, Abhinav Gupta, Alexei A. Efros, Ensemble of Exemplar-SVMs for Object Detection and Beyond, In ICCV 2011.
pdf project page

Source code (beta version) available


affordances
Abhinav Gupta, Scott Satkin, Alexei A. Efros and M. Hebert, From 3D Scene Geometry to Human Workspace. In CVPR 2011. (Oral)
pdf
   project page  ppt

Source code partly available


nips
David C. Lee, Abhinav Gupta, Martial Hebert, and Takeo Kanade, Estimating Spatial Layout of Rooms using Volumetric Reasoning about Objects and Surfaces, In NIPS 2010.
pdf

Source code available


blocksworld

Abhinav Gupta, Alexei A. Efros and M. Hebert, Blocks World Revisited: Image Understanding Using Qualitative Geometry and Mechanics. In ECCV 2010. (Oral) (Best Paper Runner Up Award)
pdf
   project page  ppt
Featured in Science Daily and ZD Net.

Source code available


al
Behjat Siddiquie and Abhinav Gupta, Beyond Active Noun Tagging: Modeling Contextual Interactions for Multi-Class Active Learning, In CVPR 2010. (Oral)
pdf    ppt


movie
Abhinav Gupta, Praveen Srinivasan, Jianbo Shi and Larry S. Davis, Understanding Videos, Constructing Plots: Learning a Visually Grounded Storyline Model from Annotated Videos, In CVPR 2009. (Oral)
pdf   ppt

Featured in an IEEE Spectrum and Discovery article. Also covered in Ethiopian Review and SiliconIndia.


beyondnouns
Abhinav Gupta and Larry S. Davis, Beyond Nouns: Exploiting prepositions and comparative adjectives for learning visual classifiers, In ECCV 2008 (Oral)
pdf ppt


dpdp`
Abhinav Gupta, Aniruddha Kembhavi and Larry S. Davis, Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition, In Trans. on PAMI (Special Issue on Probabilistic Graphical Models).
pdf
Downloads Available: Dataset

Abhinav Gupta and Larry S. Davis, Objects in Action: An Approach for Combining Action Understanding and Object Perception, In CVPR 2007
pdf
Downloads Available: Dataset


nips_shape Abhinav Gupta, Jianbo Shi and Larry S. Davis, A “Shape Aware” Model for semi-supervised Learning of Objects and its Context, In NIPS 2008 (Spotlight Poster)
pdf


Funding Sources
  • Office of Naval Research: High Level MURI, ONR Applied Research
  • NSF: IIS 1320083
  • IARPA: ALADDIN Video
  • Google: focused award (2011), faculty research awards (2012, 2014)
  • Bosch Research & Technology Center: Bosch Young Faculty Fellowship 2014, Gift Award 2013
  • YAHOO!: Cluster for NEIL, InMind
  • HighMark Grant
  • MITRE
  • DARPA Memex Program


Other Links