Ishan Misra



My research is in the field of Computer Vision and Machine Learning. I focus on visual recognition sub-areas like unsupervised, weakly supervised and semi-supervised learning.
In the past I have worked in the areas of Parallel Computing, GP-GPU programming and Operating Systems.



(New) From Red Wine to Red Tomato: Composition with Context (Oral) [pdf] [bib]
[code & models coming soon]
Ishan Misra, Abhinav Gupta and Martial Hebert.
In CVPR 2017
(New) Shuffle and Learn: Unsupervised Learning using Temporal Order Verification
[pdf] [bib] [code & models on github]
Ishan Misra, C. Lawrence Zitnick and Martial Hebert.
In ECCV 2016
(New) Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels
[pdf] [bib] [code & models on github]
Ishan Misra, C. Lawrence Zitnick, Margaret Mitchell and Ross Girshick.
In CVPR 2016
Popular Media: Microsoft Research Blog
(New) Cross-stitch Networks for Multi-Task Learning (Spotlight) [pdf] [bib] [slides]
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta and Martial Hebert.
In CVPR 2016
(New) Generating Natural Questions About an Image (Oral) [pdf] [bib]
Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin,
C. Lawrence Zitnick, Margaret Mitchell, Xiaodong He, Lucy Vanderwende.
In ACL 2016
(New) Visual Storytelling [pdf] [bib] [dataset]
Ting-Hao Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Jacob Devlin, Aishwarya Agrawal,
Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra,
C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Michel Galley and Margaret Mitchell.
In NAACL 2016
Popular Media: Venture Beat, Microsoft Research Blog MIT Technology Review
Project and Dataset Website
Learning object models from few examples (Oral)
Ishan Misra, Yuxiong Wang and Martial Hebert
In SPIE Unmanned Systems Technology XVIII, 2016
Watch and Learn: Semi-Supervised Learning of Object Detectors from Video [pdf] [bib]
Ishan Misra, Abhinav Shrivastava and Martial Hebert.
In CVPR 2015
Applying artificial vision models to human scene understanding
E. Aminoff, M. Toneva, A. Shrivastava, X. Chen, I. Misra, A. Gupta, M. Tarr.
Journal of Frontiers in Computational Neuroscience, 2015
Data-driven Exemplar Model Selection (Oral) (Best Student Paper) [pdf] [bib]
Ishan Misra, Abhinav Shrivastava and Martial Hebert.
In WACV, 2014

Old Publications


  • Microsoft Research (2015) : I spent a fantastic summer at MSR, Redmond working with Larry Zitnick, Ross Girshick and Meg Mitchell on image classification from noisy labeled data. I also helped around on projects on Visual Story-telling, Question Generation and a navigation system for the visually impaired. Our work made it into the SeeingAI api and was featured in this Microsoft Research Blog, The Next Web, Tech Insider, Ars Technica. Here's a YouTube video for the same.
  • Microsoft Research (2014) : I worked with Dr. Xian-Sheng Hua on Large Scale weakly supervised image classification. Our work resulted in two patent filings.
  • INRIA/Ecole Centrale Paris (2012) : I was an intern under Professor Iasonas Kokkinos working as a part of the INRIA Galen team. My project focused on using Shading cues to estimate surface depth. The dataset comprised of uncalibrated images in unknown general illumination. Our final formulation used combination of intrinsic images, and a new variational formulation of "minimum surface area, fixed volume surfaces" (Zhang et. al. CVPR, 2011) respecting constraints given by surface normals as obtained by Basri et. al. (CVPR, 2001).
  • Course Designer for ITWS-2 (2012) : I was a course designer under Professor Venkatesh Choppella for a freshman course which covered OOP, shell scripting, networking, python and a working knowledge of the GNU/Linux system.
  • Yale University (2011) : I worked as a part of the Dedis group under Professor Bryan Ford. My project involved developing memory management, paging and threading schemes for the Determinator Operating System (a deterministic parallel OS).

Open Source Projects

  • esvmTestCPP : A C++ implementation of the Exemplar SVM testing pipeline. It includes SIMD optimized code for computing Histogram of Oriented Gradients (HOG) features and performing spatial convolution. (Lead Developer)
  • Determinator OS : A deterministic parallel operating system developed at Yale University.


A copy (updated on Oct 2016) is available here. Contact me for a more recent version.


  • Larry Zitnick made (I helped out with the image features) this cool video to visualize 80k COCO images in 5 minutes - YouTube
  • Slides from my presentation on the various Optimization techniques used for Deep Networks [Updated in Nov 2015].

    In my free time (the amount of which keeps decimating each year) I try to juggle between singing, clicking pictures, thinking about life, customizing my Emacs, writing and dabbling in new languages. My taste in music can be generally classified as rock. I am "particular" about fonts which has led to the title of "Font Nerd". My friends have different stories of how this title came to be.


  • Academic Ancestry: (leeched from Scott Satkin's and David Fouhey's page ): Academic Ancestry
  • I have been accepted for Masters in Robotics at CMU, what do I do? Read this.
  • I am coming to Pittsburgh/CMU. Advice? Read this.
  • How do I apply for graduate studies? Read this.


  • Email (the best way to reach me) :
  • Office Address : EDSH 115, The Robotics Institute, Carnegie Mellon University
  • Postal Information (for sending me goodies) : NSH 4000B, The Robotics Institute, Carnegie Mellon University, Pittsburgh PA 15213, United States
Website proudly created in Org Mode. Template adapted from Eric Schulte