Deva Ramanan Professor Robotics Institute Carnegie Mellon University Elliot Dunlap Smith Hall (EDSH), Rm 221 deva@cs.cmu.edu 412-268-6966 Mailing address

Bio
A formal bio is here.

Research
My research focuses on computer vision, often making heavy use of machine learning techniques and often using the human visual system as inspiration. For example, temporal processing is a key component of human perception, but is still relatively unexploited in current visual recognition systems. Machine learning from big (visual) data allows systems to learn subtle statistical regularities of the visual world. But humans have the ability to learn from very few examples. Here's a old talk (from 2015) that discusses some thoughts on these issues.

I currently direct the Center for Autonomous Vehicle Research.

---

Current group members

• Postdoctoral fellows / visitors
  • Arun Vasudevan
  • Aljosa Osep
  • Jonathon Luiten
• PhD
  • Kangle Deng (joint with Jun-Yan Zhu)
  • Gautam Gare (joint with John Galeotti)
  • Tarasha Khurana
  • Zhiqiu Lin
  • Neehar Peri
  • Haithem Turki
  • Gengshan Yang
  • Jason Zhang (joint with Shubham Tulsiani)

---

Past students and postdoctoral fellows

• Postdoctoral fellow
  • Shu Kong, Texas A&M
  • Chen Huang, Apple
  • Olga Russakovsky, Princeton
  • Gregory Rogez, INRIA Rhones-Alpes
• PhD
  • Martin Li Resource-Constrained Learning and Inference for Visual Perception, 2022, Waymo
  • Peiyun Hu Robust and Scalable Perception for Autonomy, 2021, Apple
  • Ravi Mullapudi Dynamic Model Specialization for Efficient Inference, Training, and Supervision, 2021, Snorkel
  • Achal Dave Open-World Object Detection and Tracking, 2021, Amazon
  • Aayush Bansal Unsupervised Learning of the 4D Audio-Visual World, 2020, Facebook Reality Labs
  • Rohit Girdhar Learning to Understand People via Local, Global, and Temporal Reasoning, 2019, Facebook AI
  • Phuc Nguyen Visual Recognition with Limited Annotations, 2018, Google
  • James Supancic Long-Term Tracking by Decision-Making, 2017, Blizzard
  • Mohsen Hejrati Recognizing and Reconstructing Objects in 3D, 2015, Genentech
  • Dennis Park Tracking People and Their Poses, 2014, Toyota Research Institute
  • Xiangxin Zhu Sharing Information Across Object Templates, 2014, Google
  • Yi Yang Articulated Human Pose Estimation with Mixtures of Parts, 2013, DeepMind
  • Chaitanya Desai Relational Models for Human-Object Interactions and their Affordances, 2012, Amazon
  • Hamed Pirsiavash Scalable Action Recognition in Continuous Video Streams, 2012, UC Davis
• Masters/undergraduate
  • Sean Cha Retrieval-based Novel Activity Detection in Untrimmed Videos, 2020, Nvidia
  • Krishna Uppala Exemplar-Free Video Retrieval, 2020, Apple
  • Aaron Huang End-to-End Methods for Autonomous Driving in Simulation, 2020, Zoox
  • Haochen Wang Audiovisual Ontology and Robust Representations via Cross-Modal Fusion, 2020, TTI-Chicago
  • Jessica Lee MetaPix: Few Shot Video Retargeting, 2020, UC Berkeley
  • William Qi Representation Learning for Safe Autonomous Movement , 2020, Argo AI
  • Siva Mynepalli Recognizing Tiny Faces, 2019, Nimble Robotics
  • Ishan Nigam Learning with Auxillary Supervision, 2019, UT Austin
  • Vivek Krishnan Tinkering under the Hood: Interactive Zero-Shot Learning with Net Surgery, 2016, Microsoft
  • Carl Vondrick Crowdsourcing Video Annotation, 2011, Columbia
  • Goutham Patnaik A Joint Model for Tracking and Recognizing Human Actions in Video, 2009, Google

---

Teaching (prior)

• 16-892 Fall 2023, Seminar on Multimodal Foundation Models
• 16-720 Spring 2020, Spring 2021, Spring 2022, Fall 2022, Spring 2023, Graduate Computer Vision (Canvas)
• 16-720 Spring 2017, Graduate Computer Vision
• 16-899 Fall 2016, Seminar on Human Activity Analysis
• 16-720 Spring 2016, Graduate Computer Vision

---

Professional activities (prior)

• Program Chair, CVPR 2018
• Editorial Board, IJCV
• Associate Editor, IEEE TPAMI

---

Funding (prior)

• IARPA Award for "Walk-Through Rendering From Images of Varying Altitudes (2023-2027).

---

Recent publications

For a complete list, please see my Google Scholar page.
For pre-prints, please see my ArXiv page.
For older work, please see here.

• G. Yang, S. Yang, Z. Zhang, Z. Manchester, D. Ramanan. PPR: Physically Plausible Reconstruction from Monocular Videos, ICCV 2023.
• C. Song, G. Yang, K. Deng, J. Zhu, D. Ramanan. Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis, ICCV 2023.
• E, Weng, D. Ramanan, K. Kitani. Joint Metrics Matter: A Better Standard for Trajectory Forecasting, ICCV 2023.
• A. Agarwalla, X. Huang, J, Ziglar, F. Ferroni, L. Leal-Taixe, J, Hays, A. Osep, D. Ramanan. Lidar Panoptic Segmentation and Tracking without Bells and Whistles, IROS 2023.
• Z. Pang, D. Ramanan, M. Li, Y. Wang. Streaming Motion Forecasting for Autonomous Driving, IROS 2023.
• S. Cao, M. Li, J, Hays, D. Ramanan, Y. Wang, L. Gui. Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation, ICML 2023.
• J. Tan, G. Yang, D. Ramanan. Distilling Neural Fields for Real-Time Articulated Reconstruction from Video, CVPR 2023.
• H. Turki, J. Zhang, F. Ferroni, D. Ramanan. SUDS: Scalable Urban Dynamic Scenes, CVPR 2023.
• C. Thavamani, M. Li, F. Ferroni, D. Ramanan. Learning to Zoom and Unzoom, CVPR 2023.
• T. Khurana, P. Hu, D. Held, D. Ramanan. Point Cloud Forecasting as a Proxy for 4D Occupancy Forecasting, CVPR 2023.
• Z. Lin, S. Yu, Z. Kuang, D. Pathak, D. Ramanan. Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models, CVPR 2023.
• Y. Liu, S. Yanm L. Leal-Taixe, J. Hayes, D. Ramanan. Soft Augmentation for Image Classification, CVPR 2023.
• G. Yang, C. Wang, N. Reddy, D. Ramanan. RAC: Reconstructing Animatable Categories from Videos, CVPR 2023.
• X. Wu, K. Lau, F. Ferroni, A. Osep, D. Ramanan. Pix2map: Cross-modal Retrieval for Inferring Street Maps From Images, CVPR 2023.
• K. Deng, G. Yang, D. Ramanan, J. Zhu. 3D-Aware Conditional Image Synthesis, CVPR 2023.
• A. Athar, A. Hermans, J. Luiten, D. Ramanan, B. Liebe. TarViS: A Unified Approach for Target-based Video Segmentation, CVPR 2023.
• A. Athar, J. Luiten, P. Voigtlaendr, T. Khurana, A. Dave, B. Liebe, D. Ramanan. BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video, WACV 2023.
• S. Gupta, J. Kanjani, M. Li, F. Ferroni, J. Hayes, D. Ramanan. Far3Det: Towards Far-Field 3D Detection, WACV 2023.
• N. Peri, A. Dave, D. Ramanan*, S. Kong*. Towards Long Tailed 3D Detection, CORL 2022.
• V. Fomenko, I. Elezi, D. Ramanan, L. Leal-Taixe, A. Osep. Learning to Discover and Detect Objects, NeurIPS 2022.
• Z. Lin, D. Pathak, D. Ramanan*, S. Kong*. Learning With an Evolving Class Ontology, NeurIPS 2022.
• T. Khurana*, P. Hu*, A. Dave, J. Ziglar, D. Held, D. Ramanan. Differentiable Raycasting for Self-supervised Occupancy Forecasting, ECCV 2022.
• Y. Chen*, J. Shi*, Z. Ye*, C. Mertz, D. Ramanan*, S. Kong*. Multimodal Object Detection via Probabilistic Ensembling, ECCV 2022.
• J. Zhang, D. Ramanan, S. Tulsiani. RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild, ECCV 2022.
• N. Peri, J. Luiten, M. li, A. Osep, L. Leal-Taixe, D. Ramanan. Forecasting from LiDAR via Future Object Detection, CVPR 2022.
• K. Deng, A. Liu, J. Zhu, D. Ramanan. Depth-supervised NeRF: Fewer Views and Faster Training for Free, CVPR 2022.
• H. Turki, D. Ramanan, M. Satyanarayanan. Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughs, CVPR 2022.
• G. Yang, M. Vo, N. Neverova, D. Ramanan, A. Vedaldi, H. Joo. BANMo: Building Animatable 3D Neural Models from Many Casual Videos, CVPR 2022.
• Y. Liu, I. Zulfikar, J Luten, A. Dave, D. Ramanan, B. Leibe, A. Osep, L. Leal-Taixe. Opening up Open-World Tracking, CVPR 2022.
• A. Athar, J. Luiten, A. Hermans, D. Ramanan, B. Liebe. HODOR: High-level Object Descriptors for Object Re-segmentation in Video Learned from Static Images, CVPR 2022.
• S. Alshammari, Y. Wang, D. Ramanan, S. Kong. Long-Tailed Recognition via Weight Balancing, CVPR 2022.
• B. Wilson, W. Qi, T. Agarwal, J. Lambert, J. Singh, S. Khandelwal, B. Pan, R. Kumar, A. Hartnett, J. Pontes, D. Ramanan, P. Carr, J. Hayes. Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting, NeurIPS 2021 Datasets and Benchmarks.
• Z. Lin, J. Shi, D. Pathak*, D.Ramanan*. The CLEAR Benchmark: Continual LEArning on Real-World Imagery, NeurIPS 2021 Datasets and Benchmarks.
• J. Zhang, G. Yang, S. Tulsiani*, D. Ramanan*. NeRS: Neural Reflectance Surfaces for Sparse-View 3D Reconstruction in the Wild, NeurIPS 2021.
• G. Yang, D. Sun, V. Jampani, D. Vlasic, F. Cole, C. Liu, D. Ramanan. ViSER: Video Surface Embeddings for Articulated 3D Shape Reconstruction, NeurIPS 2021.
• C. Thavamani*, M. Li*, N. Cebron, D. Ramanan. FOVEA: Foveated Image Magnification for Autonomous Navigation, ICCV 2021.
• T. Khurana, A. Dave, D. Ramanan. Detecting Invisible People, ICCV 2021.
• S. Kong, D. Ramanan. OpenGAN: Open-Set Recognition via Open Data Generation, ICCV 2021. (Marr Prize, Honorable Mention). PAMI 2022 (extended version).
• R. Mullapudi, F. Poms, W. Mark, D. Ramanan, K. Fatahalian. Learning Rare Category Classifiers on a Tight Labeling Budget, ICCV 2021.
• F. Poms*, V. Sarukkai*, R. Mullapudi, N. Sohoni, W. Mark, D. Ramanan, K. Fatahalian. Low-Shot Validation: Active Importance Sampling for Estimating Classifier Performance on Rare Categories, ICCV 2021.
• G. Yang, D. Ramanan. Learning to Segment Rigid Motions from Two Frames, CVPR 2021.
• P. Hu, A. Huang, J. Dolan, D. Held, D. Ramanan. Safe Local Motion Planning with Self-Supervised Freespace Forecasting. CVPR 2021.
• G. Yang, D. Sun, V. Jampani, D. Vlasic, F. Cole, H. Chang, D. Ramanan, W. Freeman, C. Liu. LASR: Learning Articulated Shape Reconstruction from a Monocular Video. CVPR 2021.
• R. Mullapudi, F. Poms, W. Mark, D. Ramanan, K. Fatahalian. Background Splitting: Finding Rare Classes in a Sea of Background, CVPR 2021.
• V. Shankar, A. Dave, R. Roelofs, D. Ramanan, B. Recht, L. Schmidt. Do Image Classifiers Generalize Across Time? CVPR 2021.
• K. Deng, A. Bansal, D. Ramanan. Unsupervised Audiovisual Synthesis via Exemplar Autoencoders, ICLR 2021.