Junwei Liang 梁俊卫

			Junwei Liang
			(Ph.D. Candidate)      
			梁俊卫
		
			Language Technologies Institute
		
			School of Computer Science
		
			Carnegie Mellon University
		
			Office: Gates and Hillman Centers 5513
		
			Email: junweil at cs dot cmu dot edu (junweil@cs.cmu.edu)

Publications

Research

Honors

Google Scholar

Code

Introduction

Greetings! I'm a 3rd year Ph.D. student under the supervision of Prof. Alexander Hauptmann at Carnegie Mellon University.
My research interests are large-scale computer vision and natural language processing.
I am also doing research/building tools for event reconstruction from unorganized social media videos for public safety projects.

News

[09/2019] Our Shooter Localization System received Best Demo award at CBMI2019. [Project Site]
Press Coverage: , , , , ,
[06/2019] Presented Future Prediction paper at CVPR 2019. [Tweets]
[04/2019] Our CMU team's (INF & MUDSML) system achieved the best performance on the activity detection challenge (Cached) in surveillance videos hosted by NIST & IARPA. We have released our code and model for Object Detection & Tracking here.
[02/2019] Future activity & path prediction paper reported by the media with 30k+ views in a week. To be presented at CVPR 2019.
[12/2018] MemexQA paper accepted by TPAMI (IEEE Transactions on Pattern Analysis and Machine Intelligence).
[06/2018] Presented MemexQA paper at CVPR 2018. [Presentation]
[11/2016] Best performance in the NIST TRECVID 2016 Ad-hoc Video Search Challenge (no annotation track).

Research Experience

Research Assistant at Carnegie Mellon University
2015 - present

Worked on Large-scale Video Analysis and Retrieval. Studied unsupervised learning of video concept detectors from the Internet. Also participated in the development of event reconstruction tool. The project is for Synchronization and localization of noisy user-generated videos to reconstruct the event scene and timeline from unorganized social media videos, affiliates with CMU Center for Human Rights Science. I'm also the major contributor to the government-funded projects: PSCR by NIST (2017-2020), DIVA by IARPA (2017-2021) and ALADDIN by IARPA (2017). Advised by Prof. Alexander Hauptmann. [PSCR 2018 presentation, 2019]
Research Intern at Google AI
May 2019 - Aug 2019

Working on changing the world.
Student Researcher at Google Cloud AI
May 2018 - Dec 2018

Worked on activity recognition and prediction in multi-perspective streaming videos. Studied principal computer vision and high-level semantic reasoning models for interperson and person-object interaction to help AI better understand human activities. Advised by Lu Jiang.
Research Assistant at Renmin University of China
2013 - 2015

Studied semantic concept annotation on user-generated videos using audio. Participated HUAWEI semantic concept annotation of UGC videos grand challenge 2014 and ranked 3rd in the evaluation. Also worked on natural language description generation for images and videos with deep models. Ranked 1st in ImageCLEF 2015 “image to sentence” subtask in the evaluation.

Honors and Awards

Best Demo Award at CBMI2019
2019
CVPR, IJCAI, ICASSP, NIST PSCR, NIST TRECVID, Yahoo! Inmind, NIST PSCR student travel grants
2016-2019
CMU LTI Student Research Symposium Best Paper Honorable Mentions
2018
Yahoo! Fellowship (Stipend and tuition in CMU)
2016-2018
Best Undergraduate Thesis (Top 5%)
2015
Second Prize, the National Undergraduates Computer Design Competition of China
2014
National Prize (Top 10%), National Undergraduates Innovation Project
2013

Selected Publications [Google Scholar]

Peeking into the Future: Predicting Future Person Activities and Locations in Videos
Junwei Liang, Lu Jiang, Juan Carlos Niebles, Alexander Hauptmann, Li Fei-Fei

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. (Translated and reported by a Chinese media (量子位, 02/13/2019), with 30k+ views in a week.)

Paper BibTex Demo Video Project Page/Code/Model
Shooter Localization Using Social Media Videos
Junwei Liang, Jay Aronson, Alexander Hauptmann

In Proceedings of the 27th ACM International Conference on Multimedia, 2019.
(Press coverage: , , , , , , , , , )

Paper BibTex Demo Video Project Page
Focal Visual-Text Attention for Memex Question Answering
Junwei Liang, Lu Jiang, Liangliang Cao, Yannis Kalantidis, Li-Jia Li, and Alexander Hauptmann

In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). (Journal Regular Paper)

Paper BibTex Code/Model/Dataset
Focal Visual-Text Attention for Visual Question Answering
Junwei Liang, Lu Jiang, Liangliang Cao, Li-Jia Li, and Alexander Hauptmann

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. (Spotlight Paper, 6.8% acceptance rate)

Paper BibTex Code/Model Presentation
Leveraging Multi-modal Prior Knowledge for Large-scale Concept Learning in Noisy Web Data
Junwei Liang, Lu Jiang, Deyu Meng, and Alexander Hauptmann

In ACM International Conference on Multimedia Retrieval (ICMR), 2017.

Paper BibTex
Webly-Supervised Learning of Multimodal Video Detectors
Junwei Liang, Lu Jiang, and Alexander Hauptmann

In AAAI Conference on Artificial Intelligence (AAAI), 2017.

Paper BibTex Demo Video Poster
Learning to detect concepts from webly-labeled video data
Junwei Liang, Lu Jiang, Deyu Meng, and Alexander Hauptmann

In International Joint Conference on Artificial Intelligence (IJCAI), 2016.

Paper BibTex
Video Description Generation using Audio and Visual Cues
Qin Jin, and Junwei Liang

In ACM International Conference on Multimedia Retrieval (ICMR), 2016.

Paper BibTex
Temporal Localization of Audio Events for Conflict Monitoring in Social Media
Junwei Liang, Lu Jiang and Alexander Hauptmann

In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2017.

Paper BibTex Poster

Educations

Ph.D. in Language and Information Technology
2017 - present

School of Computer Science, Carnegie Mellon University

Advisor: Alexander Hauptmann
M.S. in Language Technologies
2015 - 2017

School of Computer Science, Carnegie Mellon University

Advisor: Alexander Hauptmann
B.S. in Computer Science
2011 - 2015

School of Information, Renmin University of China

Advisor: Qin Jin

Academic Service

Journal Reviewer
ACM TOMM

Neurocomputing

IEEE Transactions on Multimedia

IEEE Access

IEEE Transactions on Circuits and Systems for Video Technology
Conference Program Committee
ACM Multimedia 2017

ACM Multimedia 2019
Website Developer (2014-2015)
Designed and developed major CMS websites for organizations. Hillhouse Academy, Annual Computer Design Competition in China (>2000 users annually)