Automatic Adaptation of Person Association of Multiview Tracking

3Dtracking

Reliable markerless motion tracking of multiple people participating in complex group activity from multiple handheld cameras is challenging due to frequent occlusions, strong viewpoint and appearance variations, and asynchronous video streams. The key to solving this problem is to reliably associate the same person across distant viewpoint and temporal instances. In this work, we combine motion tracking, mutual exclusion constraints, and multiview geometry in a multitask learning framework to automatically adapt a generic person appearance descriptor to the domain videos. Tracking is formulated as a spatiotemporally constrained clustering using the adapted person descriptor. Physical human constraints are exploited to reconstruct accurate and consistent 3D skeletons for every person across the entire sequence. We show significant improvement in association accuracy (up to 18%) in events with up to 60 people and 3D human skeleton reconstruction (5 to 10 times) over the baseline for events captured "in the wild".

Publications


"Automatic Adaptation of Person Association of Multiview Tracking"
Minh Vo, Ersin Yumer, Kalyan Sunkavalli, Sunil Hadap, Yaser Sheikh, and Srinivasa Narasimhan
Tech Report.
[
PDF][Appendix][Dataset (Coming soon)]

Contribution

Discriminative and automatically domain adaptive person appearance descriptor enables the use of clustering for multiview people tracking. This is achieved by combining motion tracking, mutual exclusion constraints, and multiview geometry in a multitask learning framework to automatically adapt a generic person appearance descriptor to the domain videos

3D Tracking results

[download]

Acknowledgements


This research is supported by the NSF CNS-1446601, the ONR N00014-14-1-0595, the Heinz Endowments "Plattform Pittsburgh", a Qualcomm Innovation Ph.D. fellowship, and Adobe "Unrestricted Research Gift".