“NormalFlow: Fast, Robust, and Accurate Contact-based Object 6DoF Pose Tracking with Vision-based Tactile Sensors” by H.-J. Huang, M. Kaess, and W. Yuan. IEEE Robotics and Automation Letters, RA-L, vol. 10, no. 1, Jan. 2025, pp. 452-459.
Tactile sensing is crucial for robots aiming to achieve human-level dexterity. Among tactile-dependent skills, tactile-based object tracking serves as the cornerstone for many tasks, including manipulation, in-hand manipulation, and 3D reconstruction. In this work, we introduce NormalFlow, a fast, robust, and real-time tactile-based 6DoF tracking algorithm. Leveraging the precise surface normal estimation of vision-based tactile sensors, NormalFlow determines object movements by minimizing discrepancies between the tactile-derived surface normals. Our results show that NormalFlow consistently outperforms competitive baselines and can track low-texture objects like table surfaces. For long-horizon tracking, we demonstrate when rolling the sensor around a bead for 360 degrees, NormalFlow maintains a rotational tracking error of 2.5 degrees. Additionally, we present state-of-the-art tactile-based 3D reconstruction results, showcasing the high accuracy of NormalFlow. We believe NormalFlow unlocks new possibilities for high-precision perception and manipulation tasks that involve interacting with objects using hands.
BibTeX entry:
@article{Huang25ral,
author = {H.-J. Huang and M. Kaess and W. Yuan},
title = {Normal{F}low: Fast, Robust, and Accurate Contact-based Object
{6DoF} Pose Tracking with Vision-based Tactile Sensors},
journal = {IEEE Robotics and Automation Letters, RA-L},
volume = {10},
number = {1},
pages = {452-459},
month = jan,
year = {2025}
}