Flexible robotic hand harnesses large deformations for full-coverage human-like multimodal haptic perception.

Abstract

Integrating human-level haptic perception into soft grippers promises safer robotic grasping and improved human-robot interaction. While visual-tactile sensors promise high perception resolution at low cost, they often sacrifice compliance to maintain optical stability, hindering non-planar contact perception. We present FlexiRay, a soft gripper that integrates visual-tactile sensing with the Fin Ray Effect to achieve high compliance, broad sensory coverage, and multimodal capability. Combining a multi-layered flexible substrate, an optimized multi-mirror optical system, and a decoupled deep learning framework, FlexiRay replicates five of the seven human tactile modalities, including force, contact location, texture, temperature, and proprioception, with a single camera. It achieves 0.17 N force accuracy, 0.96 mm spatial resolution, 0.24 mm proprioception accuracy, and 1.17 °C temperature accuracy while maintaining over 90% effective coverage. FlexiRay empowers compliant grasping, safe collaboration, and intelligent teleoperation, underscoring its potential to propel service robotics toward enhanced intelligence, safety, and real-world utility.