From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations
Ruitong Tian, Fang Xu, Noah B. Wilson, Xianyao Li, Eric Jing Du · N/A · 2026
Framework
N/A
License
N/A
Stars
N/A
Summary
Unlike conventional teleoperation, wearable interfaces allow operators to collect dexterous demonstrations through their own hand motions while directly interacting with task objects.
Abstract Summary
Key Points
- Unlike conventional teleoperation, wearable interfaces allow operators to collect dexterous demonstrations through their own hand motions while directly interacting with task objects.
- This direct interaction reduces dependence on the target robot during collection, but it also makes the collection hardware part of the physical process that generates each demonstration.
- Interface geometry can influence both how a task is performed and what tactile observations are recorded for learning.
- We study two versions of a DexUMI-family exoskeleton that share the same robot command definition, mapping procedure, and tactile module type but differ in hand-side geometry.
- The revised interface reduces reported physical demand, improves selected device ratings, enables tactile access in a precision grasp that is mechanically blocked by the baseline, and produces task-dependent changes in recorded contact.
Interface Revision and Policy Inputs
We use the hand-side revision as a controlled study mechanism for changing wearable geometry while keeping the robot command and sensing pipeline fixed. The baseline interface (v1) and the revised interface (v2) share the same 12-DoF XHand control and tactile-sensing pipeline while differing in finger clearance, fingertip housing, and thumb seat and link geometry.
Both interfaces are right-hand exoskeletons that map the operator’s finger motion to a fully actuated 12-DoF XHand.
Evaluation Protocol
The participant study asks whether the hand-side revision changes operator burden and device experience. The accessibility test asks whether the revision changes which fingertip contacts can be physically recorded. The shared-task demonstration analysis asks whether the revision changes contact location or contact occurrence.
A dataset-separability diagnostic tests which tactile feature family most clearly distinguishes the two interface datasets.
Results
Shared-task policies use matched demonstration budgets per interface. All cells use one trained policy and the same 20-entry task-specific initial-state list. The revised interface reduced the reported physical burden of data collection (Fig. 4 ). Mean Physical Demand decreased from $11.0\pm 5.5$ under the baseline interface to $7.5\pm 4.3$ under the revised interface ($d_{z}=-0.61$, $p=0.017$).
Sources and Demonstrations
Figures are reproduced from Tian et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators
Juan José García Cárdenas, Alperen Kenan, Hamidreza Raei et al.
Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments.
FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity
Zhi Cao, Yining Hong, Jiaolong Yang et al.
Human demonstrations offer a scalable alternative to costly robot data collection for dexterous manipulation, but most human datasets lack force measurements.
ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
Zhengyu Tao, Xin Li, Xin Wang
Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, yet their reliance on visual perception limits robustness in contact-rich environments, where critical physical interaction states may not be visually observable.
SoTa: Soft Tactile Skins for Dexterous Manipulation
Jingyun Yang, Baiyu Shi, Timothy Yu et al.
A growing body of work suggests that tactile sensing gives robot policies contact information that complements vision in dexterous manipulation. However, visuo-tactile robot data remains scarce: dexterous demonstrations require teleoperating robots, which limits dataset scale.