From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations

From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations

Ruitong Tian, Fang Xu, Noah B. Wilson, Xianyao Li, Eric Jing Du · N/A · 2026

Framework

N/A

License

N/A

Stars

N/A

Summary

Unlike conventional teleoperation, wearable interfaces allow operators to collect dexterous demonstrations through their own hand motions while directly interacting with task objects.

Abstract Summary

Unlike conventional teleoperation, wearable interfaces allow operators to collect dexterous demonstrations through their own hand motions while directly interacting with task objects. This direct interaction reduces dependence on the target robot during collection, but it also makes the collection hardware part of the physical process that generates each demonstration. Interface geometry can influence both how a task is performed and what tactile observations are recorded for learning. We study two versions of a DexUMI-family exoskeleton that share the same robot command definition, mapping procedure, and tactile module type but differ in hand-side geometry. The revised interface reduces reported physical demand, improves selected device ratings, enables tactile access in a precision grasp that is mechanically blocked by the baseline, and produces task-dependent changes in recorded contact. Lid twisting primarily exhibits a change in contact location, whereas egg carton opening primarily exhibits a change in contact occurrence. We then train matched policies using either a binary aggregate input or a spatial-plus-force input. On lid twisting and egg carton opening, policies trained on revised-interface demonstrations achieve higher success than those trained on baseline demonstrations, with a significant pooled interface effect. Across these and two additional tasks, USB insertion and soldering tool pick and place, the spatial-plus-force input likewise outperforms binary aggregation. These results show that wearable collection hardware is part of the data-generation process for robot learning. Such interfaces should therefore be evaluated not only through operator experience, but also through the tactile interactions they make recordable and the policy performance their demonstrations support.

Key Points

  • Unlike conventional teleoperation, wearable interfaces allow operators to collect dexterous demonstrations through their own hand motions while directly interacting with task objects.
  • This direct interaction reduces dependence on the target robot during collection, but it also makes the collection hardware part of the physical process that generates each demonstration.
  • Interface geometry can influence both how a task is performed and what tactile observations are recorded for learning.
  • We study two versions of a DexUMI-family exoskeleton that share the same robot command definition, mapping procedure, and tactile module type but differ in hand-side geometry.
  • The revised interface reduces reported physical demand, improves selected device ratings, enables tactile access in a precision grasp that is mechanically blocked by the baseline, and produces task-dependent changes in recorded contact.

Interface Revision and Policy Inputs

We use the hand-side revision as a controlled study mechanism for changing wearable geometry while keeping the robot command and sensing pipeline fixed. The baseline interface (v1) and the revised interface (v2) share the same 12-DoF XHand control and tactile-sensing pipeline while differing in finger clearance, fingertip housing, and thumb seat and link geometry.

Both interfaces are right-hand exoskeletons that map the operator’s finger motion to a fully actuated 12-DoF XHand.

Interface Revision and Policy Inputs — From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations
Fig. 2: Hand-side interface revision. The baseline and revised interfaces retain the same tactile modules and robot mapping but differ in per-finger clearance, fingertip housing geometry, and thumb seat and link geometry. The baseline fingertip housing extends approximately 5.5 mm beyond the tactile pad, whereas the revised housing is approximately flush with the pad.

Evaluation Protocol

The participant study asks whether the hand-side revision changes operator burden and device experience. The accessibility test asks whether the revision changes which fingertip contacts can be physically recorded. The shared-task demonstration analysis asks whether the revision changes contact location or contact occurrence.

A dataset-separability diagnostic tests which tactile feature family most clearly distinguishes the two interface datasets.

Evaluation Protocol — From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations
Fig. 3: Wearable demonstration collection with the revised interface. Wrist-camera views from one demonstration per task; rows run forward in time: before measured fingertip contact, early in contact, and late in contact.

Results

Shared-task policies use matched demonstration budgets per interface. All cells use one trained policy and the same 20-entry task-specific initial-state list. The revised interface reduced the reported physical burden of data collection (Fig. 4 ). Mean Physical Demand decreased from $11.0\pm 5.5$ under the baseline interface to $7.5\pm 4.3$ under the revised interface ($d_{z}=-0.61$, $p=0.017$).

Results — From Wearable Interfaces to Dexterous Policies: Contact Shifts and Tactile Representations
Fig. 4: The hand-side revision reduces reported physical demand and improves selected device ratings. Paired ratings ($N=20$; mean $\pm$ SD) are shown for NASA-TLX (left, 0–20) and device items (right, 1–5). Displayed items pass the primary test or family-specific correction; higher performance, comfort, and reuse scores are favorable.

Sources and Demonstrations

Figures are reproduced from Tian et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
arXiv preprint

ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation

Zhengyu Tao, Xin Li, Xin Wang

Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, yet their reliance on visual perception limits robustness in contact-rich environments, where critical physical interaction states may not be visually observable.

manipulation vision vla reinforcement-learning learning-from-demonstration tactile
PDF Intermediate
SoTa: Soft Tactile Skins for Dexterous Manipulation
arXiv preprint

SoTa: Soft Tactile Skins for Dexterous Manipulation

Jingyun Yang, Baiyu Shi, Timothy Yu et al.

A growing body of work suggests that tactile sensing gives robot policies contact information that complements vision in dexterous manipulation. However, visuo-tactile robot data remains scarce: dexterous demonstrations require teleoperating robots, which limits dataset scale.

manipulation vision reinforcement-learning learning-from-demonstration tactile benchmark
PDF Intermediate