DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection

Amber Xie, Jiawei Fu, Xiaofeng Guo, Carmelo Sferrazza, Guanya Shi, Haozhi Qi · N/A · 2026

Summary

Abstract Summary

Collecting high-quality demonstrations remains a central bottleneck for learning dexterous manipulation. Teleoperation records executable robot actions, but it requires a complete robot setup and places the burden of coordinating many finger joints on the operator. Human demonstrations are easier to collect, but their motions must be estimated and retargeted to the robot, and they do not directly provide robot actions. In this paper, we introduce DexPort, a portable data-collection interface for high-DoF robot hands. It uses the same hand hardware for collection and deployment, avoiding a hand embodiment gap, and records both commanded joint targets and measured joint states. Our key observation is that a small set of skills can support a wide range of dexterous tasks. During collection, the operator moves the hand-held interface to specify the robot wrist motion and selects or adjusts skills that coordinate the finger motion. This removes the need to control each joint independently while retaining behaviors such as precise grasping and in-hand manipulation. We collect demonstrations across several real-world tasks and in unstructured environments without a robot arm. We then use behavior cloning to train visuomotor policies directly on the recorded wrist motions and hand commands, without skill labels. We evaluate across 4 tasks: pick-and-place with diverse objects, screwing with in-hand tool use, drawer stowage which requires simultaneous grasp and pulling, and a bimanual scanning task. We collect over 2,500 demonstrations with our portable device, in both laboratory and in-the-wild settings. Across our tasks, we see an improvement from 11% to 64% when using recorded robot commands instead of states, and an improvement from 36% to 78% for task success when using in-the-wild data.

Key Points

Collecting high-quality demonstrations remains a central bottleneck for learning dexterous manipulation.
Teleoperation records executable robot actions, but it requires a complete robot setup and places the burden of coordinating many finger joints on the operator.
Human demonstrations are easier to collect, but their motions must be estimated and retargeted to the robot, and they do not directly provide robot actions.
In this paper, we introduce DexPort, a portable data-collection interface for high-DoF robot hands
It uses the same hand hardware for collection and deployment, avoiding a hand embodiment gap, and records both commanded joint targets and measured joint states.

Method

In Sec. III-A , we introduce the hardware design of our portable data collection interface. Next, in Sec. III-B we describe our library of skills, designed to be intuitive and useful for learning dexterous policies. Finally, Sec. III-C explains how this data can be used for imitation learning. Transferable Hand-Camera Module.

Our portable data-collection interface comprises a transferable hand-camera module and a collection handle built around a Meta Quest 2 controller, as shown in Fig. 1 (a).

Method — DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection — Fig. 1: Transferable Hardware Interface and Control Mapping. (a) Transferable Module: The Wuji hand (A) and fisheye camera (B) form a transferable hand–camera module that mounts to the Quest 2 controller for data collection and to the robot wrist for deployment. Reusing the same module preserves hand embodiment and camera-to-hand extrinsics. (b) Ergonomic Redesign: The final hardware interface places the operator’s grasp near the assembly’s center of mass and adds a support strap, shortening the moment arm and reducing wrist torque. (c) Skill Control Mapping: Quest 2 inputs start and stop demonstration episodes, select hand or screw mode, and modulate thumb, grasp, and screwing skills. The joystick adjusts thumb flexion and abduction in Thumb Mode and incrementally tightens or loosens the grasp in Finger Mode.

Experiments

Do the hand commands recorded by DexPort enable policies that succeed on diverse dexterous tasks? Q2: Does co-training with demonstrations collected in unstructured environments improve robustness to unseen objects and cluttered scenes? Q3: How does the collection throughput of DexPort compare with teleoperation and human-hand demos? Robot Platform.

We deploy policies on Tianji Marvin, a bimanual robot with two 7-DoF arms. The hand–camera modules used for collection ( Sec.

Experiments — DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection — Fig. 2: Data Visualization. We show fisheye camera views recorded during data collection. For our pick-and-place task (PnP), we collect in-domain and in-the-wild data. For drawer stowing, we add a third-person camera because the grasped object occludes much of the scene. We show both camera views at three stages of the trajectory. For bimanual scanning and drawer stowing, the operator is occasionally visible, but we do not post-process the data to remove the operator.

Sources and Demonstrations

Figures are reproduced from Xie et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.