DIGIT: A Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dougal Sutherland, Alexander Melnikov, Ellen K. Li, Zhenyu Jing, Huazhe Xu, Kuang-Huei Lee, Tingfan Wu, Yuanyuan Li, Jonathan Tremblay, Stan Birchfield, Freddy Mwinyi, Karan Khare, Rene Holladay, Bill Wen, Yunhao Ba, Roberto Calandra · Meta AI / New York University · 2020
Framework
ROS / Python
License
MIT
Stars
650
Summary
An affordable, compact, high-resolution vision-based tactile sensor optimized for in-hand manipulation with real-time fingertip deformation sensing.
Abstract Summary
Key Points
- Compact 15-mm vision-based tactile sensor with sub-millimeter deformation imaging.
- Parts cost ~$25; assembled from off-the-shelf camera, gel, 3D-printed housing.
- Captures 640×480 at >60 Hz; tested for slip detection and grasp stability.
- Open-source hardware, firmware, and driver widely adopted in manipulation labs.
- Enables direct tactile-pixel training for dexterous manipulation policies.
DIGIT: A LOW COST, COMPACT, HIGH-RESOLUTION TACTILE SENSOR
While previous vision-based tactile sensors offer unparalleled high spatial resolution raw tactile sensing, they have three main limitations compared to other tactile sensors: (i) they have relatively bulky form factors, (ii) the use of soft materials at the surface of contact makes them susceptible to wear out quickly compared to other sensors, and (iii) they require a complex (largely manual) manufacturing process that leads to high variability between sensors, so that replacing a dam...
LEARNING IN-HAND MANIPULATION WITH HIGH-RESOLUTION TACTILE SENSORS
Fine-grained in-hand manipulation is a longstanding task in robotics that has been bottlenecked by the absence of appropriate tactile sensors. Prior works have shown that high-resolution tactile sensing enables fine tactile control tasks [ 29 , 17 ] , but those sensors were too bulky to demonstrate in-hand manipulation with standard-sized robotic hands.
As explained above, DIGIT is much more compact and fits comfortably onto an Allegro robotic hand.
EXPERIMENTAL RESULTS
In addition to evaluate our design in terms of the quality of tactile images produced (Fig. 3 ) and the robustness of the gel (Section III-D ), we now evaluate the DIGIT in the complex in-hand tactile manipulation task described in Section IV . First, we evaluate the video predictive model alone.
To validate our modeling choices, we measure the prediction error on a standard benchmark for video prediction, the BAIR robot pushing dataset [ 36 ] , in addition to our DIGIT tactile marble manipulation videos.
Sources and Demonstrations
Figures are reproduced from Lambeta et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
SI-Diff: A Framework for Learning Search and High-Precision Insertion with a Force-Domain Diffusion Policy
Yibo Liu, Stanko Oparnica, Simon Shewchun-Jakaitis et al.
Contact-rich manipulation is fundamental in robotics but poses significant challenges due to uncertainties in relative poses, such as misalignments and small clearances in peg-in-hole tasks.
Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation
Jiahe Pan, Stelian Coros, Jitendra Malik et al.
A primary bottleneck in contact-rich manipulation is the difficulty of collecting real-world data. Sim-to-real reinforcement learning offers a scalable alternative, but the simulation-reality gap prev
Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators
Juan José García Cárdenas, Alperen Kenan, Hamidreza Raei et al.
Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments.
DUET: Decoupled Upper- and Lower-Body Execution for Task-Agnostic Humanoid Loco-Manipulation
Kapalik Khanal, Biswash Khatiwada, Stephen Afrifa et al.
Changing a humanoid robot's manipulation policy should not require retraining the controller that maintains the robot's balance.