DIGIT: A Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation

DIGIT: A Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation

Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dougal Sutherland, Alexander Melnikov, Ellen K. Li, Zhenyu Jing, Huazhe Xu, Kuang-Huei Lee, Tingfan Wu, Yuanyuan Li, Jonathan Tremblay, Stan Birchfield, Freddy Mwinyi, Karan Khare, Rene Holladay, Bill Wen, Yunhao Ba, Roberto Calandra · Meta AI / New York University · 2020

Framework

ROS / Python

License

MIT

Stars

650

Summary

An affordable, compact, high-resolution vision-based tactile sensor optimized for in-hand manipulation with real-time fingertip deformation sensing.

Abstract Summary

DIGIT is a low-cost, compact, vision-based tactile sensor specifically engineered for robotic in-hand manipulation tasks. Developed by researchers at Meta AI and NYU, the sensor consists of a soft gel fingertip cover deformed by contact forces, a small camera underneath capturing the deformation field, and a thin elastomer membrane that creates a high-contrast pattern when pushed. By analyzing images of the interior finger surface, the system reconstructs the spatial distribution of contact forces and local geometry at sub-millimeter resolution. The design philosophy behind DIGIT prioritizes manufacturability and affordability. Each sensor costs approximately $25 in parts, can be assembled with off-the-shelf cameras and 3D-printed molds, and is small enough (15 mm diameter) to be mounted on standard robotic fingertips. Unlike previous vision-based tactile sensors that required bulky optics or complicated mounting jigs, DIGIT achieves miniaturization by using a wide-angle lens placed extremely close to the deformable gel surface, capturing roughly 640-by-480 pixel images at over 60 Hz. In evaluation, DIGIT demonstrates superior performance in grasp stability prediction, slip detection, and object surface-texture classification. When mounted on a dexterous hand, it enables precise force-controlled object reorientation, insertion, and surface following. The sensor’s output lends itself naturally to learning-based approaches: recent studies feed raw DIGIT images directly into convolutional and transformer-based policy networks, bypassing explicit force reconstruction and learning manipulation strategies from raw tactile pixels. DIGIT has become the de facto open-source tactile sensor in manipulation research. The hardware design files, embedded firmware, and driver code are publicly released, and a growing ecosystem of compatible robotic hands (including the Meta/MyoHand and several Franka-compatible grippers) has emerged. For researchers working on contact-rich tasks, adding DIGIT provides rich local contact information that is either invisible or noisy when relying solely on wrist force/torque sensing.

Key Points

  • Compact 15-mm vision-based tactile sensor with sub-millimeter deformation imaging.
  • Parts cost ~$25; assembled from off-the-shelf camera, gel, 3D-printed housing.
  • Captures 640×480 at >60 Hz; tested for slip detection and grasp stability.
  • Open-source hardware, firmware, and driver widely adopted in manipulation labs.
  • Enables direct tactile-pixel training for dexterous manipulation policies.

DIGIT: A LOW COST, COMPACT, HIGH-RESOLUTION TACTILE SENSOR

While previous vision-based tactile sensors offer unparalleled high spatial resolution raw tactile sensing, they have three main limitations compared to other tactile sensors: (i) they have relatively bulky form factors, (ii) the use of soft materials at the surface of contact makes them susceptible to wear out quickly compared to other sensors, and (iii) they require a complex (largely manual) manufacturing process that leads to high variability between sensors, so that replacing a dam...

DIGIT: A LOW COST, COMPACT, HIGH-RESOLUTION TACTILE SENSOR — DIGIT: A Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
Figure 2 : Exploded view of a single DIGIT sensor. A) elastomer, B) acrylic window, C) snap-fit holder, D) lighting PCB, E) plastic housing, F) camera PCB, G) back housing.

LEARNING IN-HAND MANIPULATION WITH HIGH-RESOLUTION TACTILE SENSORS

Fine-grained in-hand manipulation is a longstanding task in robotics that has been bottlenecked by the absence of appropriate tactile sensors. Prior works have shown that high-resolution tactile sensing enables fine tactile control tasks [ 29 , 17 ] , but those sensors were too bulky to demonstrate in-hand manipulation with standard-sized robotic hands.

As explained above, DIGIT is much more compact and fits comfortably onto an Allegro robotic hand.

LEARNING IN-HAND MANIPULATION WITH HIGH-RESOLUTION TACTILE SENSORS — DIGIT: A Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
Figure 3 : Object under test and corresponding raw measurements taken using DIGIT. The measurements taken from DIGIT clearly capture sub-millimeters structures.

EXPERIMENTAL RESULTS

In addition to evaluate our design in terms of the quality of tactile images produced (Fig. 3 ) and the robustness of the gel (Section III-D ), we now evaluate the DIGIT in the complex in-hand tactile manipulation task described in Section IV . First, we evaluate the video predictive model alone.

To validate our modeling choices, we measure the prediction error on a standard benchmark for video prediction, the BAIR robot pushing dataset [ 36 ] , in addition to our DIGIT tactile marble manipulation videos.

EXPERIMENTAL RESULTS — DIGIT: A Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulation
Figure 4 : DIGIT supports different types of elastomers which can be rapidly replaced thanks to its mechanical design. Here we show readings when touching an object ( left ) using three different elastomers: reflective, reflective with markers, and transparent with markers. Each elastomer can have different benefits for different applications, e.g., reflective to measure textures, reflective with markers to compute optical flow.

Sources and Demonstrations

Figures are reproduced from Lambeta et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers