Edge-Native Temporal--Graph Dual-Expert Inference for Dexterous Grasp Classification
Jiajian Zheng, Bin Zhang, Wen Fan, Yanan Liu · N/A · 2026
Framework
N/A
License
N/A
Stars
N/A
Summary
Multi-finger hands contact objects at different times, so a global close command can overload early-contact fingers while others are still moving. Dense tactile arrays address this problem but add hardware, calibration, bandwidth, and computation.
Abstract Summary
Key Points
- Multi-finger hands contact objects at different times, so a global close command can overload early-contact fingers while others are still moving.
- Dense tactile arrays address this problem but add hardware, calibration, bandwidth, and computation
- We present an edge-native pipeline that uses only built-in joint-angle and pre-use-calibrated actuator-load feedback to recognize the grasped object and predict an independent Stop event for each finger.
- Four samples and their first differences form a 4-by-20 contact-evolution window
- At its core is a heterogeneous dual-expert architecture, termed DualExpert, that assigns complementary tasks to a Temporal Convolutional Network (TCN) and a Finger Graph Filter Network (FGFN).
System and Problem Formulation
The system combines sparse hand feedback with local recognition and per-finger stopping. Fig. 1 shows the complementary experts. TCN96b5 denotes a 96-channel Temporal Convolutional Network with five residual blocks, while FGFN80 denotes an 80-channel Finger Graph Filter Network. The TCN models contact evolution and stopping; the FGFN exposes neighboring-finger agreement and imbalance for classification.
The platform combines the RH56 hand, an embedded inference board, and a half-duplex RS-485 link [ 29 ] .
Heterogeneous Dual-Expert Network
Fig. 2 presents the Heterogeneous Temporal–Graph Dual-Expert, abbreviated as DualExpert. Both experts share the contact window. Their class distributions are fused, while all learned Stop scores come from TCN96b5 because it is substantially stronger on that task. Firmware angle and target limits remain independent completion conditions. As shown in Fig.
1 (a), TCN96b5 projects each frame to 96 channels and applies five residual depthwise-separable blocks with $k=3$.
Embedded Deployment and Control
ST Edge AI stores model weights and operators with the firmware in Flash. At runtime, each validated RS-485 sample enters SRAM, updates the four-frame buffer, and forms the normalized $4\times 20$ input. The Cortex-M55-only or Neural-ART mixed-execution path writes class and Stop probabilities back to SRAM for immediate finger-latch updates, without returning raw windows to the computer.
The Cortex-M55-only build uses 178.08 KiB of weights, 8.46 KiB of peak activations, and 405,516 MACs per window.
Experiments
Experiments evaluate recognition, per-finger stopping, expert complementarity, and embedded execution using the platform and five object instances shown in Fig. 4 . Deployment uses an STM32N647 on the ATK-DNN647 board and evaluates both Cortex-M55-only execution and Neural-ART mixed execution [ 30 , 31 ] . The measured board-side data path is shown in Fig. 5 .
The acquisition application performs repeatable close-record-open cycles over RS-485 and saves a trial only after complete five-finger feedback and a final synchronized read.
Sources and Demonstrations
Figures are reproduced from Zheng et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
Design and Evaluation of a Touchscreen-Based Teleoperation Interface for Robotic Manipulators
Juan José García Cárdenas, Alperen Kenan, Hamidreza Raei et al.
Intuitive teleoperation interfaces are crucial for the safe and effective operation of robotic manipulators in challenging environments.
DUET: Decoupled Upper- and Lower-Body Execution for Task-Agnostic Humanoid Loco-Manipulation
Kapalik Khanal, Biswash Khatiwada, Stephen Afrifa et al.
Changing a humanoid robot's manipulation policy should not require retraining the controller that maintains the robot's balance.
FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity
Zhi Cao, Yining Hong, Jiaolong Yang et al.
Human demonstrations offer a scalable alternative to costly robot data collection for dexterous manipulation, but most human datasets lack force measurements.
SoTa: Soft Tactile Skins for Dexterous Manipulation
Jingyun Yang, Baiyu Shi, Timothy Yu et al.
A growing body of work suggests that tactile sensing gives robot policies contact information that complements vision in dexterous manipulation. However, visuo-tactile robot data remains scarce: dexterous demonstrations require teleoperating robots, which limits dataset scale.