A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration
Xinyu Liu, Qiqi Dong, Boya Jia, Yi Zhang, Binbin Lian · N/A · 2026
Framework
N/A
License
N/A
Stars
N/A
Summary
A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-robot collaboration. This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motion and IMU-based hand motion.
Abstract Summary
Key Points
- A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-robot collaboration.
- This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motion and IMU-based hand motion.
- It embeds a confidence-trend-driven dynamic fusion mechanism into BiLSTM to adaptively balance bidirectional temporal features according to real-time modality reliability.
- A confidence-guided balanced learning strategy combined with a confidence freezing mechanism is further adopted to adjust network gradients dynamically, suppress noise from low-quality modalities and mitigate cross-modal learning bias.
- A physical platform based on the UR3 collaborative robot is built for experimental validation
A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration
|Authors: Xinyu Liu, Qiqi Dong, Boya Jia, Yi Zhang, Binbin Lian
|Venue: arXiv preprint | Year: 2026
|arXiv: 2609.10339v1
Abstract
A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human intention prediction for industrial human-robot collaboration. This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motion and IMU-based hand motion. It embeds a confidence-trend-driven dynamic fusion mechanism into BiLSTM to adaptively balance bidirectional temporal features according to real-time modality reliability. A confidence-guided balanced learning strategy combined with a confidence freezing mechanism is further adopted to adjust network gradients dynamically, suppress noise from low-quality modalities and mitigate cross-modal learning bias. A physical platform based on the UR3 collaborative robot is built for experimental validation. Comparative results show that the proposed method reaches an intention recognition accuracy of 91.86% and outperforms existing multimodal fusion approaches in overall performance and stability. It also maintains satisfactory accuracy under low light and partial occlusion interference. In practical assembly tasks, the framework enables proactive and stable human-robot cooperation with strong environmental adaptability.
Key Contributions
- A confidence-aware multimodal fusion framework (CAMF) is proposed to realize reliable human inten…
- This framework fuses four heterogeneous modalities including object 6D pose, gaze, skeletal motio…
- It embeds a confidence-trend-driven dynamic fusion mechanism into BiLSTM to adaptively balance bi…
- A confidence-guided balanced learning strategy combined with a confidence freezing mechanism is f…
- A physical platform based on the UR3 collaborative robot is built for experimental validation
Topics
- human-robot-interaction
Code & Data
No code repository linked in paper metadata.
BibTeX
@article{Liu2026_260910339v1,
title = {A Confidence-Aware Multimodal Fusion Framework for Industrial Human-Robot Collaboration},
author = {Xinyu Liu and Qiqi Dong and Boya Jia and Yi Zhang and Binbin Lian},
year = {2026},
eprint = {2609.10339v1},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2609.10339v1}
}
Related Papers
A New Human-Likeness and Comfort Index for Robot Movements Along Prescribed Paths
Rosanna Coccaro, Enrico Ferrentino, Antonio Parziale et al.
As human-robot interaction rapidly spreads in numerous fields, the subject of robot acceptance gains increasing importance.
A Physics-Informed Collision Learning Framework for Collaborative Robot Motion Generation
Chen Cai, Steven Liu
Close-proximity multi-arm manipulation requires collision models that are both geometrically accurate and differentiable enough for real-time optimization.
Aim Short to Reach Far: Your Frozen World Model Can Plan Better Than You Think
Xvyuan Liu, Jianjie Fang, Chen Gao et al.
Planners built on visual world models commonly score each predicted outcome by its distance to the encoded goal image. We show that this target can limit control even with exact dynamics and globally optimal short-horizon search: reaching a goal may require actions that initially move away from it.
Assessing Physical Frailty and Fall-Risk Indicators with Social Robots: An in situ Evaluation with Older Adults
Aniol Civit, Antonio Andriella, Alba Martínez et al.
Frailty assessments are crucial to evaluate the risk of adverse events and the health and social care needs of older adults, yet their administration remains resource-intensive and typically relies on coarse clinical outcomes, such as task completion times, which may overlook biomechanical indicators of functional decline.