Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction

Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction

|Authors: Thang Tran Viet, Thanh Nguyen Canh, Huy Uong Gia, Phuc Dinh Van, Son Tran Duc, Ngoc Minh Do, Xiem HoangVan

|Venue: arXiv preprint | Year: 2026

|arXiv: 2609.05361v1

Abstract

Human-robot interaction (HRI) enables intuitive and intelligent collaboration between humans and robots in real-world environments. This paper introduces a humanoid robot prototype designed as a flexible testbed for developing and integrating artificial intelligence (AI) modules in HRI tasks. The system features a 12 degree-of-freedom (DOFs) dual-arm mechanism and a 2 DOFs head with an expressive LCD screen to express facial emotions. All hardware components are controlled by a custom-designed controller board with real-time AI processing supported by an onboard Jetson module. The system incorporates three AI modules: (1) gesture recognition using MediaPipe Pose and an LSTM classifier, (2) object detection with YOLO and 3D localization, and (3) voice-command processing through speech recognition and large language model(LLM)-based semantic parsing. The platform is validated through experiments on positioning accuracy, with results showing average manipulation errors of approximately 1.83 cm. To demonstrate its versatility, experimental results show over 90% task accuracy, with gesture recognition reaching 96%, speech recognition reaching 92%. The results confirm the effectiveness of the proposed system as a reproducible and accessible humanoid platform for research and prototyping in HRI.

Key Contributions

• Human-robot interaction (HRI) enables intuitive and intelligent collaboration between humans and …
• This paper introduces a humanoid robot prototype designed as a flexible testbed for developing an…
• The system features a 12 degree-of-freedom (DOFs) dual-arm mechanism and a 2 DOFs head with an ex…
• All hardware components are controlled by a custom-designed controller board with real-time AI pr…
• The system incorporates three AI modules: (1) gesture recognition using MediaPipe Pose and an LST…

Topics

• manipulation
• vision
• vla
• reinforcement-learning
• control
• human-robot-interaction

Code & Data

• GitHub repository: https://github.com/Uberi/speech

BibTeX

@article{Viet2026_260905361v1,
  title     = {Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction},
  author    = {Thang Tran Viet and Thanh Nguyen Canh and Huy Uong Gia and Phuc Dinh Van and Son Tran Duc and Ngoc Minh Do and Xiem HoangVan},
  year      = {2026},
  eprint    = {2609.05361v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2609.05361v1}
}