Development of a Humanoid Robot Prototype for Multimodal Human-Robot Interaction
Thang Tran Viet, Thanh Nguyen Canh, Huy Uong Gia, Phuc Dinh Van, Son Tran Duc, Ngoc Minh Do, Xiem HoangVan · N/A · 2026
Framework
N/A
License
N/A
Stars
N/A
Summary
Human-robot interaction (HRI) enables intuitive and intelligent collaboration between humans and robots in real-world environments. This paper introduces a humanoid robot prototype designed as a flexible testbed for developing and integrating artificial intelligence (AI) modules in HRI tasks.
Abstract Summary
Key Points
- Human-robot interaction (HRI) enables intuitive and intelligent collaboration between humans and robots in real-world environments.
- This paper introduces a humanoid robot prototype designed as a flexible testbed for developing and integrating artificial intelligence (AI) modules in HRI tasks.
- The system features a 12 degree-of-freedom (DOFs) dual-arm mechanism and a 2 DOFs head with an expressive LCD screen to express facial emotions.
- All hardware components are controlled by a custom-designed controller board with real-time AI processing supported by an onboard Jetson module.
- The system incorporates three AI modules: (1) gesture recognition using MediaPipe Pose and an LSTM classifier, (2) object detection with YOLO and 3D localization, and (3) voice-command processing through speech recognition and large language model(LLM)-based semantic parsing.
Methodology
This section details the design and development methodology of the proposed humanoid robot prototype, serving as a testbed for evaluating AI-based HRI algorithms. The system architecture is organized into two interconnected layers: the mechanical–electronic infrastructure, responsible for actuation and sensing, and the AI-powered perception and control layer, which enables responsive and adaptive behaviors.
The physical design of the humanoid robot is shown in Fig. 1 .
Experimental Results
To validate the effectiveness of the proposed humanoid robot system in real-world human–robot interaction (HRI) tasks, two experimental scenarios were implemented: (1) gesture-based interaction and (2) voice-command execution. All tests were conducted in an indoor laboratory environment under consistent lighting and minimal occlusion to ensure stable sensing performance.
Videos demonstrating the results are available at: https://youtu.be/NVSn821oaCY The robot prototype was placed on a fixed base in a standing posture.
Sources and Demonstrations
Figures are reproduced from Viet et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
A Phenomenology of Neural Geometry in Vision-Language-Action Models
Mozes Jacobs, Aiden Swann, Mathilde Papillon et al.
Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it?
A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.
Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.
Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
Haoran Chen, Jingtian Ji, Samuel Wheeler et al.
Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective.
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
Zhaoda Du, Michael Bowman, Xiaoli Zhang
Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.