Robotics Research Hub
Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning

Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning

Cong-Thanh Vu, Yen-Chen Liu · N/A · 2026

locomotion reinforcement-learning control human-robot-interaction
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

In the field of Human-Robot Interaction (HRI), achieving flexibility in human-accompanying within real-world environments holds great potential for various applications but also poses significant challenges.

Abstract Summary

In the field of Human-Robot Interaction (HRI), achieving flexibility in human-accompanying within real-world environments holds great potential for various applications but also poses significant challenges. Traditional methods typically restrict robots to fixed positions relative to humans, such as tracking from behind, in front, or side-by-side, which limits robot adaptability in dynamic workspaces. This study introduces a novel human-companioning strategy that uses Reinforcement Learning (DRL) to enable mobile robots to dynamically adjust their tracking positions according to varying conditions. An interaction space is defined to capture the relationship between the human and the robot while considering the environment, which serves as the basis for state spaces in DRL to assist the robot in adapting to environmental changes. A human-robot companion controller is developed by integrating Model Predictive Path Integral (MPPI) control with Control Barrier Functions (CBF), ensuring that the robot accurately follows the target's movement in both position and orientation while avoiding obstacles and enhancing social acceptance and safety. The proposed approach is evaluated in real-world scenarios, both indoors and outdoors, and compared with other studies. The results show that the proposed method improves the success rate and tracking accuracy by at least 24% and 47%, respectively, while enhancing human comfort. Experiments demonstrate the robot's ability to flexibly accompany a person walking at speeds of up to 1.7 m/s, dynamically adjusting its strategy without being confined to a fixed position. Additionally, the robot respects the human's intimate space to ensure safety, comfort, and effective obstacle avoidance.

Key Points

  • In the field of Human-Robot Interaction (HRI), achieving flexibility in human-accompanying within real-world environments holds great potential for various applications but also poses significant challenges.
  • Traditional methods typically restrict robots to fixed positions relative to humans, such as tracking from behind, in front, or side-by-side, which limits robot adaptability in dynamic workspaces.
  • This study introduces a novel human-companioning strategy that uses Reinforcement Learning (DRL) to enable mobile robots to dynamically adjust their tracking positions according to varying conditions.
  • An interaction space is defined to capture the relationship between the human and the robot while considering the environment, which serves as the basis for state spaces in DRL to assist the robot in adapting to environmental changes.
  • A human-robot companion controller is developed by integrating Model Predictive Path Integral (MPPI) control with Control Barrier Functions (CBF), ensuring that the robot accurately follows the target's movement in both position and orientation while avoiding obstacles and enhancing social acceptance and safety.

System Design and Problem Formulation

In this section, we present the design and hardware components of the human-companion robot and the problem formulation for the human-accompanying task. Fig. 1 illustrates the configuration of the human–companion robot system. The platform is equipped with a Velodyne VLP-16 Puck Lite 3D LiDAR for environmental perception and an Amfitrack Gen 3 electromagnetic tracker [ 1 ] for human tracking.

A source mounted on the robot and a sensor attached to the user enable reliable target identification while preventing confusion with nearby pedestrians.

System Design and Problem Formulation — Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning
Fig. 3 : The overall structure of the system architecture. The robot observes the proposed cost functions, total cost, collision-free space, robot velocity, human position and velocity, obstacles, and previous actions. Policy determines an action index within the collision-free space and then provides a target position for the human-companion controller.

Human-Robot Interactive Space

This section introduces a collision-free interactive space and cost functions for selecting the desired position. The interactive space $\mathcal{W}$ is defined as the set of $n$ design points $\mathbf{q}^{r}_{d,i}=[x^{r}_{d,i},y^{r}_{d,i},\theta^{r}_{d,i}]^{\text{T}}\in\mathbb{R}^{3}$ for $i=1,\ldots,n$ in the frame $\mathcal{R}$.

These points are located within an ellipse with a semi-major axis $a$ and a semi-minor axis $b$, centered at the human position.

Human-Robot Interactive Space — Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning
Fig. 4 : Illustration of the training environment.

RL-Based Human-Accompanying Strategies

Unlike previous studies that used RL to follow the human by considering only the relative position between the human and the robot, we propose a novel approach that leverages the proposed interactive space as the state space for RL. Additionally, we design a new reward function specifically for the human-accompanying task and present the policy training process. The overall system architecture is illustrated in Fig.

RL-Based Human-Accompanying Strategies — Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning
Fig. 5 : Policy performance under varying hyperparameter configurations: (a) Proposed architecture and (b) RL baseline.

Human-Companion Controller

In this section, we introduce a human-companion controller based on MPPI [ 43 ] , which integrates a CBF, adapted from [ 48 ] , as a safety mechanism to prevent the robot from entering the human’s intimate zone, thereby enhancing user safety and comfort.

Human-Companion Controller — Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning
Fig. 6 : Simulation environment used for comparison.

Sources and Demonstrations

Figures are reproduced from Vu et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

Adaptive Rejection of Unknown Hybrid Sinusoidal Disturbances with Application to Humanoid Walking under Unknown Ground Motion
arXiv preprint

Adaptive Rejection of Unknown Hybrid Sinusoidal Disturbances with Application to Humanoid Walking under Unknown Ground Motion

I-Chia Chang, Jacob Stewart, Petros A. Ioannou et al.

Legged robot control is not yet practical in non-inertial environments, where the ground and surroundings accelerate in the inertial Earth frame, mainly because of the unknown, time-varying, persistent perturbations caused by environmental motion, compounded by the associated hybrid robotic dynamics.

locomotion reinforcement-learning control
PDF Intermediate