Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport

Summary

Abstract Summary

We focus on human-robot collaborative transport, a challenging task of broad relevance spanning logistics, manufacturing, and the home, in which a user and a robot work together to relocate a large or heavy object. To act as an effective partner, the robot should reduce the user's effort by contributing to efficient relocation of the object while remaining physically responsive to them. Prior work often addresses these capabilities separately, producing robots that may move the object efficiently but resist user input, or accommodate the user but depend on continuous guidance. Our key insight is that obstacle-constrained collaborative transport requires integrating predictions of human collaborative behavior with compliant robot control. To this end, we introduce PROACT, a framework for human-robot collaborative transport that incorporates anticipation into compliant whole-body control through a learned model of human collaborative behavior. Trained on a large-scale, real-world dataset of dyadic human transport demonstrations, our transformer architecture distills collaborative behavior into predictions of future object motion. Across 108 real-world trials with a 9-DoF mobile manipulator, PROACT reduces mean interaction work by 59.2\% and 20.4\%, and mean completion time by 12.9\% and 6.9\%, relative to compliance-only and MPC baselines, respectively. Footage from our experiments can be found at https://youtu.be/qAGvQfVPjbk.

Key Points

We focus on human-robot collaborative transport, a challenging task of broad relevance spanning logistics, manufacturing, and the home, in which a user and a robot work together to relocate a large or heavy object.
To act as an effective partner, the robot should reduce the user's effort by contributing to efficient relocation of the object while remaining physically responsive to them.
Prior work often addresses these capabilities separately, producing robots that may move the object efficiently but resist user input, or accommodate the user but depend on continuous guidance.
Our key insight is that obstacle-constrained collaborative transport requires integrating predictions of human collaborative behavior with compliant robot control.
To this end, we introduce PROACT, a framework for human-robot collaborative transport that incorporates anticipation into compliant whole-body control through a learned model of human collaborative behavior.

Problem Statement

We consider a user and a robot collaborating to transport a rigid object in a workspace $\mathcal{W}\subseteq SE(3)$ containing obstacle-occupied regions $\mathcal{O}\subset\mathcal{W}$. We represent the object state as $(x^{\mathrm{obj}},\dot{x}^{\mathrm{obj}})\in\mathcal{W}\times\mathbb{R}^{6}$, where $x^{\mathrm{obj}}$ is its pose and $\dot{x}^{\mathrm{obj}}$ is its twist.

The user and robot maintain fixed grasps on opposite sides of the object throughout the task. The robot is a mobile manipulator with a holonomic base and an $N$-DoF arm.

Problem Statement — Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport — Fig. 3 : Evaluation setting with an obstacle-constrained workspace, object, and mobile manipulator. X-shaped floor markings indicate start and goal locations.

Proact: A Framework for Proactive Assistance in Collaborative Transport

We propose a framework that combines human collaborative behavior modeling in the form of object motion prediction with compliant whole-body robot control for physical human-robot collaborative transport.

The model, learned from demonstrations of dyadic human collaborative transport, predicts a future horizon of object twists describing how a collaborative team would realize a route, conditioned on recent object state history.

Proact: A Framework for Proactive Assistance in Collaborative Transport — Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport — Fig. 7 : Starting configurations during the pilot study. Top: the robot starts ahead of the user along the direction of travel. Bottom: the robot starts behind the user.

Evaluation

We conduct two complementary evaluations of our method: a component-level evaluation of model prediction performance and a pilot study assessing the full system via real-world human-robot collaborative transport trials.

Motivated by practical transport challenges such as maneuvering around corners and through constrained spaces, we design a new $4.5\text{\times}5.4\text{\,}\mathrm{m}$ workspace that requires successive turns between start and goal locations (Fig. 3 ).

Evaluation — Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport — Fig. 8 : Snapshots from our pilot study, comparing Compliance-only, MPC, and Proact across matching timestamps during trials. Proact ’s anticipation of collaborative behavior enables the team to progress more efficiently while requiring less interaction effort.

Sources and Demonstrations

Figures are reproduced from Yang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.