Body-Grounded Replanning for Physically Adaptive Manipulation

Summary

Manipulation requires not only reasoning about the external environment, but also about the robot's physical condition.

Abstract Summary

Manipulation requires not only reasoning about the external environment, but also about the robot's physical condition. A strategy may remain geometrically feasible while becoming physically unsuitable due to increased joint load or limited mobility, yet internal physical state is typically used only for low-level control. We propose body-grounded high-level replanning, which uses internal physical state to adapt manipulation strategies during execution. Body-state events trigger strategy replanning, and an LLM interprets the underlying joint-level state, recent execution statistics, and execution history to select a context-dependent alternative, while leaving the task objective and low-level controller unchanged. We evaluate the framework on a reaching task under controlled load and asymmetric mobility constraints in simulation and on a real robot. Our experiments show that body-grounded replanning maintains high task success while reducing physical effort and enabling more efficient strategy adaptation. Additional contact-rich manipulation experiments demonstrate the applicability of the same replanning interface beyond reaching. These results show that internal physical state can inform not only low-level control, but also high-level decisions about how a manipulation task should be performed.

Key Points

Manipulation requires not only reasoning about the external environment, but also about the robot's physical condition.
A strategy may remain geometrically feasible while becoming physically unsuitable due to increased joint load or limited mobility, yet internal physical state is typically used only for low-level control.
We propose body-grounded high-level replanning, which uses internal physical state to adapt manipulation strategies during execution.
Body-state events trigger strategy replanning, and an LLM interprets the underlying joint-level state, recent execution statistics, and execution history to select a context-dependent alternative, while leaving the task objective and low-level controller unchanged.
We evaluate the framework on a reaching task under controlled load and asymmetric mobility constraints in simulation and on a real robot.

Section 3 Method

We consider a manipulation task in which a robot pursues a fixed task goal using one of multiple high-level strategies. Execution is indexed by discrete control step $t=0,1,\ldots$. An environment-centered planner selects an initial strategy $g_{0}\in\mathcal{G}$ based on the task goal and external environment, where $\mathcal{G}$ denotes the set of candidate strategies.

A fixed low-level controller then executes the selected strategy.

Section 3 Method — Body-Grounded Replanning for Physically Adaptive Manipulation — Figure 2 : Body-grounded high-level replanning: internal physical state triggers strategy reconsideration, and an LLM selects a context-dependent alternative.

Section 4 Experimental Setup

We evaluate body-grounded replanning under controlled variations in the robot’s physical condition using reaching tasks in simulation and on hardware, followed by demonstrations on three additional manipulation tasks. Simulation experiments use a Franka Panda 7-DoF manipulator in robosuite [ 19 ] with MuJoCo [ 20 ] .

Hardware experiments use a CRANE-X7 7-DoF manipulator with Dynamixel actuators controlled through MoveIt [ 21 ] . Joint state and effort are recorded at $20$ Hz on both platforms.

Sources and Demonstrations

Figures are reproduced from Saito et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.