Robotics Research Hub
Shake to Learn: Dynamic Interrogation of Hidden Object Physics for Robotic Manipulation with Physical Reservoir Computing

Shake to Learn: Dynamic Interrogation of Hidden Object Physics for Robotic Manipulation with Physical Reservoir Computing

Wen Sin Lor, Jun Wang, Suyi Li · N/A · 2026

manipulation vision
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Many physical properties relevant to robotic manipulation are hidden from vision. A sealed object, for example, may reveal little about its center of mass (COM) or internal contents until it is lifted, shaken, or otherwise dynamically perturbed.

Abstract Summary

Many physical properties relevant to robotic manipulation are hidden from vision. A sealed object, for example, may reveal little about its center of mass (COM) or internal contents until it is lifted, shaken, or otherwise dynamically perturbed. This study shows that such interactions can enable a new modality of robotic perception and learning, in which interaction-induced dynamic responses are used to infer object physics that is inaccessible to conventional sensing. We implement this idea using an origami-inspired soft robotic arm that functions as a physical reservoir computer. After grasping an object, the arm is excited by a fixed shaking input at its base, and the resulting ringdown response is recorded through either camera tracking or embedded sensors. Because the input is held constant across trials, hidden object properties, such as the COM position, are encoded through their effect on the dynamics of the coupled robot-object system. A lightweight linear readout can then decode these dynamics to recover interpretable information about the hidden object physics. Using this framework, the soft robotic arm reservoir completed three tasks of increasing difficulty: inferring the orientation of the object's hidden COM, inferring the COM distance from the grasp point, and using the inferred COM information to guide a subsequent regrasp. We further develop a dynamic summary representation of the ringdown response that improves prediction accuracy. Together, these results establish shake-to-learn mechanical interrogation as a promising strategy for robotic systems to convert brief physical interactions into actionable cues about hidden object properties for downstream manipulation.

Key Points

  • Many physical properties relevant to robotic manipulation are hidden from vision
  • A sealed object, for example, may reveal little about its center of mass (COM) or internal contents until it is lifted, shaken, or otherwise dynamically perturbed.
  • This study shows that such interactions can enable a new modality of robotic perception and learning, in which interaction-induced dynamic responses are used to infer object physics that is inaccessible to conventional sensing.
  • We implement this idea using an origami-inspired soft robotic arm that functions as a physical reservoir computer.
  • After grasping an object, the arm is excited by a fixed shaking input at its base, and the resulting ringdown response is recorded through either camera tracking or embedded sensors.

Dynamic Interrogation: Task Setup and Theoretical Framework

Dynamic interrogation aims to project hidden object properties into measurable dynamical responses in the coupled robot-object system. In this study, we choose the object’s COM as the targeted latent property. To this end, we 3D-print a cross-shaped and a bar-shaped container with internal dividers, and then place a payload mass $m$ inside at a selected location $\ell$ (Figure 2 ).

Therefore, adjusting the payload’s location inside the container effectively changes the object’s COM.

Dynamic Interrogation: Task Setup and Theoretical Framework — Shake to Learn: Dynamic Interrogation of Hidden Object Physics for Robotic Manipulation with Physical Reservoir Computing
Figure 2: Experiment setup for dynamic interrogation . ( A ) Anatomy of the modular soft robotic arm/physical reservoir. ( B ) 3D-printed cross-shaped container, showing the hidden payload mass $m$ and the corresponding coordinate setup. ( C ) 3D-printed bar-shaped container. ( D ) The ringdown test procedure on a three-module robotic arm using camera-based motion tracking. ( E ) Another test on a four-module arm using embedded bending strain sensors.

Dynamic Interrogation: Results and Insights

In this study, we task the soft robotic arm with inferring the hidden COM position through three challenges of increasing complexity. Task I asks whether shake-induced ringdown dynamics carry enough information to decode the angular position of a hidden payload mass within a cross-shaped container.

This establishes the fundamental principles of robotic learning via dynamic interaction, and confirms that this learning modality is accessible through both external camera tracking and embedded sensing alone.

Dynamic Interrogation: Results and Insights — Shake to Learn: Dynamic Interrogation of Hidden Object Physics for Robotic Manipulation with Physical Reservoir Computing
Figure 3: Predicting the angular orientation of the hidden COM. ( A–C ) Experimental snapshot, tracked marker displacement, and confusion matrix using external camera tracking. With a 100 g payload, the camera-tracking method yields 44/48 correct leave-one-out predictions. ( D–F ) Experimental snapshot, recorded sensor reading, and confusion matrix using embedded bending sensors. With a 100 g payload, the embedded bending sensors yield 78/80 correct leave-one-out predictions. ( G ) With 40 g and 20 g payloads, the camera-tracking strategy yields 44/48 and 41/48 correct predictions, respectively. ( H ) With 40 g and 20 g payloads, the embedded sensors yield 68/80 and 54/80 correct predictions, respectively. ( I, J ) Frequency-resolved spectral separability of the ringdown response across different COM orientations. A larger $J_{m}(f)$ indicates that the four COM-orientation classes produce more distinguishable frequency-domain ringdown signatures, while the area under the curve, $A_{m}$, summarizes the total separable spectral content across the frequency band. Trends in $J_{m}(f)$ and $A_{m}$ support the interpretation that lighter payloads produce weaker orientation-dependent spectral signatures in the shake-induced ringdown.

Sources and Demonstrations

Figures are reproduced from Lor et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced