Safety-aware Skill Adaptation for Reinforcement Learning in Dynamic Environments

A K M Nadimul Haque, Sheila Sutjipto, Marc G. Carmichael, Teresa Vidal-Calleja · N/A · 2026

Summary

Abstract Summary

Skill adaptation frameworks based on reinforcement learning often require restrictive assumptions to maintain stability, such as fixed observations or tightly controlled exploration schedules. In cluttered and dynamic environments, however, unrestricted exploration can lead to unsafe behaviour and unstable learning, particularly when task-relevant observations lie near obstacles or involve moving objects. In this work, we present Dist-GPRL, a distance-aware and safety-guided reinforcement learning framework for structured robot skill adaptation. Building upon Gaussian Process (GP)-based skill parameterisation, our framework sequentially adapts overlapping local windows of sparse trajectory via-points rather than modifying the complete skill at every policy step. Raw policy outputs are correlated through the GP covariance structure, producing temporally coherent trajectory updates while reducing the action-space and credit-assignment difficulties associated with global trajectory adaptation. Safety is incorporated through two complementary forms of guidance. A safe-subspace prior derived from the Hausdorff Approximation Planner (HAP) biases policy exploration toward feasible regions, while dynamically updated distance field clearance and gradient rewards provide local obstacle awareness. A trajectory-kinematics similarity regulariser further preserves the demonstrated velocity and acceleration characteristics during adaptation. We evaluate the framework on two dynamic object-manipulation tasks in simulation and transfer the learned policy to real-world robot execution. Experimental results demonstrate higher task success, lower collision frequency, and more stable learning than the baselines, while preserving the kinematic characteristics of the demonstrated skill.

Key Points

Skill adaptation frameworks based on reinforcement learning often require restrictive assumptions...
In cluttered and dynamic environments, however, unrestricted exploration can lead to unsafe behav...
In this work, we present Dist-GPRL, a distance-aware and safety-guided reinforcement learning fra...
Building upon Gaussian Process (GP)-based skill parameterisation, our framework sequentially adap...
Raw policy outputs are correlated through the GP covariance structure, producing temporally coher...

Safety-aware Skill Adaptation for Reinforcement Learning in Dynamic Environments

|Authors: A K M Nadimul Haque, Sheila Sutjipto, Marc G. Carmichael, Teresa Vidal-Calleja

|Venue: arXiv preprint | Year: 2026

|arXiv: 2609.11433v1

Abstract

Key Contributions

Skill adaptation frameworks based on reinforcement learning often require restrictive assumptions…
In cluttered and dynamic environments, however, unrestricted exploration can lead to unsafe behav…
In this work, we present Dist-GPRL, a distance-aware and safety-guided reinforcement learning fra…
Building upon Gaussian Process (GP)-based skill parameterisation, our framework sequentially adap…
Raw policy outputs are correlated through the GP covariance structure, producing temporally coher…

Topics

manipulation
reinforcement-learning
control

Code & Data

No code repository linked in paper metadata.

BibTeX

@article{Haque2026_260911433v1,
  title     = {Safety-aware Skill Adaptation for Reinforcement Learning in Dynamic Environments},
  author    = {A K M Nadimul Haque and Sheila Sutjipto and Marc G. Carmichael and Teresa Vidal-Calleja},
  year      = {2026},
  eprint    = {2609.11433v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2609.11433v1}
}