Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce

Al Bashir, Partho Ghose, Yibin Wang, Robert G. Hardin, Joshua Peeples, Genhua Niu, Azlan Zahid · N/A · 2026

Summary

Robotic harvesting requires precise manipulation to ensure accurate positioning of the end-effector, especially in a constraint workspace.

Abstract Summary

Robotic harvesting requires precise manipulation to ensure accurate positioning of the end-effector, especially in a constraint workspace. This study introduces the Path-Safe Approach Proximal Policy Optimization (PSA-PPO), a target-conditioned local reaching framework designed for robotic harvesting of nutrient film technique (NFT)-hydroponic lettuce in greenhouses. The proposed PSA-PPO preserves the standard clipped PPO objective while introducing a harvesting-oriented policy formulation that integrates target-conditioned observations, corridor-aware spatial constraints, trajectory-history-based safety encoding, and task-specific success criteria for constrained robotic manipulation. PSAPPO was trained entirely in Isaac Lab and successfully transferred to a physical UR10e robotic arm without any real-world policy fine-tuning. In a controlled physical benchmark against 150 scheduled targets, PSA-PPO achieved the highest reaching rate of 94.7%, outperforming Cartesian+TOTG (86.0%), RRTConnect (88.0%), and RRT* (80.0%), while reducing median RMS jerk by 39.8 to 73.0% relative to the classical planners during the final approach segment. A formulation ablation evaluated over 200 simulation targets demonstrates that path-safety and side-corridor shaping are the principal drivers of harvesting-compatible performance: removing either term substantially degrades strict success even when terminal endpoint accuracy is preserved or improved, confirming that optimization toward the endpoint alone is an insufficient objective for constrained crop-proximal manipulation. Finally, PSA-PPO was integrated with an RGB-D vision module and an end-effector and evaluated as a complete greenhouse harvesting system across 144 lettuce harvesting attempts, achieving 94.4% vision-guided target reaching and 86.1% complete harvesting success.

Key Points

Robotic harvesting requires precise manipulation to ensure accurate positioning of the end-effector, especially in a constraint workspace.
This study introduces the Path-Safe Approach Proximal Policy Optimization (PSA-PPO), a target-conditioned local reaching framework designed for robotic harvesting of nutrient film technique (NFT)-hydroponic lettuce in greenhouses.
The proposed PSA-PPO preserves the standard clipped PPO objective while introducing a harvesting-oriented policy formulation that integrates target-conditioned observations, corridor-aware spatial constraints, trajectory-history-based safety encoding, and task-specific success criteria for constrained robotic manipulation.
PSAPPO was trained entirely in Isaac Lab and successfully transferred to a physical UR10e robotic arm without any real-world policy fine-tuning.
In a controlled physical benchmark against 150 scheduled targets, PSA-PPO achieved the highest reaching rate of 94.7%, outperforming Cartesian+TOTG (86.0%), RRTConnect (88.0%), and RRT* (80.0%), while reducing median RMS jerk by 39.8 to 73.0% relative to the classical planners during the final approach segment.

Methodology

The proposed PSA-PPO motion controller framework integrates with a previously developed lettuce cutting-point localization perception module and cut-and-hold mechanism [ 4 , 3 ] , creating a complete pipeline. The policy is entirely trained in a simulation environment and then transferred to the physical robot without any fine-tuning required in the real world.

Figure 1 illustrates the resulting perception-learning-deployment pipeline.

Methodology — Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce — Figure 1 : Overview of the proposed sim-to-real PSA-PPO harvesting framework. The robot-mounted RGB-D camera provides RGB imagery and aligned depth to the lettuce perception module, which generates temporally smoothed 3D cutting-point targets. PSA-PPO is trained entirely in Isaac Lab for target-conditioned, path-safe side-approach reaching. During physical deployment, the frozen policy and vision-derived targets are integrated through the ROS action-control layer to command the UR10e for side-approach harvesting of nutrient film technique (NFT)-grown lettuce.

Experiment

PSA-PPO training and simulation evaluation were conducted using NVIDIA Isaac Lab in Isaac Sim 5.1.0 with the RSL-RL backend. Training used GPU-parallel simulation on an Ubuntu 22.04 LTS workstation equipped with dual Intel Xeon Gold processors, 256 GB RAM, and an NVIDIA RTX A6000 GPU with 48 GB VRAM.

Candidate policies were trained without physical robot interaction, and the selected actor was evaluated deterministically on a fixed set of randomly generated targets spanning the prescribed front workspace.

Experiment — Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce — Figure 2 : Training architecture of the path-safe, side-corridor approach policy (PSA-PPO) for the UR10e manipulator. Each target $\mathbf{p}^{T}$ defines a horizontal approach direction $\mathbf{a}^{T}$ and same-height staging reference $\mathbf{p}^{S}$ and determines a geometry-conditioned safe-start configuration. The actor receives the 42-dimensional policy observation $\boldsymbol{\xi}_{t}$ and produces a six-dimensional action $\bar{\mathbf{u}}_{t}$, which is bounded, filtered, and scaled before application. The environment evaluates target closure, corridor following, tool orientation, wrist–TCP path safety, and motion regularity. Rollout data and critic estimates are used to compute returns and advantages for the clipped PPO update. The selected actor checkpoint is subsequently validated in simulation and exported for physical deployment.

Sources and Demonstrations

Figures are reproduced from Bashir et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.