Robotics Research Hub
Knowledge-Guided Hierarchical Policy Learning for High-Precision Cylindrical Assembly under Tight Tolerances

Knowledge-Guided Hierarchical Policy Learning for High-Precision Cylindrical Assembly under Tight Tolerances

Binbin Lian, Xinyu Liu, Tao Sun · N/A · 2026

reinforcement-learning learning-from-demonstration
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

A hybrid hierarchical learning framework is proposed to achieve high-precision assembly of 170mm cylindrical components with tolerance of 0.1mm.

Abstract Summary

A hybrid hierarchical learning framework is proposed to achieve high-precision assembly of 170mm cylindrical components with tolerance of 0.1mm. The lower-level network integrates expert experience through Behavior Cloning (BC), giving the robot human-like intuition, and incorporates the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to enhance training stability and robustness. The upper-level network dynamically adjusts the lower-level decisions based on heuristic rules, ensuring flexibility in operations. A simulated model is constructed to learn before transferring to real world. An efficient and safe training is allowed. Comparisons show that the reward curve converges within 500 episodes, indicating high learning efficiency. It also demonstrates better adaptability to initial conditions and pose errors, achieving satisfactory success rates even under extreme conditions. Moreover, the method exhibits good stability under Gaussian noise interference. In the real world, the assembly trajectory of the cylindrical segment shows smoother motion and less fluctuation.

Key Points

  • A hybrid hierarchical learning framework is proposed to achieve high-precision assembly of 170mm cylindrical components with tolerance of 0.1mm.
  • The lower-level network integrates expert experience through Behavior Cloning (BC), giving the robot human-like intuition, and incorporates the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to enhance training stability and robustness.
  • The upper-level network dynamically adjusts the lower-level decisions based on heuristic rules, ensuring flexibility in operations.
  • A simulated model is constructed to learn before transferring to real world
  • An efficient and safe training is allowed

Preliminary

The setup for cylinder segment assembly is shown in Fig. 1 . Segment I is installed on a six-degree-of-freedom parallel robot while Segment II is mounted on a fixture. Assembling features include peg-in-hole and connecting holes. Diameter of shaft on Segment I is $\Phi$169.90 mm and that of the hole on Segment II is $\Phi$170.00 mm.

The eight $\Phi$6.00 mm connecting holes are along the radius directions of shaft in Segment I and symmetrically distributed. Similarly, eight $\Phi$6.00 mm connecting holes are on the hole in Segment II.

Preliminary — Knowledge-Guided Hierarchical Policy Learning for High-Precision Cylindrical Assembly under Tight Tolerances
Figure 1: Cylindrical segment assembly system and docking features.

Method

The assembly process is divided into two network layers: a high-level network and a low-level network, as shown in Fig. 6 . This structure combines the advantages of supervised learning, imitation learning, and reinforcement learning. By using a hierarchical approach, the framework optimizes assembly control, enabling efficient and precise task execution.

The high-level network uses LSTM to learn from expert decision-making experiences.

Method — Knowledge-Guided Hierarchical Policy Learning for High-Precision Cylindrical Assembly under Tight Tolerances
Figure 2: Assembly system coordinate system.

Simulation

The key to building the simulation model is accurately simulating the effects of actions on the system state to ensure alignment with real-world behavior. The model should prioritize functional integration for future expansion and applications, as shown in Fig. 7 . Initialization: Set transformation matrices for segment I, segment II, and the moving platform.

Initialize positions of six encoded points and define the rod lengths of the parallel robot. Pose Determination: Construct frame $\{C_{1}\}$ and frame $\{C_{2}\}$ using encoded points.

Simulation — Knowledge-Guided Hierarchical Policy Learning for High-Precision Cylindrical Assembly under Tight Tolerances
Figure 3: Robot mechanism diagram.

Experiments and analysis

Simulation environment is precisely built according to the robotic system, leading to high accuracy of simulation to reality mapping. During the transfer process, coordinate mapping is used to align the coordinates of encoded points in the real environment with those in the simulation environment.

This involves aligning the coordinate systems of the simulation and real environments to ensure consistency between the source domain (simulation) and target domain (reality). The effectiveness of this mapping is demonstrated in Fig. 11 .

Experiments and analysis — Knowledge-Guided Hierarchical Policy Learning for High-Precision Cylindrical Assembly under Tight Tolerances
Figure 4: Acquisition of expert experience.

Sources and Demonstrations

Figures are reproduced from Lian et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
AHEAD: Anticipatory Hand-Driven Teleoperation via Human Intent Prediction
arXiv preprint

AHEAD: Anticipatory Hand-Driven Teleoperation via Human Intent Prediction

Seok Joon Kim, Junho Lee, Federica Spinola et al.

Direct hand-driven teleoperation maps an operator's hand motion to robot end-effector commands at every frame, enabling precise control, but it requires constant monitoring and correction during approach, grasp, and placement, which can be slow and fatiguing.

manipulation reinforcement-learning planning control learning-from-demonstration
PDF Intermediate