Robotics Research Hub
Robotic Optical Coherence Tomography-Guided Geometric Modeling of Soft Tissue: 3D Representations for Data-Driven Closed-Loop Shape Control

Robotic Optical Coherence Tomography-Guided Geometric Modeling of Soft Tissue: 3D Representations for Data-Driven Closed-Loop Shape Control

Chaoyu Zhang, Guangshen Ma, Mark Draelos · N/A · 2026

manipulation vision reinforcement-learning control learning-from-demonstration
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

In medical robotics, data-driven soft-tissue shape control requires deforming tissue toward desired morphologies through sequential physical interactions.

Abstract Summary

In medical robotics, data-driven soft-tissue shape control requires deforming tissue toward desired morphologies through sequential physical interactions. This challenge is particularly important in microscale manipulation, where localized deformation is difficult to observe, represent, and control, while physics-based models can be computationally expensive and difficult to adapt to realistic environments. Optical coherence tomography (OCT) provides high-resolution volumetric perception of tissue deformation, but target-conditioned control requires effective representations of both observed and desired tissue states. In this work, we introduce an OCT-guided data-driven framework for tissue shape control, learned from user-guided demonstrations and executed with closed-loop OCT feedback. We study full-volume, tissue-masked sparse-volume, and voxelized point-cloud representations, with or without residual conditioning, using deterministic (Explicit) and generative diffusion (Diffusion) policies. Across 12 configurations and 576 phantom rollouts on 3-DoF depth and 6-DoF directional tasks, OCT-derived 3D representations support both Explicit and Diffusion policies, with performance depending on representation and residual conditioning. Residual conditioning reduces directional Shape RMSE from 0.748 to 0.555 mm for full-volume Diffusion, while voxelized point-cloud Explicit with residuals achieves 0.534 mm for depth and 0.443 mm for directional control, with comparable accuracy at intermediate unseen targets and consistent performance across deployments. We further evaluate the framework in 60 ex vivo porcine rollouts, extending to biological tissue with final geometric errors comparable to human demonstrations. These results establish volumetric OCT as current and goal representations for learned closed-loop tissue shape control across deterministic and generative action models.

Key Points

  • In medical robotics, data-driven soft-tissue shape control requires deforming tissue toward desired morphologies through sequential physical interactions.
  • This challenge is particularly important in microscale manipulation, where localized deformation is difficult to observe, represent, and control, while physics-based models can be computationally expensive and difficult to adapt to realistic environments.
  • Optical coherence tomography (OCT) provides high-resolution volumetric perception of tissue deformation, but target-conditioned control requires effective representations of both observed and desired tissue states.
  • In this work, we introduce an OCT-guided data-driven framework for tissue shape control, learned from user-guided demonstrations and executed with closed-loop OCT feedback.
  • We study full-volume, tissue-masked sparse-volume, and voxelized point-cloud representations, with or without residual conditioning, using deterministic (Explicit) and generative diffusion (Diffusion) policies.

OCT-Guided Tissue Shape Control Framework

We formulate soft-tissue shape control as a goal-conditioned robot learning problem with closed-loop volumetric OCT feedback. A policy learned from user-guided demonstrations predicts tooltip motions conditioned on current OCT observations and a target tissue morphology.

The framework integrates parameterized target generation (Appendix A.2 ), 3D OCT representations, goal-conditioning schemes, and Explicit and Diffusion policies within a unified closed-loop formulation.

OCT-Guided Tissue Shape Control Framework — Robotic Optical Coherence Tomography-Guided Geometric Modeling of Soft Tissue: 3D Representations for Data-Driven Closed-Loop Shape Control
Figure 1: Closed-loop OCT-guided tissue shape control. The OCT captures volumetric tissue deformation induced by the robotic tooltip, while parameterized target generation constructs desired morphologies from predeformed OCT states. Current and target OCT volumes are encoded as full volumes, tissue-masked volumes, or voxelized point clouds, with optional current–target residual conditioning. These representations condition an Explicit or Diffusion policy to predict tooltip motions, with the first action executed before an OCT scan updates the state for closed-loop prediction.

Real-Robot Evaluation

We evaluated the proposed framework on physical tissue phantoms and ex vivo porcine tissue to determine whether learned policies (i) achieve desired tissue morphologies, (ii) adapt actions to different targets, and (iii) generalize to intermediate targets unseen during training.

The platform combines a 6-DoF UR5e robot, a printed palpation tooltip, a 1064-nm swept-source OCT system operating at 400 kHz, and an RGB camera (Fig. 1 ). Experiments use a soft silicone 40A phantom with an internal empty layer, together with ex vivo porcine tissue.

Real-Robot Evaluation — Robotic Optical Coherence Tomography-Guided Geometric Modeling of Soft Tissue: 3D Representations for Data-Driven Closed-Loop Shape Control
Figure 2: Representation-specific residual conditioning. Full volumes use voxelwise intensity differences, sparse tissue volumes use intensity and occupancy differences over occupied coordinates, and voxelized point clouds use pooled feature differences. Values are schematic.

Sources and Demonstrations

Figures are reproduced from Zhang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
arXiv preprint

DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation

Yunchao Yao, Zhuxiu Xu, Tianqi Zhang et al.

Building general-purpose dexterous manipulation policies requires benchmarks that go beyond isolated tasks to systematically evaluate policies across diverse interaction modes, sensory conditions, and robot embodiments.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
PDF Advanced