Robotics Research Hub
Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments

Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments

Alperen Kenan, Juan José García Cárdenas, Adriana Tapus, Paul Bremner, Manuel Giuliani · N/A · 2026

vision reinforcement-learning control learning-from-demonstration benchmark
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Multi-robot systems that combine the complementary capabilities of specialised platforms are critical for complex missions in challenging, inaccessible environments.

Abstract Summary

Multi-robot systems that combine the complementary capabilities of specialised platforms are critical for complex missions in challenging, inaccessible environments. Current deployments assign an operator or operator team to each robot, with collaboration relying on operator-to-operator communication to manually coordinate heterogeneous platforms with incompatible interfaces. This operator-dependent approach increases cognitive load and limits scalability across robots from different manufacturers. This study designs and evaluates a shared-visualisation collaborative teleoperation interface to improve operator coordination, situational awareness, and the task performance in multi-operator, multi-robot systems. A comparative user study with 16 participants compared three teleoperation conditions: co-located operators with a shared interface, remote operators with a shared interface, and co-located operators with individual interfaces. Participants controlled two simulated robots in a nuclear power plant digital twin to complete disaster response tasks. Performance metrics, physiological data, and behavioural measures were recorded to assess task performance, cognitive load, and operator awareness across conditions. The co-located shared-interface condition achieved the fastest mission completion (292.8 s), outperforming remote operation (351.6 s) and individual interfaces (385.1 s), with the fewest collisions (0.9 per trial) and highest situational awareness accuracy (91.1 %). Physical co-presence was associated with reduced cognitive load, with NASA-TLX scores increasing from 38.2 in co-located to 57.7 in remote operation, supported by consistent physiological indicators. These results demonstrate that co-presence and shared interfaces provide complementary benefits, supporting workload regulation and mutual awareness respectively in multi-operator, multi-robot teleoperation.

Key Points

  • Multi-robot systems that combine the complementary capabilities of specialised platforms are critical for complex missions in challenging, inaccessible environments.
  • Current deployments assign an operator or operator team to each robot, with collaboration relying on operator-to-operator communication to manually coordinate heterogeneous platforms with incompatible interfaces.
  • This operator-dependent approach increases cognitive load and limits scalability across robots from different manufacturers.
  • This study designs and evaluates a shared-visualisation collaborative teleoperation interface to improve operator coordination, situational awareness, and the task performance in multi-operator, multi-robot systems.
  • A comparative user study with 16 participants compared three teleoperation conditions: co-located operators with a shared interface, remote operators with a shared interface, and co-located operators with individual interfaces.

Methodology

To evaluate the proposed collaborative interface for robot teleoperation, a comparative user study was conducted under three different conditions (Fig. 2 ). The objective was to assess usability, cognitive load, and task performance. Tasks were modelled on realistic disaster response scenarios such as radiation detection and search and rescue, which required collaboration between operator pairs.

Details of the simulation and the tasks are explained in Sec IV .

Methodology — Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments
Fig. 1 : Illustration of the collaborative teleoperation interface

Digital Twin Simulation Implementation

The environment is a digital twin of the Zwentendorf Nuclear Power Plant (NPP) in Austria, developed by the authors as part of the RAICAM team during the 5th European Robotics Hackathon using video and point cloud data. For the current research, the digital twin was extended with mobile robots, robot control and interaction algorithms, a collaborative User Interface (UI), and task scenarios.

Further details on the hackathon participation and digital twin creation process can be found in [ 10 ] .

Digital Twin Simulation Implementation — Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments
Fig. 2 : Three conditions used during the user studies

Hypotheses

Building on research in collaborative HRI, three hypotheses are formulated: Participants in condition C1 (co-located, shared interface) will have lower cognitive load and better performance than condition C2 (remote, shared interface). Participants in condition C1 (co-located, shared interface) will have lower cognitive load and better performance than condition C3 (co-located, individual interfaces).

Hypotheses — Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments
Fig. 3 : Ground-Floor Map of the Nuclear Plant.

Results

All eight operator pairs completed the three experimental conditions. Unless otherwise stated, condition effects were first tested using one-way repeated-measures ANOVAs with condition as the within-subject factor. For task-performance and communication outcomes, the operator pair was treated as the unit of analysis; for questionnaire and physiological outcomes, the operator was treated as the unit of analysis.

Results — Design and Evaluation of a Collaborative Teleoperation Interface for Multi-Operator Multi-Robot Systems in Challenging Environments
Fig. 4 : Mobile robots in initial positions inside the safe zone.

Sources and Demonstrations

Figures are reproduced from Kenan et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
arXiv preprint

DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation

Yunchao Yao, Zhuxiu Xu, Tianqi Zhang et al.

Building general-purpose dexterous manipulation policies requires benchmarks that go beyond isolated tasks to systematically evaluate policies across diverse interaction modes, sensory conditions, and robot embodiments.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
PDF Advanced