Robotics Research Hub
Can People Distinguish Human and AI Agency in Humanoid Teleoperation? A Preliminary Study of Agency Perception

Can People Distinguish Human and AI Agency in Humanoid Teleoperation? A Preliminary Study of Agency Perception

Xiang Li, Koya Dendo, Keigo Minamida, Yuto Nakamura, Per Ola Kristensson, Jun Rekimoto · N/A · 2026

vision reinforcement-learning control learning-from-demonstration
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Can people distinguish between human and AI agency in humanoid teleoperation?

Abstract Summary

Can people distinguish between human and AI agency in humanoid teleoperation? To explore this question, we developed \textit{Ghost-in-the-Loop}, a teleoperation framework that supports both human-operated and AI-generated control of a robot's voice, facial expressions, and gestures while maintaining a consistent embodiment. We conducted a preliminary online study ($N=50$) in which participants viewed short interaction clips generated by either a Human Operator or an AI Control and judged the perceived source of control. Results suggest that participants often struggled to distinguish between the two conditions in brief interactions. Qualitative responses indicate that judgments were primarily influenced by perceived naturalness, temporal coordination, and consistency across speech, facial expression, and gesture. These findings provide initial insights into agency perception in embodied human--AI communication and motivate future investigations of blended human--AI telepresence systems.

Key Points

  • Can people distinguish between human and AI agency in humanoid teleoperation?
  • To explore this question, we developed \textit{Ghost-in-the-Loop}, a teleoperation framework that supports both human-operated and AI-generated control of a robot's voice, facial expressions, and gestures while maintaining a consistent embodiment.
  • We conducted a preliminary online study ($N=50$) in which participants viewed short interaction clips generated by either a Human Operator or an AI Control and judged the perceived source of control.
  • Results suggest that participants often struggled to distinguish between the two conditions in brief interactions.
  • Qualitative responses indicate that judgments were primarily influenced by perceived naturalness, temporal coordination, and consistency across speech, facial expression, and gesture.

Ghost-in-the-Loop

Ghost-in-the-Loop is a humanoid teleoperation framework that supports switching between human-operated and AI-generated behavior while maintaining a consistent robotic embodiment. The system is implemented on a Unitree G1 humanoid robot equipped with a head-mounted facial display and onboard camera.

Human and AI control share the same output channels, for example, voice, facial expression, and upper-body motion, allowing both control sources to be presented through a visually consistent interface.

Ghost-in-the-Loop — Can People Distinguish Human and AI Agency in Humanoid Teleoperation? A Preliminary Study of Agency Perception
Figure 2. voice2hand : generating hand gestures from speech. A diagram of a speech-to-gesture model. Audio input is converted into a log-Mel spectrogram and fed into a generator network composed of temporal convolutional blocks with coarse and detail branches. The generator outputs left- and right-hand 6DoF motion sequences. A multi-scale discriminator evaluates the generated motion as real or fake, and a reconstruction loss is applied during training.

Preliminary Evaluation

To explore whether people can distinguish between human and AI agency in humanoid teleoperation, we conducted a preliminary online perception study using interaction clips. We recruited 50 participants through Prolific. To generate the study stimuli, we first used Ghost-in-the-Loop to record 8 short humanoid interaction clips under 2 control conditions.

In the Human Operator condition, the robot was directly controlled by a human teleoperator.

Sources and Demonstrations

Figures are reproduced from Li et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —