Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation

Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation

|Authors: Yu Qi, Zhang Ye, Xinyi Xu, Yuxuan Lu, Amitoj Sandhu, Boce Hu, Haojie Huang, Jonathan Tremblay, Lawson L. S. Wong

|Venue: arXiv preprint | Year: 2026

|arXiv: 2607.21582v1

Abstract

Compositional generalization is essential for robot to follow diverse instructions. However, pretrained policies are known to take shortcuts, deferring to salient cues rather than grounding language. We introduce a diagnostic framework that localizes this failure to individual \textit{instruction factors}, \textit{e.g.,} reusable semantic components such as color, verb, object, size, and spatial attribute. Our framework formalizes instruction factor bias, the tendency of fine-tuned policies to over-rely on dominant factors as shortcuts, and quantifies it through two metrics: Factor Dominance Rate (FDR), capturing pairwise bias between factors, and Factor Dominance Hierarchy (FDH), aggregating these into a global ranking. Evaluation on six foundation policies reveals broadly consistent ordering, \textit{i.e.}, color $\geq$ object $\geq$ spatial $\geq$ verb $\geq$ size, with color dominant, and verb and size most under-grounded. We further show the diagnosis is actionable: a bias-aware data collection strategy that reallocates a fixed budget toward under-grounded factors outperforms baselines in simulation and on a real robot using half the demonstrations, thereby enabling more sample-efficient and generalizable policy learning.

Key Contributions

• Compositional generalization is essential for robot to follow diverse instructions
• However, pretrained policies are known to take shortcuts, deferring to salient cues rather than g…
• We introduce a diagnostic framework that localizes this failure to individual \textit{instruction…
• Our framework formalizes instruction factor bias, the tendency of fine-tuned policies to over-rel…
• Evaluation on six foundation policies reveals broadly consistent ordering, \textit{i.e.}, color $…

Topics

• manipulation
• reinforcement-learning
• learning-from-demonstration
• benchmark

Code & Data

No code repository linked in paper metadata.

BibTeX

@article{Qi2026_260721582v1,
  title     = {Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation},
  author    = {Yu Qi and Zhang Ye and Xinyi Xu and Yuxuan Lu and Amitoj Sandhu and Boce Hu and Haojie Huang and Jonathan Tremblay and Lawson L. S. Wong},
  year      = {2026},
  eprint    = {2607.21582v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2607.21582v1}
}