Evidence-Gated Task and Motion Planning with Vision-Language Models

Tsunehiko Tanaka, Matthew Stephenson, Alistair Macvicar, Edgar Simo-Serra · N/A · 2026

Summary

Abstract Summary

Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. However, under partial observability, the availability of goal-relevant objects may be uncertain. In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) may generate subgoals that rely on the VLM's prior knowledge without observational support, leading to execution failures or unintended outcomes. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual evidence through VLM-generated exploratory subgoals and TAMP-based execution. EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire further evidence, or halt. Our experiments show that, in cooking tasks with ambiguous object use, EAFG improves recipe completion by discovering task-relevant objects before planning. For instructions requiring an absent object, EAFG promotes appropriate halt decisions and reduces repeated attempts to manipulate that object.

Key Points

Robots executing long-horizon manipulation tasks from natural-language instructions must reason a...
However, under partial observability, the availability of goal-relevant objects may be uncertain
In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Plannin...
We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual e...
EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire fur...

Evidence-Gated Task and Motion Planning with Vision-Language Models

|Authors: Tsunehiko Tanaka, Matthew Stephenson, Alistair Macvicar, Edgar Simo-Serra

|Venue: arXiv preprint | Year: 2026

|arXiv: 2608.20084v1

Abstract

Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. However, under partial observability, the availability of goal-relevant objects may be uncertain. In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) may generate subgoals that rely on the VLM’s prior knowledge without observational support, leading to execution failures or unintended outcomes. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual evidence through VLM-generated exploratory subgoals and TAMP-based execution. EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire further evidence, or halt. Our experiments show that, in cooking tasks with ambiguous object use, EAFG improves recipe completion by discovering task-relevant objects before planning. For instructions requiring an absent object, EAFG promotes appropriate halt decisions and reduces repeated attempts to manipulate that object.

Key Contributions

Robots executing long-horizon manipulation tasks from natural-language instructions must reason a…
However, under partial observability, the availability of goal-relevant objects may be uncertain
In such cases, approaches that combine Vision-Language Models (VLMs) with Task and Motion Plannin…
We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework that acquires visual e…
EAFG then applies a feasibility gate to decide whether to proceed with task planning, acquire fur…

Topics

manipulation
vision
vla
reinforcement-learning
planning

Code & Data

No code repository linked in paper metadata.

BibTeX

@article{Tanaka2026_260820084v1,
  title     = {Evidence-Gated Task and Motion Planning with Vision-Language Models},
  author    = {Tsunehiko Tanaka and Matthew Stephenson and Alistair Macvicar and Edgar Simo-Serra},
  year      = {2026},
  eprint    = {2608.20084v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2608.20084v1}
}