EnactOpt: Diagnosis-Guided Harness Evolution for Embodied Agents

ZhiYuan Feng, Zhuoshi Pan, Zhaolu Kang, Zhongxia Chen, Shuo Yang, Sen Ye, Heng Wang, Han Li · N/A · 2026

Summary

Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions.

Abstract Summary

Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions. Failures can arise not only from limited model capabilities, but also from how the harness presents evidence and guides its use. These model-external mechanisms provide an adaptation target without updating model parameters, but task-specific interaction experience can be costly to collect. Task-level outcomes alone do not reveal whether relevant evidence was missing from the model input or was available but inadequately used. Moreover, these two aspects are coupled: effective action guidance may require evidence that the current context omits, while exposing that evidence alone does not ensure an appropriate response. We introduce EnactOpt, which uses limited execution feedback to jointly evolve reusable textual skills and context-construction code while keeping the underlying models and runtime fixed. It diagnoses adjacent decisions by relating action feedback to the actual model input and subsequent action. Structured episode summaries guide joint revisions through prompt-level edit operators, and candidate harnesses undergo compatibility checks and validation-based retention. Across VLABench, ALFWorld, and Embodied Agent Interface (EAI), EnactOpt improves task success by approximately 10-27 percentage points over the initial harnesses and outperforms the evaluated GEPA, TextGrad, and Trace2Skill baselines. These gains extend to task conditions not used for harness evolution or candidate selection. The results support a complementary route to embodied adaptation: turning limited execution experience into reusable behavioral improvements by jointly refining how evidence is presented and acted upon.

Key Points

Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions.
Failures can arise not only from limited model capabilities, but also from how the harness presents evidence and guides its use.
These model-external mechanisms provide an adaptation target without updating model parameters, but task-specific interaction experience can be costly to collect.
Task-level outcomes alone do not reveal whether relevant evidence was missing from the model input or was available but inadequately used.
Moreover, these two aspects are coupled: effective action guidance may require evidence that the current context omits, while exposing that evidence alone does not ensure an appropriate response.

Problem Setup

We consider an embodied task $q$ specified by an environment $\mathcal{E}_{q}$ with initial conditions, an instruction $\ell_{q}$, and a task-level success criterion. At decision step $t$, the agent selects an action $a_{t}$ using information drawn from $\ell_{q}$, the current observation $o_{t}$, and the available interaction history $h_{t}$ of prior observations, actions, and feedback.

Executing $a_{t}$ yields the next observation $o_{t+1}$ and any available execution feedback $f_{t+1}$; interaction continues until termination.

EnactOpt: Diagnosis-Guided Harness Evolution

Figure 2 summarizes EnactOpt’s iterative evolution loop. Starting from $H_{0}=(S_{0},C_{0})$, EnactOpt maintains a pool of retained harnesses while keeping $\mathcal{M}$ and $\mathcal{R}$ fixed.

At evolution iteration $r$, a source harness $H=(S,C)$ is selected from $\mathcal{P}_{r-1}$ to execute a batch $B_{r}$ from the training set $\mathcal{D}_{\mathrm{train}}$, producing rollouts and their native execution outcomes. Each rollout is analyzed through adjacent-decision diagnoses, which are aggregated into a structured episode summary.

EnactOpt: Diagnosis-Guided Harness Evolution — EnactOpt: Diagnosis-Guided Harness Evolution for Embodied Agents — Figure 2: Overview of EnactOpt. (a) A source harness $H=(S,C)$ executes training batch $B_{r}$, recording actual model inputs, actions, observations, and episode outcomes. (b) Adjacent-decision diagnoses and episode summaries form its feedback $F_{r}$. (c) A joint optimizer uses the same $H$ and $F_{r}$ to propose candidates $\mathcal{C}_{r}$ through prompt-guided structured edit operators. Compatible candidates are compared with the previous pool $\mathcal{P}_{r-1}$ on validation tasks to form $\mathcal{P}_{r}$. Scenes are illustrative, not experimental results.

Experiments

Benchmarks and protocol. We evaluate separately evolved harnesses on VLABench ( Zhang et al., 2025b ) , ALFWorld ( Shridhar et al., 2020 ) , and EAI’s VirtualHome Action Sequencing ( Li et al., 2024 ) . All MLLM components use GPT-5.4 mini ( OpenAI, 2026 ) ; VLABench additionally uses the benchmark-released $\pi_{0}$ policy ( Black et al., 2024 ) , fixed during evolution.

ALFWorld and EAI use high-level action interfaces. Train/validation/test sizes are 30/30/920 for VLABench and 30/30/282 for EAI.

Experiments — EnactOpt: Diagnosis-Guided Harness Evolution for Embodied Agents — Figure 3: Generalization on VLABench T3/T4/T6 and ALFWorld Unseen. Bars show task success rates (%); horizontal dashed lines denote the Naive baseline. Adaptation is performed separately for each benchmark. The plotted results come from the same evaluation runs as Table 1 , which reports overall success rates on the full test sets.

Sources and Demonstrations

Figures are reproduced from Feng et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.