Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

|Authors: Zikui Cai, Kaushal Janga, Tan Dat Dao, Seungjae Lee, Shivin Dass, Mingyo Seo, Kaiyu Yue, Mintong Kang, Nandhu Pillai, Monte Hoover, Aadi Palnitkar, Ruchit Rawal, Ruijie Zheng, Bo Li, Yuke Zhu, Roberto Martín-Martín, Tom Goldstein, Furong Huang

|Venue: arXiv preprint | Year: 2026

|arXiv: 2607.21571v1

Abstract

Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. However, real-world robots operate continuously and must accumulate, retain, and selectively reuse information acquired from prior interactions. Despite this practical requirement, the architectural mechanisms needed to support sequential memory in EQA remain underexplored. In this work, we investigate how different memory architectures behave when EQA agents are evaluated sequentially, with multiple questions answered in the same scene while memory is carried forward across queries. We find that simply preserving existing memory is often insufficient. Agents that retain only traversability information, such as 2D occupancy maps, remember where the robot has explored but not the visual-semantic evidence needed for later questions. Agents trained on short-horizon episodic data face a different challenge: when exposed to continuous, multi-query histories, their inherited context suffers from severe temporal mismatch, rather than forming a reusable scene representation. To overcome this architectural bottleneck, we highlight the necessity of structured, spatially grounded memory: architectures that map persistent visual observations onto metric 3D geometry preserve visual-semantic evidence in a coherent scene representation. Extensive experiments in simulated environments reveal that this form of memory breaks the accuracy-efficiency tradeoff in sequential settings, simultaneously achieving higher answer accuracy and lower navigation costs. We further validate these findings on a real-world mobile robot, demonstrating that spatially grounded visual memory is critical for enabling continuous, intelligent operation in physical environments.

Key Contributions

• Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where…
• However, real-world robots operate continuously and must accumulate, retain, and selectively reus…
• Despite this practical requirement, the architectural mechanisms needed to support sequential mem…
• In this work, we investigate how different memory architectures behave when EQA agents are evalua…
• We find that simply preserving existing memory is often insufficient

Topics

• vision
• reinforcement-learning
• planning
• benchmark

Code & Data

No code repository linked in paper metadata.

BibTeX

@article{Cai2026_260721571v1,
  title     = {Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering},
  author    = {Zikui Cai and Kaushal Janga and Tan Dat Dao and Seungjae Lee and Shivin Dass and Mingyo Seo and Kaiyu Yue and Mintong Kang and Nandhu Pillai and Monte Hoover and Aadi Palnitkar and Ruchit Rawal and Ruijie Zheng and Bo Li and Yuke Zhu and Roberto Martín-Martín and Tom Goldstein and Furong Huang},
  year      = {2026},
  eprint    = {2607.21571v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2607.21571v1}
}