iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains

Vincenzo Guarino, Emanuele Musumeci, Vincenzo Suriani, Daniele Nardi · N/A · 2026

Summary

Abstract Summary

Agentic AI based on Large Language Model generalization capabilities offers a wide range of potential applications, including planning for embodied tasks. For example, embodied agents based on Foundation models can generate plausible plans in autonomous robotics scenarios. Due to limited context windows or hallucinatory phenomena in the next-token prediction formulation, behaviors may be generated without establishing whether the deployed robot and the observed environment actually support the requested operation, in what we call a "grounding failure". Thanks to the recent improvements in reasoning capabilities of foundation models, autonomous robot behavior generation problem can be formulated as a code generation problem. We present iAm.md, a Markdown standard and generation framework, that allows anchoring this process in complementary forms of deployment evidence. Through open-vocabulary semantic mapping, we combine local vision-language detections and object segmentation and refer them to persistent object records in this intermediate standardized representation, allowing agentic introspection. We then study this new technique on a simulated TIAGo, on navigation-and-manipulation tasks, showing how this standardized representation jointly supports skill self-assessment and executable task generalization.

Key Points

Agentic AI based on Large Language Model generalization capabilities offers a wide range of potential applications, including planning for embodied tasks.
For example, embodied agents based on Foundation models can generate plausible plans in autonomous robotics scenarios.
Due to limited context windows or hallucinatory phenomena in the next-token prediction formulation, behaviors may be generated without establishing whether the deployed robot and the observed environment actually support the requested operation, in what we call a "grounding failure".
Thanks to the recent improvements in reasoning capabilities of foundation models, autonomous robot behavior generation problem can be formulated as a code generation problem.
We present iAm.md, a Markdown standard and generation framework, that allows anchoring this process in complementary forms of deployment evidence.

Methodology

An embodied code generation system places a tool-calling foundation-model agent between a task request and the deployed robot. Persistent robot facts enter the agent context through iAm.md , whereas the current pose and camera observation describe the state in which the request must be executed. One general command tool connects this context to the live ROS 2 environment Macenski et al.

(2022) , allowing the agent to observe the execution state, write executable programs, and run them in the deployment that supplied the supporting evidence.

Methodology — iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains — Figure 1: System architecture of the robot-description generation and embodied-agent execution stages. The generation agent extracts resolved robot evidence from the ROS 2 environment and records it in iAm.md . During task execution, the embodied agent combines this description with the semantic map and NL request, produces code for successive subgoals, and uses execution outcomes for introspective revision.

Experimental Results

Twelve objects in a simulated scene were independently annotated as labelled spheres in the map frame through an interactive RViz interface.

An estimated record is compatible with an annotation when its centroid lies inside the sphere and its label matches; a maximum one-to-one bipartite matching over compatible pairs yields object-level Found (fraction of annotations matched) and Correct (fraction of estimates matched) scores, following Hydra Hughes et al. (2022) .

Experimental Results — iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains — Table 1: (A) Phase 1 construction with and without iAm.md ; the condition without the document reports the effort before the 110-minute budget was reached. (B) The documented agent across construction (P1) and reuse (P2).

Sources and Demonstrations

Figures are reproduced from Guarino et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.