LLM Robotics

4 papers

Large language models integrated into robotic reasoning and planning.

vla llm-robotics foundation-models manipulation imitation-learning planning zero-shot

Papers

Sorted by year (newest first)

OpenVLA: An Open-Source Vision-Language-Action Model

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti et al. · arXiv · Jun 2024

A 7B-parameter open-source Vision-Language-Action model pre-trained on 970k real-world robot demonstrations, achieving strong generalization across robots and tasks.

vla llm-robotics foundation-models manipulation

Code PDF Intermediate

GitHub ★ 6,260 Code updated: May 2026

RT-2: Vision-Language-Action Models That Generalize to Novel Tasks

Anthony Brohan, Noah Brown, Justice Carbajal et al. · ICRA 2024 · May 2024

Google DeepMind's VLA model combining a vision-language foundation model with robot action outputs, showing emergent generalization to novel objects, backgrounds, and semantic instructions far beyond training data.

llm-robotics foundation-models manipulation

Code Advanced

GitHub ★ 1,853 Code updated: May 2026

RT-1: Robotics Transformer for Real-World Control at Scale

Anthony Brohan, Noah Brown, Justice Carbajal et al. · RSS · Jul 2023

A large transformer model trained on 130k episodes of real robot manipulation to output discretized arm-and-gripper actions from RGB images and natural language instructions.

llm-robotics foundation-models imitation-learning

Code PDF Advanced

GitHub ★ 1,723 Code updated: May 2026

VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

Wenlong Huang, Chen Wang, Ruohan Zhang et al. · CoRL 2023 · Jul 2023

VoxPoser uses LLMs to compose 3D value maps from free-form language instructions, enabling zero-shot closed-loop manipulation planning with model-based trajectory synthesis over dynamic environments.

llm-robotics manipulation planning zero-shot

Code PDF Intermediate

GitHub ★ 803 Code updated: May 2026

Suggested Learning Path

Read these papers in order to build expertise in LLM Robotics.

1
OpenVLA: An Open-Source Vision-Language-Action Model
Moo Jin Kim, Karl Pertsch et al. · 2024
2
RT-2: Vision-Language-Action Models That Generalize to Novel Tasks
Anthony Brohan, Noah Brown et al. · 2024
3
RT-1: Robotics Transformer for Real-World Control at Scale
Anthony Brohan, Noah Brown et al. · 2023
4
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models
Wenlong Huang, Chen Wang et al. · 2023