evaluation

12 papers

Evaluation methodologies and benchmarks.

Papers

Sorted by year (newest first)
Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think
arXiv preprint

Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha et al.

Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference.

ar-vr evaluation manipulation
PDF Intermediate
Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think
arXiv preprint

Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha et al.

Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference.

ar-vr evaluation manipulation
PDF Intermediate
Invertible Neural Network Adapter for One-Step Flow Matching in Robot Manipulation
arXiv preprint

Invertible Neural Network Adapter for One-Step Flow Matching in Robot Manipulation

Yu Zhang, Kangyi Ji, Yongxiang Zou et al.

This paper presents an invertible neural network adapter for general robotic manipulation, designed to generate precise high-dimensional actions conditioned on multimodal observations, including visual, linguistic, and proprioceptive inputs, through a one-step denoising process.

ar-vr evaluation generation
PDF Intermediate
Invertible Neural Network Adapter for One-Step Flow Matching in Robot Manipulation
arXiv preprint

Invertible Neural Network Adapter for One-Step Flow Matching in Robot Manipulation

Yu Zhang, Kangyi Ji, Yongxiang Zou et al.

This paper presents an invertible neural network adapter for general robotic manipulation, designed to generate precise high-dimensional actions conditioned on multimodal observations, including visual, linguistic, and proprioceptive inputs, through a one-step denoising process.

ar-vr evaluation generation
PDF Intermediate
PhAIL: A Real-Robot VLA Benchmark and Distributional Methodology
arXiv preprint

PhAIL: A Real-Robot VLA Benchmark and Distributional Methodology

Sergey Arkhangelskiy

Real-world evaluation of vision-language-action (VLA) policies still rests on binary success rate at a fixed timeout with N <= 25 rollouts per condition, almost always without confidence intervals or

benchmarking vla manipulation evaluation
PDF Intermediate

Suggested Learning Path

Read these papers in order to build expertise in evaluation.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6

…and 6 more papers.