Robotics Research Hub
A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle

A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle

Gustavo Claudio Karl Couto, Eric Aislan Antonelo, Gabriel George Zipperer · N/A · 2026

vision reinforcement-learning sim-to-real planning control learning-from-demonstration
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

This paper presents a low-cost, open experimental platform for research in end-to-end autonomous driving with miniature Ackermann vehicles.

Abstract Summary

This paper presents a low-cost, open experimental platform for research in end-to-end autonomous driving with miniature Ackermann vehicles. The platform combines a physical vehicle, a printed urban track, data collection tools, trajectory registration, and a Webots digital twin, enabling controlled experiments that connect simulation-based autonomous-driving methods to real-world execution. As a first baseline, we implement command-conditioned behavior cloning, in which a neural policy receives an on-board camera image and a high-level navigation command and outputs steering and speed. The system is evaluated both on the physical vehicle and in simulation. In real closed-loop experiments, the learned policy follows lanes and executes commanded turns, reaching a mean cross-track error of 6.1 cm with respect to the reference route, close to the 4.7 cm observed in human demonstrations. In the digital twin, camera field of view has a strong effect on performance, reducing the mean cross-track error from 35.6 to 3.3 cm when widened from 58 to 120 degrees. Using the digital twin to generate synthetic driving data and a learned sim-to-real image translator to reduce the appearance gap, we further show that a higher-capacity policy trained on this synthetic data combined with real demonstrations is the only configuration that completes all four track routes in closed loop, whereas the compact baseline and the same network trained on real data alone complete fewer. These results establish the open platform as a practical testbed for sim-to-real studies and provide an initial command-conditioned imitation-learning baseline; we release it to support reproducible research.

Key Points

  • This paper presents a low-cost, open experimental platform for research in end-to-end autonomous driving with miniature Ackermann vehicles.
  • The platform combines a physical vehicle, a printed urban track, data collection tools, trajectory registration, and a Webots digital twin, enabling controlled experiments that connect simulation-based autonomous-driving methods to real-world execution.
  • As a first baseline, we implement command-conditioned behavior cloning, in which a neural policy receives an on-board camera image and a high-level navigation command and outputs steering and speed.
  • The system is evaluated both on the physical vehicle and in simulation
  • In real closed-loop experiments, the learned policy follows lanes and executes commanded turns, reaching a mean cross-track error of 6.1 cm with respect to the reference route, close to the 4.7 cm observed in human demonstrations.

Command-Conditioned Behavior Cloning Baseline

Given an RGB image $I_{i}$ from the on-board camera and a high-level command $c_{i}$, the policy $\pi_{\theta}(I_{i},c_{i})$ predicts a continuous steering command $\hat{\delta}_{i}$ and a signed speed command $\hat{v}_{i}$. The expert label for each frame is $a_{i}=(\delta_{i},v_{i})$, collected during teleoperation.

Both outputs are bounded to $[-1,1]$ by a $\tanh$ activation and then denormalized to the physical actuator limits.

Command-Conditioned Behavior Cloning Baseline — A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle
Figure 2: Experimental setup, split into the on-board mini-vehicle and the host PC: the ESP32-S3 micro-ROS board closes the $100\,\mathrm{Hz}$ speed PID loop and drives the servo, while a separate ESP32 camera streams images to the host PC, which runs the policy and the extended Kalman filter and returns steering and speed commands.

Policy Architectures

The policy input is an RGB image from the vehicle’s on-board camera, resized to $160\times 120$ pixels, together with the one-hot navigation command. Frames are kept in RGB channel order and scaled to $[0,1]$, with no lens undistortion applied; the higher-capacity variant instead takes $320\times 240$ frames with the top $35\%$ cropped to remove the region above the road.

Policy Architectures — A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle
Figure 3: The four ideal routes on the rectified mat used as the Webots floor texture. Each panel shows one route, driven in a single direction from the shared start marker (white circle). Numbered markers indicate the labelled intersections used for command annotation.

Experiments

The vehicle is an off-the-shelf Ackermann chassis (Yahboom ROSMASTER R2L class) with two rear drive motors, a servo-steered front axle, and a forward camera on a short mast (Fig. 2 ). Its wheelbase is $0.2354\,\mathrm{m}$, its track width $0.1911\,\mathrm{m}$, and steering is limited to $\pm 45^{\circ}$. An ESP32-S3 runs an ESP-IDF firmware with a micro-ROS client [ 2 , 11 ] .

Experiments — A Low-Cost, Open Platform for End-to-End Autonomous Driving on a Miniature Ackermann Vehicle
Figure 5: Ground-plane homography. Because the track is planar, camera intrinsics and pose define a single $3\times 3$ homography between metric map coordinates $(X,Y)$ and image pixels $(u,v)$. The projection tool uses this mapping, or its inverse, to relate on-board camera observations to the rectified map; the figure illustrates the geometry in the Webots digital twin. The blue and green markers are the same physical points on the track and in the camera image.

Sources and Demonstrations

Figures are reproduced from Couto et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Dexterity from Supervised Play
arXiv preprint

Dexterity from Supervised Play

Botao He, Yiming Dou, Yiannis Aloimonos et al.

How can robots autonomously collect the contact-rich experience needed to learn dexterous manipulation?

manipulation reinforcement-learning sim-to-real planning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —