Bringing Autonomous Driving  RL to OpenEnv and TRL

CARLA, a 3D autonomous driving simulator built on Unreal Engine 5.5, has been integrated into OpenEnv to facilitate reinforcement learning (RL) training for language models (LLMs) and vision-language models (VLMs) in autonomous driving scenarios. This project allows models to interact with a realistic virtual environment where decision-making is tested through scenarios such as the trolley problem and maze navigation, using tools like observe, lane_change, and emergency_stop. Enhancements in OpenEnv include vision support, free-roam navigation, and rubric-based rewards, enabling more nuanced RL training. The simulator requires substantial computational resources, typically utilizing NVIDIA Tesla T4 GPUs via HF Spaces, but it can also be deployed on personal infrastructure. The project demonstrates the potential for LLMs to learn driving strategies through text descriptions and camera images, with successful training showcased in scenarios like the trolley_micro_escape_exists, where models learn to safely navigate around pedestrians.