We are excited to share that our survey “Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis” has been accepted by the IEEE Open Journal of Intelligent Transportation Systems!
This work is a joint effort between multiple institutions and industry partners including Technical University of Munich (TUM), Stanford University, Nvidia, Austrian Institute of Technology (AIT), Audi AG, Technical University of Darmstadt, fortiss GmbH and University of the Bundeswehr Munich.
🚗🔬 We provide a comprehensive overview of how Foundation Models (FMs), including Large Language Models (LLMs), Vision–Language Models (VLMs), Multimodal LLMs (MLLMs), Diffusion Models (DMs), and World Models (WMs), are reshaping scenario generation and scenario analysis in autonomous driving.
In total, we systematically review 348 papers, including
- 93 on scenario generation
- 56 on scenario analysis
- 58 datasets
- 21 simulators
- 25 benchmark challenges.
Key highlights of the paper:
- The first systematic survey exclusively dedicated to FM-based scenario generation & scenario analysis
- A unified taxonomy across model types, inputs, tasks, and evaluation metrics
- A comprehensive review of public datasets, simulation platforms, and benchmark challenges
- Identification of key challenges and promising future research directions
📄 IEEE (Early Access): https://ieeexplore.ieee.org/document/11370877
💻 GitHub Repository: https://github.com/TUM-AVS/FM-AD-Survey
🎥 Introduction Video: www.youtube.com/watch