We develop integrated autonomy software that uses modern AI methods and algorithms—including foundation models, reinforcement learning, and physics-informed neural networks—to enable mobile robots to perceive, reason, plan, and control their motion in complex and interactive environments.
This theme investigates how artificial intelligence can be integrated throughout the autonomy software stack, from multimodal perception and semantic world modeling to behavior reasoning, motion planning, and vehicle control. The work combines foundation and vision-language models, video world models, diffusion methods, reinforcement and imitation learning, differentiable optimization, model-structured neural networks, and classical planning and control. A central scientific question is how learned representations and policies can be connected to physical constraints, real-time computation, uncertainty, and safety requirements
PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
Y. Gao, S. Müller, M. Piccinini, M. Kaufeld, Y. Zhang, F. R. Schäfer, Q. Song, J. Betz
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026
doi: , PDF, Website, Code, Dataset
GaussianFusionOcc: A Seamless Sensor Fusion Approach for 3D Occupancy Prediction Using 3D Gaussians
T. Pavkovic, M. Mahani, J. Niedermayer, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2026
doi: , PDF
Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving
Y. Gao, W. Miao, M. Piccinini, H. Wang, Q. Song, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2026
doi: , PDF, Website, Code, Dataset
G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance
H. Yu, Y. Jin, A. Canevaro, J. Schmidt, J. Jordan, P. Li, M. Kaufeld, S. Lindner, J. Betz, W. Stork
IEEE International Conference on Intelligent Robots and Systems (IROS), 2026
doi: , PDF,
EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving
F. R. Schäfer, Y. Gao, D. Wang, T. Stauner, S. Günnemann, M. Piccinini, S. Schmidt, and J. Betz
European Conference on Computer Vision (ECCV), 2026
doi: , Website, Code
Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?
D. Wang, Z. Liang, H. Ye, Z. Sun, Z. Lu, Y. Zhang, Y. Gao, M. Seegert, F. Schäfer, H. Qin, W. Lei, L. Palmieri, F. Jahncke, M. Piccinini, J. Betz
European Conference on Computer Vision (ECCV), 2026
doi: , PDF, Website, Code, Dataset
Learning to Sample: Reinforcement Learning-Guided Sampling for Autonomous Vehicle Motion Planning
K. Moller, R. Stroop, M. Piccinini, A. Langmann, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Code, Video
Can VLMs Unlock Semantic Anomaly Detection? A Framework for Structured Reasoning
R. Brusnicki, D. Pop, Y. Gao, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Website, Dataset
How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study
R. Brusnicki, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF
Talk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecars
D. Henel, F. Werner, A. Langmann, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF
Benchmarking Empirical and Learning-Based Approaches for Feedforward Steering Control in Autonomous Racing
G. Jank, M. Piccinnini, S. Wenk. P. Pitschi, J. Betz, B. Lohmann
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF
Vehicle Dynamics Learning from Physics Priors with Model-Structured Neural Networks
A. Mugello, F. Jahncke, S. Santini, J. Betz, G. Papini, M. Piccinini
IEEE Open Journal of Intelligent Transportation System (OJ-ITS), 2026
doi: 10.1109/OJITS.2026.3685078, PDF, Code, Video
Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation
A. Zhang, H. Chen, Y. Burkhardt, Y. Zhong, J. Betz, H. Oleynikova, S. Leutenegger
IEEE International Conference on Robotics and Automation (ICRA) 2026
doi: , PDF, Code, Video
NuRisk: A Visual Question Answering Dataset for Agent-Level Risk Assessment in Autonomous Driving
Y. Gao, M. Piccinini, R. Brusnicki, Y. Zhang, J. Betz
IEEE International Conference on Robotics and Automation (ICRA) 2026
doi: , PDF, Website, Code, Dataset
Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis
Y. Gao, M. Piccinini, Y. Zhang, D. Wang, K. Moller, R. Brusnicki, B. Zarrouki, A. Gambi, J. Totz, K. Storms, S. Peters, A. Stocco, B. Alrifaee, M. Pavone, J. Betz
IEEE Open Journal of Intelligent Transportation Systems (OJ-ITS), 2026
doi: 10.1109/OJITS.2026.3660686, PDF, Code,Video
Beyond Known Objects: A Novel Framework for Open-Set Object Detection using Negative-Aware Norm
Y. Zhang, Y. Lu, J. Betz
IEEE Intelligent Vehicles Symposium (IV), 2026
doi: , PDF, Code, Video
Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Driving
M. Seegert, K. Moller, J. Betz
IEEE Intelligent Vehicles Symposium (IV), 2026
doi: , PDF, Code, Video
Differentiable Weights-Varying Nonlinear MPC via Gradient-based Policy Learning: An Autonomous Vehicle Guidance Example
F. Jahncke, B. Zarrouki, M. Piccinini, S. Bae, J. D'sa, D. Isele, J. Betz
IEEE Robotics and Automation Letters, 2026
doi: 10.1109/LRA.2026.3662644, PDF, Website, Code, Video
Data-driven reachability analysis for piecewise affine systems
P. Xie, J. Betz, D. Raimondo, A. Alanwar
IEEE Conference on Decision and Control (CDC), 2025
doi: 10.1109/CDC57313.2025.11312136, PDF
Unsupervised Learning for Detection of Rare Driving Scenarios
L. Dat, T. Manhardt, M. Venator, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423214 , PDF
Model-Structured Neural Networks to Control the Steering Dynamics of Autonomous Race Cars
M. Piccinini, A. Mungiello, G. Jank, GPR. Papini, F. Biral, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423721, PDF, Video
A Learning-based Planning and Control Framework for Inertia Drift Vehicles
B. Zhou, Z. Li, L. Xie, H. Su, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423578 , PDF
MP-RBFN: Learning-based Vehicle Motion Primitives using Radial Basis Function Networks
M. Kaufeld, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423753 , PDF, Code, Video
From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
Y. Gao, M. Piccinini, K. Moller, A. Alanwar, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423486, PDF, Code, Video
DualAD: Dual-Layer Planning for Reasoning in Autonomous Driving
D. Wang, M. Kaufeld, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2025
doi: 10.1109/IROS60139.2025.11246084, PDF, Code, Video
Safe Reinforcement Learning with a Predictive Safety Filter for Motion Planning and Control: A Drifting Vehicle Example
B. Zhou, B. Zarrouki, M. Piccinini, C. Hu, L. Xie, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2025
doi: 10.1109/IROS60139.2025.11246272, PDF, Video
MIND-Stack: Modular, Interpretable, End-To-End Differentiability for Autonomous Navigation
F. Jahncke, J. Betz
IEEE Intelligent Vehicles Symposium (IV), 2025
doi: 10.1109/IV64158.2025.11097814 , PDF, Video, Code
Adaptive learning-based model predictive control strategy for drift vehicles
B. Zhou, C. Hu, J. Zeng, Z. Li, J. Betz, L. Xie, H. Su
Robotics and Autonomous Systems, vol. 188. Elsevier BV, p. 104941, 2025
doi: 10.1016/j.robot.2025.104941, PDF
A safe reinforcement learning driven weights-varying model predictive control for autonomous vehicle motion control
B. Zarrouki, M. Spanakakis, J. Betz
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588747, PDF, Video, Code
A reinforcement learning-boosted motion planning framework: Comprehensive generalization performance in autonomous driving
R. Trauth, A. Hobmeier, J. Betz
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588750, PDF, Code
R2NMPC: A Real-Time Reduced Robustified Nonlinear Model Predictive Control with Ellipsoidal Uncertainty Sets for Autonomous Vehicle Motion Control
B. Zarrouki, J. Nunes, J. Betz
8th IFAC Conference on Nonlinear Model Predictive Control (NMPC 2024)
doi: https://doi.org/10.1016/j.ifacol.2024.09.048 , Paper, Video, Code
Adaptive Stochastic Nonlinear Model Predictive Control with Look-ahead Deep Reinforcement Learning for Autonomous Vehicle Motion Control
B. Zarrouki, C. Wang, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS) 2024
doi: 10.1109/IROS58592.2024.10801876, Paper, Video, Code
A Stochastic Nonlinear Model Predictive Control with an Uncertainty Propagation Horizon for Autonomous Vehicle Motion Control
B. Zarrouki, C. Wang, J. Betz
IEEE American Control Conference (ACC) 2024
doi: 10.23919/ACC60939.2024.10645032 , Paper, Video, Code
ESP: Extro-Spective Prediction for Long-term Behavior Reasoning in Emergency Scenarios
D. Wang, L. Zheyuan, Y. Li, Y. Wu, Y. Ma, J. Betz, R. Yang, W. Li
IEEE International Conference on Robotics and Automation (ICRA) 2024
doi: 10.1109/ICRA57147.2024.10610002, Paper, Video, Project Website, Dataset
Bypassing the Simulation-to-Reality Gap: Online Reinforcement Learning Using a Supervisor
B. D. Evans, J. Betz, H. Zheng, H. A. Engelbrecht, R. Mangharam, and H. W. Jordaan
IEEE 21st International Conference on Advanced Robotics (ICAR)
doi: 10.1109/icar58858.2023.10406465, Paper, Video, Code
CamRaDepth: Semantic Guided Depth Estimation Using Monocular Camera and Sparse Radar for Automotive Perception
F. Sauerbeck, D. Halperin, L. Connert, J. Betz
IEEE Sensors Journal, pp. 1–1, 2023
doi: 10.1109/jsen.2023.3321886, PDF, Video
Learn to See Fast: Lessons Learned from Autonomous Racing on How to Develop Perception Systems
F. Sauerbeck, S. Huch, F. Fent, P. Karle, D. Kulmer, J. Betz
IEEE Access, pp. 1–1, 2023
doi: 10.1109/access.2023.3272750, PDF
DeepSTEP - Deep Learning-Based Spatio-Temporal End-To-End Perception for Autonomous Vehicles
S. Huch, F. Sauerbeck, J. Betz
IEEE Intelligent Vehicles Symposim (IV), June 2023
doi: 10.1109/IV55152.2023.10186768, PDF, Video
A Benchmark Comparison of Imitation Learning-Based Control Policies for Autonomous Racing
X. Sun, M. Zhou, Z. Zhuang, S. Yang, J. Betz, R. Mangharam
IEEE Intelligent Vehicles Symposim (IV), June 2023
doi: 10.1109/IV55152.2023.10186780, PDF
Learning and Adapting Behavior of Autonomous Vehicles Through Inverse Reinforcement Learning
R. Trauth, M. Kaufeld, M. Geisslinger, J. Betz
IEEE Intelligent Vehicles Symposim (IV), June 2023
doi: 10.1109/IV55152.2023.10186668, PDF
RGB-L: Enhancing Indirect Visual SLAM using LiDAR-based Dense Depth Map
F. Sauerbeck, B. Obermeier, M. Rudolph, J. Betz
IEEE 3rd International Conference on Computer, Control and Robotics (ICCCR), March 2023
doi: 10.1109/icccr56747.2023.10194045, PDF
Local_INN: Implicit Map Representation and Localization with Invertible Neural Networks
Z. Zang, H. Zheng, J. Betz, R. Mangharam
2023 IEEE International Conference on Robotics and Automation (ICRA), May 2023
doi: 10.1109/ICRA48891.2023.10161015, PDF, Video
Combinatorial and Parametric Gradient-Free Optimization for Cyber-Physical System Design
H. Zheng, J. Betz, A. Ramamurthy, H. Jin, and R. Mangharam
2022 IEEE Workshop on Design Automation for CPS and IoT (DESTION), May 2022
doi: 10.1109/destion56136.2022.00012, PDF
Scenario Understanding and Motion Prediction for Autonomous Vehicles—Review and Comparison
P. Karle, M. Geisslinger, J. Betz, and M. Lienkamp,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10., pp. 16962–16982, Oct. 2022
doi: 10.1109/tits.2022.3156011, PDF
A Combined LiDAR-Camera Localization for Autonomous Race Cars
F. Sauerbeck, L. Baierlein, J. Betz, M. Lienkamp
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 1, Jan. 2022
doi: 10.4271/12-05-01-0006, PDF
Watch-and-Learn-Net: Self-supervised Online Learning for Probabilistic Vehicle Trajectory Prediction
M. Geisslinger, P. Karle, J. Betz, and M. Lienkamp, “
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021.
doi: 10.1109/smc52423.2021.9659079, PDF
Radar Voxel Fusion for 3D Object Detection
F. Nobis, E. Shafiei, P. Karle, J. Betz, M. Lienkamp
Applied Sciences, vol. 11, no. 12, p. 5598, 2021
doi: 10.3390/app11125598, PDF
Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation
F. Nobis, F. Fent, J. Betz, and M. Lienkamp
Applied Sciences, vol. 11, no. 6, p. 2599, 2021
doi: 10.3390/app11062599, PDF
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
S. Huch, A. Ongel, J. Betz, and M. Lienkamp
Sensors, vol. 21, no. 4, p. 1039, 2021
doi: 10.3390/s21041039, PDF
Virtual Strategy Engineer: Using Artificial Neural Networks for Making Race Strategy Decisions in Circuit Motorsport
A. Heilmeier, A. Thomaser, M. Graf, J. Betz
Applied Sciences, vol. 10, no. 21, p. 7805, Nov. 2020
doi: 10.3390/app10217805, PDF
End-to-End Neural Network for Vehicle Dynamics Modeling – Best Paper Award
L. Hermansdorfer, R. Trauth, J. Betz, M. Lienkamp
3rd IEEE Conference on Optimization and Modeling of Complex Systems, Agadir, Morocco, December 2020, 2020
doi: 10.1109/cist49399.2021.9357196, PDF
Exploring the Capabilities and Limits of 3D Monocular Object Detection – A Study on Simulation and Real World Data
F. Nobis, F. Brunhuber, S. Janssen, J. Betz, and M. Lienkamp
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294625, PDF
Persistent Map Saving for Visual Localization for Autonomous Vehicles: An ORB-SLAM 2 Extension
F. Nobis, O. Papanikolaou, J. Betz, and M. Lienkamp
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243094, PDF
A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection
F. Nobis, M. Geisslinger, M. Weber, J. Betz, and M. Lienkamp
2019 Sensor Data Fusion: Trends, Solutions, Applications (SDF), 2019
doi: 10.1109/SDF.2019.8916629, PDF
A Model-Free Algorithm to Safely Approach the Handling Limit of an Autonomous Racecar
A. Wischnewski, J. Betz, and B. Lohmann
2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), 2019
doi: 10.1109/ICCVE45908.2019.8965218, PDF
We develop motion planning, reasoning & decision-making algorithms that explicitly recognize uncertainty, anticipate hazards, and preserve safety in rare, occluded, interactive, and previously unseen situations.
In this research theme, we investigate how autonomy can remain dependable when perception is incomplete, other agents behave unpredictably, models are uncertain, or the system approaches failure. We work on probabilistic collision prediction, risk-aware planning, occlusion-aware reasoning, pedestrian-aware navigation, and online safeguarding. Increasingly, language and foundation models are used not merely for perception but to construct, analyze, and challenge safety-critical scenarios. The scientific significance lies in moving beyond average-case performance toward autonomy whose limitations are measurable, whose decisions account for uncertainty, and whose behavior can be systematically evaluated before and during real-world deployment.
PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
Y. Gao, S. Müller, M. Piccinini, M. Kaufeld, Y. Zhang, F. R. Schäfer, Q. Song, J. Betz
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026
doi: , PDF, Website, Code, Dataset
Chat2Scenic: An Iterative RAG-Based Framework for Scenario Generation in Autonomous Driving
Y. Gao, W. Miao, M. Piccinini, H. Wang, Q. Song, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2026
doi: , PDF, Website, Code, Dataset
A2RL Vmax: The A2RL autonomous racing dataset for long-range, high-speed perception and multi-vehicle interaction
M. Klemp, D. Ebner, C. Schröder, D. Malvezzi, L. Turanyi, R. Donati, I. Schminik, Y. Ning, Y. Zhou, M. Flagg, C. Stiller, M. Lienkamp, M. Bertogna, G. Bari, A. Birk, R. Jin, C. Lv, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2026
doi: , PDF, Website, Dataset, Video
G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance
H. Yu, Y. Jin, A. Canevaro, J. Schmidt, J. Jordan, P. Li, M. Kaufeld, S. Lindner, J. Betz, W. Stork
IEEE International Conference on Intelligent Robots and Systems (IROS), 2026
doi: , PDF,
EgoDyn-Bench: Evaluating Ego-Motion Understanding in Vision-Centric Foundation Models for Autonomous Driving
F. R. Schäfer, Y. Gao, D. Wang, T. Stauner, S. Günnemann, M. Piccinini, S. Schmidt, and J. Betz
European Conference on Computer Vision (ECCV), 2026
doi: , Website, Code
A Hybrid Sampling-Based Trajectory Planner with Game-Theoretic Guidance for Autonomous Racing
A. Langmann, F. de Araujo, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Code, Video
Learning to Sample: Reinforcement Learning-Guided Sampling for Autonomous Vehicle Motion Planning
K. Moller, R. Stroop, M. Piccinini, A. Langmann, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Code, Video
Real-Time Minimal Risk Maneuver Planning for Autonomous Driving on Embedded Hardware
K. Moller, L. Jürgens, M. Seegert, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Code, Video
Can VLMs Unlock Semantic Anomaly Detection? A Framework for Structured Reasoning
R. Brusnicki, D. Pop, Y. Gao, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Website, Dataset
How Well Do Vision-Language Models Understand Sequential Driving Scenes? A Sensitivity Study
R. Brusnicki, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF
Risk-Aware Motion Planning with Learned Trajectory Primitives and Probabilistic Safety
M. Kaufeld, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2026
doi: , PDF, Code, Video
SR-VIO: Leveraging Structural Regularity for Robust Initialization and Accurate Visual-Inertial Odometry
L. Zhang, H. Yin, W. Ye, H. Zhang, J. Betz
IEEE/ASME Transactions on Mechatronics
doi: 10.1109/TMECH.2026.3669015, PDF
Actron3D: Learning Actionable Neural Functions from Videos for Transferable Robotic Manipulation
A. Zhang, H. Chen, Y. Burkhardt, Y. Zhong, J. Betz, H. Oleynikova, S. Leutenegger
IEEE International Conference on Robotics and Automation (ICRA) 2026
doi: , PDF, Code, Video
NuRisk: A Visual Question Answering Dataset for Agent-Level Risk Assessment in Autonomous Driving
Y. Gao, M. Piccinini, R. Brusnicki, Y. Zhang, J. Betz
IEEE International Conference on Robotics and Automation (ICRA) 2026
doi: , PDF, Website, Code, Dataset
Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysis
Y. Gao, M. Piccinini, Y. Zhang, D. Wang, K. Moller, R. Brusnicki, B. Zarrouki, A. Gambi, J. Totz, K. Storms, S. Peters, A. Stocco, B. Alrifaee, M. Pavone, J. Betz
IEEE Open Journal of Intelligent Transportation Systems (OJ-ITS), 2026
doi: 10.1109/OJITS.2026.3660686, PDF, Code,Video
Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Driving
M. Seegert, K. Moller, J. Betz
IEEE Intelligent Vehicles Symposium (IV), 2026
doi: , PDF, Code, Video
Pedestrian-Aware Motion Planning for Autonomous Driving in Complex Urban Scenarios
K. Moller, T. Nyberg, J. Tumova, J. Betz
IEEE Open Journal of Intelligent Transportation Systems (OJ-ITS), 2026
doi: 10.1109/OJITS.2026.3655468, PDF, Code, Video
RoGER-SLAM: A Robust Gaussian Splatting SLAM System for Noisy and Low-light Environment Resilience
H. Yin, Z. Yang, L. Zhang, G. Rigoll, J. Betz
IEEE Transactions on Instrumentation & Measurement, 2026
doi: 10.1109/TIM.2026.3657530 , PDF, Video
Towards Safe Autonomous Driving: A Real-Time Safeguarding Concept for Motion Planning Algorithms
K. Moller, R. Neher, M. Seegert, J. Betz
IEEE International Conference on Vehicular Electronics and Safety (ICVES), 2025
doi: 10.1109/ICVES65691.2025.11376522, PDF, Code, Video
HSR-VINS: Hybrid Structural Regularities for Robust Visual-Inertial Navigation Systems
H. Yin, P. Zhen, L. Zhang, J. Betz
IEEE Transactions on Aerospace and Electronic Systems
doi: 10.1109/TAES.2025.3623951, PDF
Unsupervised Learning for Detection of Rare Driving Scenarios
L. Dat, T. Manhardt, M. Venator, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423214 , PDF
MP-RBFN: Learning-based Vehicle Motion Primitives using Radial Basis Function Networks
M. Kaufeld, M. Piccinini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423753 , PDF, Code, Video
MultiDrive: A Co-Simulation Framework Bridging 2D and 3D Driving Simulation for AV Software Validation
M. Kaufeld, K. Moller, A. Gambi, P. Arcaini, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi:10.1109/ITSC60802.2025.11423490 , PDF, Code, Video
From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios
Y. Gao, M. Piccinini, K. Moller, A. Alanwar, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2025
doi: 10.1109/ITSC60802.2025.11423486, PDF, Code, Video
DualAD: Dual-Layer Planning for Reasoning in Autonomous Driving
D. Wang, M. Kaufeld, J. Betz
IEEE International Conference on Intelligent Robots and Systems (IROS), 2025
doi: 10.1109/IROS60139.2025.11246084, PDF, Code, Video
Coherent Online Road Topology Estimation and Reasoning with Standard-Definition Maps
K. Pham, C. Witte, J. Behley, J. Betz, C. Stachniss
IEEE International Conference on Intelligent Robots and Systems (IROS), 2025
doi: 10.1109/IROS60139.2025.11247353 , PDF, Video, Project Website
From Shadows to Safety: Occlusion Tracking and Risk Mitigation for Urban Autonomous Driving
K. Moller, L. Schwarzmeier, J. Betz
IEEE Intelligent Vehicles Symposium (IV), 2025
doi: 10.1109/IV64158.2025.11097529 , PDF, Video, Code
Open-Loop and Feedback Nash Trajectories for Competitive Racing with iLQGames
M. Rowold, A. Langmann, B. Lohmann, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), 2024
doi: 10.1109/ITSC58415.2024.10920189, PDF
FRENETIX: A High-Performance and Modular Motion Planning Framework for Autonomous Driving
R. Trauth, K. Moller, G. Würsching, J. Betz
IEEE Access, vol. 12, pp. 127426–127439, 2024
doi: 10.1109/ACCESS.2024.3436835, PDF, Video, Code
DP-VINS: Dynamics Adaptive Plane-Based Visual-Interatial SLAM
L. Zhang, H. Yin, W. Ye, J. Betz
IEEE Transactions on Instrumentation and Measurement, vol. 73, pp.1-16, 2024, Art no. 5036516, 2024
doi: 10.1109/TIM.2024.3476615, PDF
Accelerating Autonomy: Insights from Pro Racers in the Era of Autonomous Racing-An Expert Interview Study
F. Werner, R. Oberhuber, J. Betz
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588407, PDF, Code
FlexMap Fusion: Georeferencing and Automated Conflation of HD Maps with OpenStreetMap
M. Leitenstern, F. Sauerbeck, D. Kulmer, J. Betz
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588414, PDF, Code
Investigating driving interactions: A robust multi-agent simulation framework for autonomous vehicles
M. Kaufeld, R. Trauth, J. Betz
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588423, PDF, Video, Code
Overcoming blind spots: Occlusion considerations for improved autonomous driving safety
K. Moller, R. Trauth, J. Betz
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588481, PDF, Video, Code
A new Taxonomy for Automated Driving: Structuring Applications based on their Operational Design Domain, Level of Automation and Automation Readiness
J. Betz, M. Lutwitzi, S. Peters
IEEE Intelligent Vehicles Symposium (IV), June 2024
doi: 10.1109/IV55156.2024.10588711, PDF, Video
Loosely Coupled Stereo VINS Based on Point-Line Features Tracking with Feedback Loops
Z. Linchuan, Y. Huilin, Y. Wei, Y. Jun, Z. Hao, J. Betz
IEEE Transactions on Vehicular Technology
doi: 10.1109/TVT.2024.3370732, PDF
ESP: Extro-Spective Prediction for Long-term Behavior Reasoning in Emergency Scenarios
D. Wang, L. Zheyuan, Y. Li, Y. Wu, Y. Ma, J. Betz, R. Yang, W. Li
IEEE International Conference on Robotics and Automation (ICRA) 2024
doi: 10.1109/ICRA57147.2024.10610002, Paper, Video, Project Website, Dataset
Toward Safer Autonomous Vehicles: Occlusion-Aware Trajectory Planning to Minimize Risky Behavior
R. Trauth, K. Moller, and J. Betz
IEEE Open Journal of Intelligent Transportation Systems, vol. 4, pp. 929–942, 2023
doi: 10.1109/ojits.2023.3336464, Paper, Video
Multi-LiDAR Localization and Mapping Pipeline for Urban Autonomous Driving
F.Sauerbeck, D. Kulmer, M. Pielmeier, M. Leitenstern, C. Weiß, J. Betz
IEEE Sensors 2023
doi: 10.1109/sensors56945.2023.10325207, Paper, Video
Ground-optimized SLAM with Hierarchical Loop Closure Detection in Large-scale Environment
H. Yin, M. Sun, L. Zhang, J. Yan, J. Betz
IEEE International Conference on Intelligent Transportation Systems (ITSC), In Print
doi: 10.1109/ITSC57777.2023.10422438, Paper
Applying ethical theories to the decision-making of self-driving vehicles: A systematic review and integration of the literature
F. Poszler, M. Geisslinger, J. Betz, C. Lütge
Technology in Society, vol. 75. Elsevier BV, p. 102350, Nov. 2023
doi: 10.1016/j.techsoc.2023.102350, PDF, Video
A Benchmark Comparison of Imitation Learning-Based Control Policies for Autonomous Racing
X. Sun, M. Zhou, Z. Zhuang, S. Yang, J. Betz, R. Mangharam
IEEE Intelligent Vehicles Symposim (IV), June 2023
doi: 10.1109/IV55152.2023.10186780, PDF
Deriving Spatial Policies for Overtaking Maneuvers with Autonomous Vehicles
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2022 14th International Conference on COMmunication Systems & NETworkS (COMSNETS), 2022
doi: 10.1109/COMSNETS53615.2022.9668548, PDF
Stress Testing Autonomous Racing Overtake Maneuvers with RRT
S. Bak, J. Betz, A. Chawla, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
doi: 10.1109/IV51971.2022.9827237, PDF
Drive Right: Autonomous Vehicle Education through an Integrated Simulation Platform
Z. Qiao, H. Loeb, V. Gurrla, M. Lebermann, J. Betz, and R. Mangharam,
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 4., Apr. 13, 2022
doi: 10.4271/12-05-04-0028, PDF
Scenario Understanding and Motion Prediction for Autonomous Vehicles—Review and Comparison
P. Karle, M. Geisslinger, J. Betz, and M. Lienkamp,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10., pp. 16962–16982, Oct. 2022
doi: 10.1109/tits.2022.3156011, PDF
A Combined LiDAR-Camera Localization for Autonomous Race Cars
F. Sauerbeck, L. Baierlein, J. Betz, M. Lienkamp
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 1, Jan. 2022
doi: 10.4271/12-05-01-0006, PDF
Watch-and-Learn-Net: Self-supervised Online Learning for Probabilistic Vehicle Trajectory Prediction
M. Geisslinger, P. Karle, J. Betz, and M. Lienkamp, “
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021.
doi: 10.1109/smc52423.2021.9659079, PDF
Track based offline policy learning for overtaking maneuvers with autonomous racecars
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2021 IEEE International Conference on Robotics and Automation (ICRA 2021) – Workshop Opportunities and Challenges With Autonomous Racing, 2021
doi: arxiv.org/abs/2107.09782, PDF
Autonomous Driving Ethics: from Trolley Problem to Ethics of Risk
M. Geisslinger, F. Poszler, J. Betz, C. Luetge and M. Lienkamp
Philosophy & Technology, vol. 34, no. 4. Springer Science and Business Media LLC, pp. 1033–1055, Apr. 12, 2021
doi: 10.1007/s13347-021-00449-4, PDF
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
S. Huch, A. Ongel, J. Betz, and M. Lienkamp
Sensors, vol. 21, no. 4, p. 1039, 2021
doi: 10.3390/s21041039, PDF
Virtual Strategy Engineer: Using Artificial Neural Networks for Making Race Strategy Decisions in Circuit Motorsport
A. Heilmeier, A. Thomaser, M. Graf, J. Betz
Applied Sciences, vol. 10, no. 21, p. 7805, Nov. 2020
doi: 10.3390/app10217805, PDF
Application of Monte Carlo Methods to Consider Probabilistic Effects in a Race Simulation for Circuit Motorsport
A. Heilmeier, M. Graf, J. Betz, M. Lienkamp
Applied Sciences, vol. 10, no. 12, p. 4229, Jun. 2020
doi: 10.3390/app10124229, PDF
Minimum Race-Time Planning-Strategy for an Autonomous Electric Racecar
T. Herrmann, F. Passigato, J. Betz, and M. Lienkamp
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294681, PDF
An Open-Source Scenario Architect for Autonomous Vehicles
T. Stahl and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243029, PDF
Persistent Map Saving for Visual Localization for Autonomous Vehicles: An ORB-SLAM 2 Extension
F. Nobis, O. Papanikolaou, J. Betz, and M. Lienkamp
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243094, PDF
Importance of Contextual Information for the Detection of Road Damages
K. Riedl, S. Huber, M. Bomer, J. Kreibich, F. Nobis, and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9242954, PDF
A Concept for Estimation and Prediction of the Tire-Road Friction Potential for an Autonomous Racecar
L. Hermansdorfer, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/ITSC.2019.8917024, PDF
We develop autonomy algorithms that perceive, plan, and act reliably at the physical and computational limits of dynamic motion.
This theme investigates how autonomous vehicles and mobile robots can perform agile, precise, and safety-critical maneuvers near the boundaries of their dynamic capabilities. The work integrates motion planning, optimal and predictive control, vehicle dynamics, reinforcement learning, and game-theoretic decision-making. Particular attention is given to changing dynamics, interaction with other agents, three-dimensional trajectories, and real-time computational constraints. Autonomous racing and drifting provide demanding experimental environments in which methods can be tested beyond nominal operation. The scientific goal is to create autonomy that can reason about and exploit the limits of motion while retaining stability and robustness.
Deriving Spatial Policies for Overtaking Maneuvers with Autonomous Vehicles
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2022 14th International Conference on COMmunication Systems & NETworkS (COMSNETS), 2022
doi: 10.1109/COMSNETS53615.2022.9668548, PDF
Stress Testing Autonomous Racing Overtake Maneuvers with RRT
S. Bak, J. Betz, A. Chawla, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
doi: 10.1109/IV51971.2022.9827237, PDF
Autonomous vehicles on the edge: A survey on autonomous vehicle racing
J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, R. Mangharam
IEEE Open Journal of Intelligent Transportation Systems, vol. 3., pp. 458–488 2022
doi: 10.1109/ojits.2022.3181510, PDF
Scenario Understanding and Motion Prediction for Autonomous Vehicles—Review and Comparison
P. Karle, M. Geisslinger, J. Betz, and M. Lienkamp,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10., pp. 16962–16982, Oct. 2022
doi: 10.1109/tits.2022.3156011, PDF
Optimization-Based Real-Time-Capable Energy Strategy for Autonomous Electric Race Cars
T. Herrmann, F. Sauerbeck, M. Bayerlein, J. Betz, and M. Lienkamp
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 1., pp. 45–59, Jan. 10, 2022
doi: 10.4271/12-05-01-0005, PDF
Watch-and-Learn-Net: Self-supervised Online Learning for Probabilistic Vehicle Trajectory Prediction
M. Geisslinger, P. Karle, J. Betz, and M. Lienkamp, “
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021.
doi: 10.1109/smc52423.2021.9659079, PDF
Track based offline policy learning for overtaking maneuvers with autonomous racecars
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2021 IEEE International Conference on Robotics and Automation (ICRA 2021) – Workshop Opportunities and Challenges With Autonomous Racing, 2021
doi: arxiv.org/abs/2107.09782, PDF
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
S. Huch, A. Ongel, J. Betz, and M. Lienkamp
Sensors, vol. 21, no. 4, p. 1039, 2021
doi: 10.3390/s21041039, PDF
Minimum Race-Time Planning-Strategy for an Autonomous Electric Racecar
T. Herrmann, F. Passigato, J. Betz, and M. Lienkamp
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294681, PDF
An Open-Source Scenario Architect for Autonomous Vehicles
T. Stahl and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243029, PDF
A Software Architecture for the Dynamic Path Planning of an Autonomous Racecar at the Limits of Handling
J. Betz, A. Wischnewski, A. Heilmeier, F. Nobis, T. Stahl, L. Hermansdorfer, T. Herrmann, M. Lienkamp
2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), 2019
doi: 10.1109/ICCVE45908.2019.8965238, PDF
Energy Management Strategy for an Autonomous Electric Racecar using Optimal Control
T. Herrmann, F. Christ, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/ITSC.2019.8917154, PDF
Multilayer Graph-Based Trajectory Planning for Race Vehicles in Dynamic Scenarios
T. Stahl, A. Wischnewski, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/itsc.2019.8917032, PDF
Minimum curvature trajectory planning and control for an autonomous race car
A. Heilmeier, A. Wischnewski, L. Hermansdorfer, J. Betz, M. Lienkamp, B. Lohmann
Vehicle System Dynamics, vol. 58, no. 10, pp. 1497–1527, 2019
doi: 10.1080/00423114.2019.1631455, PDF
A Quasi-Steady-State Lap Time Simulation for Electrified Race Cars
A. Heilmeier, M. Geisslinger, and J. Betz
2019 Fourteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2019
doi: 10.1109/EVER.2019.8813646, PDF
We are transforming autonomy algorithms into time-predictable, experimentally validated software that operates on real vehicles and mobile robots, creating reproducible research at scale.
This research theme allows us to bring autonomous systems out of simulation and into physically demanding, unstructured, and dynamic environments. Our group operates a diverse fleet of experimental platforms—including road vehicles, autonomous racecars, small-scale vehicles, humanoids, quadrupeds, and other mobile robots—to study how algorithms behave when confronted with environmental disturbances, physical constraints, and human interaction. Field experiments generate data that are used to validate assumptions, identify failure modes, quantify performance, and reveal research questions that cannot be observed in simulation alone. Insights from these deployments feed directly back into our algorithm development.
An Analysis of Software Latency for a High-Speed Autonomous Race Car – A Case Study in the Indy Autonomous Challenge
T. Betz, P. Karle, F. Werner, J. Betz
SAE International Journal of Connected and Automated Vehicles, vol. 6, no. 3. SAE International, 2023.
doi: 10.4271/12-06-03-0018, PDF
TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge
J. Betz, T. Betz, F. Fent, M. Geisslinger, A. Heilmeier, L. Hermansdorfer, et al.
Journal of Field Robotics, 1– 27, 2023.
doi: https://doi.org/10.1002/rob.22153, PDF
Combinatorial and Parametric Gradient-Free Optimization for Cyber-Physical System Design
H. Zheng, J. Betz, A. Ramamurthy, H. Jin, and R. Mangharam
2022 IEEE Workshop on Design Automation for CPS and IoT (DESTION), May 2022
doi: 10.1109/destion56136.2022.00012, PDF
Indy Autonomous Challenge - Autonomous Race Cars at the Handling Limits
A. Wischnewski, M. Geisslinger, J. Betz, et al.
Pfeffer, P. (eds) 12th International Munich Chassis Symposium 2021. Proceedings. Springer Vieweg, Berlin, Heidelberg., pp. 163–182, 2022
doi: 10.1007/978-3-662-64550-5_10, PDF
Winning the 3rd Japan Automotive AI Challenge--Autonomous Racing with the Autoware. Auto Open Source Software Stack
Z. Zang, R. Tumu, J. Betz, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
Doi: 10.1109/IV51971.2022.9827162, PDF
Stress Testing Autonomous Racing Overtake Maneuvers with RRT
S. Bak, J. Betz, A. Chawla, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
doi: 10.1109/IV51971.2022.9827237, PDF
Drive Right: Autonomous Vehicle Education through an Integrated Simulation Platform
Z. Qiao, H. Loeb, V. Gurrla, M. Lebermann, J. Betz, and R. Mangharam,
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 4., Apr. 13, 2022
doi: 10.4271/12-05-04-0028, PDF
Autonomous vehicles on the edge: A survey on autonomous vehicle racing
J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, R. Mangharam
IEEE Open Journal of Intelligent Transportation Systems, vol. 3., pp. 458–488 2022
doi: 10.1109/ojits.2022.3181510, PDF
Online Verification Concept for Autonomous Vehicles – Illustrative Study for a Trajectory Planning Module
T. Stahl, M. Eicher, J. Betz, and F. Diermeyer
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294703, PDF
An Open-Source Scenario Architect for Autonomous Vehicles
T. Stahl and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243029, PDF
Benchmarking of a software stack for autonomous racing against a professional human race driver
L. Hermansdorfer, J. Betz, and M. Lienkamp
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9242926, PDF
A Software Architecture for the Dynamic Path Planning of an Autonomous Racecar at the Limits of Handling
J. Betz, A. Wischnewski, A. Heilmeier, F. Nobis, T. Stahl, L. Hermansdorfer, T. Herrmann, M. Lienkamp
2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), 2019
doi: 10.1109/ICCVE45908.2019.8965238, PDF
A Software Architecture for an Autonomous Racecar
T. Stahl, A. Wischnewski, J. Betz, and M. Lienkamp
2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), 2019
doi: 10.1109/VTCSpring.2019.8746367, PDF
What can we learn from autonomous level 5 Motorsport?
J. Betz, A. Wischnewski, A. Heilmeier, F. Nobis, T. Stahl, L. Hermansdorfer, B. Lohmann M. Lienkamp
Proceedings, Springer Fachmedien Wiesbaden, 2018
doi: 10.1007/978-3-658-22050-1_12, PDF