We use visual information to optimize the planning and decision-making process for autonomous vehicles. We use data from different sensors (e.g. a camera) and process them either individually or fused to generate information about the dynamic environment of the vehicle, such as the position of objects. This information is then processed by the vehicle's trajectory and behaviour planning to make better and more dynamic decisions for the final trajectory. Our goal is to enable the autonomous vehicle to navigate complex and dynamic environments autonomously and make more efficient and safe decisions based on causal relationships and visual interpretations.
RGB-L: Enhancing Indirect Visual SLAM using LiDAR-based Dense Depth Map
F. Sauerbeck, B. Obermeier, M. Rudolph, J. Betz
IEEE 3rd International Conference on Control, Automation, Robotics (ICCCR), In Print
Local_INN: Implicit Map Representation and Localization with Invertible Neural Networks
Z. Zang, H. Zheng, J. Betz, R. Mangharam
2023 IEEE International Conference on Robotics and Automation (ICRA), In Print
Scenario Understanding and Motion Prediction for Autonomous Vehicles—Review and Comparison
P. Karle, M. Geisslinger, J. Betz, and M. Lienkamp,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10., pp. 16962–16982, Oct. 2022
doi: 10.1109/tits.2022.3156011, PDF
A Combined LiDAR-Camera Localization for Autonomous Race Cars
F. Sauerbeck, L. Baierlein, J. Betz, M. Lienkamp
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 1, Jan. 2022
doi: 10.4271/12-05-01-0006, PDF
Watch-and-Learn-Net: Self-supervised Online Learning for Probabilistic Vehicle Trajectory Prediction
M. Geisslinger, P. Karle, J. Betz, and M. Lienkamp, “
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021.
doi: 10.1109/smc52423.2021.9659079, PDF
Radar Voxel Fusion for 3D Object Detection
F. Nobis, E. Shafiei, P. Karle, J. Betz, M. Lienkamp
Applied Sciences, vol. 11, no. 12, p. 5598, 2021
doi: 10.3390/app11125598, PDF
Kernel Point Convolution LSTM Networks for Radar Point Cloud Segmentation
F. Nobis, F. Fent, J. Betz, and M. Lienkamp
Applied Sciences, vol. 11, no. 6, p. 2599, 2021
doi: 10.3390/app11062599, PDF
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
S. Huch, A. Ongel, J. Betz, and M. Lienkamp
Sensors, vol. 21, no. 4, p. 1039, 2021
doi: 10.3390/s21041039, PDF
Exploring the Capabilities and Limits of 3D Monocular Object Detection – A Study on Simulation and Real World Data
F. Nobis, F. Brunhuber, S. Janssen, J. Betz, and M. Lienkamp
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294625, PDF
Persistent Map Saving for Visual Localization for Autonomous Vehicles: An ORB-SLAM 2 Extension
F. Nobis, O. Papanikolaou, J. Betz, and M. Lienkamp
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243094, PDF
Importance of Contextual Information for the Detection of Road Damages
K. Riedl, S. Huber, M. Bomer, J. Kreibich, F. Nobis, and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9242954, PDF
A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection
F. Nobis, M. Geisslinger, M. Weber, J. Betz, and M. Lienkamp
2019 Sensor Data Fusion: Trends, Solutions, Applications (SDF), 2019
doi: 10.1109/SDF.2019.8916629, PDF
A Concept for Estimation and Prediction of the Tire-Road Friction Potential for an Autonomous Racecar
L. Hermansdorfer, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/ITSC.2019.8917024, PDF
SemanticDepth: Fusing Semantic Segmentation and Monocular Depth Estimation for Enabling Autonomous Driving in Roads without Lane Lines
P. R. Palafox, J. Betz, F. Nobis, K. Riedl, and M. Lienkamp
Sensors, vol. 19, no. 14, p. 3224, 2019
doi: 10.3390/s19143224, PDF
ROS-based localization of a race vehicle at high-speed using LIDAR
T. Stahl, A. Wischnewski, J. Betz, and M. Lienkamp
E3S Web of Conferences, 2019
doi: 10.1051/e3sconf/20199504002, PDF
Autonomous Racing: A Comparison of SLAM Algorithms for Large Scale Outdoor Environments
F. Nobis, J. Betz, L. Hermansdorfer, and M. Lienkamp
ICVARS ’19: 2019 the 3rd International Conference on Virtual and Augmented Reality Simulations, 2019
doi: 10.1145/3332305.3332319, PDF
The goal is to enable a tight coupling between reason about the influence of the surrounding agents on the ego vehicles trajectory while maintaining full capabilities of the ego vehicles dynamic limitations. This research creates a new decision maker that can achieve safe, reliable and high dynamic vehicle actions in complex, multi vehicle environments. With this research we can balance performance and Risk efficiently.
Deriving Spatial Policies for Overtaking Maneuvers with Autonomous Vehicles
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2022 14th International Conference on COMmunication Systems & NETworkS (COMSNETS), 2022
doi: 10.1109/COMSNETS53615.2022.9668548, PDF
Stress Testing Autonomous Racing Overtake Maneuvers with RRT
S. Bak, J. Betz, A. Chawla, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
doi: 10.1109/IV51971.2022.9827237, PDF
Autonomous vehicles on the edge: A survey on autonomous vehicle racing
J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, R. Mangharam
IEEE Open Journal of Intelligent Transportation Systems, vol. 3., pp. 458–488 2022
doi: 10.1109/ojits.2022.3181510, PDF
Scenario Understanding and Motion Prediction for Autonomous Vehicles—Review and Comparison
P. Karle, M. Geisslinger, J. Betz, and M. Lienkamp,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10., pp. 16962–16982, Oct. 2022
doi: 10.1109/tits.2022.3156011, PDF
Optimization-Based Real-Time-Capable Energy Strategy for Autonomous Electric Race Cars
T. Herrmann, F. Sauerbeck, M. Bayerlein, J. Betz, and M. Lienkamp
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 1., pp. 45–59, Jan. 10, 2022
doi: 10.4271/12-05-01-0005, PDF
Watch-and-Learn-Net: Self-supervised Online Learning for Probabilistic Vehicle Trajectory Prediction
M. Geisslinger, P. Karle, J. Betz, and M. Lienkamp, “
IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2021.
doi: 10.1109/smc52423.2021.9659079, PDF
Track based offline policy learning for overtaking maneuvers with autonomous racecars
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2021 IEEE International Conference on Robotics and Automation (ICRA 2021) – Workshop Opportunities and Challenges With Autonomous Racing, 2021
doi: arxiv.org/abs/2107.09782, PDF
Multi-Task End-to-End Self-Driving Architecture for CAV Platoons
S. Huch, A. Ongel, J. Betz, and M. Lienkamp
Sensors, vol. 21, no. 4, p. 1039, 2021
doi: 10.3390/s21041039, PDF
Minimum Race-Time Planning-Strategy for an Autonomous Electric Racecar
T. Herrmann, F. Passigato, J. Betz, and M. Lienkamp
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294681, PDF
An Open-Source Scenario Architect for Autonomous Vehicles
T. Stahl and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243029, PDF
A Software Architecture for the Dynamic Path Planning of an Autonomous Racecar at the Limits of Handling
J. Betz, A. Wischnewski, A. Heilmeier, F. Nobis, T. Stahl, L. Hermansdorfer, T. Herrmann, M. Lienkamp
2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), 2019
doi: 10.1109/ICCVE45908.2019.8965238, PDF
Energy Management Strategy for an Autonomous Electric Racecar using Optimal Control
T. Herrmann, F. Christ, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/ITSC.2019.8917154, PDF
Multilayer Graph-Based Trajectory Planning for Race Vehicles in Dynamic Scenarios
T. Stahl, A. Wischnewski, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/itsc.2019.8917032, PDF
Minimum curvature trajectory planning and control for an autonomous race car
A. Heilmeier, A. Wischnewski, L. Hermansdorfer, J. Betz, M. Lienkamp, B. Lohmann
Vehicle System Dynamics, vol. 58, no. 10, pp. 1497–1527, 2019
doi: 10.1080/00423114.2019.1631455, PDF
A Quasi-Steady-State Lap Time Simulation for Electrified Race Cars
A. Heilmeier, M. Geisslinger, and J. Betz
2019 Fourteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2019
doi: 10.1109/EVER.2019.8813646, PDF
We are focusing on adaptive and dynamic motion planning and control algorithms used in robotics and autonomous systems. Our goal is to enable the system to adjust its behavior based on changing environments and conditions. These algorithms use feedback and sensory information to continually update and optimize the motion plan, allowing for more robust and flexible operation in real-world scenarios. Examples include Model Predictive Control, Reinforcement Learning, and Probabilistic Roadmaps.
TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge
J. Betz, T. Betz, F. Fent, M. Geisslinger, A. Heilmeier, L. Hermansdorfer, et al.
Journal of Field Robotics, 1– 27, 2023.
doi: https://doi.org/10.1002/rob.22153, PDF
Deriving Spatial Policies for Overtaking Maneuvers with Autonomous Vehicles
J. Bhargav, J. Betz, H. Zheng, and R. Mangharam
2022 14th International Conference on COMmunication Systems & NETworkS (COMSNETS), 2022
doi: 10.1109/COMSNETS53615.2022.9668548, PDF
Winning the 3rd Japan Automotive AI Challenge--Autonomous Racing with the Autoware. Auto Open Source Software Stack
Z. Zang, R. Tumu, J. Betz, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
Doi: 10.1109/IV51971.2022.9827162, PDF
TireEye: Optical On-board Tire Wear Detection
S. Huber, P. Preindl, and J. Betz
Annual Conference of the PHM Society, vol. 14, no. 1. PHM Society, Oct. 28, 2022.
doi: 10.36001/phmconf.2022.v14i1.3242, PDF
Autonomous vehicles on the edge: A survey on autonomous vehicle racing
J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, R. Mangharam
IEEE Open Journal of Intelligent Transportation Systems, vol. 3., pp. 458–488 2022
doi: 10.1109/ojits.2022.3181510, PDF
End-to-End Neural Network for Vehicle Dynamics Modeling – Best Paper Award
L. Hermansdorfer, R. Trauth, J. Betz, M. Lienkamp
3rd IEEE Conference on Optimization and Modeling of Complex Systems, Agadir, Morocco, December 2020, 2020
doi: 10.1109/cist49399.2021.9357196, PDF
Real-Time Learning of Non-Gaussian Uncertainty Models for Autonomous Racing
A. Wischnewski, J. Betz, and B. Lohmann
2020 59th IEEE Conference on Decision and Control (CDC), 2020
doi: 10.1109/cdc42340.2020.9304230, PDF
Minimum Race-Time Planning-Strategy for an Autonomous Electric Racecar
T. Herrmann, F. Passigato, J. Betz, and M. Lienkamp
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294681, PDF
Importance of Contextual Information for the Detection of Road Damages
K. Riedl, S. Huber, M. Bomer, J. Kreibich, F. Nobis, and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9242954, PDF
A Model-Free Algorithm to Safely Approach the Handling Limit of an Autonomous Racecar
A. Wischnewski, J. Betz, and B. Lohmann
2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), 2019
doi: 10.1109/ICCVE45908.2019.8965218, PDF
A Concept for Estimation and Prediction of the Tire-Road Friction Potential for an Autonomous Racecar
L. Hermansdorfer, J. Betz, and M. Lienkamp
2019 IEEE Intelligent Transportation Systems Conference – ITSC, 2019
doi: 10.1109/ITSC.2019.8917024, PDF
Minimum curvature trajectory planning and control for an autonomous race car
A. Heilmeier, A. Wischnewski, L. Hermansdorfer, J. Betz, M. Lienkamp, B. Lohmann
Vehicle System Dynamics, vol. 58, no. 10, pp. 1497–1527, 2019
doi: 10.1080/00423114.2019.1631455, PDF
Vehicle Dynamics State Estimation and Localization for High Performance Race Cars – Young Author Award
A. Wischnewski, T. Stahl, J. Betz, and B. Lohmann
IFAC-PapersOnLine, vol. 52, no. 8, pp. 154–161, 2019
doi: 10.1016/j.ifacol.2019.08.064, PDF
Based on additional ethical studies and interdisciplinary research projects we want to incoperate ethical and responsible behavior in autonomous systems. The goal is to conduct technical-ethical evaluations on autonomous functions. These evaluations will show the interaction of humans and machines in decision-making processes, display bias in the algorithm and give insights into the enhancement of software regarding ethical theories and the possibility of creation responsible behavior.
Autonomous Driving Ethics: from Trolley Problem to Ethics of Risk
M. Geisslinger, F. Poszler, J. Betz, C. Luetge and M. Lienkamp
Philosophy & Technology, vol. 34, no. 4. Springer Science and Business Media LLC, pp. 1033–1055, Apr. 12, 2021
doi: 10.1007/s13347-021-00449-4, PDF
Autonomous Driving—A Crash Explained in Detail
J. Betz, A. Heilmeier, A. Wischnewski, T. Stahl and M. Lienkamp
Applied Sciences, vol. 9, no. 23, p. 5126, 2019
doi: 10.3390/app9235126, PDF
All the algorithm we develop in the AVS Lab are aimed to be tested extensively in real world scenarios with actual vehicle hardware. With this we can test the algorithm in combination with Software Stacks, assure their performance – especially of applied Machine Learing algorithms and explore the multi domains of hardware and software at the same time.
An Analysis of Software Latency for a High-Speed Autonomous Race Car – A Case Study in the Indy Autonomous Challenge
T. Betz, P. Karle, F. Werner, J. Betz
SAE International Journal of Connected and Automated Vehicles, vol. 6, no. 3. SAE International, 2023.
doi: 10.4271/12-06-03-0018, PDF
TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challenge
J. Betz, T. Betz, F. Fent, M. Geisslinger, A. Heilmeier, L. Hermansdorfer, et al.
Journal of Field Robotics, 1– 27, 2023.
doi: https://doi.org/10.1002/rob.22153, PDF
Combinatorial and Parametric Gradient-Free Optimization for Cyber-Physical System Design
H. Zheng, J. Betz, A. Ramamurthy, H. Jin, and R. Mangharam
2022 IEEE Workshop on Design Automation for CPS and IoT (DESTION), May 2022
doi: 10.1109/destion56136.2022.00012, PDF
Indy Autonomous Challenge - Autonomous Race Cars at the Handling Limits
A. Wischnewski, M. Geisslinger, J. Betz, et al.
Pfeffer, P. (eds) 12th International Munich Chassis Symposium 2021. Proceedings. Springer Vieweg, Berlin, Heidelberg., pp. 163–182, 2022
doi: 10.1007/978-3-662-64550-5_10, PDF
Winning the 3rd Japan Automotive AI Challenge--Autonomous Racing with the Autoware. Auto Open Source Software Stack
Z. Zang, R. Tumu, J. Betz, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
Doi: 10.1109/IV51971.2022.9827162, PDF
Stress Testing Autonomous Racing Overtake Maneuvers with RRT
S. Bak, J. Betz, A. Chawla, H. Zheng, R. Mangharam
IEEE Intelligent Vehicles Symposium (IV 22), 2022
doi: 10.1109/IV51971.2022.9827237, PDF
Drive Right: Autonomous Vehicle Education through an Integrated Simulation Platform
Z. Qiao, H. Loeb, V. Gurrla, M. Lebermann, J. Betz, and R. Mangharam,
SAE International Journal of Connected and Automated Vehicles, vol. 5, no. 4., Apr. 13, 2022
doi: 10.4271/12-05-04-0028, PDF
Autonomous vehicles on the edge: A survey on autonomous vehicle racing
J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, R. Mangharam
IEEE Open Journal of Intelligent Transportation Systems, vol. 3., pp. 458–488 2022
doi: 10.1109/ojits.2022.3181510, PDF
Online Verification Concept for Autonomous Vehicles – Illustrative Study for a Trajectory Planning Module
T. Stahl, M. Eicher, J. Betz, and F. Diermeyer
2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020
doi: 10.1109/ITSC45102.2020.9294703, PDF
An Open-Source Scenario Architect for Autonomous Vehicles
T. Stahl and J. Betz
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9243029, PDF
Benchmarking of a software stack for autonomous racing against a professional human race driver
L. Hermansdorfer, J. Betz, and M. Lienkamp
2020 Fifteenth International Conference on Ecological Vehicles and Renewable Energies (EVER), 2020
doi: 10.1109/ever48776.2020.9242926, PDF
A Software Architecture for the Dynamic Path Planning of an Autonomous Racecar at the Limits of Handling
J. Betz, A. Wischnewski, A. Heilmeier, F. Nobis, T. Stahl, L. Hermansdorfer, T. Herrmann, M. Lienkamp
2019 IEEE International Conference on Connected Vehicles and Expo (ICCVE), 2019
doi: 10.1109/ICCVE45908.2019.8965238, PDF
A Software Architecture for an Autonomous Racecar
T. Stahl, A. Wischnewski, J. Betz, and M. Lienkamp
2019 IEEE 89th Vehicular Technology Conference (VTC2019-Spring), 2019
doi: 10.1109/VTCSpring.2019.8746367, PDF
What can we learn from autonomous level 5 Motorsport?
J. Betz, A. Wischnewski, A. Heilmeier, F. Nobis, T. Stahl, L. Hermansdorfer, B. Lohmann M. Lienkamp
Proceedings, Springer Fachmedien Wiesbaden, 2018
doi: 10.1007/978-3-658-22050-1_12, PDF