Virtual Reality for Autonomous Vehicles: A Review of Safety, Training, and Human–Machine Interaction
DOI:
https://doi.org/10.65542/djei.v1i1.8Keywords:
Virtual Reality, Autonomous Vehicles, Human–Machine Interaction, Driver Training, SimulationAbstract
Virtual Reality (VR) enables immersive, repeatable, and risk-free simulation of haz-ardous driving scenarios. VR is increasingly applied in autonomous vehicle (AV) re-search for safety evaluation, takeover training, and human–machine interaction (HMI) prototyping. This review synthesizes empirical work to assess the contributions of VR to takeover performance, training efficacy, ecological validity, and user ac-ceptance. Evidence suggests that VR interventions can reduce takeover reaction times compared to conventional instruction, support iterative HMI design, and enhance user familiarity and calibrated trust in AV systems. Persistent limitations include simulator sickness, sample homogeneity, fidelity gaps between VR and on-road performance, and inconsistent reporting of scenario design. We recommend standardized reporting (PRISMA flow plus scenario metadata), longitudinal transfer studies, cross-cultural samples, and hybrid VR–AR validation methods to strengthen transferability and reg-ulatory acceptance.
References
Aldoski, Z.N.; Koren, C. Assessment of Traffic Sign Retroreflectivity for Autonomous Vehicles: A Comparison between Handheld Retroreflectometer and LiDAR Data. Arch. Transp. 2024, 70(2), 7–26. https://doi.org/10.61089/aot2024.qxy24g93.
SAE International. Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems. SAE Stand. J. 2021, 3016, 1. Available online: https://www.sae.org/standards/content/j3016_202104 (accessed on 12 July 2025).
Ondruš, J.; Kolla, E.; Vertaľ, P.; Šarić, Ž. How Do Autonomous Cars Work? Transp. Res. Procedia 2020, 44, 226–233. https://doi.org/10.1016/j.trpro.2020.02.049.
Sportillo, D.; Paljic, A.; Ojeda, L. Get Ready for Automated Driving Using Virtual Reality. Accid. Anal. Prev. 2018, 118, 102–113. https://doi.org/10.1016/j.aap.2018.06.003.
Zou, X.; et al. On-Road Virtual Reality Autonomous Vehicle (VRAV) Simulator: An Empirical Study on User Experience. Transp. Res. Part C Emerg. Technol. 2021, 126, 103090. https://doi.org/10.1016/j.trc.2021.103090.
Harari, R.E.; Lamb, R.; Fathi, R.; Hulme, K. Virtual Reality Tour for First-Time Users of Highly Automated Cars: Comparing the Effects of Virtual Environments with Different Levels of Interaction Fidelity. Appl. Ergon. 2021, 90, 103226. https://doi.org/10.1016/j.apergo.2020.103226.
Kalra, N.; Paddock, S.M. Driving to Safety: How Many Miles of Driving Would It Take to Demonstrate Autonomous Vehicle Reliability? Transp. Res. Part A Policy Pract. 2016, 94, 182–193. https://doi.org/10.1016/j.tra.2016.09.010.
Malik, S.; Khan, M.A.; El-Sayed, H. CARLA: Car Learning to Act—An Inside Out. Procedia Comput. Sci. 2021, 198, 742–749. https://doi.org/10.1016/j.procs.2021.12.316.
Shanu, S.; Narula, D.; Nayana; Pathak, L.K.; Mahato, S. AR/VR Technology for Autonomous Vehicles and Knowledge-Based Risk Assessment. In Virtual and Augmented Reality for Automobile Industry: Innovation Vision and Applications; Hassanien, A.E., Gupta, D., Khanna, A., Slowik, A., Eds.; Springer International Publishing: Cham, Switzerland, 2022; pp. 87–109. https://doi.org/10.1007/978-3-030-94102-4_5.
Tran, T.T.M.; Parker, C.; Hoggenmüller, M.; Wang, Y.; Tomitsch, M. Exploring the Impact of Interconnected External Interfaces in Autonomous Vehicles on Pedestrian Safety and Experience. In Proceedings of the CHI Conference on Human Factors in Computing Systems; ACM: New York, NY, USA, 2024; pp. 1–18. https://doi.org/10.1145/3613904.3642118.
Meftah, L.H.; Braham, R. VSim-AV: A Virtual Simulation Platform for Autonomous Vehicles. In Lecture Notes in Networks and Systems; Springer: Cham, Switzerland, 2022; Volume 418, pp. 379–388. https://doi.org/10.1007/978-3-030-96308-8_35.
Cao, L.; Feng, X.; Liu, J.; Zhou, G. Automatic Generation System for Autonomous Driving Simulation Scenarios Based on PreScan. Appl. Sci. 2024, 14, 1354. https://doi.org/10.3390/app14041354.
Fry, A.; Fidan, I.; Wooldridge, E. Advancing Foundry Training through Virtual Reality: A Low-Cost, Immersive Learning Environment. Inventions 2025, 10, 38. https://doi.org/10.3390/inventions10030038.
Li, V.; Siniosoglou, I.; Sarigiannidis, P.; Argyriou, V. Enhancing Manufacturing Training through VR Simulations. In Proceedings of the 2025 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC); IEEE: Piscataway, NJ, USA, 2025; pp. 1–9. https://doi.org/10.1109/ICE/ITMC65658.2025.11106519.
Candela, E.; Feng, Y.; Angeloudis, P.; Demiris, Y. Quantitative Risk Indices for Autonomous Vehicle Training Systems. arXiv 2021, arXiv:2104.12945. Available online: http://arxiv.org/abs/2104.12945 (accessed on 28 July 2025).
Ajao, Q.; Oludamilare, O.; Prio, M.H. Safety Challenges and Analysis of Autonomous Electric Vehicle Development: Insights from On-Road Testing and Accident Reports. SSRN Electron. J. 2023, 1–9. https://doi.org/10.2139/ssrn.4454486.
Riedmaier, S.; Schneider, D.; Watzenig, D.; Diermeyer, F.; Schick, B. Model Validation and Scenario Selection for Virtual-Based Homologation of Automated Vehicles. Appl. Sci. 2021, 11, 35. https://doi.org/10.3390/app11010035.
Riegler, A.; Riener, A.; Holzmann, C. A Systematic Review of Virtual Reality Applications for Automated Driving: 2009–2020. Front. Hum. Dyn. 2021, 3, 689856. https://doi.org/10.3389/fhumd.2021.689856.
Chen, L.; Fang, J.; Li, J.; Xie, J. Research on the Effectiveness of Driving Simulation Systems in Risky Traffic Environments. Systems 2025, 13, 329. https://doi.org/10.3390/systems13050329.
Garikapati, D.; Shetiya, S.S. Autonomous Vehicles: Evolution of Artificial Intelligence and the Current Industry Landscape. Big Data Cogn. Comput. 2024, 8, 42. https://doi.org/10.3390/bdcc8040042.
Samak, T.V.; Samak, C.V.; Li, B.; Krovi, V. When Digital Twins Meet Large Language Models: Realistic, Interactive, and Editable Simulation for Autonomous Driving. arXiv 2025, arXiv:2507.00319. Available online: http://arxiv.org/abs/2507.00319 (accessed on 2 August 2025).
Mirzarazi, F.; Danishvar, S.; Mousavi, A. The Safety Risks of AI-Driven Solutions in Autonomous Road Vehicles. World Electr. Veh. J. 2024, 15, 438. https://doi.org/10.3390/wevj15100438.
Ebnali, M.; Lamb, R.; Fathi, R.; Hulme, K. Virtual Reality Tour for First-Time Users of Highly Automated Cars: Comparing the Effects of Virtual Environments with Different Levels of Interaction Fidelity. Appl. Ergon. 2021, 90, 103226. https://doi.org/10.1016/j.apergo.2020.103226.
Zhu, Y.; Wang, J.; Meng, F.; Liu, T. Review on Functional Testing Scenario Library Generation for Connected and Automated Vehicles. Sensors 2022, 22, 7735. https://doi.org/10.3390/s22207735.
Awasthi, S.S.; Imran, M.I.I.S.; Arrigoni, S.; Braghin, F. Bayesian Optimization Applied for Accelerated Virtual Validation of the Autonomous Driving Function. arXiv 2025, arXiv:2507.22769. Available online: http://arxiv.org/abs/2507.22769 (accessed on 15 August 2025).
Zhao, X.; Salako, K.; Strigini, L.; Robu, V.; Flynn, D. Assessing Safety-Critical Systems from Operational Testing: A Study on Autonomous Vehicles. Inf. Softw. Technol. 2020, 128, 106393. https://doi.org/10.1016/j.infsof.2020.106393.
Razi, M. Autonomous Vehicle Testing Using a Model-Based Approach. Master’s Thesis, San José State University, San José, CA, USA, 2022. Available online: https://scholarworks.sjsu.edu/etd_theses/5348 (accessed on 30 July 2025).
Mütsch, F.; Gremmelmaier, H.; Becker, N.; Bogdoll, D.; Zofka, M.R.; Zöllner, J.M. From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Driving. arXiv 2023, arXiv:2305.13960. Available online: http://arxiv.org/abs/2305.13960 (accessed on 25 July 2025).
Piazzoni, A.; Cherian, J.; Dauwels, J.; Chau, L.P. PEM: Perception Error Model for Virtual Testing of Autonomous Vehicles. IEEE Trans. Intell. Transp. Syst. 2024, 25, 670–681. https://doi.org/10.1109/TITS.2023.3311633.
Yang, Y.; et al. AuthSim: Towards Authentic and Effective Safety-Critical Scenario Generation for Autonomous Driving Tests. arXiv 2025, arXiv:2502.21100. Available online: http://arxiv.org/abs/2502.21100 (accessed on 8 August 2025).
Zarghani, A.; Ebrahimi, A.; Malekesfandiari, A. Multimodal Framework for Explainable Autonomous Driving: Integrating Video, Sensor, and Textual Data for Enhanced Decision-Making and Transparency. arXiv 2025, arXiv:2507.07938. Available online: http://arxiv.org/abs/2507.07938 (accessed on 20 August 2025).
Atakishiyev, S.; Salameh, M.; Yao, H.; Goebel, R. Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions. IEEE Access 2024, 12, 101603–101625. https://doi.org/10.1109/ACCESS.2024.3431437.
Rzig, D.E.; Iqbal, N.; Attisano, I.; Qin, X.; Hassan, F. Characterizing Virtual Reality Software Testing. arXiv 2022, arXiv:2211.01992. Available online: http://arxiv.org/abs/2211.01992 (accessed on 18 July 2025).
Otto, L.M.; Kaiser, M.; Seebacher, D.; Müller, S. Validation of AI-Based 3D Human Pose Estimation in a Cyber-Physical Environment. arXiv 2025, arXiv:2506.23739. Available online: http://arxiv.org/abs/2506.23739 (accessed on 18 August 2025).
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Dasinya Journal for Engineering and Informatics
This work is licensed under a Creative Commons Attribution 4.0 International License.
Dasinya journal is licensed under a 