Start of funding 01.07.2024

Multimodal Large Language Models for Analysis of Volumetric Radiology Images

Prof. Dr. Nassir Navab
Technische Universität München
Fakultät für Informatik

Prof. Dr. Akshay Chaudhari
Stanford University
Stanford Center for Artificial Intelligence in Medicine and Imaging (AIMI)



This collaborative project between the Technical University of Munich's CAMP and Stanford University's MIMI research groups focuses on developing multimodal large language models (MLLMs) for the analysis of volumetric radiology images. The research aims to extend current conversational radiology assistants by creating models that can process 3D images, such as CT scans, alongside text like patient history and clinical metadata. By integrating these diverse data types, the project seeks to enhance the interpretation and analysis of volumetric medical images for a holistic patient understanding. The team will evaluate the models' robustness and clinical correctness to ensure their suitability for real-world applications. This collaboration lays the groundwork for advanced conversational assistants in radiology, potentially improving diagnostic accuracy and efficiency in medical imaging interpretation.