Start of funding 01.01.2026

AI-guided automatic CT segmentation and colorization for mixed-reality path-tracing visualization

Prof. Dr. Daniel Roth
Technische Universität München
TUM University Hospital

Dr. Christoph Leuze
Stanford University
WU Tsai Insitute



CT scans, routinely taken in medical procedures, are typically inspected as grayscale slices. This limits intuitive spatial perception. High-quality 3D anatomy visualizations can provide more realistic depictions and improve understanding. A major barrier is designing the transfer functions that map CT density values to color.

We propose automating this transfer-function creation using deep learning. The goal is to allow users to assign colors at the anatomy level rather than by raw densities. Our pipeline will first segment CT volumes into anatomical structures and infer density ranges and. Combined with supplemental data like anatomical information and user supplied color, we can automatically generate the low-level transfer function.

The system will be able to optimize visualizations for objectives such as maximal contrast or while preserving user stylistic choices. We plan to automatically adapt parameters like color and brightness for different mixed-reality device types. We will integrate this with prior anatomy visualization work and provide a reproducible pipeline and code to facilitate research and clinical use.