Medicine Without Boundaries: Generative AI for Translating Medical Data Across Modalities

Generative artificial intelligence (AI) is transforming digital health by enabling synthetic data generation, enhancement, and multimodal integration. In this work, we investigate the role of generative models in clinical data translation, introducing contributions for intra-modality, inter-modality, and any-to-any translations. Intra-modality methods focus on improving data quality within a single modality, e.g., for example, through denoising, super-resolution, or harmonization. Inter-modality models enable translation between distinct modalities, e.g., generating PET from CT or CESM from mammography, supporting safer or more accessible diagnostics. Finally, any-to-any frameworks combine heterogeneous data types, including imaging, text, and signals, within shared latent spaces to enable multimodal synthesis and digital twin construction. We provide representative examples for each category, discuss their clinical relevance and methodological underpinnings, and outline the challenges that must be addressed to integrate generative AI into healthcare.