His-MMDM: Multi-domain and Multi-omics Translation of Histopathology Images with Diffusion Models

Abstract

AbstractGenerative AI (GenAI) has advanced computational pathology through various image translation models. These models synthesize histopathological images from existing ones, facilitating tasks such as color normalization and virtual staining. Current models, while effective, are mostly dedicated to specific source-target domain pairs and lack scalability for multi-domain translations. Here we introduce His-MMDM, a diffusion model-based framework enabling multi-domain and multi-omics histopathological image translation. His-MMDM can translate images across an unlimited number of categorical domains, enabling new applications like the translation of tumor images across various tumor types, while performing comparably to dedicated models on previous tasks such as transforming cryosectioned images to formalin-fixed paraffin-embedded (FFPE) ones. Additionally, it can perform genomics- and/or transcriptomics-guided editing of histopathological images, illustrating the impact of driver mutations and oncogenic pathway alterations on tissue histopathology. These versatile capabilities position His-MMDM as a versatile tool in the GenAI toolkit for future pathologists.