A tumor inpainting framework for MRI using automated masks based on channel-specific conditions across the volume

Abstract

Despite significant advancements in deep learning for medical screening, generating realistic annotation labels for specific disease groups remains a critical challenge. To address this, we propose an inpainting framework that employs realistic annotation masks, integrating channel-specific conditions across the image volume to produce high-quality paired datasets. Specifically, free-form annotation masks for 2D normal MR images were generated using a variational autoencoder (VAE) adjusted along the z-axis. Additionally, localized areas were inpainted using generative adversarial networks (GANs) with a cascaded generator featuring mask- guided boundary attention and a discriminator operating in a hyperspherical embedding space. This approach ensures the synthesis of high-coverage tumors with seamless integration into adjacent tissues. The proposed framework outperformed existing models, achieving a low FID score of 32.43, demonstrating its ability to generate highly realistic datasets. Moreover, when applied to data augmentation for downstream segmentation tasks, it improved the Dice score from 0.749 to 0.780, demonstrating its potential to enhance segmentation performance. These results underscore the framework's effectiveness in addressing critical challenges in building paired datasets in the medical domain. Our trained model and inference code are available on github.com/kyurikeem/Tumor-Inpainting.