Multimodal Semantic Segmentation in Yangtze River Economic Belt: Outcome of the 2024 IEEE WHISPERS MMSeg-YREB Challenge

Abstract

With the growing availability of remote sensing (RS) data from diverse platforms, multimodal RS techniques have emerged as a transformative solution for large-scale semantic segmentation. In response, we developed MMSeg-YREB, a specialized framework that integrates complementary RS modalities, such as multispectral and synthetic aperture radar data from Sentinel-1/2 sources, to enhance the accuracy and robustness of land use and land cover mapping across urban and regional landscapes within the Yangtze River Economic Belt (YREB). By leveraging extensive geographic coverage and heterogeneous data sources, MMSeg-YREB supports a wide range of applications, from precise urban planning to comprehensive environmental monitoring. Utilizing state-of-the-art artificial intelligence methodologies, this framework aims to develop highly generalizable and scalable semantic segmentation models, driving methodological advancements and accelerating the adoption of Earth observation technologies across diverse regions. As part of this initiative, the multimodal semantic segmentation challenge, i.e., MMSeg-YREB, is organized in conjunction with the 14th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing 2024. To foster further research and innovation, all datasets and code will be publicly released online for the sake of reproducibility, contributing to the broader Earth observation and RS communities.