Data Augmentation for Low-Level Vision: CutBlur and Mixture-of-Augmentation

Abstract

Data augmentation (DA) is an effective way to improve the performance of deep networks. Unfortunately, current methods are mostly developed for high-level vision tasks (eg, image classification) and few are studied for low-level (eg, image restoration). In this paper, we provide a comprehensive analysis of the existing DAs in the frequency domain. We find that the methods that largely manipulate the spatial information can hinder the image restoration process and hurt the performance. Based on our analyses, we propose CutBlur and mixture-of-augmentation (MoA). CutBlur cuts a low-quality patch and pastes it to the corresponding high-quality image region, or vice versa. The key intuition is to provide enough DA effect while keeping the pixel distribution intact. This characteristic of CutBlur enables a model to learn not only “how” but also “where” to reconstruct an image. Eventually, the model understands “how much” to restore given pixels, which allows it to generalize better to unseen data distributions. We further improve the restoration performance by MoA that incorporates the curated list of DAs. We demonstrate the effectiveness of our methods by conducting extensive experiments on several low-level vision tasks on both single or a mixture of distortion tasks. Our results show that CutBlur and MoA consistently and significantly improve the performance especially when the model size is big and the data is collected under real-world environments. Our code is available at https://github.com/clovaai/cutblur.