Rain-component-aware capsule-GAN for single image de-raining

Fei Yang, Jianfeng Ren, Zheng LU, Jialu Zhang, Qian Zhang

Research output: Journal PublicationArticlepeer-review

3 Citations (Scopus)

Abstract

Images taken in the rain usually have poor visual quality, which may cause difficulties for vision-based analysis systems. The research aims to recover clean image content from a single rainy image by removing rain components without introducing any artifacts. Existing rain removal methods often model the rain component as noise, but it obviously has clear patterns instead of random noise. Motivated by this, we raise the idea to build modules to capture rain patterns for de-raining. A Rain-Component-Aware (RCA) network is proposed to capture the characteristics of the rain. We then integrate it into an image-conditioned generative adversarial network (image-cGAN) as a RCA loss to guide the generation of rainless images. This results in the proposed two-branch cGAN, where one branch aims at improving the image visual quality after de-raining, and the other aims at extracting rain patterns so that the rain could be effectively removed. To better capture the spatial relationship of different objects within an image, we incorporate the capsule structure in both generator and discriminator of cGAN, which further improves the quality of generated images. The proposed approach is hence named as RCA-cGAN. Benefited by the RCA loss based two-branch optimization and the capsule structure, RCA-cGAN achieves good de-raining effect. Extensive experimental results on several benchmark datasets show that the RCA network is effective to capture rain patterns and the proposed approach could produce much better de-raining images in terms of both subjective visual quality inspection and objective quantitative assessment.

Original languageEnglish
Article number108377
JournalPattern Recognition
Volume123
DOIs
Publication statusPublished - Mar 2022

Keywords

  • Capsule
  • De-raining
  • Generative adversarial network
  • Rain-component-aware network

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Rain-component-aware capsule-GAN for single image de-raining'. Together they form a unique fingerprint.

Cite this