Region-aware RGB and near-infrared image fusion

Jiacheng Ying, Can Tong, Zehua Sheng, Bowen Yao, Si Yuan Cao, Heng Yu, Hui Liang Shen

Research output: Journal PublicationArticlepeer-review

6 Citations (Scopus)

Abstract

This paper proposes a region-aware fusion method, called RaIF, for RGB and near-infrared (NIR) outdoor scenery image fusion. The method is motivated by the observation that current fusion approaches produce gray appearance in overexposed sky regions and distortion in vegetation regions. RaIF generates the region probability maps by exploiting their specific characteristics in the visible and NIR spectra. It recovers the overexposed sky regions by employing the intrinsic channel correlation between RGB and NIR images, and enhances the vegetation regions in an adjustable manner. RaIF formulates image fusion problem as a gradient-domain optimization problem with luminance and chromaticity regularizations. Experimental results validate the superiority of RaIF that produces fused images with improved appearance in the sky and vegetation regions, and achieves the state-of-the-art performance quantitatively and qualitatively. Furthermore, RaIF can act as a refinement module that improves the fusion results of current deep learning based approaches. It is also capable of recovering specular highlight regions other than sky overexposure.

Original languageEnglish
Article number109717
JournalPattern Recognition
Volume142
DOIs
Publication statusPublished - Oct 2023

Keywords

  • gradient-domain optimization
  • Image fusion
  • overexposed sky recovery
  • RGB and near-infrared
  • vegetation enhancement

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Region-aware RGB and near-infrared image fusion'. Together they form a unique fingerprint.

Cite this