Learning to Correct Unevenly Exposed Face Images Using RGB-NIR Pairs

We propose the face image correction network, named FICNet, to tackle the unevenly exposure problem by enhancing underexposure and restoring overexposure. Our work is inspired by the observation that facial images may suffer from uneven exposure in portrait photography and facial selfies under the conditions of complicated indoor lighting. Previous works rely on a single image to deal with this abnormal exposure problem, but they meet the problem of lost textures recovery. In contrast, considering the spectrum irradiance of indoor lights, this work introduces an additional NIR image, and uses the complementary characteristics of the RGB-NIR pair to correct the unevenly exposed face image. To adjust the abnormal luminance distribution, our FICNet introduces an initial exposure correction (IEC) module to adjust the RGB input features. In addition, to recover the lost textures in overexposed regions, FICNet employs cascaded deformable fusion (CDF) modules to effectively aggregate RGB and NIR branch features via spatial non-adjacent sampling and offset transferring. Furthermore, for robust model training, we devise three kinds of abnormal exposure simulation using linear and Gaussian exposure models and S-shaped mapping on face images to generate diverse training data from limited captured images. Experimental results validate that our method outperforms the state-of-the-art image fusion, guided restoration, and exposure correction approaches. Furthermore, FICNet can also improve the effect of face parsing and face recognition.