Recently, makeup transfer task has been widely explored with the development of deep learning. However, existing methods have shortcomings in more complex lighting situations in the real world because they do not consider the interference of lighting factors on facial features. To solve the above problem, we propose a local highlight and shadow adaptively repairing GAN for illumination-robust makeup transfer. We first map the 2D face images to UV representations and perform makeup transfer in the UV texture space, which explicitly removes the spatial misalignment to achieve pose and expression invariant makeup transfer. Furthermore, we take advantage of the face symmetry in the UV texture space to design an illumination repair module. It can adaptively repair the features affected by asymmetric local highlight and shadow based on a process of flipping and multi-layer attention fusion. In addition, the multi-layer attention maps are obtained by a pre-trained illumination classification network and hence have the ability to indicate local highlight and shadow areas. Comprehensive experiment results demonstrate the consistent effectiveness and clear advantages of our method, which significantly improve the robustness against local light effects and generate natural transfer results.
Research Article
Open Access