Multi Task-Based Facial Expression Synthesis with Supervision Learning and Feature Disentanglement of Image Style

Wenya Lu, Zhibin Peng, Cheng Luo, Weicheng Xie, Jiajun Wen, Zhihui Lai, Linlin Shen

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

Image-to-Image synthesis paradigms have been widely used for facial expression synthesis. However, current generators are apt to either produce artifacts for largely posed and non-aligned faces or unduly change the identity information like AdaIN-based generator. In this work, we suggest to use image style feature to surrogate the expression cues in the generator, and propose a multi-task learning paradigm to explore this style information via the supervision learning and feature disentanglement. While the supervision learning can make the encoded style specifically represent the expression cues and enable the generator to produce correct expression, the feature disentanglement of content and style cues enables the generator to better preserve the identity information in expression synthesis. Experimental results show that the proposed algorithm can well reduce the artifacts for the synthesis of posed and non-aligned expressions, and achieves competitive performances in terms of FID, PNSR and classification accuracy, compared with four publicly available GANs. The code and pre-trained models are available at https://github.com/lumanxi236/MTSS.

Original languageEnglish
Title of host publication2023 IEEE International Conference on Image Processing, ICIP 2023 - Proceedings
PublisherIEEE Computer Society
Pages1005-1009
Number of pages5
ISBN (Electronic)9781728198354
DOIs
Publication statusPublished - 2023
Externally publishedYes
Event30th IEEE International Conference on Image Processing, ICIP 2023 - Kuala Lumpur, Malaysia
Duration: 8 Oct 202311 Oct 2023

Publication series

NameProceedings - International Conference on Image Processing, ICIP
ISSN (Print)1522-4880

Conference

Conference30th IEEE International Conference on Image Processing, ICIP 2023
Country/TerritoryMalaysia
CityKuala Lumpur
Period8/10/2311/10/23

Keywords

  • expression style learning
  • Facial expression synthesis
  • multi-task learning
  • style and content disentanglement

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint

Dive into the research topics of 'Multi Task-Based Facial Expression Synthesis with Supervision Learning and Feature Disentanglement of Image Style'. Together they form a unique fingerprint.

Cite this