MoVE-CNNs: Model aVeraging Ensemble of Convolutional Neural Networks for Facial Expression Recognition

Jing Xuan Yu, Kian Ming Lim, Chin Poo Lee

Research output: Journal PublicationArticlepeer-review

5 Citations (Scopus)


Facial expression is a powerful non-verbal communication that can express emotions and messages without saying a single word. In view of the prominence of facial expression, we propose a model averaging ensemble of Convolutional Neural Networks (CNN) that consolidates multiple pre-trained CNN models. Each pre-trained CNN model first undergoes transfer learning with the classification layer substituted with a multilayer perceptron. The newly formed model is then fine-tuned on the facial expression datasets and adapted to facial expression recognition. The predictions returned by all models are combined by model averaging to determine the final class probability distributions. The proposed model averaging ensemble of CNNs is evaluated on three facial expression datasets: FER-2013, modified CK+ and RAF-DB. Since the modified CK+ dataset is a small dataset, data augmentation is leveraged to increase the size and diversity of data. Apart from that, oversampling is adopted to address the class imbalance challenge in RAF-DB. The empirical results demonstrate that the proposed model averaging ensemble of CNNs outperforms the individual ensemble model at the test accuracy of 77.70%, 94.10% and 87.50% in FER 2013, modified CK+ and RAF-DB datasets, respectively.

Original languageEnglish
Pages (from-to)1-5
Number of pages5
JournalIAENG International Journal of Computer Science
Issue number3
Publication statusPublished - 2021
Externally publishedYes


  • convolutional neural network
  • data augmentation
  • ensemble
  • facial expression
  • facial expression recognition
  • model averaging
  • oversampling
  • transfer learning

ASJC Scopus subject areas

  • General Computer Science


Dive into the research topics of 'MoVE-CNNs: Model aVeraging Ensemble of Convolutional Neural Networks for Facial Expression Recognition'. Together they form a unique fingerprint.

Cite this