Facial expression recognition using a hybrid CNN-SIFT aggregator

Tee Connie, Mundher Al-Shabi, Wooi Ping Cheah, Michael Goh

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

91 Citations (Scopus)


Deriving an effective facial expression recognition component is important for a successful human-computer interaction system. Nonetheless, recognizing facial expression remains a challenging task. This paper describes a novel approach towards facial expression recognition task. The proposed method is motivated by the success of Convolutional Neural Networks (CNN) on the face recognition problem. Unlike other works, we focus on achieving good accuracy while requiring only a small sample data for training. Scale Invariant Feature Transform (SIFT) features are used to increase the performance on small data as SIFT does not require extensive training data to generate useful features. In this paper, both Dense SIFT and regular SIFT are studied and compared when merged with CNN features. Moreover, an aggregator of the models is developed. The proposed approach is tested on the FER-2013 and CK+ datasets. Results demonstrate the superiority of CNN with Dense SIFT over conventional CNN and CNN with SIFT. The accuracy even increased when all the models are aggregated which generates state-of-art results on FER-2013 and CK+ datasets, where it achieved 73.4% on FER-2013 and 99.1% on CK+.

Original languageEnglish
Title of host publicationMulti-disciplinary Trends in Artificial Intelligence - 11th International Workshop, MIWAI 2017, Proceedings
EditorsSomnuk Phon-Amnuaisuk, Swee-Peng Ang, Soo-Young Lee
PublisherSpringer Verlag
Number of pages11
ISBN (Print)9783319694559
Publication statusPublished - 2017
Externally publishedYes
Event11th Multi-disciplinary International Workshop on Artificial Intelligence, MIWAI 2017 - Gadong, Brunei Darussalam
Duration: 20 Nov 201722 Nov 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10607 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference11th Multi-disciplinary International Workshop on Artificial Intelligence, MIWAI 2017
Country/TerritoryBrunei Darussalam


  • CNN
  • Dense SIFT
  • Facial expression recognition
  • SIFT

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Facial expression recognition using a hybrid CNN-SIFT aggregator'. Together they form a unique fingerprint.

Cite this