SCQT-MaxViT: Speech Emotion Recognition with Constant-Q Transform and Multi-Axis Vision Transformer

Kah Liang Ong, Chin Poo Lee, Heng Siong Lim, Kian Ming Lim, Takeki Mukaida

Research output: Journal PublicationArticlepeer-review

1 Citation (Scopus)

Abstract

Speech emotion recognition presents a significant challenge within the field of affective computing, requiring the analysis and detection of emotions conveyed through speech signals. However, existing approaches often rely on traditional signal processing techniques and handcrafted features, which may not effectively capture the nuanced aspects of emotional expression. In this paper, an approach named 'SCQT-MaxViT' is proposed for speech emotion recognition, combining signal processing, computer vision, and deep learning techniques. The method utilizes the Constant-Q Transform (CQT) to convert speech waveforms into spectrograms, providing high-frequency resolution and enabling the model to capture intricate emotional details. Additionally, the Multi-axis Vision Transformer (MaxViT) is employed for further representation learning and classification of the CQT spectrograms. MaxViT incorporates a multi-axis self-attention mechanism, facilitating both local and global interactions within the network and enhancing the ability of the model to learn meaningful features. Furthermore, the dataset is augmented using random time masking techniques to enhance the generalization capabilities. Achieving accuracies of 88.68% on the Emo-DB dataset, 77.54% on the RAVDESS dataset, and 62.49% on the IEMOCAP dataset, the proposed SCQT-MaxViT method exhibits promising performance in capturing and recognizing emotions in speech signals.

Original languageEnglish
Pages (from-to)63081-63091
Number of pages11
JournalIEEE Access
Volume11
DOIs
Publication statusPublished - 2023
Externally publishedYes

Keywords

  • constant-Q transform
  • Emo-DB
  • IEMOCAP
  • multi-axis vision transformer
  • RAVDESS
  • spectrogram
  • Speech
  • speech emotion
  • speech emotion recognition
  • vision transformer

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering

Fingerprint

Dive into the research topics of 'SCQT-MaxViT: Speech Emotion Recognition with Constant-Q Transform and Multi-Axis Vision Transformer'. Together they form a unique fingerprint.

Cite this