A feature covariance matrix with serial particle filter for isolated sign language recognition

Kian Ming Lim, Alan W.C. Tan, Shing Chiang Tan

Research output: Journal PublicationArticlepeer-review

57 Citations (Scopus)

Abstract

As is widely recognized, sign language recognition is a very challenging visual recognition problem. In this paper, we propose a feature covariance matrix based serial particle filter for isolated sign language recognition. At the preprocessing stage, the fusion of the median and mode filters is employed to extract the foreground and thereby enhances hand detection. We propose to serially track the hands of the signer, as opposed to tracking both hands at the same time, to reduce the misdirection of target objects. Subsequently, the region around the tracked hands is extracted to generate the feature covariance matrix as a compact representation of the tracked hand gesture, and thereby reduce the dimensionality of the features. In addition, the proposed feature covariance matrix is able to adapt to new signs due to its ability to integrate multiple correlated features in a natural way, without any retraining process. The experimental results show that the hand trajectories as obtained through the proposed serial hand tracking are closer to the ground truth. The sign gesture recognition based on the proposed methods yields a 87.33% recognition rate for the American Sign Language. The proposed hand tracking and feature extraction methodology is an important milestone in the development of expert systems designed for sign language recognition, such as automated sign language translation systems.

Original languageEnglish
Pages (from-to)208-218
Number of pages11
JournalExpert Systems with Applications
Volume54
DOIs
Publication statusPublished - 15 Jul 2016
Externally publishedYes

Keywords

  • Feature covariance matrix
  • Serial particle filter
  • Sign language recognition

ASJC Scopus subject areas

  • General Engineering
  • Computer Science Applications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'A feature covariance matrix with serial particle filter for isolated sign language recognition'. Together they form a unique fingerprint.

Cite this