FACLSTM: ConvLSTM with focused attention for scene text recognition

Qingqing Wang, Ye Huang, Wenjing Jia, Xiangjian He, Michael Blumenstein, Shujing Lyu, Yue Lu

Research output: Journal PublicationArticlepeer-review

23 Citations (Scopus)

Abstract

Scene text recognition has recently been widely treated as a sequence-to-sequence prediction problem, where traditional fully-connected-LSTM (FC-LSTM) has played a critical role. Owing to the limitation of FC-LSTM, existing methods have to convert 2-D feature maps into 1-D sequential feature vectors, resulting in severe damages of the valuable spatial and structural information of text images. In this paper, we argue that scene text recognition is essentially a spatiotemporal prediction problem for its 2-D image inputs, and propose a convolution LSTM (ConvLSTM)-based scene text recognizer, namely, FACLSTM, i.e., focused attention ConvLSTM, where the spatial correlation of pixels is fully leveraged when performing sequential prediction with LSTM. Particularly, the attention mechanism is properly incorporated into an efficient ConvLSTM structure via the convolutional operations and additional character center masks are generated to help focus attention on right feature areas. The experimental results on benchmark datasets IIIT5K, SVT and CUTE demonstrate that our proposed FACLSTM performs competitively on the regular, low-resolution and noisy text images, and outperforms the state-of-the-art approaches on the curved text images with large margins.

Original languageEnglish
Article number120103
JournalScience China Information Sciences
Volume63
Issue number2
DOIs
Publication statusPublished - 1 Feb 2020
Externally publishedYes

Keywords

  • convolutional LSTM
  • focused attention
  • scene text recognition
  • sequential prediction
  • spatial correlation

ASJC Scopus subject areas

  • Computer Science (all)

Fingerprint

Dive into the research topics of 'FACLSTM: ConvLSTM with focused attention for scene text recognition'. Together they form a unique fingerprint.

Cite this