Spatial Context-Aware Object-Attentional Network for Multi-Label Image Classification

Jialu Zhang, Jianfeng Ren, Qian Zhang, Jiang Liu, Xudong Jiang

Research output: Journal PublicationArticlepeer-review

16 Citations (Scopus)

Abstract

Multi-label image classification is a fundamental but challenging task in computer vision. To tackle the problem, the label-related semantic information is often exploited, but the background context and spatial semantic information of related objects are not fully utilized. To address these issues, a multi-branch deep neural network is proposed in this paper. The first branch is designed to extract the discriminant information from regions of interest to detect target objects. In the second branch, a spatial context-aware approach is proposed to better capture the contextual information of an object in its surroundings by using an adaptive patch expansion mechanism. It helps the detection of small objects that are easily lost without the support of context information. The third one, the object-attentional branch, exploits the spatial semantic relations between the target object and its related objects, to better detect partially occluded, small or dim objects with the support of those easily detectable objects. To better encode such relations, an attention mechanism jointly considering the spatial and semantic relations between objects is developed. Two widely used benchmark datasets for multi-labeling classification, MS COCO and PASCAL VOC, are used to evaluate the proposed framework. The experimental results demonstrate that the proposed method outperforms the state-of-the-art methods for multi-label image classification.

Original languageEnglish
Pages (from-to)3000-3012
Number of pages13
JournalIEEE Transactions on Image Processing
Volume32
DOIs
Publication statusPublished - 2023

Keywords

  • Multi-label image classification
  • adaptive patch expansion
  • object clustering
  • spatial context-aware object detection
  • spatial semantic attention

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Spatial Context-Aware Object-Attentional Network for Multi-Label Image Classification'. Together they form a unique fingerprint.

Cite this