SPFusionNet: Sketch segmentation using multi-modal data fusion

Fei Wang, Shujin Lin, Hefeng Wu, Hanhui Li, Ruomei Wang, Xiaonan Luo, Xiangjian He

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

16 Citations (Scopus)

Abstract

The sketch segmentation problem remains largely unsolved because conventional methods are greatly challenged by the highly abstract appearances of freehand sketches and their numerous shape variations. In this work, we tackle such challenges by exploiting different modes of sketch data in a unified framework. Specifically, we propose a deep neural network SPFusionNet to capture the characteristic of sketch by fusing from its image and point set modes. The image modal component SketchNet learns hierarchically abstract ro-bust features and utilizes multi-level representations to produce pixel-wise feature maps, while the point set-modal component SPointNet captures local and global contexts of the sampled point set to produce point-wise feature maps. Then our framework aggregates these feature maps by a fusion network component to generate the sketch segmentation result. The extensive experimental evaluation and comparison with peer methods on our large SketchSeg dataset verify the effectiveness of the proposed framework.

Original languageEnglish
Title of host publicationProceedings - 2019 IEEE International Conference on Multimedia and Expo, ICME 2019
PublisherIEEE Computer Society
Pages1654-1659
Number of pages6
ISBN (Electronic)9781538695524
DOIs
Publication statusPublished - Jul 2019
Externally publishedYes
Event2019 IEEE International Conference on Multimedia and Expo, ICME 2019 - Shanghai, China
Duration: 8 Jul 201912 Jul 2019

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2019-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2019 IEEE International Conference on Multimedia and Expo, ICME 2019
Country/TerritoryChina
CityShanghai
Period8/07/1912/07/19

Keywords

  • Deep neural network
  • Multi-modal fusion
  • Sketch segmentation

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'SPFusionNet: Sketch segmentation using multi-modal data fusion'. Together they form a unique fingerprint.

Cite this