Tracking and fusion for multiparty interaction with a virtual character and a social robot

Zerrin Yumak, Jianfeng Ren, Nadia Magnenat Thalmann, Junsong Yuan

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

7 Citations (Scopus)

Abstract

To give human-like capabilities to artificial characters, we should equip them with the ability of inferring user states. These artificial characters should understand the users' behaviors through various sensors and respond back using multimodal output. Besides natural multimodal interaction, they should also be able to communicate with multiple users and among each other in multiparty interactions. Previous work on interactive virtual humans and social robots mainly focuses on one-to-one interactions. In this paper, we study tracking and fusion aspects of multiparty interactions. We first give a general overview of our proposed multiparty interaction system and mention how it is different from previous work. Then, we provide the details of the tracking and fusion component including speaker identification, addressee detection and a dynamic user entrance/leave mechanism based on user re-identification using a Kinect sensor. Finally, we present a case study with the system and provide a discussion on the current capabilities, limitations and future work.

Original languageEnglish
Title of host publicationSIGGRAPH Asia 2014 Autonomous Virtual Humans and Social Robot for Telepresence, SA 2014
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450332439
DOIs
Publication statusPublished - 24 Nov 2014
Externally publishedYes
EventSIGGRAPH Asia 2014 Workshop on Autonomous Virtual Humans and Social Robot for Telepresence, SA 2014 - Shenzhen, China
Duration: 3 Dec 20146 Dec 2014

Publication series

NameSIGGRAPH Asia 2014 Autonomous Virtual Humans and Social Robot for Telepresence, SA 2014

Conference

ConferenceSIGGRAPH Asia 2014 Workshop on Autonomous Virtual Humans and Social Robot for Telepresence, SA 2014
Country/TerritoryChina
CityShenzhen
Period3/12/146/12/14

Keywords

  • Interactive virtual human
  • Multimodal fusion
  • Multiparty interaction
  • Social robot

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Fingerprint

Dive into the research topics of 'Tracking and fusion for multiparty interaction with a virtual character and a social robot'. Together they form a unique fingerprint.

Cite this