Evaluating user experience of adaptive digital educational games with Activity Theory

Effie Lai Chong Law, Xu Sun

Research output: Journal PublicationArticlepeer-review

80 Citations (Scopus)

Abstract

Adaptive digital educational games (DEGs) providing players with relevant interventions can enhance gameplay experience. This advance in game design, however, renders the user experience (UX) evaluation of DEGs even more challenging. To tackle this challenge, we developed a four-dimension evaluation framework (i.e.; gaming experience, learning experience, adaptivity, and usability) and applied it to an empirical study with a DEG on teaching geography. Mixed-method approaches were adopted to collect data with 16 boys aged 10-11. Specifically, a so-called Dyadic User Experience Tests (DUxT) was employed; participants were paired up to assume different roles during gameplay. Learning efficacy was evaluated with a pre-post intervention measurement using a domain-specific questionnaire. Learning experience, gaming experiences and usability were evaluated with intensive in situ observations and interviews guided by a multidimensional scheme; content analysis of these transcribed audio data was supplemented by video analysis. Effectiveness of adaptivity algorithms was planned to be evaluated with automatic logfiles, which, unfortunately, could not be realised due to some technical problem. Nonetheless, the user-based data could offer some insights into this issue. Furthermore, we attempted to bridge the existing gap in UX research - the lack of theoretical frameworks in understanding user experience - by adopting Engeströms (1987) extended framework of Activity Theory (AT) that provides contextual information essential for understanding contradictions and breakdowns observed in the interactions between the game players. The dyadic gameplay setting allows us to explore the issue of group UX. Implications for further applications of the AT framework in the UX research, especially the interplay between evaluation and redesign (i.e.; downstream utility of UX evaluation methods), are discussed.

Original languageEnglish
Pages (from-to)478-497
Number of pages20
JournalInternational Journal of Human Computer Studies
Volume70
Issue number7
DOIs
Publication statusPublished - Jul 2012

Keywords

  • Activity Theory
  • Adaptivity
  • Breakdowns
  • Contradictions
  • Digital educational game
  • Downstream utility
  • Usability
  • User experience

ASJC Scopus subject areas

  • Software
  • Human Factors and Ergonomics
  • Education
  • Engineering (all)
  • Human-Computer Interaction
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Evaluating user experience of adaptive digital educational games with Activity Theory'. Together they form a unique fingerprint.

Cite this