The Audience Effect: Do Observations Change Outcomes in HCI Studies?

Observational studies are widely used in Human-Computer Interaction (HCI) research to evaluate usability and user experience with technologies. However, the act of observation may influence participant behaviour and performance, threatening the validity of study findings. This paper investigates the impact of three observation types on participant outcomes in a simulated HCI study context. Participants completed Sudoku puzzles under baseline (no observation), human observation, sensor-based observation, and combined human/sensor conditions. Performance was assessed by puzzle completion rates. The mental workload was measured via NASA-TLX surveys, heart rate, galvanic skin response, and infrared thermal imaging. Results showed observations negatively impacted performance versus baseline, with human observers inducing the greatest distraction. Experienced participants were more influenced than novices. Task medium also affected engagement and observation reactivity. Findings demonstrate observations introduce bias in HCI research, emphasising careful consideration of observation methods to improve result validity.