Understanding the role of driver fatigue in takeover performance using non-contact multimodal sensing

Abstract

Conditionally automated driving (SAE Level 3) requires drivers to resume manual control upon a takeover request (TOR), creating a safety-critical transition in which driver state plays a decisive role. While existing driver monitoring systems primarily focus on detecting acute drowsiness through behavioural cues, they provide limited insight into the gradual accumulation of task-related mental fatigue that emerges during prolonged automated supervision. This thesis develops and validates a non-contact, multimodal, and temporally continuous framework for modelling driver fatigue and evaluating its behavioural relevance for takeover performance.
Across three interlinked studies, the research progresses from physiological validation to behavioural verification. Study 1 establishes the engineering feasibility of a non-contact sensing architecture integrating facial thermal imaging and millimetre-wave radar, and provides preliminary construct-related validity evidence by examining whether sensor-derived features show physiologically plausible and theoretically consistent patterns alongside subjective fatigue and observer-rated drowsiness. Under a 40-minute monotonous Level 3 simulation, significant time-on-task effects were observed in subjective fatigue and observer-rated drowsiness. Thermal features from central facial regions (e.g., forehead and nasal tip) and radar-derived cardiopulmonary indicators demonstrated physiologically plausible trends, supporting their sensitivity to autonomic modulation. Notably, subjective fatigue and behavioural drowsiness exhibited weak individual-level correlation, suggesting that fatigue operates as a partially latent psychophysiological state.
Study 2 advances a hierarchical temporal modelling framework to estimate fatigue as a continuous trajectory rather than a binary state. A Multilayer Perceptron (MLP) regression model trained under Leave-One-Subject-Out validation achieved strong alignment with observer-rated fatigue (r ≈ 0.79), outperforming ensemble alternatives in trajectory coherence. Incorporating temporal derivatives and stacked meta-features significantly improved the detection of high-risk states, demonstrating the importance of modelling fatigue accumulation dynamics.
Study 3 evaluates the behavioural validity of model-derived fatigue indices by linking event-level predicted fatigue to objective takeover performance metrics. Predicted fatigue scores were significantly associated with increased post-TOR vehicle control instability, operationalised through a composite Driving Roughness index (r ≈ 0.49). In contrast, isolated reaction-time measures were comparatively insensitive. These findings indicate that fatigue primarily degrades control quality rather than response initiation speed, underscoring the limitations of takeover time as a sole safety metric.
Collectively, this thesis contributes a coherent sensing–modelling–validation pipeline that reframes driver fatigue as a continuous latent process with measurable safety consequences. By integrating non-contact physiological sensing with temporal trajectory modelling and behavioural validation, the proposed framework advances driver monitoring beyond discrete drowsiness detection toward proactive, safety-oriented state estimation in conditionally automated vehicles.

Date of Award	18 Jul 2026
Original language	English
Awarding Institution	University of Nottingham
Supervisor	Xu Sun (Supervisor) & Jiang Wu (Supervisor)