Multimodal Fusion-Based Analysis and Detection of Fatigue Driving

Abstract

Fatigue driving is a major hazard to road safety and a common cause of traffic accidents. Developing effective methods to monitor and identify driver fatigue for early warnings is a critical research focus. Traditional detection methods often rely on monitoring specific physiological parameters, but these can be less effective due to environmental interferences like poor image quality, complex lighting, or unstable equipment. In contrast, computer vision-based methods offer accurate, non-invasive detection and a better user experience, though they still face challenges such as high error rates and limited robustness due to individual variations and the ambiguous nature of fatigue. Addressing these issues, this paper introduces a fusion model that combines facial detection and head posture analysis using a multi-task convolutional neural network (MTCNN). This model, utilizing a multi-task cascade architecture, simultaneously performs comprehensive facial posture analysis and facial detection. Additionally, a lightweight network cascade based on SqueezeNet is employed to detect facial keypoints, identifying 72-point coordinates crucial for assessing fatigue. By extracting fatigue features from these keypoints, a multimodal fusion approach to determine fatigue is proposed. The paper concludes with the development of an online fatigue detection and early warning system, showcasing practical applications of these research findings