Adaptive Pilot Fatigue Detection Using Hybrid Computer Vision-EEG Fusion Models for Autonomous AI Co-Pilot Activation in Next-Generation Cockpit Systems

Pilot fatigue remains one of the most critical human-factor risks in modern aviation, responsible for degraded situational awareness, delayed reaction times, and a substantial proportion of human-induced incidents. Traditional monitoring relies heavily on subjective assessments and isolated biometric indicators, which fail to capture the multidimensional neuro-cognitive state of pilots in real time. This study introduces a novel hybrid fatigue-detection framework integrating deep computer vision networks with advanced EEG-driven neurocognitive modeling to form a unified, high-fidelity state-estimation engine capable of activating an autonomous AI co-pilot module during high-risk conditions. The proposed system leverages a dual-stream architecture combining (۱) a spatiotemporal facial-microexpression and ocular-dynamics encoder based on ۳D-CNN + Vision Transformer fusion, and (۲) an EEG transformer using cross-channel self-attention to extract early neural signatures of cognitive decline. A Bayesian Multimodal Fusion Layer is introduced to model uncertainty, dynamically adjust modality weights based on signal quality, and produce a continuous fatigue-probability index. To enable safe cockpit autonomy, this index is tightly coupled with a Reinforcement Learning-based Autonomous Co-Pilot (RL-ACP) that modulates assistance levels-from subtle stabilization to full trajectory control-under strict aviation safety constraints. A closed-loop simulation using X-Plane ۱۲ and NeuroSim demonstrates that the proposed system reduces undetected fatigue intervals by up to ۴۷%, and decreases high-risk control deviations by ۳۴% compared to existing unimodal methods. This research contributes a fully integrated theoretical-computational architecture, validated through high-resolution simulations, and represents a significant step toward next-generation AI-assisted cockpit systems where neurophysiological insight and real-time visual analytics jointly enhance flight safety and pilot well-being.

کلیدواژه ها:

Pilot Fatigue Detection ، EEG Analysis ، Computer Vision ، Vision Transformer ، Hybrid Multimodal Fusion ، Autonomous Co-Pilot Systems ، Reinforcement Learning ، Cognitive State Estimation ، Aviation Safety ، Neuroergonomics