A Novel Transformer-Based Multiscale Fusion Network for Fetal Diseases Diagnosis During Pregnancy

Early detection of fetal anomalies such as neural tube defects (NTD) and congenital heart diseases (CHD) is one of the important challenges in prenatal medicine. Traditional methods such as ultrasound and MRI, despite their widespread use, suffer from operator dependency and limited diagnostic accuracy in complex cases. This paper introduces a novel architecture named Multimodal Multiscale Fusion Network, a deep learning model that integrates three heterogeneous data sources: spatio-temporal features of ultrasound images, ۳D/۴D MRI images using an improved U-Net with residual connections, and genetic features with an MLP network. These features are then processed by a Vision Transformer (ViT) with a multi-head attention mechanism for adaptive inter-modal fusion. Grad-CAM is used to generate interpretable attention maps. The model was implemented in TensorFlow and evaluated on a dataset including ultrasound images, MRI and corresponding genetic profiles using ۱۰-fold cross-validation and extensive ablation studies. The results show that it achieves ۹۷.۴% accuracy, ۹۴.۲% sensitivity, ۹۸.۸% specificity, and an AUC-ROC of ۰.۹۹۲, which shows better performance than single-modality models (CNN, U-Net, FINE). This architecture is an important advancement toward standardization and intelligentization of diagnosis during pregnancy that can have clinical application.

کلیدواژه ها:

Multimodality Integratio ، Deep learning ، Vision Transformer ، Fetal ultrasound ، Fetal MRI