A Transformer-Based Approach for Class-Imbalanced Binary Classification

Class imbalance is a pervasive challenge in machine learning, often leading to biased models that prioritize majority classes while neglecting critical minority samples. This study investigates the effectiveness of a transformer-based neural network for binary classification on imbalanced data, leveraging self-attention mechanisms to dynamically capture feature relationships. The proposed architecture integrates multiple transformer blocks with multi-head attention, layer normalization, and dropout to enhance feature representation and prevent overfitting. Class weights are incorporated during training to address imbalance without altering the dataset distribution. The model's performance is evaluated using standard metrics, including precision, recall, F۱-score, and area under the ROC curve (AUC), while training dynamics are visualized through accuracy and loss trajectories across epochs. Results demonstrate the model's robustness in handling class imbalance, achieving high discriminative performance with minimal preprocessing. The transformer's ability to weigh feature importance adaptively proves particularly advantageous for imbalanced datasets, where traditional models may struggle. Furthermore, the integration of regularization techniques ensures stable training and generalization. The visual analysis of training metrics underscores the model's consistent convergence and balanced learning across classes.

کلیدواژه ها:

Transformer ، Class Imbalance ، Attention Mechanism ، Binary Classification ، Deep Learning