AttentionPool-CNN: Enhancing RoBERTa-Based Sentiment Analysis Through Adaptive Feature Aggregation and Multi-Kernel Convolution

Sentiment classification of user-generated text has become a cornerstone problem in applied natural language processing, with practical applications spanning consumer analytics, healthcare feedback mining, and financial market monitoring. The emergence of large-scale pre-trained transformer models has substantially elevated the performance ceiling for this task, yet the design of lightweight, interpretable, and accurate task-specific classifiers that sit atop these encoders remains an active area of investigation. This paper presents a systematically improved architecture for binary sentiment classification, building upon and empirically surpassing the R-CNN model that achieved ۹۱.۵۰% accuracy on the IMDB Large Movie Review Dataset (Maas et al., ۲۰۱۱). Three targeted modifications are proposed: (i) selective unfreezing of the two uppermost RoBERTa encoder layers to enable domain-specific adaptation while preserving the broad linguistic knowledge captured during pre-training; (ii) replacement of a single-width convolutional filter with three parallel filters of kernel sizes ۳, ۴, and ۵, allowing the model to detect sentiment-bearing n-gram patterns at multiple linguistic granularities simultaneously; and (iii) substitution of Global Max Pooling with a Self-Attention Pooling mechanism that computes a context-sensitive weighted aggregation of the post-convolutional feature sequence. Comprehensive experiments under strictly controlled conditions demonstrate that the proposed model achieves ۹۲.۳۲% test accuracy and an Area Under the Receiver Operating Characteristic Curve (AUC) of ۰.۹۷۶۴—an absolute accuracy improvement of ۰.۸۲ percentage points over the reference system. Beyond raw discriminative performance, the learned attention weights provide token-level interpretability, enabling inspection of which lexical regions most influence each prediction. The confusion matrix analysis confirms symmetric improvement across both sentiment classes, and the ROC curve indicates strong discriminability across all operating thresholds. These results demonstrate that well-motivated, targeted architectural changes to an established baseline can produce meaningful and consistent gains.

کلیدواژه ها:

Sentiment analysis ، RoBERTa ، Convolutional neural networks ، Self-attention pooling ، Multi-scale feature extraction ، Partial fine-tuning ، IMDB dataset ، Natural language processing ، Binary text classification