A New Method for Detecting Emotions in Video Using a Hybrid CNN-RNN-RBM Network

Facial expressions are one of the most important techniques of non-verbal communication that humans use to display their emotional states and establish non-verbal communication. In recent research in the field of emotion recognition based on facial expression changes, deep learning networks are used. In this regard, an appropriate network is selected according to the type of data. In the present research, video data is used, and for recognizing facial expressions, recurrent networks are employed. Subsequently, to achieve better results, a hybrid network consisting of Convolutional, Recurrent, and Restricted Boltzmann Machine networks has also been used. Finally, in this paper, using deep learning techniques and the Python language, an algorithm based on a hybrid CNN-LSTM-RBM neural network for emotion recognition through facial expressions in a video is presented. In this method, seven emotional states (happy, sad, disgust, excitement, contempt, fear, and anger) are detected. The CK+۴۸ dataset has been used to train and test the proposed model. The experimental results demonstrate that the proposed Hybrid CNN-LSTM-GRBM model achieves a training accuracy of ۹۹.۰۶%. Crucially, the model attains ۹۱.۱۶% test accuracy and an F۱-score of ۸۹.۲۷% on the test set, outperforming existing baseline methods and confirming its robustness in handling spatiotemporal facial features.

کلیدواژه ها:

Emotion Recognition ، Hybrid Neural Network ، Deep Learning ، Convolutional neural network ، Recurrent network ، Restricted Boltzmann Machine