Topic classification of social networks contents: Text and graphical features fusion using transformer-based architecture

The content produced in social networks may have different textual, visual, or audio structures. Each of these structures can be used to classify generated content. A significant number of produced contents have both textual and graphical features. Some of them, such as the stories published on Instagram, have the usual text and graphical features. In addition to text features, background color, text color, and font as graphical features can be used to improve the accuracy of the classification model. In this research, our ۳۶۶۰ Persian data published in Instagram stories have been used for the dataset. The data has been divided into ۱۸ different classes by human supervision. The ۸۰% of the data has been used for training and ۲۰% remaining for testing the learning model. The approach of this research is to use transformer architecture and a multilingual model for text classification and a neural network for graphical features classification and then combine these two classification models in one model based on ensemble learning. The obtained results of proposed method show about ۱۰% improvement in accuracy and F۱-score respected to text classification.