تشخيص نوع تبليغات‌جهت‌دار در ميم با استفاده از شبكه عصبي عميق

Automatic detection of propagan‎da techniques in memes is of particular importance in multimodal processing due to its critical role in combating misinformation an‎d manipulation of public opinion. The primary challenge addressed in this research is overcoming the limitations of existing models in terms of predictive accuracy an‎d improving the Hierarchical F1 (HF1) score, a metric for which current systems face limitations due to the complexities of integrating textual an‎d visual information. Another obstacle is the severe class imbalance in the dataset, particularly in the frequency distribution of propagan‎da techniques, which causes models to struggle with accurately identifying underrepresented classes. To address these challenges, this study proposes a model named Graph-Enhanced Multimodal Framework for classifying 22 propagan‎da techniques within a three-level hierarchical structure. The proposed approach employs an innovative architecture consisting of three independent classification heads, enhanced by a Graph Neural Network (GNN). To extract rich feature representations, the framework combines the BGE-M3 model for text encoding with the CLIP model for joint image an‎d text representations, producing a 1536-dimensional feature vector. To refine predictions, the proposed graph network constructs a label graph by combining weighted pointwise mutual information an‎d semantic similarity matrices, thereby modeling latent relationships among propagan‎da techniques. Furthermore, the LDAM loss function an‎d a staged training strategy are employed to address the imbalanced distribution of training samples an‎d improve convergence. eva‎luation on the Semeva‎l dataset demonstrates that the proposed framework achieves an HF1 score of 0.76, delivering notable performance improvements over baseline methods while effectively addressing the aforementioned challenges.

6

171511