يك سيستم بازيابي اطلاعات خود نظارتي در زبان فارسي با استفاده از روش¬هاي يادگيري عميق

Natural Language Processing , Information Retrieva‎l , Dense Retrievers , Self Supervised Learning , Data Augmentation

A Self-Supervised Information Retrieva‎l System for Persian Language using Deep Learning

مهندسي نرم افزار

With the advancement of technology and the increasing popularity and accessibility of the World Wide Web, the volume of documents and information available to users is growing day by day. This necessitates the development of systems to retrieve information relevant to usersʹ needs. Traditional information retrieva‎l systems like BM25 are basic methods that perform retrieva‎l based on the apparent matching of query and document terms. These systems face the problem of lexical gaps, leading to a decline in the performance of information retrieva‎l systems. With the advent of neural networks and deep learning approaches, dense information retrieva‎l systems have emerged. One of the most commonly used algorithms in dense information retrieva‎l systems is supervised learning, which requires a significant amount of labeled training data. Accessing such datasets is challenging in low-resource languages and information domains, such as Persian. The aim of this research is to present an unsupervised dense information retrieva‎l system for low-resource languages and information domains. To this end, the BERT language model, a pre-trained language model, has been utilized. Then, to fine-tune this language model for creating an information retrieva‎l system, a selective Data Augmentation (SDA) approach is used to overcome the limitations of accessing labeled datasets. Based on this approach, new labeled samples are generated for the retrieva‎l dataset. After fine-tuning the retrieva‎l model, the proposed approach conducts an initial eva‎luation and then expands the samples in the training dataset where the dense retrieva‎l model shows more weaknesses, which in subsequent stages, fine-tuning on the new data improves the modelʹs performance. For generating new samples, new queries are derived from existing documents using abstractive summarization methods. The proposed solution is fine-tuned on the PersianQuAD dataset in Persian and compared with the base language model in metrics such as recall and Mean Reciprocal Rank (MRR). The results of the proposed solution show an improvement of 9.8% and 5.067% in the top three retrieved documents, respectively, compared to the initial state. Then, to eva‎luate the proposed solution in a language other than Persian, the Ms-Marco dataset in English is used. By fine-tuning the proposed model with this dataset, an improvement of 5.61 and 3.31 in recall and MRR metrics is achieved compared to the base and initial state, indicating the effectiveness and improvement of the proposed solution regardless of the language.

6