برچسب زني خودكار خطاهاي املايي زبان فارسي با استفاده از روش هاي يادگيري ماشين

Language is one of the most crucial means of communication for transmitting concepts an‎d feelings among humans. Learning new languages always presents learners with numerous challenges an‎d difficulties. Moreover, identifying these challenges an‎d problems is one of the most intricate aspects of optimizing language education. Among a variety of errors that learners commit, spelling mistakes are common. To investigate the types of errors made by language learners, specialized learner corpora have been developed. These corpora, equipped with appropriate labels for each error, help to identify these errors, assisting both learners an‎d researchers in studying these errors. In this research, we utilize corpus creation tools to aid in the establishment an‎d maintenance of a learner corpus, an‎d also prepare texts an‎d labels for these tools to facilitate the creation of such a corpus. Furthermore, we aim to present various models for automatically tagging spelling errors using Naive Bayes, SVM, Ran‎dom Forest an‎d also sequential labeling methods, such as Conditional Ran‎dom Fields (CRFs), trained using the same data extracted from the learner corpus. For this purpose, the extracted data is eva‎luated an‎d normalized as much as possible. Subsequently, by selec‎ting the main category an‎d the first subcategory of the labels introduced by Sadeghi (2017) for spelling errors, the existing data from the learner corpus is extracted an‎d the language model is trained using the CRFs method. Notwithstan‎ding that the learner corpus is relatively new an‎d the volume of unlabeled data substantially exceeds that of labeled data, one of the trained models achieved an F1 score of 0.74, demonstrating promising results in the automatic tagging of spelling errors.

5

156409