توسعه يك رويكرد يادگيري عميق سبك‌وزن براي بازشناسي خودكار گفتار فارسي

Today, speech recognition models based on deep learning have achieved high performance. However, to enhance accuracy an‎d efficiency, most of these models tend to be large an‎d complex, which introduces two main challenges in speech recognition: (1) increased processing time an‎d (2) the need for powerful hardware resources. The primary solution to address these challenges is model compression using various techniques such as knowledge distillation, network pruning, quantization, neural architecture search, an‎d the design of low-parameter processing layers. In this dissertation, among the available compression techniques, the combination of knowledge distillation an‎d network pruning has been chosen as the primary approach for compressing the speech recognition model due to their proven effectiveness in reducing the size of large language models. In this regard, various methods for transferring knowledge from the intermediate layers of the larger (teacher) model to the lightweight (student) model have been explored, both individually an‎d in combination. Additionally, different techniques have been proposed to enhance network pruning. Furthermore, hybrid approaches integrating knowledge distillation an‎d network pruning have been introduced an‎d comprehensively analyzed to improve their effectiveness in compressing speech recognition models. The findings of this research indicate that leveraging the latent information in intermediate layers during the knowledge distillation process, combined with optimized network pruning methods an‎d a well-designed strategy for integrating these two techniques, can enhance model efficiency while preserving its performance. Specifically, this approach increases processing speed by 50% while reducing computational complexity an‎d memory consumption by 32% an‎d 20%, respectively. These improvements enable the deployment of the model in environments with limited computational resources.

6

عليرضا درويشي