عنوان‌دهي تصوير با استفاده ازيك مدل انتها به انتها بر پايه مبدل بينايي

In todayʹs world, with increasing the growth of visual concepts such as images an‎d increasing the application of them in various fields, the need for a system that can describe these concepts in natural language is necessary more than ever. Image captioning with the aim of producing accurate, concise an‎d useful descriptions of images is one of the attractive fields for researchers. Image captioning faces two major challenges, namely image encoding an‎d language models that each of them is eva‎luated in many studies by itself. Recently, the use of transformers in various fields has resulted in a new trajectory in machine vision problems, including image captioning. Most of the current transformer-based models have two major problems: 1- Most of these transformer-based models use fixed-size symbols, which is not appropriate for vision applications. 2- The resolution of pixels in images is much higher than that of words in text. In present study, In order to solve these problems an‎d also improving the current methods, a new end-to-end image captioning model has been proposed. This model uses transformer-based models as encoders that supports regional features an‎d local features in a hierarchical structure. We also extracted global features from regional features to better support the context information. Finally, a decoder that uses both local features an‎d global features produced accurate an‎d descriptive quality titles. Experimental tests based on well-known eva‎luation criteria such as (CIDEr, BELU, METEOR) on the MSCOCO dataset showed that the proposed method was better compared to the other new methods. For example, according to the CIDEr metric, it has a 3.18% increase compared to the PureT model an‎d a 6.5% increase compared to the DLCT model.

6

150197