ادغام تصوير مادون قرمز و مرئي به كمك يادگيري عميق

With the advent of image fusion technology, image fusion has become a significant research area of interest, as it enables the creation of fused images containing more useful information than each of the original images. This study focuses on the fusion of infrared and visible images, a prominent type of image fusion. The data captured by visible and infrared sensors can complement each other, and their fusion can result in a single image that includes significant thermal targets and richer textures compared to each of the original images. In this research, we review various approaches to infrared and visible image fusion and propose a novel deep learning-based method. Our proposed framework employs a generative adversarial network (GAN) with a triple discriminator to integrate critical information from both infrared and visible images. The triple discriminator is utilized to identify essential elements such as details, appropriate intensity, and infrared targets in the generated image. Texture and detail extraction is achieved using gradient maps, and adaptive fusion of these maps through an attention mechanism, considering their statistical correlation, effectively preserves the details in the fused image. The proposed framework improves deep learning-based fusion approaches when incorporated within this structure. Furthermore, loss functions are defined based on key information such as common gradient maps, appropriate intensity from the original images, and infrared targets along with the visible image background, which contributes to maintaining information integrity and reducing susceptibility to potential noise in the infrared images. We also examine objective eva‎luation metrics to assess the quality of the fusion. Quantitative and qualitative analyses are provided, and a proposed scoring method addresses existing eva‎luation system shortcomings by better highlighting the differences among fusion approaches based on the desired information in the fused image. Extensive quantitative and qualitative experiments on publicly available datasets demonstrate the superiority of our approach over those proposed by other researchers

5