Algorithm of infrared-polarization image fusion based on fireworks algorithm

Because of the shortcomings of traditional infrared-polarization image fusion algorithm, such as low intelligence and single optimization index, this paper proposes an intelligent infrared-polarization image fusion optimization algorithm based on fireworks algorithm. Based on the strong complementarity between infrared-intensity image and degree of linearpolarization (DOLP) image and the explosive optimization of fireworks algorithm, the problem model of weighted fusion algorithm is established, and the fitness function based on root mean square error (RMSE) is constructed to calculate the optimal weight of source image. In the fusion experiment of long-wave infrared-intensity image and DOLP image, this method is compared with the common fusion algorithms. The results show that this method can effectively fuse the infrared-intensity and degree of polarization information, and the evaluation indexes of standard deviation, spatial frequency, mutual information, structural similarity, peak signal-to-noise ratio and information entropy of the fusion image are better than the comparison algorithm. In the future, cooperated with the long-wave infrared-polarization imaging system, this method can be applied to improve the infrared detection ability in complex environment.


INTRODUCTION
For a long time, infrared imaging technology is a powerful means of tactical reconnaissance in modern war, but with the increasingly complex battlefield environment, the traditional infrared imaging technology is difficult to meet the needs of new photoelectric countermeasures in complex environment.Infrared-polarization imaging detection can obtain polarization information different from the background according to the target characteristics, and make up for the shortcomings of traditional imaging technology by adding polarization dimension information [1].Therefore, the fusion of infrared-intensity image and DOLP image can improve the ability of target detection and recognition, which plays an important role in weapon reconnaissance and anti-reconnaissance.How to describe the target characteristics accurately through the rich scene information of fusion images is the key problem in the process of image fusion.
In general image fusion algorithms, the weight of source image is determined by setting a threshold value, and the consistency of evaluation indexes is poor.To solve this problem, some scholars proposed an intelligent fusion algorithm based on maximizing the evaluation index to obtain the weight of the source image.Kanwal [2] proposed an intelligent multi-scale exposure fusion algorithm based on image segmentation to preserve the details of the darkest and brightest areas.Dai [3] proposed a fusion method of infrared image and visible image, which can realize the optimization of fuzzy region feature frame.The performance of different intelligent fusion algorithms is very different.For example, when the acceleration constant of particle swarm optimization algorithm [4] is different, it may only be local optimal, and flower pollination algorithm [5] is easy to fall into "dimension disaster" when solving high-dimensional optimization problems.In order to have the same advantages in terms of convergence speed and computing performance, Tan Ying from Peking University proposed the Fireworks Algorithm [6] (FWA).
In view of the excellent performance of fireworks algorithm in the field of image processing, this paper introduces fireworks algorithm into infrared-polarization image fusion process, and proposes an intelligent fusion algorithm of infrared-polarization image based on fireworks algorithm.The optimization problem model is proposed based on the weighted fusion idea of the source image.According to the root mean square error (RMSE) of the infrared-intensity image, the DOLP image and the fused image, the fitness function is constructed to calculate the optimal weight of the source image.The experimental results show that the algorithm introduced in this paper has a higher degree of intelligence, by which several image fusion indexes have been achieved at their best, and the image fusion quality and visual effect get greatly improved.

IMAGE FUSION
Fireworks algorithm, as a new swarm intelligence optimization algorithm, uses explosion search mechanism to simulate the possible solution of fireworks optimization problem, takes the location of fireworks explosion as the center to generate a certain number of sparks within the set range, and selects the sparks within the optimal range to ignite the next generation of fireworks through the defined fitness function.Sparks fitting the fitness function are searched in the center of fireworks, and the next generation fireworks are detonated to generate multiple iterative optimization effects.In this process, information sources and interactive information are distributed intelligently, and the optimal solution of the solution space is finally obtained.In this paper, the fireworks algorithm is applied to the fusion of infrared-intensity image and DOLP image.The main steps of the algorithm include problem model, fitness function and algorithm flow.
(1) Problem model Infrared-polarization image fusion usually obtains the weights of infrared-intensity image and DOLP image according to the fusion rules, and then fuses the source image through the weight matrix.
Where, f I is the fused image, l I is the infrared-intensity image, p I is the DOLP image, 1  and 2  is the weight of the source image.In order to prevent the energy spillover of the fusion image, the weighted average method is introduced, so Equation ( 8) is rewritten as: The weighted average method affects the quality of the fused image by controlling the weight of the source image.Therefore, the essence of image fusion is the optimization of the weight of the source image.In the fireworks algorithm, the optimal solution is the parameter value of the fitness function minimization.The optimization problem model is represented by the following formula: min , , , , 0 1 Where,   (2) Fitness function The RMSE is an index to measure the deviation of experimental data from the real value, which is used to characterize the difference of gray information distribution between the source image and the fused image.RMSE is expressed by the following formula: Where, ( , ) X i j is the gray value of current image point , ( , ) Y i j is the gray value of reference image point , and M and N are the height and width of the image respectively.In the process of image fusion, the more information the fusion image extracts from the source image, the better the fusion effect.The smaller the RMSE value is, the smaller the difference between the two images is, indicating that the fused image retains more information from the source image.Based on the above analysis, the fitness function of the optimization problem is constructed by using the root mean square error of the gray values of the two images.
According to the definition of fitness function, the smaller the fitness value is, the closer it is to the solution of the optimization problem.The fusion of infrared-polarization image is composed of one infrared-intensity image and one DOLP image.Due to the different contribution of the source image to the fusion image evaluation index, a comprehensive root mean square error function is established for the two source images and the fusion image, and the comprehensive root mean square error function is expressed as: (5) Where, FL RMSE is the root mean square error of the fused image F and the infrared-intensity image L; According to the problem model and the comprehensive root mean square error function), FLP RMSE can be expressed as: , , , Therefore, equation ( 6) is selected as the fitness function of image fusion optimization problem.The weights 1  and 2  are introduced into the problem model, and the fitness function is constructed to introduce the contribution weights 1 v and 2 v .Therefore, the problem space dimension suitable for fireworks algorithm is 4. The optimal weights needed for fusion image are obtained by calculation.Finally, the fusion image of the infrared-intensity image and the DOLP image with the best fitness value is obtained by fusion.
(3) Process of optimization algorithm be the population of N fireworks, and , , ,  in the 4-dimensional problem domain space.The algorithm process is simplified as follows: 1) Initialize the system and set parameters, select N locations randomly, and assume the initial number of fireworks in this paper is 10.
2) Ten fireworks were detonated at different positions.The sparks produced by the fireworks explosion are shown in Figure 1.

Spark
Searching space Figure 1.Sparks from fireworks explosions 3) Calculate the fitness values of all fireworks explosions according to the fitness function.
4) Calculate the number of sparks generated by the explosion of each firework i  , which is defined as follows: Where, m is the parameter controlling the number of sparks generated by fireworks explosion in each generation, is the fitness value of the individual with the worst performance in the current population, and  is a minimum constant to prevent errors in the operation of the formula.

5) Calculate the explosion radius i
A generated by the i-th fireworks explosion, which is defined as follows: Where, A is the maximum explosion radius, is the fitness value of the individual with the optimal performance in the current population.
6) Fireworks explode to produce i  regular sparks, and calculate the fitness of all sparks in the current group.

7)
In order to improve the diversity of the population, Gaussian variation spark [7] is introduced.The formula is: Where, e is a random variable of Gaussian distribution, Algorithm iteration is an important feature of fireworks algorithm, which can decide whether to continue the iteration step according to the maximum number of iterations or the best fitness value.In order to optimize the algorithm flow and reduce the amount of calculation, the principle of maximum iteration number is adopted.In this algorithm, the variable dimension is set to 4 dimensions, and the maximum number of iterations is 100.All new sparks are evaluated by fitness function, and the best, the worst and the other two randomly selected sparks are selected to form the next generation fireworks.As can be seen from figure 3, the fireworks algorithm can converge quickly and accurately when searching for the group optimal solution.

EXPERIMENT AND QUALITY ANALYSIS
Fireworks algorithm is to simulate the process of spark ignition fireworks to get the optimal solution.In order to verify the effectiveness of the fitness function based on root mean square error in solving the fusion weight problem and evaluate the image fusion ability of fireworks algorithm, this paper obtains the infrared-intensity image and the DOLP image through the laboratory equipment long-wave infrared-polarization imaging system, after image preprocessing work such as removing the cold reflection and reducing the image sensitivity, and fuses the DOLP image.the same time, in order to evaluate the performance of the proposed algorithm, the proposed method is compared with other fusion methods.

Experimental environment
Experimental environment: Intel Core i7-5500u, 3.0GHz, 12GB memory, 64-bit win10 operating system and MATLAB 2019a running software.Main parameters of polarization imaging system: The detection wavelength ranges from 7500 nm to 14000 nm, and the extinction ratio of polarizer is 10000:1.

Image objective evaluation index
The objective evaluation of image is mainly composed of the following 6 indicators to evaluate the results of image processing and fusion quantitatively.
(1) Average gradient (AG) is an evaluation index describing the gradient information of the fused image.The larger the Average gradient value is, the clearer the image details and texture information will be.The calculation formula is as follows: Where, ( , ) F i j is the gray value of the pixel points in the i row and j column of the image.
(2) Standard deviation (STD) is an evaluation index to describe the gray level distribution and contrast of fused image.The larger the Standard deviation is, the more kinds of fused image information are.The formula is as follows: ( ( , ) ) Where, M and N respectively represent the height and width of the image,  is the gray mean of the fusion image.
(3) Spatial frequency (SF) reflects the richness of the image spectrum.The formula is as follows:

SF RF CF  
(12) Where, RF and CF correspond to image row and column frequencies respectively.
(4) Information entropy (IE) is used to calculate the average Information quantity of the fused image.The bigger the Information Entropy is, the bigger the average Information quantity is.The formula is as follows: Where, ( ) p i is the proportion of pixels with gray value of in the image.
(5) Peak signal to noise ratio (PSNR) is a full reference image quality evaluation index.The calculation formula is as follows: Where, n is the number of bits per pixel, which is 8 in this paper.The larger the Peak signal to noise ratio, the smaller the distortion is, which can be used to evaluate the quality of image fusion.
(6) Mutual information (MI) reflects the degree of association between the source image and the fused image, which is expressed by the following formula: Where,

, ( , )
X F h i j is the normalized joint histogram distribution formed by the source image X and the fusion image F; ( )

X h i and ( )
F h i represent normalized edge histogram distribution of source image and fusion image respectively; L is the gray scale series.(7) Structural similarity index measure (SSIM) is an index to evaluate the loss and distortion degree of the fused image, and its calculation formula is as follows: Where, x  and f  respectively represent the gray values of image X and fused image F; x  and y  respectively represent the standard deviation of image X and fused image F; xf  is the gray covariance of source image X and fused image F. The function of parameters C 1 and C 2 is to keep the algorithm stable, and their values are usually set as 0. The greater the structural similarity is, the smaller the SSIM between the two images is, and the more information inherited from the source image.In this paper, the structural similarity index is the sum of the structural similarity values of the fused image and the two source images.

Fusion experiments and quality analysis
To validate verification infrared-polarization image fusion algorithm proposed in this paper, the experiment selects three groups of the infrared image and the DOLP image intensity under the natural background, which is later compared with several other methods, such as Brovery transform fusion algorithm (Brovery), principal component transformation fusion algorithm (PCA), Laplace pyramid fusion algorithm (Laplace), discrete wavelet transformation fusion algorithm (DWT), variance based fusion algorithm (Var).Figure 4 shows the infrared-intensity image and DOLP image of the car (Car) obtained by the long-wave infraredpolarization imaging system.The infrared-intensity image clearly shows the temperature difference of various parts of the car.The higher temperature of the car's intake grille can be easily seen.From the DOLP image, the reflection of the tree on the Car's glass and smooth front baffle can be distinguished, and the gap between the tree tops at the back of the car is also obvious.From the fusion results, FWA can better integrate the rich information of infrared-intensity image and DOLP image, and the important information on the air intake grille and glass is preserved.Although other algorithms can successfully fuse the image, the imaging visual effect is not good.By observing, it can be found that the images of Brovey and DOLP are similar, but more infrared information is lost; the fusion images of Laplace, DWT and Var are dark as a whole, which integrate the information of light intensity and DOLP, but the visual effect is poor; PCA fusion images produce more noise, and the fusion quality is the worst.As can be seen from the tables above Table 1, 2, 3 and Figure 7, the fireworks algorithm is slightly worse than the comparison methods in some fusion evaluation indexes, but on the whole it shows better superiority.The PSNR, SF, SSIM and IE are the best in 3 groups, and the stabilities of STD and MI are generally good.In Car image fusion experiment, our method is the best in SF, SSIM and PSNR, the fusion result of Laplace is slightly better than that of MI and IE, but the fusion image of Laplace has artifacts; the STD Index of this method is lower than that of PCA, but the PCA fusion image has more noise and distortion, and the proposed method is better than the comparison methods in general.In the experiment of Kettle image fusion, our method is the best in MI, SSIM, PSNR and IE, and PCA and Brovery are slightly better than this method in STD and SF respectively, but these two methods fluctuate greatly in other indexes, the overall performance is poor.In the experiment of Three Cars image fusion, Brovery method is slightly better than FWA in MI and SSIM indexes, but the performance of this method is not consistent, for example, the worst result is obtained in STD, SF and IE Indexes.Our method is the best in other indicators, and obtain the best visual effect.

CONCLUSION
Focusing on the characteristics of information redundancy and complementary features between infrared-intensity image and DOLP image, we propose a fusion method of infrared-polarization image based on fireworks algorithm.The image performance is improved to various degrees.In the aspect of infrared-intensity image and DOLP image fusion, the minimum root mean square error is taken as the fitness function of fireworks algorithm, and the optimal weight of source image is calculated through the optimal fitness function to realize image fusion, and the subjective visual effect and objective index are evaluated.The experimental results show that the proposed method is superior to those comparison algorithms in subjective and objective evaluation indexes, by which the fusion results are good, the infrared characteristics and polarization characteristics of the source image are retained, and the imaging quality is greatly improved.
image quality evaluation relation, and is also the fitness function of the optimization problem of the weighted function.

1 v and 2 v
square error of the fused image F and the DOLP image P; respectively represent the contribution of the infrared-intensity image and the DOLP image to the comprehensive root mean square error FLP RMSE .

Figure 3 .
Figure 3. Optimization process of RMSE fitness

Figure 5 .
Figure 5. Fusion result of Kettle image(a) S0; (b) DOLP; (c) FWA; (d) Brovery; (e) PCA; (f) Laplace;(g) DWT; (h) VarFigure5shows the fusion result of infrared-intensity image and DOLP image of hot kettle (Kettle).In the infrared-intensity image, the infrared information of Kettle is stronger, while in the infrared-polarization image, the handle and the plastic part of Kettle have better contrast.From the perspective of subjective visual effect, all the six algorithms can fuse the main feature information of the source image, FWA and Laplace fusion images have better clarity, but Laplace fusion image loses most of the DOLP information; Brovey and PCA fusion images lose more infrared information, the contrast and overall gray value are low; DWT and Var fusion images produce unsatisfactory gray area, poor visual effect.

Figure 6 .
Figure 6.Fusion results of Three Cars image (a) S0; (b) DOLP; (c) FWA; (d) Brovery; (e) PCA; (f) Laplace;(g) DWT; (h) VarFigure6shows the fusion result of infrared-intensity image and polarization degree image of three outdoor cars (Three Cars).The infrared-intensity image depicts the overall appearance of Three Cars, while the edge contour of Three Cars is highlighted in the DOLP image, and the frame and glass are distinguished from each other.Compare to other fusion methods, FWA has the best visual effect, which can better fuse the information of infrared-intensity image and DOLP image, for example, the residual temperature at the bottom of the middle vehicle is well preserved; Brovery fusion image loses more infrared information; PCA and Var fusion image mainly retains the contour information of three vehicles, but most areas are fuzzy; Laplace and DWT fusion image quality is poor Good, dark as a whole, not distinguishable from the background.

Figure 7 .
Figure 7. Contrast effect image of fusion image evaluation index homogenization.(a) Car; (b) Kettle; (c) Three Cars

Table 1 .
Objective evaluation of the fusion results of Car

Table 2 .
Objective evaluation of the fusion results of Kettle

Table 3 .
Objective evaluation of the fusion results of Three Cars