Multiresolution approach on medical image fusion by modified local energy

Human and machine perception data that are not redundant are very important in medical field for diagnosis and treatment. Existing fusion methods lack in complexity and are time-consuming. The proposed work fuses medical images to extract valuable necessary information from dissimilar images to a single image in the wavelet domain using a novel modified local energy (MLE) fusion rule termed MLE image fusion. Modified local energy helps to provide edge characteristics more clearly in the fusion outcome than a single pixel-based fusion rule. The denoising property of the local energy is the additional advantage of the proposed fusion. The similarity of the fused image with the source images is improved by B-Spline registration in the pre-processing stage. Finally, the fused image is created with all the corresponding coefficients by transforming the inverse wavelet. SVM-based contouring of the lesion part helps observe to identify the lesion part from the fused image. The proposed approach assists medical professionals in the diagnosis of lesions or anomalies in tissues. Experiments on real-time and standard datasets with expert radiologist subjective evaluation and quantitative analysis are carried out by well-known non-reference performance measures.

image fusion. The images that are fused provide information better than a single image of the source [3,4].
The simplest methodology for image fusion is the mean of the source images and drawbacks in reduced variance. Wavelets are better positioned both in the time domains and frequency domains, but the Fourier transforms are located only in the transform domain [5]. Wavelets provide better signal delineation using multiresolution processes [6]. For multi-resolution image fusion, wavelets and pyramid-based image fusion have been utilized. Decompose complex image details by wavelets into elementary forms at different locations and sizes, and then, construct it back with high precision of fused images [6].
Several methods of image fusion [7] advanced from the previous decades. Current fusion approaches were categorized onto the principle of the stage at which the fusion was carried out. The three levels of fusion are fusion at pixel values called pixel level [7] or lower level of fusion, fusion based on a feature called feature level [8], and fusion based on decision called decision level [9]. Fusion at the decision level, the highest level, utilizes voting, statistics, heuristics, or fuzzy logic for decision-making prior to fusion. Fusion with pixel intensity values is the easiest and most effective method among the fusion methods [7]. Currently, optimization-based image fusion techniques also exist [10].
Wavelets are the enhanced version of Fourier transform (FT) [11] and are expressed as small sine functions. Fourier transform was unable to analyze non-stationary signal elements, although wavelets resolve this disadvantage [12]. The choice of coefficients based on local energy(LE) is useful for the capture of edge details. LE is efficient when images are impaired by unwanted signals.
The research intends to accomplish the following objectives: • Develop a novel modified local energy (MLE) fusion rule for obtaining better information and saliency features of the fused image. • Develop B-spline-based registration method for maximizing the similarity of the fused image. • Propose a work that decreases the probability of selecting a noise component. • Enhance the superiority of edges details in the final fused image. • Develop contouring on fused images to help observers to identify the abnormal area in the fused image.
To preserve the edge details along with the LF components, the HF components are also fused by MLE. To obtain a better-fused result, B-Spline-based registered images are utilized in the wavelet domain as the wavelets can resolve the limitations of the spatial domain. Support vector machine (SVM)-based contouring helped observers to identify the abnormal area in the fused image.
This work is arranged accordingly. Section 2 explains the methods and techniques utilized for the work. Objective evaluation measures used in the work are detailed in Sect. 3, and results and discussion are covered in Sect. 4. Section 5 deals with the conclusion of the work.

Methodology
The proposed fusion methodology includes source image pre-processing, image registration [7], image decomposition, novel fusion rule by implementing fusion algorithms performed through the proposed MLE fusion method to gather useful data from multi-sensor medical imaging and image reconstruction. The proposed system architecture provides an outline of the entire work flow such as image pre-processing, registration, image fusion, and segmentation of tumor part from the fused image as illustrated in Fig. 1a. In Fig. 1b, the source images are decomposed into HF and LF components. The decomposed LF and HF components of source images are fused by MLE fusion rule, and the fused

Source image pre-processing
The pre-processing work takes place in three operations, such as Gaussian smoothing, edge sharpening, and the rescaling of the images as shown in Fig. 1a for image enhancement, noise reduction, and resizing. The pre-processing steps are as follows: 1. A 5*5 mask is used to compute Gaussian smoothing, G S Gσ (u, v) as derived: where (u, v) = {-2, -1, 0, 1, 2} and σ = [0. 1,5]. The total sum, Tot G , of the entire mask should be equal to 1. Each element must therefore be normalized among 25 elements as: 2. Apply edge sharpening using the MATLAB function 'imsharpen' for improving edge quality of image and rescaling with MATLAB function to obtain similar dimensions of source images.

B-spline registration
For geometrical alignment of source images, the B-splinebased registration [7] is used. In the registration phase, the normalized and re-scaled input images are aligned spatially according to the co-ordinates. To register the image, two sets of images of the same patient, the same organ taken at different times, are considered. One image is considered as the moving image, and the second is the static image. In B-Spline registration, map point u in the moving image, Mi(u), to point T c(u), in the static image Si(u), using deformation coefficient for geometric transformation, Tc. Let D be the divergent function. The transformation functionτ minimizes D.
where M i and S i are the moving and static images, respectively. Hence, the transformation functions,τ , minimize D for the best registration of a moving image with the reference image. The deformation of any control point, ϑ, in the moving image is interpolated using kernel function as is the kernel function of cubic B-spline. ∂ b is the control points nearest to u. Now the defined deformation, C(ϑ | ∂), coefficient is applied over test image given as where ϑ = [u, v, w] z position in the reference image and ∂ b is the set of deformation function. The transformation function, τ , is τ = ∂ b The evaluation output of image registration is discussed in Sect. 4.2, and different sets of images are compared with the pixel-based registration model.

Discrete wavelet transform
The discrete wavelet transform(DWT) decomposes images into sub-bands [5]. The wavelets are obtained through dilatation and translation from the mother or prototype wavelet, ψ(t).
where p and q are the scaling and shifting parameters, respectively. DWT is simple to implement and computationally less complex compared to other advanced transformations [13].

Fusion rule
Both the LF and HF coefficients are fused with modified local energy fusion. HF coefficients provide features of edge details more clearly. To preserve the edge details along with the LF components, the HF components are also fused by MLE. The proposed algorithm utilized a modified local energy-based fusion rule for better visualization of tissue structure or parenchyma. In the MLE fusion rule, the fused coefficient is based on the neighboring coefficients. Two or more pixels are usually spread on the edge. So, when a coefficient signifies the edge, its adjacent pixels also signify the edge. The consideration of a neighboring coefficient provides edge details more clearly. When the images are distorted by the unwanted signal, this MLE is useful for capturing the edge details properly. Noise is usually isolated, and thus, the noisecorrupted neighboring coefficients can have a low absolute value. So fusion with MLE provides better edge details and information after the fusion process. Let C be the transformation coefficient matrix as shown in Fig. 2a. Let the window size be 3 × 3 along with a specific component value. The center pixel local energy is the resultant of the square values of all neighboring coefficients within the window. The LE can be calculated as shown in Eq. (7): The center pixel and its neighboring pixels are more likely to match to the edge when the local energy of the center coefficient is large. Calculate proposed MLE as shown in Eq. (8): where AvgC is the average of e 1 (i, j) and e 2 (i, j) and MaxC is the maximum among the e 1 (i, j) and e 2 (i, j). The complete fusion process of the proposed fusion algorithm is demonstrated in Fig. 2b.

SVM tumor segmentation
The contouring of the abnormal region is achieved by the level set method [14]. Hyperplane in SVM can be defined by where α is the weight vector and α 0 is the bias. The successful hyperplane is described by a vast number of different paths by scaling α and α 0 where z is the closest hyperplane training example. Support vectors are usually the closest examples of hyperplane training. Use the geometry result that differs from a point z to a hyperplane(α, α 0 ): For the canonical hyperplane, the numerator is equal to one and the difference is: The two times the difference from the nearest example is the following equation, called the margin Mrg. Maximizing Mrg problems is the same as minimizing R(α) problems with multiple confines. The hyperplane confined model designed to correctly classify all training examples z is: where each label of the training examples is represented by y i .

Experimental results and analysis
Two types of quantitative evaluations are carried out for the performance evaluations of the image pairs: comparison of normal fusion rule with proposed MLE fusion rule and comparison of existing algorithms with the proposed MLE algorithm.
All the input images are of grayscale in nature. The size of the source images used is 256 × 256 and is in JPEG format. The source image details are shown in Table 1 and in Fig. 3. Tests were performed to examine the quantitative analysis of fused images with three existing algorithms (A1 [18], A2 [14], A3 [1]), A4 [16], A5 [19], A6 [20], and our proposed algorithm A7. The analysis of the prominent result is indicated in bold letters in Tables 3 and 4.

Comparison of registration outcome
To identify a better registration of the input images, compared with existing pixel-level registration [13] and B-spline registration [17]. In pixel-level registration, the moving image is aligned based on the intensity values of pixels with the static image [13]. B-spline registration is based on control points. Based on mutual information(MI), Pixel-based and B-Spline-based registration is compared for 5 image sets as shown in Table 2 and concluded that the latter is showing the better result.    Table 3. In Fig. 4a

Comparison of existing algorithms and proposed MLE algorithm
The objective evaluation comparison for the four sets of images is shown in Table 4. E and M I values for the proposed  Table 4. The E values of source images and fused images are calculated with the proposed fusion algorithm, and further the same is graphically represented in Fig. 5b Fig. 4. The fusion result of A4 is giving tissue structure more clearly than other algorithms. From the outcome of the visual and quantitative analysis, A4 is showing a better result for all the sets of images.

Segmentation results
Set-5 fused image with various α values producing contouring result is shown in Fig. 6i. The segmented result of the contoured tumor part for different α values is shown in Fig. 6ii. The subjective evaluation for the segmented results is evaluated by expert radiologists from HCG hospital, Bangalore. The evaluation results are illustrated graphically in Fig. 5a. It is observed from the sub-

Conclusion
The paper proposed a novel fusion rule with modified local energy fusion to obtain the saliency features of the image in the resultant fusion. The novel modified local energy fusion rule from the proposed fusion process enhances the edge details, and thereby, more information can be obtained from the fused image. The proposed method decreases the probability of selecting a noise component and enhances the superiority of edges details from the final fused image. Prior to fusion, the algorithm utilizes a B-Spline registration for the geometrical alignment of the source images which increases accuracy of the fusion result. For diagnosis and treatment purposes, tumor segmentation in the post-fusion process work is implemented. Quantitative and visual evaluation of the proposed fusion method has been demonstrated. In the future, the algorithm can be optimized with advanced registration algorithms to overcome the drawback of minor loss of information with more pairs of images.
Funding No funds, grants, or other support was received.
Data availability Not available.