Infrared Small Target Detection Based on Multidirectional Cumulative Measure

Robustness of small target detection is a researchable hotspot in infrared (IR) surveillance system. The residual phenomenon of background clutter is universal in current local comparison methods. The algorithm of sparse low-rank decomposition restoration cannot be applied to the actual situations due to the long time consumption. This letter proposes a multi-directional cumulative measure (MDCM) to enhance the saliency and effectiveness of weak-small target detection. First, multi-directional cumulative mean difference is implemented in central layer and background layer to estimate the background, while multi-directional cumulative derivative multiplying (MDCDM) is calculated in central-active layer to characterize the overall target’s heterogeneity, and then the technology of image fusion is adopted to eliminate the interference of false target. Finally, a simple adjudicative technology is employed toward separated target region from complex scenes. Compared to up-to-date existing approaches, extensive simulational testing on four public datasets prove that the proposed approach is capable of separating small targets efficiently from an irregular background in a single-scale window and achieving a comparable or even better accuracy.


Infrared Small Target Detection Based on
Multidirectional Cumulative Measure Guofeng Zhang , Askar Hamdulla, and Hongbing Ma Abstract-Robustness of small target detection is a researchable hotspot in infrared (IR) surveillance system. The residual phenomenon of background clutter is universal in current local comparison methods. The algorithm of sparse low-rank decomposition restoration cannot be applied to the actual situations due to the long time consumption. This letter proposes a multi-directional cumulative measure (MDCM) to enhance the saliency and effectiveness of weak-small target detection. First, multi-directional cumulative mean difference is implemented in central layer and background layer to estimate the background, while multi-directional cumulative derivative multiplying (MDCDM) is calculated in central-active layer to characterize the overall target's heterogeneity, and then the technology of image fusion is adopted to eliminate the interference of false target. Finally, a simple adjudicative technology is employed toward separated target region from complex scenes. Compared to up-to-date existing approaches, extensive simulational testing on four public datasets prove that the proposed approach is capable of separating small targets efficiently from an irregular background in a single-scale window and achieving a comparable or even better accuracy.

I. INTRODUCTION
T HE technology of infrared (IR) small target detection has extensive and realistic significance in IR search and tracking system, precision-guided weapons, IR surveillance, and other fields [1], [2]. IR small target detection has always been a vital research content of IR image processing. Remote small target detection and tracking will be transformed into decision superiority in future information warfare. Currently, there are two main approaches of target detection at home and abroad: sequential-based and single-frame-based methods [1]. Compared with sequential image detection, single frame detection has fewer restrictions and lower requirements in the field of guidance system. Accordingly, the goal of this letter is to boost the detection accuracy of single-frame image and adaptability in engineering field. In recent years, a number of researchers are committed to small target detection, and put forward a variety of single-frame methods in past predecessor achievement foundation, and have made outstanding contributions to solve the problem under different conditions. At present, commonly used single-frame small target detection algorithms is roughly capable of separating into methods founded on background suppression, human vision system (HVS), and low-rank and sparse matrix decomposition (LRSMD). Maximum median/mean filtering and morphological transformation [2] are typical methods based on background suppression. When the background is homogeneous and simple, these methods are relatively applicable. However, in various complex scenes, the performance of filtering degrades rapidly. The method using LRSMD considers IR image as the combination of low-rank background matrix and sparse target matrix. For example, Zhang et al. [3] proposed a novel method based on nonconvex rank approximation minimization joint l 2 , 1 norm (NRAM) to describe the background, Yao et al. [4] proposed facet kernel and random walker (FKRW), and Zhang and Peng [5] proposed partial sum of the tensor nuclear norm (PSTNN) method based on IR patch tensor (IPT) model. These methods can achieve good results in the case of high target intensity or relatively obvious contrast. Yet, it is difficult to eliminate the strong edge and sharp noise in complex backgrounds and has poor real-time performance. Approaches based on HVS make use of contrast mechanism in target region and local neighborhood to distinguish the target. Typical algorithms such as Chen et al. [6] proposed an effective local contrast measure (LCM) with nested filter structure in eight directions, which has a good detection ability and strong anti-interference to enhance the target areas. But LCM amplifies random noise of single pixel, resulting in high-intensity clutter of background also produces enhancement in the output image. Therefore, numerous improved methods based on LCM have been proposed. Relative LCM (RLCM) [7] is a typical multi-scale local ratio and difference combination algorithm. Multi-scale patch contrast measure (MPCM) [8] utilizes differences in diagonal background window units to detect dark and bright targets, Lu et al. [9] proposed the weighted double LCM (WDLCM) utilizing a novel window detection framework to improve the signal-to-noise ratio (SNR). However, some contrast measures of pixels may not be suitable for clutter suppression by selecting the maximum contrast measure from a multi-scale saliency image. simultaneously, it extends computation time and introduces important redundancy. To solve this problem, a three-layer window local contrast mechanism (TTLCM) [10] using single-scale window calculations to realize multi-scale detection problems is proposed. An enhanced closest-mean background estimation (ECMBE) using matching filters and the closest to the mean principle is proposed by Han et al. [11] to suppress high-brightness backgrounds. Although these algorithms can enhance small targets and suppress the background, they ignore the direction information. When the small targets are at the edge of the highlighted background or in heterogeneous areas, the high-contrast information filtering is easy to cause the targets to be submerged. Multi-scale subwindow technology restricts time efficiency. In order to solve the above problems, direction gradient is introduced in this letter, and a multi-directional cumulative measure (MDCM) algorithm is proposed to optimize the detection performance of IR weak targets under complex backgrounds.
The main contributions can be summarized as follows. 1) An enhanced closest to the mean principle with multi-directional cumulative mean difference (MDCMD) is proposed to estimate the background to the maximum extent.
2) A multi-directional cumulative derivative multiplying (MDCDM) is employed to suppress the background clutter with sharp edges.
3) Direction gradient threshold is introduced to eliminate random noise in the active layer.
4) A single-scale detection algorithm is used to replace the multi-scale window, which improves the calculation efficiency.

II. PROPOSED METHODS
Most IR small target detection approaches ignore the directional information in these fields; on an ECMBE basis, a new MDCMD algorithm is constructed in center-background layer to repress the participation of the highlighted background. Meanwhile, an MDCDM algorithm is constructed in central-active layer and directional gradient (DG) is calculated to enhance potential targets. The diagonal derivative multiplying can remarkably enhance target which is stronger than background or weaker than background. Fig. 1 displays the realization process of the MDCM approach. Fig. 2 exhibits a layer structure diagram in different directions.

A. Gaussian Pre-Enhance Operation
IR small targets have dispersion characteristics from the center to the periphery, and diffusing gradient direction of small target is almost comparable in all directions, so true target is closest to the Gaussian-like near its center. In a matched filter according to the filter theory [12], SNR is capable of improvement best while selecting filter kernel is identical to the shape of signal.
So a contrivable normalized Gaussian kernel (GK) is employed to raw IR image to pre-enhance target and suppress pixel-like noise high brightness (PNHB). GK template . (1) The pixel of central layer at (i, j) is expressed as where I represents original input image, G(u, v) represents the GK of (1), and I c (i, j) represents pre-enhance result after Gaussian filtering.

B. Multi-Directional Cumulative Mean Difference
In the background layer, we divide cell window into eight orientations. Average intensity of every cell window can be estimated using pixels in three directions around it. Cell window is mainly employed for the purpose of capturing local neighborhood background pixels as accurately as possible.
For every directions, to accurately estimate the background, cumulative mean gray in three directions for each subcell is proposed in this letter as BL where (x, y) represents a coordinate position in the central layer, i is number of subcell, and represents intensity level of the kth pixel in the jth direction, and n represents quantities of pixels in every direction. So as to better restrain interference of highlighted clutter to dim-small targets, the final background estimation (FBE) is calculated using the principle of the closest-mean center pixel instead of the max-mean criterion. MDCMD between center pixel and background pixels in three directions for each subcell is defined as

C. Multi-Directional Cumulative Derivative Multiplying
After MDCMD operations, some highlighted background clutter and random noise are still present and residual. Researchers have discovered that the peak value of intensity level of a real target attenuates exponentially with distance [12]. DGs of real small target area are always present and variable in different directions. The gradient coverage of the real target always points roughly to the center of the target area [12]. When the window of central pixel moves to the true target (TT) center, the active layer is still the larger area where the target energy is concentrated. Multi-directional mean estimation of active layer pixels in three directions for each subcell is defined as D i represents average intensity in the ith subcell for active layer and n represents quantities of pixels in every direction. represents intensity lever of the qth pixel in the sth direction. In this letter, to distinguish background edge and TTs, a new variable called DG threshold is introduced after multi-directional mean estimation operation In addition, in order to obtain light or dark targets, calculating the multiplying and difference are applied in active layer So MDCDM by constraining condition between center layer pixel and active layer pixels in three directions for each subcell is defined as MDCDM(x, y) = 1, if AD i (x, y) > τ and η DG < ς 0, otherwise.

D. Calculation of MDCM
After calculating the MDCMD and MDCDM, the MDCM of one of the pixels in the original IR input image is calculated by the following: Saliency map named MDCM is calculated for each pixel from left to right and top to bottom pixel by pixel in MDCMD and MDCDM operation.
III. DISCUSSIONS AND THRESHOLD OPERATION Next, when calculating MDCM map in each pixel, there are five different pixel types that need to be discussed. 1) If (x,y) Is a TT: The pixel locates the target center, since the target is Gaussian-like with a positive local contrast, so MDCMD and MDCDM are larger at the same time.
Finally, the result of MDCM is large. 2) If (x,y) Locates a Pure Background Area (PB): PB always appears as a large and homogeneous area, so there is a small gradient around the pixel in the PB. MDCMD will approximate zero. Therefore, the final MDCM will be much smaller than the target. 3) If (x,y) Is a High Brightness Background (HBB): It may have a big intensity level than TT, according to closest-mean center pixel principle, the background estimate is small and MDCMD is large. However, the DG with maximum value and minimum value differ greatly in active layer, which is probably greater than gradient threshold, so MDCDM is labeled utmost as 0. 4) If (x,y) Is an Edge of Background (EB): Although the gradient of several MDCMD is still large, the others are kept small, so the value of DG will be large than DG threshold by setting. So MDCDM is calculated as 0, and EB could be well restrained. 5) If (x,y) Is a PNHB: That would probably result in value of AD being less than 0, not participating in the next operation, and MDCDM still is 0. Through the analysis of the above different situations, massive interference pixels can be further restrained, TT will be evidently prominent and is significantly enhanced after the target saliency map is calculated by MDCM. To thoroughly segment the target from MDCM map, an adaptive decision threshold can be obtained as where MDCM max and MDCM represent maximum and average values of MDCM map, respectively. µ represents a parameter in the range 0-1. That in the range of the deviation among the parameter µ, the relation among 0.5 and 0.8 will is more optimal test samples in this letter.

IV. EXPERIMENTAL RESULTS AND DISCUSSIONS A. Experimental Datasets and Qualitative Evaluation
For the sake of proving the reliability of the algorithm under complex background interference, Table I displays a detailed description of the test data. Database of seq. 1-3 [13] is derived from "plane" of thermal image sequences using domestic in recent public IR datasets and seq. 4 is derived from the "plane" of thermal image sequence using a network dataset. We compared the proposed model to several recent current models with RLCM [7], MPCM, TLLCM [10], FKRW [4], PSTNN [5], NRAM [3], WDLCM [9], and ECMBE [11]. Fig. 3 presents the qualitative comparison results using the proposed method and up-to-date approaches. The first column demonstrates representative frames of every raw scene, and columns 2-10 are compared results, namely RLCM, MP-CM, TLLCM, FKRW, PSTNN, NRAM, WDLCM, ECMBE, and proposed method. In seq. 1, all the methods are capable of detecting small target, especially NRAM, and the proposed algorithm can not only enhance the target significantly, but also are extraordinary superior in suppressing the highlighted background clutter. RLCM, TLLCM, and WDLCM all contain tiny amounts of clutter with pixel size; RLCM has better-enhanced performance to target area; MPCM, FKRW, PSTNN, and ECMBE have high brightness noise points at some regions; and threshold separation is likely to fail to eliminate false targets. FKRW and PSTNN significantly have patches of speckled clutter, this is due to the influence of continuous highlight buildings under the ground background, and the background suppression abilities are relatively weak. In seq. 2, clutter suppression ability of RLCM, MPCM, TLLCM, and NRAM is also superior, the targets protrude obviously, but there are some dark and faint noise points. Small targets cannot be detected in FKRW, The proposed algorithm also has strong anti-clutter performance, and the performance of target enhancement is more obvious in spite of existing dark noise points. Threshold separation can eliminate false noise points. PSTNN has more speckled clutter and poor antiinterference ability. ECMBE and WDLCM have better ability to enhance the small target, but capacity of clutter suppression is limited in background edge regions, which is rather less positive for threshold separation to extract small targets. In seq. 3, the target enhancement of the proposed algorithm is prominent under banded highlighted interference, which is due to the diagonal multiplying principle proposed for dark target detection (8). The target detected by RLCM, MPCM, TLLCM, and FKRW is extremely weak, clutter completely submerges the targets, and threshold separations are impossible to extract small targets. The target detected by PSTNN, ECMBE, and WDLCM is relatively prominent, low intensity clutter is more, threshold separation is still possible to filter small targets, resulting in leak detection. In seq. 4, the proposed algorithm has excellent ability whether in enhancing target or clutter suppression. In PSTNN, NRAM, and WDLCM, the brightness of the interfering pixels is slightly lower than the target, and this is not conducive to threshold separation. ECMBE has more noise point interference with high brightness which is difficult for target extraction. FKRW is also relatively excellent for background suppression, and the target is also prominent. RLCM, MPCM, and TLLCM have more clutter points and target extraction is quite difficult. All in all, in different scenarios MDCM shows strong robustness to target enhancement and background suppression, which is more salient or exceeds all comparison methods.

B. Quantitative Evaluation
In quantitative evaluation, BSF and SCRG indicators are specially used to evaluate the algorithm's ability to restrain background clutter and enhance the target signal. The higher the value of the two, the better the performance of the algorithm BSF = σ in σ out , SCRG = SCR out SCR in (13) where σ in and σ out represent the standard deviation of raw input image and the corresponding enhanced map, respectively. SCR in and SCR out represent signal to clutter ratio (SCR) of the raw input image and corresponding enhanced map, respectively. Table II shows the value of BSF and SCRG, it can be seen that criteria of the proposed algorithm are obviously higher than comparative algorithms simultaneously no matter in ground background, sky-ground, or heavy floccus cloud background. The higher value of the two indices, the better the method performance compared to other state-of-the-art approaches. Even though in more complex scenario 3, the small target is completely submerged, the proposed model acquires a great advancement comprehensively compared with the up-to-date algorithms in both criteria of BSF and SCRG. Comparative approaches could not attain a great BSF as well as SCRG at the same time. Bold data represent the maximum in Table II. From what has been discussed above, we may safely draw the conclusion that the proposed model can advance the saliency of the targets and suppress complicated backgrounds remarkably. In Fig. 4, a 3-D receiver operating characteristic (3-D ROC) curve [15] is introduced to assess the performance of the model to predict detection outcome, area under curve (AUC) has also been introduced. It describes a relation among probability of detection (Pd) with false alarm rate (Fa) as judgment in each whole sequence. They are, respectively, defined as P d = the number of detected true targets total number of true targets × 100% F a = the number of nontarget pixels total number of pixels in tested frames × 100%. (15) Fig. 4 shows that the AUC values of 3-D ROC curve of the proposed algorithm are almost larger than other comparison algorithms except for scenario 4 (as shown in Table II). On the same false alarm probability, the detection rates are higher under four different scenarios. Especially when the targets suffer from banded highlighted interference clutters in seq. 3, the detection rate can reach 94.7%, and NRAM also achieves a high detection rate; however, it is slightly worse than the proposed algorithm. Other algorithms are not robust and the detection rate is unstable. From the perspective of quantitative indicators, our algorithm basically tends to be consistent, reaching the level of similar or much higher than similar comparison algorithms, showing strong anti-interference ability against different backgrounds as a whole, and achieving the balance of detection rate.
V. CONCLUSION This letter proposed an IR small target detection approach derived from MDCM. First, an application of typical Gaussian filter kernel for the central layer improves the SNR of image. Next, in the background layer, a closest-mean principle with MDCMD is proposed to utmost estimate the background. Meanwhile, using MDCDM suppress background clutters with sharp edge, in active layer, DG proposed to eliminate random noise. Then, the fusion of MDCMD and MDCDM further suppresses the background clutter and enhances the target region. Finally, an adaptive decision threshold is applied for the sake of achieving target region. The single-scale detection algorithm greatly reduces the complexity of computation instead of using multi-scale windows. Extensive simulated experiments using public IR datasets demonstrate the saliency and robustness of the proposed algorithm in irregular and complicated types of background clutter. The performance analysis compared with other algorithms demonstrates the proposed algorithm can efficiently suppress the sensitivity of clutter interfere and achieve a comparable and robust accuracy.