Spectral-spatial detection of buildings with special roong in hyperspectral images

One of the analyses performed on hyperspectral images is target detection. Given the recent developments and the creation of images with high spatial resolution, the need for both use of spectral and spatial information in the detection of hyperspectral images has increased. The present research was conducted to introduce a new method for spectral-spatial detection of hyperspectral images. In the proposed method, the spectral image was primarily segmented using the watershed algorithm. Afterwards, for the objects resulting from segmentation, five spatial properties of area, perimeter, strength, meaning intensity, and entropy were extracted. Finally, the detection operation was performed utilizing the marker-based minimum spanning forest (MSF) algorithm. The above-mentioned techniques were applied to two sets of CASI sensor image data taken from the urban area of Toulouse in southern France. The results of quantitative and qualitative evaluations showed that the proposed method improved the kappa coefficient by 40% and 34% in comparison with the spectral angle measurement (SAM) algorithm in the two tested images.


Introduction
Hyperspectral remote sensing technology has seen a significant progress over the past two decades. This progress has been very evident in the design and construction of sensors and in the development and implementation of data processing methods. The complexity and large volume of the data from hyperspectral sensors have led to the consideration of further specialized and advanced methods of data analysis in order to extract information (Akbari, 2020). The methods employed in hyperspectral data processing all belong to different groups of pattern recognition methods. Based on this and according to the level of knowledge used, certain methods, such as discovery, classification, identification, and separation of objects, could be mentioned (Bhattacharya et al., 2019;Dos Reis Salles et al., 2017). Representation spaces of this data for calculations include image space, spectral space, and feature space. In this regard, depending on which computing space is used, different computational algorithms were used to extract information from these data (Ren et al., 2015).
One of the analyses performed on hyperspectral images is target detection (Akbari et al., 2014;Hou et al., 2016;Zhang et al., 2015). In this study, roofs with special coverage as a target were detected in an urban environment through a series of hyperspectral images. Since an urban environment has complex physical, geometrical, and elemental properties used in buildings, hyperspectral data effectively helps identify, extract, and generate a map of the building blocks of an urban environment (Carvalho and Meneses, 2002;Chang, 2003). Identifying building materials in urban environments is of great importance in various applications, such as mobile phone communications, virtual reality, urban architecture and modeling, urban planning, and management (Bhattacharya et al., 2019;Cheng and Han, 2016;Freitas et al., 2018;Kanjir et al., 2018;Yadav et al., 2018).
Numerous studies have been done in the last two decades on spectral target detection. In 2002, Chang and Chiang introduced the Mahalanobis distance method and matched filter (MF) as anomaly detection methods (Chang and Chiang, 2002) and implemented them on AVIRIS hyperspectral images. In 2003, Homayouni and Roux evaluated the three methods of spectral angle measurement detection (SAM), correlation, and constrained energy minimizing (CEM) (Homayouni and Roux, 2003). Another study was conducted in 2004 by Du et al. on the AVIRIS image employing SAM algorithms and spectral information divergence (SID) as well as their combination for spectral detection (Du et al., 2004). Emami and Afary, in 2007, classified AVIRIS image of an agricultural area using pixel-based classification methods, including SAM methods and maximum similarity as well as spectral analysis method (subpixels) (Emami and Afary, 2007). In another study, entitled "Study of different methods of hyperspectral image detection", conducted by Akbari et al.,14 spectral detection algorithms in the form of four general categories of classical methods, definitive measurement, statistical measurements and sub-pixel measurements were implemented (Akbari et al., 2008). In 2010, Tarabalka et al. proposed a marker-based minimum spanning forest (MSF) algorithm to classify hyperspectral images (Tarabalka et al., 2010). They selected pixels with a high degree of belonging to each class as a marker with a support vector machine (SVM) classification map.
For this purpose, primarily, on the SVM classification map, the labeling analysis of the connected components was performed. For large areas, p percent of the pixels with the highest probability were then created and for small areas, pixels with a higher probability degree above the specified thresholds, as markers, were considered. In 2017, a SVM algorithm was first used to classify the image (Akbari, 2017). Then, the marker-based MSF algorithm was applied to improve the accuracy for classes with low accuracy. In 2019, Freitas et al. applied the neural network (NN) algorithm to detect floats in hyperspectral images and the SAM algorithm to compare the results (Freitas et al., 2019). In 2020, Jha and Nidamanuri first simulated multispectral and hyperspectral data considering various criteria, such as platform and radiation angle, and then, using statistical algorithms, detected different targets from the simulated images (Jha and Nidamanuri, 2020). In all these investigations, different methods of target spectral detection were created and evaluated. The problem here is the serious error in the detection of the target spectrum, which different researchers have tried to reduce through different detection methods. Certain factors, such as the high volume of useless information caused by sensor and atmospheric noise and spectral dependence, will increase the detection error in hyperspectral images. In this research, we presented a new method for hyperspectral image detection based on object-based classification methods and with the help of markerbased MSF algorithm, which, in comparison with most spectral-spatial classification algorithms in hyperspectral images, reached more acceptable results.
In the object-based classification methods, spatial information is used in different techniques (Tarabalka et al., 2010). One of the most common of these techniques is segmentation methods, in which the objects in the image (a set of pixels with the same property) are identified based on various characteristics such as uniformity (Tarabalka et al., 2010).
Object-based classification methods do not work directly on image pixels but are applied to objects created by segmentation (Lu and Weng, 2007). Various methods have been proposed for segmentation of hyperspectral images. One of the most common of these methods is the watershed segmentation method (Stawiaski, 2008). Due to the complex nature of land cover in urban areas, it is difficult to classify hyperspectral images with high spatial resolution using only spectral information (Rajadell et al., 2009). Conventional detection methods, such as maximum likelihood, which uses only spectral information, are not able to separate classes in urban areas with a high degree of accuracy. Therefore, methods that use spatial information in addition to spectral information are needed to accurately detect urban areas. To this end, there is a great deal of spatial information, such as object features, shape features, texture (Selvarajah and Kodituwakku, 2011). Generally, the spatial information used in object-based classification is one of the object features obtained from the segmentation image. In the proposed method, the watershed algorithm was initially segmented. Subsequently, for the created objects, five spatial characteristics of area, perimeter, strength, meaning intensity, and entropy, which had the best results among other object features, were extracted, and ultimately, employing the marker-based MSF algorithm, the detection of buildings with special roofing was done.

The proposed method
In this study, spectral-spatial detection of hyperspectral images was performed. Figure (1) shows the steps of the proposed method. As could be seen, the proposed method consisted of three main stages: segmentation, extraction of spatial features, and object-based detection. In the following, each step of the proposed method is explained.

Watershed segmentation
In the watershed conversion, the image is considered as a topological surface where the values f (x, y) represent the height. In fact, the above-mentioned transformation to create different purposes determines catchment basins and ridge lines in the image (Gonzalez and Woods, 2002).
In watershed conversion, the gradient image must be first defined, which is an image with pixels with the maximum values at the edges of the targets and with the minimum values in other parts. The watershed conversion is then calculated from the gradient image. It should be noted that the resulting segmentation image has the over-segmentation problem. To this end, the gradient image must be uniform prior to calculating the watershed conversion. Therefore, the neighborhood closing operator and h-minima converter were used. The closing neighborhood operator fills the empty space between the pixels and makes their outer edges uniform (Stawiaski, 2008).

Extraction of spatial features
In this research, the following spatial features were utilized to add to the spectral feature space.
1-Area: the area in the segmented image was equal to the number of pixels in each area (Selvarajah and Kodituwakku, 2011). For example, even though the Meadow and Tree or Water and Shadow classes have similar spectral signatures, the areas covered by meadow and water are larger than tree and shadow covered areas respectively.
2-Perimeter: this included the border distance of each area (Li et al., 2007).
3-Strength: it was equal to the ratio of the area of each area to the number of pixels of the smallest rectangle containing that area (Li et al., 2007). 4-Meaning intensity: it equaled the meaning intensity in each area (Selvarajah and Kodituwakku, 2011).

5-Entropy:
In the case of Meadow and Tree classes, the area covered by a meadow appears far more homogeneous than tree-covered areas. There are several statistical texture measures that can enhance the discrimination among these classes. However, the entropy measure has the most potential in this regard (Nghi and Mai, 2008). Entropy is one of the texture properties in the image defined as Equation (1).
In this relation, L represents the number of distinct gray levels, Z represents the gray levels of the image, and P ( ) represents the normalized histogram.

Object-based detection
Spatial information extracted from targets could reduce the errors in classifying similar spectral classes (Tzotsos, 2006). In this study, the marker-based MSF algorithm was applied to classify the objects obtained from segmentation. In the MSF algorithm, each pixel was considered as a vertex of the graph G = (V, E), in which V and E are the set of vertices and edges of the graph, respectively; accordingly, each edge e i,j ∈ E connects the two vertices i and j of the neighboring pixels ( Van der Meer, 2006). In addition, each edge e i,j has a weight w i,j that indicates the similarity of the respective vertices. To create a marker-based MSF, n vertices (t i i = 1, … , n) are firstly added to the graph (n is equal to the number of problem classes).
Zero-weight edges are then formed from class one to vertex one, class two to vertex two and thus, class n to vertex n (Tarabalka et al., 2010). In the next step, another vertex, such as r, is added to the set and from the n vertices added to the previous step, edges with a weight of zero are attached to it.

Experimental results and discussions
In this section, the characteristics of the test data are first described. Afterwards, the test results are presented qualitatively and quantitatively.

Hyperspectral data
CASI sensor image data was used in this study. CASI is a hyperspectral sensor with spectral resolution or a maximum number of bands of 228, which of course can be changed according to the user's needs. The bands of these hyperspectral images cover the spectral range of 0.4 to 1 μm. The spatial resolution of the sensor also depends on the height of the carrier platform, that is the aircraft, and can range from 1 to 10 meters.
The images processed in this study included two images with 32 spectral bands and a resolution of 2 meters, taken in May 2001 from the urban area of Toulouse in southern France.
In these images, different areas including the targets, needed for identification, were selected and evaluated. Figure (2) depicts the false color combination of two images, including 128 by evaluation and calculation of the error matrix, by performing accurate visual interpretation and observation of the spectrum of different materials, the ground true map of the region containing the pixels belonging to the target class (building roofs) was extracted for two images ( Figure   (3)).

Experimental results
After performing the segmentation operation with the watershed algorithm, the five spatial properties mentioned in Section 2.2 were extracted for the obtained objects. Subsequently, the marker-based MSF algorithm was used to classify the segments. In this study, SVM classification map and Gaussian radial base kernel were used to select the markers (Cristianini and Shawe-Taylor, 2000). The values of the two parameters of penalty (C) and Gaussian kernel (γ) in SVM algorithm were determined using cross validation technique. The values of the above parameters in the first image were equal to C=128 and γ=0.02 and in the second image, they were equal to C =32 and γ=0.1. The labeling analysis of the connected components was then performed based on eight neighborhood pixels on the SVM classification map and for the areas with more than M=20 pixels, P=5% of the pixels with the highest probability of belonging to a class were selected as marker pixels. For the small areas of less than 20 pixels, the pixels with a degree of probability greater than one threshold were selected as the marker pixels. The selected threshold was equal to the lowest probability among =2% of the highest probabilities of the whole image ( Van der Meer, 2006).  In order to evaluate the results, a ground true map must have been prepared, in which the areas or pixels associated with the target could be identified. The detection result was then compared to it and the error matrix (Table (1)) was obtained. In the error matrix, commission and omission error were determined for the respective purpose, then the criteria of overall accuracy (Equation (2)) and kappa coefficient (Equation (3)) were extracted (Rosenfield and Fitzpatric-Lins, 1986).
In addition to the error matrix, the ROC curve, which shows the frequency of detection error on the check data, was employed to investigate the performance of algorithms and their decision accuracy in target detection (Bradley, 1997). This curve was plotted based on the concepts of detection power or positive probability and false alarm probability. The ROC curve compared image detection results for different thresholds with ground true information. In practice, a number of thresholds were considered between the minimum and maximum values of the actual information. Afterwards, for each threshold, two ROC curves could be considered, one showed the detection probability curve versus the false alarm probability and the other one detected the probability curve versus the threshold (Figure (7)).
(b) (a) The output (share image) could be converted to a binary image considering a value for the false alarm probability and by selecting a suitable threshold through the ROC curves. In order to compare the results of the proposed approach, we implement independently the SAM, MF, SID, CEM, NN, marker-based MSF, and MSF-SVM algorithms. Figures (8) and (9) represent the detection maps obtained applying the above-mentioned methods in two hyperspectral images used.  (8) and (9) are obtained selecting a suitable threshold. To determine the optimal threshold, this goal should be achieved by creating an error matrix and then forming ROC curves. Taking into account 1000 numbers as the threshold in the range of zero to one, the error matrix was formed for each threshold. The kappa coefficient value was then calculated for each error matrix. The maximum value of the kappa coefficient determined the optimal threshold. In fact, ROC curves compared the results of image classification for different thresholds to ground true information. As exhibited in Figures (8) and (9), the maps related to the proposed method have further uniformity and less noise.

Binary images of buildings roofs in Figures
Table (2) shows the values of the overall accuracy and the kappa coefficient parameters calculated to assess the results of the implemented tests on two hyperspectral images. As could be seen in Table (2), the proposed approach in the first image resulted in higher OA rates up to 20, 17, 15, 13, 9, 7, and 4%, and (10) and (11) show that global accuracy improved when using the proposed approach in the first and second images, respectively.   By comparing the eight proposed detection algorithms, it could be concluded that since the ground true map for each image was the same in all the algorithms, the observed accuracy difference was due to the performance of the algorithm. On the other hand, the reason behind not obtaining 100% was the accuracy of the ground true map and the ability of the utilized detection algorithm. Furthermore, one of the reasons behind the low overall accuracy of the results in the second image compared to the first image was the high complexity of this image and also the low accuracy of its ground true map. As mentioned earlier, the ground true map in the second image was selected based on three bands and in the first image based on the total Overall accuracy Kappa coefficient number of bands (32 bands). According to the obtained results, entering spatial information in general increases the detection accuracy.

Conclusions
The current research aimed to investigate spatial information strategy along with spectral information to improve target detection in hyperspectral image analysis. In the proposed method, which was based on object-based classification methods, the image was initially segmented using the watershed algorithm. Subsequently, suitable spatial properties were extracted for the created objects and finally, employing the marker-based MSF spectral-spatial algorithm, each object was assigned to a specific class. Seven spectral detection algorithms were considered to evaluate the proposed approach. In most of the selected algorithms, the speed and accuracy parameter was considered; thus, the most accurate results could be achieved in the shortest time possible. The proposed method was applied to two hyperspectral images.
The experiments showed the quantitative and qualitative superiority of this method in comparison with the seven detection algorithms SAM, MF, SID, CED, NN, marker-based MSF and MSF-SVM, which indicates the importance of using spatial information in the detection process. Herein, we intended to reduce the amount of error in spectral-spatial detection, which could be advantageous for future research. Conditions for creating mixed pixels, such as the overlap of ground phenomena, the heterogeneity of most phenomena, and consequently the increase in the internal variance of the target, were found to increase the detection error in hyperspectral images. Therefore, we tried to reduce the above-mentioned errors applying different methods.