Multi-modal deep learning detection of deep-seated gravitational slope deformation by Typhoon Talas in 2011

doi:10.21203/rs.3.rs-1308955/v1

Download PDF

Research Article

Multi-modal deep learning detection of deep-seated gravitational slope deformation by Typhoon Talas in 2011

https://doi.org/10.21203/rs.3.rs-1308955/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

There has been an increasing demand in recent years for detailed and accurate landslide maps and inventories in disaster-prone areas of subtropical and temperate zones, particularly in Asia. Most standard mapping methods require detailed fieldwork to be conducted by knowledgeable, skilled professionals. When predicting landslides, it is important to understand past landslide cases and prepare for situations in which the same phenomena occur. Developing automatic analysis methods using deep learning can contribute to the sophistication and cost of screening. This case study analyzed the potential of using deep learning convolutional neural networks (CNNs) for landslide detection with digital elevation models (DEMs) before the slide for deep-seated landslides (DLs) that occurred during Typhoon Talas. Here, we created 36,985 pieces of learning data from topographic information, which were then applied to a CNN using a multi-modal learning model. Eight types of numerical analysis images were created using the DEM as the learning data. The learning outcome achieved an accuracy of >0.856 for a 50 × 50 pixel window size CNN. This indicates that the decrease in the number of numerical analysis image types influenced the outcome. This study uses data from a limited range of DL sites in a topography specific to the accretionary zone. Although this CNN model is still in the initial stages of development, it accumulated many collapse cases and could contribute to disaster location screening, risk assessment, and hazard mapping during disasters.

Deep-seated landslide

deep-seated gravitational slope deformations

eigenvalue ratio

TWI

When predicting landslides, it is important to understand past landslide cases and prepare for similar case scenarios. By accumulating frequently occurring slope disaster cases, researchers have determined that landslide behaviors change according to various conditions, including geological and topographical characteristics, rainfall characteristics, and landslide progression. Thus, no system has been proposed in which these various conditions can be discussed in a unified manner. The relationship between gravitational deformation of a mountain body and deep-seated collapse has long existed in the European Alps. The gravitational deformation topography is useful in understanding the deformation process. This has been applied to landslide prediction in an increasing number of heavy rainfall disasters. However, it is difficult to collect several disaster cases in the same area, i.e., with the same geological distribution and similar topography, because of the probability of a landslide of the same area is low. A method using deep learning—popular in recent years—is quantitative, reproducible, deeply analytical, and cost effective. Learning the location of the collapse is objective, as opposed to the conventional method of topographic map reading by skilled engineers. In this study, we conducted deep learning using 30 landslides with mountain gravity deformation and their surrounding non-landslide areas that occurred in 2011.

Deep-seated landslide (DLs) have become a focal point since recent disastrous rockslides, including those on Leyte Island in the Philippines in 2006 (Evans et al. 2007; Guthrie et al. 2009) and at Xiaolin Village in Taiwan in 2009 (Tsou et al. 2011) and have characteristic properties. In these disasters, deep-seated gravitational slope deformation (DGSD) has been reported as a DL precursor (Chigira et al. 2003; Crosta et al. 2006). DGSDs are related to mass movements caused by gravitational forces. DGSDs indicate deformation in the deep subsurface, whereas mass rock creep indicates deformation not limited to the subsurface but also continuous deformation that extends to the rocks near the ground surface (e.g., Chigira and Kiho 1994). Typical examples of DGSDs are zagging (Zischinsky 1966) and double ridges.

In the Kii Peninsula, which is the target area of this study, DLs have been frequently observed in recent disasters. With Typhoon Talas in 2011, approximately a few million m³ of massive landmass moved in rapidly with successive DLs, and in most of the slope failures, DGSDs were determined to be their precursors (Chigira et al. 2013). To predict potential DL locations, it is therefore necessary to understand DGSD formation and determine their progression manner. Hiraishi and Chigira (2009) and Tsou et al. (2017) noted that large-scale knickpoint formation and gentle slopes influence DGSD formation. Knickpoints form when rapid downward erosion occurs, whereas gentle slope formation is related to groundwater and surface water. Thus, DL sites can be identified using known characteristics extracted from DEM and numerical analysis.

When sliding occurs in a landslide, observing its behavior and investigating its characteristics allows displacement to be determined. An effective way to conduct a numerical analysis for the displacement is to estimate its dynamic structure (Wakai et al. 2020). However, predictions cannot be made until the slide becomes apparent. In contrast, for making landslide predictions by focusing on geographical features, such as DGSDs, landslide inventory maps (LIMs) and landslide susceptibility maps (LSMs, Can et al. 2019) can be used. High-resolution satellite images and photographs taken by unmanned aerial vehicles (UAVs), as well as terrestrial and UAV-mounted LiDAR (light detection and ranging) data, have become available in recent years. These data can easily be arranged chronologically, and geographic data for the same location at different times can be used for analysis. Much of the geographic data are complex, and when dealing with large scales, they can be difficult to analyze and some data may be inadvertently overlooked. Machine learning has been a method for complementing multivariate analyses and has been used in LIMs and LSMs. LSM-based landslide location prediction has been conducted by Lee (2004), who using the sigmoid function. Dahal et al. (2008) used weights-of-evidence modeling conducted using eight elements to predict landslides. In contrast, Chen et al. (2018) reported creating an LSM using an entropy model and support vector machine (SVM).

In deep learning analyses, considering that environmental factors about landslide susceptibility are uncorrelated or have non-linear correlations, 27 environmental factors were analyzed using a fully connected spare auto-encoder for landslide susceptibility prediction (LSP) (Huang et al. 2020). Liu and Wu (2016) used optical satellite imagery in their analysis using an Artificial Neural Network (ANN).

Images have also been used in analyses that use a convolutional neural network (CNN) instead of gridded data (Table 1). In the dataset, various other elements have been used (Wang et al. 2019; Wang et al. 2021) besides optical images (Ghorbanzadeh et al. 2019), and their effectiveness has been confirmed. However, because the scale of the mesh is large, it is still difficult to predict specific DLs. The application of CNNs with DL images has been reported by Sakita et al. (2019) and Kikuchi et al. (2019). They combined the slope and wavelet analysis maps in a transparent overlay as the learning data for the CNN. Therefore, the interpretations' accuracy exceeded 80%. One challenge we faced was that the learning data were divided into two highly simplified categories (landslide and outside the landslide) and that the numerical analysis of the images was a composite figure, so few variations were observed.

Table 1

Deep Learning using convolutional neural network (CNN)
Reference	Study Area	Training Strategy	Learning Data*	Area Scale (training)	Window Size
Wang, Y. et al. (2019)	Yanshan County, China	CNN (1~3D, LeNet)	1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12	50 × 60 km	25 × 25-m mesh
Ghorbanzadeh et al. (2019)	Ganga, Himalayas	CNN	13	2 × 2 km	32~80 × 80 pixels
Wang, H. et al. (2020)	Lantau Island, Hong Kong	CNN (Layer stacking and data extraction)	1, 2, 3, 4, 5, 6, and 14	20 × 8 km	11 × 11 cells
This study	Totsugawa, Japan	CNN (Multi modal)	1, 3, 4, 5, 15, 16, 17, and 18	400 × 400 m × 35 sites (Average)	50 × 50 pixels (m/pix)
*Data: 1: Slope, 2: Geological map, 3: Elevation, 4: Curvature, 5: TWI (Topological wetness index), 6: Aspect, 7: SPI (Split information), 8: STI (Stream sediment transport index), 9: Distance to fault, rivers, road, 10: Rainfall, 11: Landuse, 12: NDVI (Normalized difference vegetation index), 13: Optical data (UAV), 14: SDOIF (Step duration orographic intensification factor), 15: Wavelet, 16: Eigenvalue ratio, 17: Underground openness, 18: Overground openness

The goals of this study were to: 1) use DLs with the same geological characteristics over a wide area that can serve as a valuable case study because they allow for effective learning of real-world collapse cases, which till now is considered difficult; 2) use proven CNNs for image analysis, which combines existing and the latest methods and can provide new knowledge for landslide prediction using numerical analysis tools; and 3) clarify the analytical design by utilizing many variations of learning data derived from a single DEM, and devise a way to obtain an effective accuracy. The purpose of this deep learning was to determine the possibility of automatically recognizing DGSDs specific to DLs. Finding the same terrain as the collapsed area can predict landslides by deciding in a 50 m × 50 m area before gravity deformation affects the entire DL.

2.1. Outline of the rainstorm hazard

Typhoon Talas (also known as Typhoon #12 in Japan) made landfall and crossed western Japan during September 2–5, 2011, and brought over 2,000 mm of precipitation to the Kii Mountains. According to Chigira et al. (2013), over 50 DLs occurred because of the typhoon. These DLs occurred when the total precipitation reached 600–1,200 mm. However, regions that received over 1,200 mm of precipitation did not experience landslides. Matsushi et al. (2012) examined the rainfall and catastrophic landslides and found that DLs occurred when the accumulated rainfall reached over 600 mm within a brief period (48–72 h).

2.2. Geologic setting

Mount Hakkeiga-take (elevation: 1,915 m) is the highest peak in the Kii Mountains, as shown in Fig. 1, with mountains of similar elevations forming the ridge. The elevation of the study area ranges from 220 to 1,915 m. A paleo-surface is in the southern and the western parts of the Kumano-gawa basin. The study area comprises the Shimanto Belt on the western part of the Kii Peninsula. The Gobo-Hagi tectonic line bisects the belt, the northern part of which comprises a Cretaceous accretionary tectonic complex, whereas the southern part of the belt contains a Paleogene accretionary tectonic complex (Kimura 1986). The upper Totsugawa River catchment, which is the focus of this study, is located in the northern belt and contains the Hidakagawa Group that comprises five formations that trend ENE–WSW: (from north to south) the Hanazono, Yukawa, Miyama, Ryujin, and Nyunokawa formations (Hara and Hisada 2007). The Hidakagawa Group is predominantly underlain by the Cretaceous to lower Paleozoic Shimanto accretionary complex, with minor amounts of Miocene granite and sedimentary rocks. The complex comprises foliated mudstone, sandstone, acid tuff, chert, and greenstones (Kumon et al. 1988; Hashimoto and Kimura 1999). Following the surveys conducted after the disaster, studies of the micro-topography and geology related to the DLs were conducted. At Akatani and Nagatono which formed a barrier lake, DLs occurred along thrust faults with dominant mudstone layers (Arai and Chigira 2019). Hiraishi and Chigira (2009) geographically analyzed the upstream section of the Nakaharagawa River in the catchment and found that in the Kii Mountains, the knick line at lower elevations that formed because of downward erosion was a crucial element that caused the landslides and collapses. In regions where many collapses occurred, knick lines (as predispositions to DLs) were observed near the unit boundary within the Miyama formations of the Hidaka Group in the Shimanto Belt on slopes dipping N to NW.

2.3. DL setting

We conducted this study using supervised learning because we knew the regions in which landslides occurred when Typhoon Talas made landfall. The learning data were created by turning the pre-slide images of the landslide locations into tiles. Airborne laser surveys of the catchment before and after the disaster were conducted by the Kii Mountain Area Sabo Office of the Kinki Regional Development Bureau of the Ministry of Land, Infrastructure, Transport, and Tourism. The measurements were taken before the disaster in July 2009 and after the disaster from December 2011 to February 2012. The measurements were acquired as 1-m mesh DEM using an airborne laser scanner. Thirty-eight locations were identified as having landslides that amounted to over 1,000 m² of collapsed area (Erosion Control and Disaster Prevention Division, Prefectural Land Management Department, Nara Prefecture, 2012) (Fig. 2; the red dots). We have presented the data in the Appendix Table A1. To focus on DGSDs, ID numbers were assigned to topographic features and areas in that order. Among the DL occurrence sites, IDs 5, 7, 30, 32, 35, and 37 by Chigira et al. (2013) reported that deep collapse occurred along thrust faults in mudstone dominant layers. Fig. 2 (red dots) shows the geological map of the area where the collapse occurred and the geological map of the target area. However, not all unique DLs share the same characteristics. As shown in Fig. 3, these slides were determined by comparing the before and after images of the slides and determining the topographic features of the terrain before the slides occurred.

The average pre-slide slope angles (average) in the 38 regions in the catchment that collapsed were 36.9°, and the maximum slope inclination directions were primarily in the NE–N–NW directions. The pre-slide topographic features in the landslide areas included minor scarps, flanks, terminal cliffs, gullies, and irregular undulations (Kikuchi et al. 2019). These features indicate gravitational deformation, which can cause landslides; thus, the data were effective for deep learning. Fig. 3 shows the regions in which the slides occurred. For ID 13, irregular undulations and terminal cliffs were identified within the slide area, and gravitational deformation was still in progress, although it was not active. For IDs 15 and 16, minor scarps, terminal cliffs, irregular undulations, and gullies were identified within the slide area. ID 37 contained all the topographic features.

However, in the areas surrounding the slides, regions were identified as having topographic features consistent with DGSDs, but no collapses had occurred (“non-slide”). These non-slides indicate that, whereas these regions did not collapse, they possessed geographical features required for landslide occurrence. Sixty-three regions fitting these characteristics were identified and extracted (Fig. 4 and Appendix Table A2). According to Kikuchi et al. (2019), the average non-slide slope angles in the 63 regions in the catchment that collapsed were 34.6°, which was smaller than that of the collapsed area. The maximum slope inclination directions were primarily in the same NE–N–NW directions. The pre-slide topographic features in the landslide areas included main scarp, flank— notably more than the slide (IDs 43, 44, 69, 88, and 100)—terminal cliffs, gullies, and irregular undulations (IDs 43, 44, 69, and 88).

It is difficult to determine the areas to be classified as slide/non-slide areas using visual interpretations alone. Here, we created learning data that could objectively classify the areas inside the slides (slide, y0), inside the non-slides (non-slide, y1), outside the slides, and outside the non-slides (outside landslide, y2).

Deep learning is an analytical method that uses a neural network, which is a multi-layer structure modeled after the neural circuits of the human brain. We conducted an analysis using a CNN, which is highly effective for image recognition. A CNN is a machine learning method that contains multiple convolutional layers within the structure of the neural net that convolve and generate the feature values for each layer and pooling layers that compress these values. The convolutional layer of the CNN has a good perception of the local characteristics of the image and can sense the relationship between the pixel of interest and the surrounding pixels (Simonyan 2016).

Image recognition has been used extensively in previous studies. However, identifying images of gravitational deformation in mountainous terrain is a complex and difficult process. The range of collapsed areas is a topographic element that leaves various DGSD evidence at each juncture, and all are different. The slight differences in the ground surfaces with a complex history may not be discernible in a single image. Therefore, in this study, we used different analysis diagrams for images that shared the same coordinates. We gave the images multiple perspectives as modal information (Fig. 5), called the multi-modal method (Ngiam et al. 2011; Srivastava et al. 2012). This model is used for feature extraction and pre-training in deep neural networks and obtains common and high-level feature representations from different modal information by connecting the hidden layers of two networks or by sharing latent variables (Baltrušaitis et al. 2018). We determined the possibility of identifying ground surface features that contain multiple factors and chronological information using this method.

Images that comprise multiple types of explanatory variables (x) were the input (learning data). The goal was to clarify the characteristics of the sites where landslides would occur. Learning data contained not only two-dimensional information but also spatial information through the joints. By connecting the three types of color tomes (RGB) to eight types of numerical analysis, we obtained data that has 24 potential spatial dimensions. The target variables can be set in the classification layer and multi-class classifications conducted.

In this study, we used the pre-disaster LiDAR data taken in July 2009. We used a 1-m mesh data, i.e., 1 m resolution. The data were cut into a rectangle with an area about twice as large as the long axis of the landslide area. The reason for this cropping was to balance the total number of slides (y0), non-slides (y1), and not-landslides (y2) when creating the learning dataset. The clippings created this way were assigned IDs used in Tables 2 and 3, respectively. This was used for the numerical analysis.

Table 2

Objective variables and parameters of numerical analysis
Numerical analysis	Objective variable	Parameters	Citation
Slope angle	θ	(x1, y1, z1)...(xn, yn, zn), n = 9 (3 × 3 grid)	Burrough et al. (2015)
Eigenvalue ratio	Er	Unit vector(x1, y1, z1)...(xn, yn, zn), n = 9 (3 × 3 grid)	MacKean and Roering (2004)
Curvature	H	h(x,y): Plane orthogonal to ± 10 grid apart	Nishida et al. (1997)
Overground openness	Φ_L	φ₁: Maximum zenith angle in eight direction	Yokoyama et al. (1999)
Underground openness	Ψ_L	ψ₁: Minimum angle from vertically downwards in eight direction	Yokoyama et al. (1999)
TWI	W	As: Area of the upstream region per unit width, β: Slope angle	Beven and Kirkby (1979)
Wavelet	C	z(x, y),s = 1	Booth et al. (2012)
Elevation	El	z(x, y)	-
TWI, Topological wetness index.

Table 3

Number of data (tiles) used for training and validation
Data type		Slide (y'0)	Non-slide (y'1)	Outside Landslide (y'2)	Total
Learning	Training	13985	2969	8935	25889
	Validation	6666	1072	3358	11096
	Total	20651	4041	12293	36985
Evaluation		38	33	150	221

3.1. Generation of numerical analysis data

In this study, instead of creating learning data from topographic features based on human interpretation, we created learning data with objective features from multiple types of numerical analyses. The numerical analysis effectively selected the landslide conditioning parameters. Applying multiple analysis techniques to learning data is a basic procedure in machine learning. Three key parameters (topography, geology, and environment) proposed by Aleotti and Chowdhury (1999) were used here. These parameters had eight types: slope angle, Eigenvalue ratio, curvature, overground openness, underground openness, topographic wetness index (TWI), wavelet, and elevation. The area subjected to numerical analysis was approximately twice the range of the slide area. The DEM dataset was a 1-m mesh. The procedures for converting the data for the numerical methods are summarized in the Appendix. The resolution of the initial data is 1 m DEM. The parameters used in the model calculations are summarized in Table 2.

The numerical analysis data were rasterized. The color tone of the raster data was determined based on the expectation it would allow for effective identification. As shown in Fig. 6, the rasterized data processed multiple times clarified the geographical features and emphasized any local differences.

The slope angle (slope) is the geomorphic quantity that indicates the gradient in relation to the horizontal plane of each DEM mesh. For the wavelet analysis diagram (wavelet), we conducted an analysis using a two-dimensional continuous wavelet transform (Booth et al. 2012). This technique is effective for emphasizing unevenness of the ground surface.

The eigenvalue ratio is an index that indicates ground surface disturbances. The three-dimensional disturbances near the ground surface of the study area were defined by Woodcock (1977) and Woodcock and Naylor (1983) as the differences relative to the surrounding ground surface. MacKean and Roering (2004) found that the eigenvalue ratio expresses the roughness of a surface and suggest that it can be an index to estimate landslide activity. In locations where active ground surface activity occurs, the ground surface becomes rough, and the value decreases (Kasai et al. 2009).

The average curvature is an index of the unevenness of the terrain (Nishida et al. 1997) and is defined by connecting the shortest distance between two points on a curve. The calculation consideration range was calculated in a range of 10 m (10 pixels) from the target point on each side. Values near zero indicate nearly flat surfaces. Profile curvature, which is the vertical plane parallel to the slope direction of downslope flows, also affects erosion and deposition (He et al. 2012; Kannan et al. 2013).

The overground openness is the average of the maximum values of the zenith angle that can be seen in the sky from eight directions within a distance of 50 m from the point of interest. Likewise, the underground openness is an average of the eight directions based on the maximum values of the nadir angle in which the underground can be seen. (Yokoyama et al. 1999).

The TWI (Beven and Kirkby 1979) is based on geological conditions and shows the watershed and water storage amounts. This indicates the total area of the catchment and the valleys, which become the pathway for water to flow and store upstream of each point of interest.

3.2. Creation of tiles for learning data

The raster data obtained through the numerical analyses were divided into squares to turn into learning data for the machine learning process according to the method described by Kikuchi et al. (2018). These jpg image files are called tiles—we cut the analysis images from the pre-landslide into 50 × 50 pixel jpg images starting from the NW corner (Fig. 7). To classify the tiles, the objective variables were labeled as y0 when the image contained over 80% slide area, y1 when the image contained over 80% non-slide area and y2 for all others. Notably, 50 pixels equal 50 m, which is the minimum range required to interpret micro-geography. Each tile was replicated with a version rotated 90° and another that was mirrored. Creating the rotated images meant we would lose information related to slide direction. The landslides and collapses in this region occurred on north-facing slopes. However, rather than the directional features, it was necessary to learn more about the gravitational deformation features of mountainous bodies, and we increased the learning data by implementing those rotations.

In creating the tiles, out of the 38 slide locations, 35 were used as learning data, whereas the remaining three locations were used as evaluation data for unknown areas. These tiles were allocated randomly for training and verification purposes, as shown in Table 3. The total number of slide tiles eventually became 20,651 images. The tiles of the 63 locations labeled as non-slide and the outside landslide topography were processed in the same manner. There were 4,041 and 12,293 images, respectively, resulting in using a combined total of 36,985 images for the training process.

3.3. Model creation

Model construction comprised a learning and an evaluation phase. The hold-out method was used for the learning phase and comprised training and validation. In the training stage, 70% of the tiles were used with the creation of weighted parameters to construct the model. Next, in the validation stage, the remaining 30% of the tiles were used for verification, referring to the model construction method proposed by Kohavi (1995); each parameter and layer structure was constructed by simulating whether the tiles were judged correctly. The model created was called a “trained model.” The final evaluation phase determined the accuracy of the model using the unknown data by classifying each tile into one of three categories: DL occurred (slide y0), DL did not occur but included landslide topography (non-slide y1) or outside landslide (y2). Probability values were assigned to each tile, and the results of the binary classification were given as correct/incorrect judgment accuracies. A positive judgment was made for values of 0.5 or greater, whereas a negative judgment was made for values of less than 0.5.

Neural network console (Sony Corporation, Tokyo, Japan) was used in this study. In this application, model construction and parameter adjustment were automatically performed using graphical user interfaces (GUIs). The results of the automatic search for the model structure are presented in Appendix Table-A3 and Fig-A2. For comparison with previous methods, we also performed the analysis using slope and wavelet (Sakita et al. 2019) for the learning data.

4.1. Learning model structure and accuracy

Figure 8 shows the multi-layer structure of the trained model after training and validation. The trained model was evaluated using a confusion matrix. A confusion matrix summarizes the results of multi-class classifications (Eq. (1) and (2)) and is a measure of machine learning model performance. Four standard metrics were used to evaluate the performance. These metrics were calculated based on three numbers measured during the test: 1) true positives (TPs, correct detections), 2) false negatives (FNs, missed targets), 3) false positives (FPs, incorrect detections), and 4) true negatives (TNs, correct detections). Recall expresses the proportion of each explanatory variable detected (e.g., 5885 of 6069 for slide (y'0) were detected). In the best case, recall is equal to 1, so the CNN detected all category labels (y'0, y'1, and y'2) in the test set. Precision describes the percentage of category labels correctly detected and classified.

\(Recall=\frac{TP}{TP+FN}\times 100,\frac{TP}{FP+TN}\times 100\) (1)

\(Accuracy=\frac{TP+TN}{TP+FP+FN+TN}\) (2)

As shown in Table 4, for slide areas, recalls were 96% (yʹ0), 52% (yʹ1), and 81% (yʹ2) for each explanatory variable, whereas the overall accuracy was 0.856. Good precision was obtained for slide (y'0) and outside landslide (y'2). The non-slide (y'1) was low, which may result from the number of learning data. The results indicate an increase in the recall ratio and accuracy (except for non-slide(y'1) recall) over those of Sakita et al. (2019), who used two numerical images (Table 5).

Table 4

Correct answer rate from verification results using eight parameters
Data Type	Slide (yʹ0)	Non-slide (yʹ1)	Outside Landslide (yʹ2)	Recall Rate
Slide	5811	178	80	95.7%
Non-slide	391	658	220	51.9%
Outside landslide	464	236	3058	81.4%
Total	7521	1966	6636
Accuracy	0.856

Table 5

Correct answer rate from verification results using two parameters
Data Type	Slide (yʹ0)	Non-slide (yʹ1)	Outside Landslide (yʹ2)	Recall Rate
Slide	5489	176	404	90.4%
Non-slide	231	697	341	54.9%
Outside landslide	755	348	2655	70.6%
Total	7461	2266	6396
Accuracy	0.797

Evaluation as “Unknown data” using the trained model was conducted for three of the 38 slide sites (IDs 8, 20, and 23) and non-slide areas in the northern part of the study area (4 × 1.5 km) that were not used for training and validation. These results are shown in Table 6 and Fig. 9(a) (ex. only non-slide). For the results comparing evaluation and training, the recall of slide (y'0) decreased from 95.7–50.0%. The recall for non-slide (y'2) decreased from 51.9–21.2%. However, the recall of the Outside the landslide(y'2) increased from 81.4–87.6%. The results indicate that the learning was implemented efficiently for slide (y'0) and non-slide (y'1) areas.

Table 6

Results of analysis from the evaluation phase by eight parameters
Data Type	Slide (yʹ0)	Non-slide (yʹ1)	Outside Landslide (yʹ2)	Recall Rate
Slide	12	4	8	50.0%
Non-slide	18	65	223	21.2%
Outside landslide	90	188	1963	87.6%
Total	120	257	2194
Accuracy	0.793

The results using the two types of numerical analysis images are shown in Table 7. and Fig. 9(b) (ex. only non-slide). Compared to the eight types of numerical images (Table 6 and Fig. 9(a)), slide (y'0) increased from 37.5–50.0%. Non-slide (y'1) also decreased, but outside landslide (y'2) increased from 70.4–87.6%. Accuracy also increased from 0.639 to 0793. This indicates that an increase in the number of numerical analysis images had an effect. Additionally, Fig. 10 compares the recall ratios and accuracies of the slide sites (IDs 8, 20, and 23) and non-slide images. The recall ratio improved in four out of eight items, the accuracy improved in three out of four items, and the total accuracy improved in all items.

Table 7

Results of analysis from the evaluation phase using two parameters
Data Type	Slide (yʹ0)	Non-slide (yʹ1)	Outside Landslide (yʹ2)	Recall Rate
Slide	9	8	7	37.5%
Non-slide	118	81	142	23.8%
Outside landslide	444	209	1553	70.4%
Total	571	298	1702
Accuracy	0.639

Unknown data face generalization issues that result from introducing new elements. Specifically, the results express the non-identity of topography and geology, which present import issues in terms of predicting landslides. In contrast, the outside landslide (y'2) represents a general topography not involved with a landslide and could therefore be trained well.

4.2. Evaluation of slide areas (slide)

First, we selected three of the 38 collapse sites (IDs 8, 20, and 23) and evaluated them as unknown data for the collapse sites (data were not used as learning data). These three locations were selected to show different locations and different scale and topographic characteristics as shown in Fig. 2 and Appendix Table A1. Fig. 11 shows an example of ID 8. Fig. 11(a) shows the slope angle diagram before the slide, the approximate location of the slide, the tiles identified as slide areas (y'0), and tiles judged to be uncollapsed landslide topography (y'1). Colorless tiles were identified as having no connection to landslides (y'2). Fig. 11(b) shows the slope angle diagram after sliding.

In ID 8, the NW–SE ridge slid W at the top of the head. Besides terminal cliffs, minor scarps were observed within the slide area. These are DGSDs, and the arcuate main scarp is intermittent, suggesting that DL development was ongoing. Tiles identified as slide (y'0) selected most of the collapsed regions, and the slide areas were considered having the same properties as in the learning data. The northeastern slope of the ridge to the SE of the collapsed site was also selected as slide (y'0). This site did not slide, but because it had the same properties as the learning data, it was considered a candidate for future collapse. In contrast, uncollapsed landslide topography tiles (y'1) were partially selected in the surrounding areas. Thus, the trained model can identify sites with the same characteristics as collapse sites in non-slide areas, which are candidates for predicting future collapses. Fig. 11(c) shows the results of the learning model using slope and wavelet (Sakita et al. 2019). The slide(y'0) was more effective than the two types of analysis (Fig. 10).

Figure 12 shows an example of ID 20. Figure 12(a) shows the slope angle diagram after sliding. Figure 12(b) shows the slope angle diagram before the slide, the approximate location of the collapse, tiles identified by the model as slide areas (y'0), and tiles identified as non-slide landslide topography (y'1). Colorless tiles were identified as having no connection to landslides (y'2). At ID 20, the NW–SE ridge slid at the top of the head, trending NE. Terminal cliffs appeared to develop in the slide area. The minor scarp and arcuate but indistinct main scarp formed intermittently, suggesting that the DL development was ongoing, or that sliding occurred after development. Tiles identified as slides (y'0) were selected for the slide areas. The upper ridge adjacent to the collapse site was selected. However, uncollapsed landslide topography tiles (y'1) were partially selected in the surrounding areas. The trained model could select all slide areas in ID 20. Additionally, sites with minor scarps and irregular undulations were selected as slide areas (y'0), indicating that the learning data were effective. Figure 12(c) shows the results of the learning model using the slope and wavelet (Sakita et al. 2019). Here, the selection of slide (y'0) and non-slide (y'1) were not appropriate. This indicates that using multiple types of numerical analysis can be effective.

Figure 13 shows an example of ID 23. Figure 13(a) shows the slope angle diagram after sliding. Figure 13(b) shows the slope angle diagram before the slide, the approximate location of the collapse, the tiles identified as slide areas (y'0), and tiles judged to be uncollapsed landslide topography (y'1). Colorless tiles were identified as having no connection to landslides (y'2). In ID 23, the NE–SW ridge slid from the top of the head to the NE. Irregular undulations, minor scarps, and terminal cliffs were also confirmed in the slide area. These are DGSDs, but an arcuate main scarp did not form, suggesting this site was in the initial stages of DL development. Tiles identified as slide (y'0) in the analysis results were selected in several locations around the boundary of the slide area. Although the full extent of the slide area was not selected, minor scarps with DGSD characteristics were selected, indicating a certain effect. They did not exhibit a consolidated area; however, they are considered unknown data and have properties other than those of learned tiles, exposing the generality problem for the model. The northwestern part of the collapse exhibited a large triangulated irregular network, with a poor-quality DEM. Owing to its low accuracy, these data were treated as y2 before the learning data. Although this is an error in identifying non-slides within the landslide area, they can be evaluated as a product of learning. Figure 13(c) shows the results of the learning model using the slope and wavelet (Sakita et al. 2019). However, for slides (y'0), the selection accuracy was improved for outside the slide, whereas non-slide (y'1) and outside the landslide (y'2) selection accuracies also improved (Fig. 10).

General micro-topographic interpretations of landslide areas focus on major scarps, terminal cliffs, minor cliffs, and irregular undulations. However, in this study, slopes that experienced actual collapses were used as objective tiles in the learning data. These included terrain with no topographic features. This indicates that the learning data were not complete. However, even before the gravitational deformation affects the whole DL, we can find the same topography as the collapsed area in the 50 × 50 m range.

An important aspect of DLs prediction is the automatic recognition of DGSDs specific to the DL. Finding the same terrain as the collapse site with high probability can predict landslides. The significance of deep learning here is judged by examining whether the topographic features specific to slide areas can be identified automatically.

A single tile used for automatic identification in this study contained topographic information for a 50-m² area. Actual collapse sites are considered as a result of a complex combination of a wider range of micro-topography, leading to collapse. The minimum size of the DL learning data was 1,400 m² (Table 2, ID 27), and the possibility of DLs was limited to cases where over five meshes were concentrated.

5.1. Evaluation of non-slide areas

The location selected as a non-slide area to the north of the learning data (reported by Hiraishi and Chigira 2009) has a knick line in the low-elevation region formed by undercutting. The potential for DLs was noted based on the topographic features; however, no DLs occurred during the 2011 typhoon. In Fig. 14, most of the inside slide (yʹ0) was judged as slides in 1–2 independent locations. Because no more than five tiles were recognized in the slide examples (IDs 8, 20, and 23) described in the previous section, we interpreted this to mean that no DLs occurred during the 2011 heavy rainfall. Fig. 14 (a) and (b) were identified as having four adjacent slides (yʹ0) within a certain range. This site is on a gentle slope with a water catchment area, which is the upper slope of the terminal cliff. Although this area did not collapse during the 2011 torrential rains, it had the same properties as the slide area and was therefore at high risk. This follows the findings of Hiraishi and Chigira (2009) and indicates the potential for a future landslide.

In addition, as shown in Fig. 14(c), 10 sites were identified as having landslide topography without having collapsed (National Research Institute for Earth Science and Disaster Resilience 2015). Many of these sites had distinct main scarps and included topography where sliding had stopped or finished. Although y'1 did not entirely include a landslide topography, it captured the topographic features of the head within a consolidated area, demonstrating the results of learning.

5.2. Limitations and model generalization

This model is still in the initial stages of achieving the ability to be generalized, because this study used data from a limited range of collapse occurrence sites in topography specific to the accretionary zone, and various settings for numerical analysis were applied on a trial-and-error basis. The misinterpretations connected to the present conditions can be explained by two reasons. First, the collapsed areas contain judgment results that lack topographical features. This may have been due to the mixing of similar terrain within the decay area during the learning data selection process. Second, a single tile marked as a slide does not necessarily indicate a slide, because the collapse areas are divided by a mesh. Given the minimum size of the learned collapse (1400 m²), the likelihood of DLs was limited to a concentration of five or more meshes. Regarding building the CNN model, the method used a complex algorithm based on the images of numerical results generated from a single DEM. Here, the independence of the variables used in each analysis, which depends on the input parameter values, must be checked. In this respect, the application scope of this study is limited to similar topographic and geologic structures where the DLs occurred.

This study targeted points where DGSDs manifested and shows that it is possible to estimate the locations of DLs by utilizing image analysis and CNN methods. Although it took 59 h to validate and construct the model with a multi-layered structure in this study, the optimized model can analyze a 1.5 km × 4 km area in about 6 min. This increased speed of analysis will help predict where disasters will be derived. Although the conditions in this study are limited, additional case studies and further data accumulation regarding new pre-slide images can be used as learning data to achieve further improvements in accuracy and generalization in the future. However, using several numerical methods leads to decreased analytical accessibility, which is a trade-off for improving the accuracy of the analysis. Deep learning can improve the accuracy of landslide prediction if the effects of different designs and of augmentation strategies that artificially increase the existing sample size are better understood, and if sufficient learning data exist.

We analyzed unknown collapse sites at three locations using a CNN structure as a learning machine. The results of the analysis using an unknown non-slide site had an accuracy rate of 0.856. The model we constructed is not fully generalizable. Based on the results, the cause of this problem can be improved by trying the following solutions. First solution is by creating teacher data. In this study, it was effective to use the actual slide data; however, the accuracy was limited by the teacher data containing tiles that were not DGSDs. The solution to this problem is to create a refinement of the teacher data limited to DGDGs. However, this would reduce the number of teacher data and thus require more pre-collapse terrain data.

Second, the performance of an image-based CNN depends heavily on its design (i.e., layer depth, input window size, and learning data strategy). It is debatable whether it is better to include many methods or to select only those that are effective. This point requires discussion because it depends on the topographical features of interest. A fixed optimal range of 50 pixels was also considered unique to each area. These issues require further study. The CNN model, which solved this problem and accumulated many collapse cases, can specify more specific DL predictions in subtropical regions, including Japan. This can contribute to screen disaster locations, risk assessments, and hazard mapping during disasters. Deep learning could improve landslide prediction if the effects of learning model designs that include a variety of learning data and multi-modality are better understood, learning data with sufficient decay histories exist, and the current understanding of the effects of augmentation strategies that artificially increase the existing sample size improves.

Acknowledgements

The aerial laser survey data used in this study were provided by the Geospatial Information Authority of Japan. We are deeply grateful to them for providing these valuable data.

Funding: The authors declare that no funds, grants, or other support were received during the preparation of this manuscript.

Competing Interests: The authors have no relevant financial or non-financial interests to disclose.

Author contributions: Teruyuki Kikuchi conceived the study, identified the topographic features, established the CNN model, and drafted the manuscript. Koki Sakita carried out the numerical analyses. Kenichi Takahashi conducted the geological survey. Satoshi Nishiyama participated in study design and coordination. All authors read and approved the final manuscript.

Aleotti P, Chowdhury R (1999) Landslide hazard assessment: summery review and new perspective. Bull Eng Geol Env 58(1):28–44. https://doi.org/10.1007/s100640050066
Arai N, Chigira M (2019) Distribution of gravitational slope deformation and deep-seated landslides controlled by thrust faults in the Shimanto accretionary complex. Eng Geol 260:105236. https://doi.org/10.1016/j.enggeo.2019.105236
Baltrušaitis T, Ahuja C, Morency LP (2018) Multimodal machine learning: A survey and taxonomy. IEEE PAMI 41(2):423–443. https://doi.org/10.1109/TPAMI.2018.2798607
Beven KJ, Kirkby MJ (1979) A physically based, variable contributing area model of basin hydrology. Hydrol Sci Bull 24:43–69. https://doi.org/10.1080/02626667909491834
Booth AM, Roering JJ, Prron JT (2012) Automated landslide mapping using spectral analysis and high-resolution topographic data: Puget Sound Lowlands, Washington, and Portland Hills, Oregon. Geomorphology 109:132–147. https://doi.org/10.1016/J.GEOMORPH.2009.02.027
Burrough PA, McDonnell RA, Lloyd CD (2015) Principles of geographical information systems. Oxford university press
Can A, Dagdelenler G, Ercanoglu M, Sonmez H (2019) Landslide susceptibility mapping at Ovacık-Karabük (Turkey) using different artificial neural network models: comparison of training algorithms. Bull Eng Geol Environ 78:89–102. https://doi.org/10.1007/s10064-017-1034-3
Chen W, Pourghasemi HR, Naghibi SA (2018) A comparative study of landslide susceptibility maps produced using support vector machine with different kernel functions and entropy data mining models in China. Bull Eng Geol Environ 77(2):647–664. https://doi.org/10.1007/s10064-017-1010-y
Chigira M, Kiho K (1994) Deep-seated rockslide-avalanches preceded by mass rock creep of sedimentary rocks in the Akaishi Mountains, central Japan. Eng Geol 38(3-4):221–230. https://doi.org/10.1016/0013-7952(94)90039-6
Chigira M, Wang WN, Furuya T, Kamai T (2003) Geological causes and geomorphological precursors of the Tsaoling landslide triggered by the 1999 Chi-Chi Earthquake, Taiwan. Eng Geol 68:259–273. https://doi.org/10.1016/S0013-7952(02)00232-6
Chigira M, Tsou CY, Matsushi Y, Hiraishi N, Matsuzawa M (2013) Topographic precursors and geological structures of deep-seated catastrophic landslides caused by Typhoon Talas. Geomorphology 201(1):479–493. https://doi.org/10.1016/j.geomorph.2013.07.020
Crosta GB, Chen H, Frattini P (2006) Forecasting hazard scenarios and implications for the evaluation of countermeasure efficiency for large debris avalanches. Eng Geol 83:236–253. https://doi.org/10.1016/j.enggeo.2005.06.039
Dahal RK, Hasegawa S, Nonomura A, Yamanaka M, Dhakal S, Paudyal P (2008) Predictive modelling of rainfall-induced landslide hazard in the Lesser Himalaya of Nepal based on weights-of-evidence. Geomorphology 102(3-4):496–510. https://doi.org/10.1016/j.geomorph.2008.05.041
Erosion Control and Disaster Prevention Division, Prefectural Land Management Department, Nara Prefecture (2012) Nara Prefecture Kii Peninsula Flood and Large-Scale Landslide Disaster Archive. http://www3.pref.nara.jp/doshasaigai/saboarchive1. (Last accessed on 8 December 2021)
Evans SG, Guthrie RH, Roberts NJ, Bishop NF (2007) The disastrous 17 February 2006 rockslide-debris avalanche on Leyte Island, Philippines: a catastrophic landslide in tropical mountain terrain. Nat Hazards Earth Syst Sci 7:89–101. https://doi.org/10.5194/nhess-7-89-2007
Ghorbanzadeh O, Blaschke T, Gholamnia K, Meena SR, Tiede D, Aryal J (2019) Evaluation of different machine learning methods and deep-learning convolutional neural networks for landslide detection. Remote Sens 11(2):196. https://doi.org/10.3390/rs11020196
Guthrie RH, Evans SG, Catane SG, Zarco MAH, Saturay Jr. RM (2009) The 17 February 2006 rock slide-debris avalanche at Guinsaugon Philippines: a synthesis. Bull Eng Geol Environ 68:201–213. https://doi.org/10.1007/s10064-009-0205-2
Hara H, Hisada K (2007) Tectono-metamorphic evolution of the Cretaceous Shimanto accretionary complex, central Japan: Constraints from a fluid inclusion analysis of syn-tectonic veins. Island Arc 16(1):57–68. https://doi.org/10.1111/j.1440-1738.2007.00558.x
He S, Pan P, Dai L, Wang H, Liu J (2012) Application of kernel-based Fisher discriminant analysis to map landslide susceptibility in the Qinggan River delta, three Gorges, China. Geomorphology 171:30–41. https://doi.org/10.1016/J.GEOMORPH.2012.04.024
Hiraishi N, Chigira M (2009) Topographic evolution indicated by the distributions of knickpoints and slope breaks in the tectonically active Kii Mountains, southwestern Japan, EGU General Assembly 2009, held 19-24 April 2009 in Vienna, Austria. http://meetings.copernicus.org/egu2009, p.6722. Aaccessed 25 May 2021
Huang F, Zhang J, Zhou C, Wang Y, Huang J, Zhu L (2020) A deep learning algorithm using a fully connected sparse autoencoder neural network for landslide susceptibility prediction. Landslides 17(1):217–229. https://doi.org/10.1007/s10346-019-01274-9
Kannan M, Saranathan E, Anabalagan R (2013) Landslide vulnerability mapping using frequency ratio model: a geospatial approach in Bodi-Bodimettu Ghat section, Theni district, Tamil Nadu. India Arab J Geosci 6(8):2901–2913. https://doi.org/10.1007/s12517-012-0587-5
Kasai M, Ikeda M, Asahina T, Fujisawa K (2009) LiDAR-derived DEM evaluation of deep-seated landslides in a steep and rocky region of Japan. Geomorphology 113:57–69. https://doi.org/10.1016/j.geomorph.2009.06.004
Kikuchi T, Hatano T, Nishiyama S (2019) Verification of microtopographic features of landslide or non-landslide area in Typhoon Talus in 2011. Journal of the Japan Landslide Society, 56(4), 141–152 (in Japanese with English abstract). https://doi.org/10.3313/jls.56.141
Kimura K (1986) Stratigraphy and paleogeography of the Hidakagawa Group of the Northern Shimanto Belt in the southern part of Totsugawa village, Nara Prefecture, southwest Japan. J Geol Soc Japan 92(3):185–203 (in Japanese with English abstract). https://doi.org/10.5575/geosoc.92.185
Kohavi R (1995) A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai 14(2):1137–1145
Lee S (2004) Application of likelihood Ratio and logistic Regression Models to Landslide Susceptibility Mapping Using GIS. Environ Management 34(2):223–232. https://doi.org/10.1007/s00267-003-0077-3
Liu Y, Wu L (2016) Geological disaster recognition on optical remote sensing images using deep learning. Proced Comp Sci 91:566–575. https://doi.org/10.1016/j.procs.2016.07.144
McKean J, Roering J (2004) Objective landslide Detection and surface morphology mapping using high-resolution airborne laser altimetry. Geomorphology 57:331–351. https://doi.org/10.1016/S0169-555X(03)00164-8
Matsushi Y, Chigira M, Yamada M, Hiraishi N, Matsuzawa M (2012) Location and timing of deep-seated landslides in Kii Mountains at the 2011 disaster: an approach from rainfall history. Characterization, prediction, and management of deep-seated catastrophic landslides, Disaster Prevention Research Institute, Kyoto University. 43–45 (in Japanese with English abstract). http://www.slope.dpri.kyoto-u.ac.jp/symposium/DPRI_20120218proceedings.pdf
Ngiam J, Khosla A, Kim M, et al (2011) Multimodal deep learning. In: Proceedings of the 28th International Conference on International Conference on Machine Learning. Omnipress, Bellevue, Washington, USA, pp 689–696
Nishida K, Kobashi S, Mizuyama T (1997) DTM-based topographical analysis of landslides caused by an earthquake, J Japan Soc Erosion Control Eng 49(6):9–16 (in Japanese with English abstract). https://doi.org/10.11475/sabo1973.49.6_9
Sakita K, Nishiyama S, Kikuchi T, Song J, Ohnishi Y (2019) Quantitative Interpretation of Landslide Occurrence Site by AI Technology of Image Classification and Topographical Analysis Map Created by Aerial Laser Data. Proceedings of SEE-Bangkok 541–546. ISBN: 978-4-909106032 C3051
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556
Srivastava N, Salakhutdinov R (2012) Multimodal Learning with Deep Boltzmann Machines. In Proceedings of NIPS'12. Curran Associates Inc., Red Hook, NY, 2222–2230
Tsou CY, Chigira M, Matsushi Y, Hiraishi N, Arai N (2017) Coupling fluvial processes and landslide distribution toward geomorphological hazard assessment: a case study in a transient landscape in Japan. Landslide 14:1901–1914. https://doi.org/10.1007/s10346-017-0838-3
Tsou CY, Feng Z, Chigira M (2011) Catastrophic landslide induced by Typhoon Morakot, Shiaolin, Taiwan. Geomorphology 127(15):166–178. https://doi.org/10.1016/j.geomorph.2010.12.013
Varnes DJ (1958) Landslide types and processes. In: Eckel E.B. (Ed.) Landslides and engineering practice, Highway Res Board Spec Rep 29:20–47
Wakai A, Bhat DR, Kotani K, Osawa S (2020) Numerical Simulation of a Creeping Landslide Case in Japan. In: Workshop on World Landslide Forum, Springer Cham. 273–280. https://doi.org/10.1007/978-3-030-60706-7_26
Wang H, Zhang L, Yin K, Luo H, Li J (2021) Landslide identification using machine learning. Geosci Frontiers 12(1):351–364. https://doi.org/10.1016/j.gsf.2020.02.012
Wang Y, Fang Z, Hong H (2019) Comparison of convolutional neural networks for landslide susceptibility mapping in Yanshan County, China. Sci Tot Environ 666:975–993. https://doi.org/10.1016/j.scitotenv.2019.02.263
Woodcock NH (1977) Specification of fabric shapes using an eigenvalue method. GSA Bull 88(9):1231–1236. https://doi.org/10.1130/0016-7606(1977)88<1231:SOFSUA>2.0.CO;2
Woodcock NH, Naylor MA (1983) Randomness testing in three-dimensional orientation data. J Structural Geol 5(5):539-548. https://doi.org/10.1016/0191-8141(83)90058-5 12)
Yokoyama R, Shirasawa M and Kikuchi Y (1999) Representation of topographical features by opennesses. Photogramm Eng Remote Sens 38:26–34. https://doi.org/10.4287/jsprs.38.4_26
Zischinsky Ü (1966) On the deformation of high slopes. In: Proceedings 1st Conference of International Society for Rock Mechanics, Lisbon, Sect. 2:179–185

Appendix.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

Multi-modal deep learning detection of deep-seated gravitational slope deformation by Typhoon Talas in 2011

Status:

Version 1

Abstract

Figures

1. Introduction

2. Geologic Setting And Its Disaster Status

2.1. Outline of the rainstorm hazard

2.2. Geologic setting

2.3. DL setting

3. Methods

3.1. Generation of numerical analysis data

3.2. Creation of tiles for learning data

3.3. Model creation

4. Results: Evaluating Slide/non-slide Areas

4.1. Learning model structure and accuracy

4.2. Evaluation of slide areas (slide)

5. Discussion

5.1. Evaluation of non-slide areas

5.2. Limitations and model generalization

6. Conclusions

Declarations

References

Supplementary Files

Status:

Version 1