Multi-modal Landslide Monitoring Data Fusion Algorithm Based on Resistivity Imaging

In the process of landslide deformation monitoring, the indicators of monitoring system based on surface 3 displacement cannot accurately reflect the deformation evolution law of deep geotechnical body. Although the joint 4 time curve of deep displacement monitoring of borehole and related monitoring data can reflect the deformation 5 characteristics inside the slope body, it cannot spatially describe and explain the overall deformation process of 6 geotechnical body completely due to the limitation of technical conditions such as borehole. In this paper, using the 7 characteristics of resistivity imaging technology with fast and accurate collection of electrical information of 8 subsurface medium and multi-dimensional imaging, we take resistivity imaging data as complete modal data and 9 fuse deep displacement and groundwater level and other modal data. Through joint depth matrix decomposition and 10 optimization, layer-by-layer modal semantic matching and updating, the distribution and representation differences 11 of modal data are compensated, and the analysis tasks such as classification and clustering of incomplete 12 multimodal data are completed, and the inversion results of resistivity data are updated according to the output 13 modal shared eigenvalues to realize effective multidimensional imaging monitoring of the internal deformation 14 process of landslide geological bodies. 15


Introduction 17
Landslide geological hazards are complex physical systems with a long time evolutionary process (Xu et al. 18 2008). Studied in the direction of evolutionary mechanism, landslides are the result of the joint action of 19 fundamental, action and coupled fields generated by the structural, seepage, stress, chemical and temperature fields 20 Geological hazards are characterized by frequency, hazard and complexity, and it is difficult for a single 31 monitoring means to accurately reflect the landslide evolution process, and the integrated analysis and fusion of 32 data obtained using different methods such as multi-temporal and multi-scale displacement monitoring, 33 hydro-meteorological, and geological monitoring, and geotechnical interaction monitoring of the sky-lands is of 34 great significance for predicting geological hazards (Lin et  Multimodal information can describe the same data instance from different sides, and effective analysis of 48 multimodal complementary information can obtain a more reasonable representation of data characteristics. The 49 main causative factor of landslide generation is the weakening of soil shear strength in the process of rainwater 50 infiltration caused by atmospheric precipitation, and the slope-sliding force is greater than the soil shear resistance. 51 In this process, different stratigraphic structures within the geotechnical body, with the infiltration of rainwater, will 52 form obvious resistivity differences near the slip surface. The resistivity imaging technique, based on the significant 53 differences between the material composition, porosity, structure, and water content of the landslide weak body 54 (face) and the surrounding rock (Carlo et al. 2013;Yin et al. 2018), measures the electrical conductivity 55 information of the subsurface medium by scanning a large-area electrode array and can obtain a complete 56 multidimensional electrical data set, reflecting the internal structure of the geological body. Therefore, a 57 multimodal dataset consisting of monitoring data such as resistivity imaging data, deep horizontal displacement, 58 soil moisture, groundwater level and rainfall can effectively reflect the process of obtaining structural deformation 59 inside the landslide geotechnical body (Shao et al. 2013;Yin et al. 2017). 60 The multi-source data fusion technology can comprehensively analyze and reasonably utilize the multi-source 61 heterogeneous data of landslide monitoring, eliminate the possible redundancy and mutual exclusion between data, 62 and make all kinds of data complement and cooperate with each other, thus effectively improving the reliability of 63 landslide monitoring data and increasing the utilization rate of landslide monitoring data (Qiu 2017; Zhao et al. In large landslide-monitoring sites, large-area, multi-dimensional resistivity data collection is required. electrodes (data transmission between mainframe and sub-stations through CAN bus, the number of connected 130 sub-stations can be expanded as needed). For different monitoring environments, the layout of the host, sub-stations 131 and smart electrodes can be flexibly adjusted (Fig.3), is a layout designed for the need of long-distance monitoring 132 of high slopes. 133 Aiming at the migration process of underground seepage field in landslide geological disaster evolution, which 134 is often irregular and its transport speed and direction have the characteristics of sudden change, the initial scanning 135 and collection of resistivity is performed in a large range (cross-electrode power supply and data collection) by 136 rapidly changing the collection area. After determining the range of landslide-hidden trouble spots, the intelligent 137 electrodes are encrypted and the electrode network is automatically coded, and the working state of electrodes is set 138 according to the measurement needs, and the measurement is realized by completing the conversion of each 139 electrode state through the host control to realize the resistivity monitoring data collection with real-time and 140 dynamic variable monitoring point density and multi-dimensional resistivity dynamic collection grid structure in 141 In the actual situation of the landslide geological hazard monitoring process, there is a large correlation 151 between resistivity imaging technology, deep displacement and related monitoring data, and the modal feature 152 information of each monitoring data can describe the same data instance from different sides, but it is difficult for 153 various monitoring data to constitute a modal data set with complete feature values in time and space due to 154 technical conditions, to achieve effective analysis of multi-modal complementary information and be able to allow 155 for effective analysis of multimodal complementary information and a more reasonable representation of data 156 characteristics. The multimodal data fusion algorithm for landslide geology body deep displacement monitoring 157 ensures the local similarity of each modal data by encoding the geometric structure of the data with graph 158 regularization factors, constructs a deep semantic matching model that fuses modal deep neural networks and 159 incomplete multimodal matrix decomposition, and then updates and optimizes the model ( . By jointly training and optimizing the modal private depth network and the base matrix, as well as 165 the modal consistent encoding matrix, multimodal depth semantic shared features in the subspace will be obtained. 166 The flowchart is shown in Fig.5. 167 To ensure the consistency of each modal data with its geometric structure in the potential subspace, the learned 168 shared encoding matrix is C P represented regularly by the invariant graph model. Assuming that there are two data 169 instances close to each other in the original data space The incomplete multimodal deep semantic matching model can be represented as 185 By sharing the characteristics C P and )   direction, and about 120 m wide in the north-south direction, and the natural slope angle of the slope surface is 209 30°-37°. The cover of the front edge is mainly residual slope deposits and crumbling slope deposits, with a small 210 number of fully weathered mudstone chips, while the middle and back edge cover are mudstone with different 211 degrees of weathering respectively. The lithology is dominated by calcium-bearing mudstone in the upper part, 212 calcareous siltstone in the middle part, interspersed with calcium-bearing mudstone, and conglomerate, sandstone 213 and conglomerate-bearing sandstone in the bottom part. This stratigraphic structure is favorable for rainwater to 214 continuously replenish groundwater from top to bottom and infiltrate into the lower mudstone, and the muddy 215 debris and weak interlayer in the rock layer are immersed in the water for a long time, which will cause the strength 216 of the soil body to decrease. Under the influence of multiple effects such as self-weight of the landslide body, 217 rainfall infiltration, and vibration caused by human engineering activities, the cohesive force inside the landslide 218 body gradually decreases, i.e., the slide force continues to increase due to rainfall infiltration and other effects, 219 while the anti-slip force decreases rapidly due to shear damage, the landslide body shows increased cumulative 220 deformation and contributes to further weakening of the weak zone inside the landslide body. A monitoring profile 221 was set up in the middle of the slope in the form of the profile shown in Fig.6, and monitored for 182 consecutive 222 days from June to December 2019. 223 The rainfall-monitoring point is arranged at the leading edge of the slope YL1, rainfall, and deep displacement 224 data monitoring, sampling frequency (triggered acquisition). 182 consecutive days of daily average rainfall 225 monitoring data at point YL1 are shown in Fig.7. 226 There is an obvious continuous precipitation process on the 90th-100th monitoring day, because the surface 227 displacement rate of the landslide correlates well the rainfall, while the deep displacement rate of the landslide has 228 a good correlation with the rate of deep displacement of the landslide has a certain lag with the amount of rainfall, 229 which shows a greater influence on the deformation of the soil at the trailing edge and the central slip zone in the 230 slope of this experiment. The monitoring frequency is 0.5 times/hour, and the monitoring period is 182 days. Figure 8 shows the 236 average daily deformation results of ZK2 monitoring on monitoring days 85-120.From the 36-day continuous 237 observation curve of the central monitoring hole ZK2 (Fig.8), it can be seen that the displacement is basically 238 generated by the 0-5m hole section, the maximum sliding displacement at the mouth of the hole is 16.15mm, the 239 curve forms a more obvious sliding surface at 3m, the sliding displacement above the sliding surface is larger, 240 while the lower displacement is smaller, and the landslide is dominated by shallow overall sliding.
There are more than 10 kinds of acquisition devices commonly used in resistivity imaging technology, and the 242 Wenner device and Wenner-Schlumber device are used as experimental devices in this paper. Among them: 243 Wenner device: AM=MN=NB, A, M, N, B move to the right at the same time point by point, with the increase of 244 pole spacing, the depth through which the profile inversion is interpreted also gradually increases, the electric field 245 distribution of Wenner device is mainly directly below the center of the device, and the sensitivity function 246 becomes horizontal distribution. Wenner device is more sensitive to the vertical change of resistivity, used to detect 247 horizontal target body; Wenner-Schlumberger device-running pole way: this device between Wenner and 248 Schlumberger, the interval layer is 3a (a is the standard pole spacing), in 1-3 layer Schlumberger method running 249 poles, 4-6 layer MN interval becomes 3a, 7-9 layer MN electrode spacing becomes 5a, and so on, to get an inverted 250 trapezoidal cross-sectional map. Its high sensitivity value appears directly below between the measuring electrodes, 251 but the detection depth is small. The slip surface of landslide geological hazards is located within 30m below the 252 surface, and this depth is just within the sensitivity range of Wenner-Schlumber device with pole spacing (a=1m, 253 a=0.5m) (reducing the pole spacing can effectively improve the monitoring accuracy), and it is a more 254 ideal-monitoring device for landslide geological hazards because it takes into account both the horizontal and 255 vertical resolution. 256 Resistivity data collection was performed from the top to the bottom of the slope along the profile direction as 257 shown in Fig.6 with the measurement line , electrode spacing a=4 m, using a Wenner device (which has better 258 sensitivity to lateral structures) for resistivity data collection, the number of measurement electrodes was 60, the 259 supply voltage was 90 V, the maximum supply distance was AB=236 m, the effective measurement depth was 32 260 m, and on the 85th and 120th day of the monitoring process using DEM-3 distributed direct current meter with 261 smart electrodes was used to measure this profile 4 times/day. The Swedish high-density processing software 262 RES2Dinv was applied for topographic correction and data inversion processing, and the resistivity inversion 263 results were obtained as shown in Fig.9 for the 85th monitoring day (before rainfall) and the 120th monitoring day 264 (after continuous rainfall) in Fig.10 .   Fig.9, Fig.10, Fig.13, and Fig.14, it can be seen that: ① with the increase of rainfall, the overall 289 structural resistivity value of this slope has a significant decrease, and the data fusion results near the two boreholes ZK1 290 and ZK2, with three kinds of modal data fusion near the surface (0-5 m), the error is less than the deep data fusion results; 291 ② at 6-8 m of ZK1 and 12-14 m of ZK2 are slip surface, the structure of the two parts above and below the slip surface 292 are different, which leads to the obvious difference of discontinuity in resistivity data, and the error of the fusion result 293 reaches 0.36%, which is lower than the fusion result of other depth data of uniform medium, indicating that the fusion 294 algorithm proposed in this paper can effectively monitor the overall deformation and displacement of the slip surface. 295 Fig.11 and Fig.12 show the measured data, fusion results and error analysis of the electrical data before and after the rain 296 in the horizontal fourth layer (depth of 8 m), respectively.

297
As can be seen from Fig.11, the electrical data of the slope as a whole at the resistivity data collection points 3~8 298 and 13~40 at 8 m below the ground surface produced significant changes with rainfall infiltration, and the different water 299 saturation of the rock body led to obvious differences in the electrical data. The results correspond to the main slip 300 surface shown in Fig.6. From Fig.12, it can be seen that the error of the results of the fusion before the rain (-1.7% to 301 4.2%) is significantly larger than that after the rain (0.4% to 2.9%), and the error of data fusion is around 1% near both 302 ZK1 (collection point 10) and ZK2 (collection point 28). Fig.13 and Fig.14 show the 2D inversion effect of the output 303 after updating the resistivity imaging technique data by data fusion.

304
The deep displacement monitoring of the borehole can provide the most direct and effective correction and 305 supplement to the resistivity imaging data, although continuous measurement cannot be achieved in space. From Fig.12   306 and Fig.13, it can be seen that: ① comparing Fig.9 and Fig.12, the results before and after data fusion at the leading edge

Conclusion 320
More than 85% of landslide geological hazards are caused by the dynamic changes of soil seepage field caused by 321 atmospheric precipitation and its resulting deep displacement, so the study of internal deformation evolution mechanism 322 of landslide geological body is the key to landslide monitoring and prediction, and when the modal distribution or 323 characteristics differ greatly, it is difficult to ensure the fusion by only using a linear or nonlinear transformation to 324 compensate for the semantic deviation between multi-modal data for monitoring internal structural changes of landslide 325 body The validity of the results. The depth semantic matching multimodal data fusion algorithm for landslide geology 326 monitoring based on resistivity imaging technology uses the depth semantic matching mechanism of incomplete modal 327 data, explores the depth semantic sharing features of modal data, and establishes multilayer nonlinear correlation among 328 multimodal data by jointly optimizing the fused modal private depth network and the graph regularization-based 329 incomplete modal data learning model, and then obtains the depth semantic matching features of multimodal data. The 330 deep semantic matching features of multimodal data can effectively compensate for the large semantic bias between 331 modalities and obtain more accurate data sharing semantics. In the later stage, by combining multiple surface 332 displacement monitoring data sets, heterogeneous modal data migration fusion with multi-layer semantic matching can 333 obtain the overall three-dimensional dynamic changes of landslide geological body, which provides powerful technical 334 support for landslide geological disaster monitoring and prediction. Figure 1 The arbitrary quadrupole device (Fig.1), with a topographic correction, enables the inversion of 2D pro les of geological bodies with different topography.

Figure 2
In large landslide-monitoring sites, large-area, multi-dimensional resistivity data collection is required. The system controls multiple electrical measurement sub-stations (main functions include: measurement , control the collection sequence and data upload) through the host computer, and each sub-station controls the smart electrodes connected to it (the electrodes internally realize the function conversion between power supply A, B, and measurement M, N), and the collected data are stored in the electrical measurement sub-stations and transferred to the host computer (Fig. 2).

Figure 3
For different monitoring environments, the layout of the host, sub-stations and smart electrodes can be exibly adjusted (Fig.3), is a layout designed for the need of long-distance monitoring of high slopes.

Figure 4
Fig.4a shows a schematic of the dynamic moving electrode grid when scanning the hidden area over a large area. In which, the solid circle on the left side is de ned as the scanning area, and the dashed circle on the right side is de ned as the area to be scanned. When a hidden spot is found, it can be switched to the encrypted scanning mode shown in Fig. 4b. Since the effective depth and accuracy of the inversion of resistivity imaging has been depending on the spacing, the exible electrode grid layout can effectively reduce the pole spacing and improve the accuracy of the complete modal data set based on resistivity imaging data and the reliability of the multidimensional imaging of the internal structure of the fused landslide.

Figure 5
By jointly training and optimizing the modal private depth network and the base matrix, as well as the modal consistent encoding matrix, multimodal depth semantic shared features in the subspace will be obtained. The owchart is shown in Fig.5.  The rainfall-monitoring point is arranged at the leading edge of the slope YL1, rainfall, and deep displacement data monitoring, sampling frequency (triggered acquisition). 182 consecutive days of daily average rainfall monitoring data at point YL1 are shown in Fig.7.

Figure 8
The monitoring frequency is 0.5 times/hour, and the monitoring period is 182 days. Figure 8 shows the average daily deformation results of ZK2 monitoring on monitoring days 85-120.From the 36-day continuous observation curve of the central monitoring hole ZK2 (Fig.8), it can be seen that the displacement is basically generated by the 0-5m hole section, the maximum sliding displacement at the mouth of the hole is 16.15mm, the curve forms a more obvious sliding surface at 3m, the sliding displacement above the sliding surface is larger, while the lower displacement is smaller, and the landslide is dominated by shallow overall sliding.

Figure 9
The Swedish high-density processing software RES2Dinv was applied for topographic correction and data inversion processing, and the resistivity inversion results were obtained as shown in Fig.9 Figure 10 for the 85th monitoring day (before rainfall) and the 120th monitoring day (after continuous rainfall) in Fig.10 .

Figure 11
As can be seen from Fig.11, the electrical data of the slope as a whole at the resistivity data collection points 3~8 and 13~40 at 8 m below the ground surface produced signi cant changes with rainfall in ltration, and the different water saturation of the rock body led to obvious differences in the electrical data. Figure 12 show the measured data, fusion results and error analysis of the electrical data before and after the rain in the horizontal fourth layer (depth of 8 m), respectively. Figure 13 the error of data fusion is around 1% near both ZK1 (collection point 10) and ZK2 (collection point 28). show the 2D inversion effect of the output after updating the resistivity imaging technique data by data fusion.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. renamedbb46f.doc