Research and application of 3D laser scanning in transparent longwall

With the rapid development of science and technology, the development of coal mining in China is stepping into intelligent mining stage from the mechanized automatic mining stage. And the research of intelligent mining is also upgrading to the self-adaptive automation mining from visual remote intervention. In 2019, the first practice of self-adaptive digital mining technology, which is based on transparent longwall theory, was performed in #43101 longwall of Yujialiang coal mine and made notable gains. The 3D laser scanning technology, which played an important role in technology architecture of Yujialiang coal mine’s transparent longwall practice, transformed the mined longwall space information into digital format and then provided reliable basic data for cutting template calculation. This paper introduces application of 3D laser scanning in Yujialiang coal mine in detail, including principle of 3D laser scanning, detection of intersecting curve between longwall’s coal wall and roof, neighbor point-clouds splicing, transformation for longwall pointcloud from local space coordinate system to 3D geological model’s global space coordinate system. The experiment result in #43101 longwall of Yujialiang coal mine demonstrated that 3D laser scanning technology, which is able to quickly and precisely capture mined longwall space information, is a important sensing technology involved by self-adaptive automation mining.


Introduction
Coal, which is often referred to as food for industry, dominates energy consumption structure in China at present. The coal industry is inextricably linked to energy production, logistics transportation, steel and chemical industry. Compared with the other industries, it is categorized as a high-risk industry however. Due to arduous working environment, aging practitioners, less professional talents, it becomes more difficult to mine coal resources. Besides, because of increased coal utilization, slowed down economic growth, the coal demand will decrease and progressively arrive peak in the future (Tian, 2016). Therefore, the action plan about revolution and innovation of energy technology in 2016-2030 was complied by the National Development and Reform Commission and National Energy Board in 2016. The action plan explicitly pointed out that by 2050, a safe, green efficient and intelligent mining technology system will be fully established to achieve safe, green and efficient coal mining (Hu, 2016). Therefore, considering about current industry situation, market requirements, na-tional policies and so on, the coal mining industry will develop comprehensively, integrated and synergistically in the future. Traditional coal mining method will be replaced by intelligent unmanned mining technology (Zhang and Li, 2014).
The first visual remote intervention mining demonstration in Huangling coal mine in 2014 started intelligent mining practice in China. On the basic of the shearer memory cutting technology and combined with visual remote intervention, this method demonstrated its effectiveness in simple geological condition's longwall. It is able to transfer miner from dangerous longwall to safer roadway monitoring center. However, because shearer memory cutting technology could not fit complicated geological condition, it exposed many problems when was applied to complicated geological condition, such as frequent remote intervention, serious human decision dependent. As a result, the coal mining industry requires a self-adaptive automation mining technology which is able to fit complicated geological condition.
Researcher Li Shoubin from Beijing Tiandi-Marco Electro-Hydraulic Control System Company proposed concept of transparent longwall (Li, 2019) in International intelligent mine innovation forum in 2018. On the basis of the real mining longwall system prototype, the transparent longwall adopts mining engineering, internet of things, information science and technology, big data, artificial intelligence and other theoretical basis. Meanwhile, taking the National geodetic coordinate system as absolute coordinate system, it is supported by geological exploration, computer modeling, 3D simulation and so on technology. The intelligent analysis and decision system model during the coal mining is built, on the basic of collected geological data and supervised mining state. Finally, the digital longwall model is generated Wang and Du, 2019).
In 2019, a new self-adaptive digital mining technology based on the transparent longwall theory was proposed and implemented in Yujialiang coal mine. By comparing high-precision 3D geological model and 3D laser scanning model of longwall, it calculates shearer cutting template and sends the template to shear. Then shearer works by the template which is able to fit changing geological condition. It was the first practice of self-adaptive intelligent mining in China, even in the world Wang and Du, 2019;Gu and Zhao, 2020). In this practise, 3D laser scanning technology played an important role in digitizing mined longwall space and provided reliable basic data for cutting template calculation. Upcoming sections cover 3D laser scanning technology and its application in Yujialiang coal mine in detail.

Principle of 3D laser scanning technology
The 3D laser scanning, also known as 3D LIDAR(3-Dimensional Light Detection and Ranging) is able to emit one or more lasers into space. The information, including position (x,y,z) and intensity (Heritage and Large, 2009), is got by comparing emitted laser and reflected one, as shown in equation(1). The 3D laser scanning can reconstruct the digital model about surrounding by contactless active measurement. R is the distance from start point of laser to the detected object. and are pitch angle and horizontal azimuth angle of laser.Principle of 3D laser scanning is shown in Fig.1.

Typical LIDAR products
There are some typical manufacturers, which are engaged in the 3D laser scanner production and related technologies, such as Ocular from Australia, Leica and Velodyne from America, Optech from Canada, LeiShen and SLAMTEC from China, and so on. Their products and technologies cover various application fields, such as airborne, vehicular and ground 3D laser scanners. Based on the different scanning method, 3D LIDARs are divided into single-layer LIDAR, multi-layer LIDAR and full-field single-layer LIDAR which is improved on basic of single-layer LIDAR.
1)Single-layer LIDAR emits only single laser light. The laser light source, which emit laser into space, is installed on a motor. The 2D point-cloud plane could be got by rotating laser light. At present, single-layer LIDAR is mainly used on the robotics field, mostly service robotic. It can help robot to cross obstacles. One of typical single-layer LIDAR products is RPLIDAR A3 of SLAMTEC in China.
2)Multi-layer LIDAR emits multi-layer laser lights. There are 4-layer, 8-layer, 16-layer, 32-layer, 64-layer and 128-layer LIDAR in market currently. The multi-layer LIDAR could get height information of object and reconstruct 3D digital model of surrounding. This technology mainly serves unmanned driving. One of typical multi-layer LIDAR products is VLP-16 of Velodyne in America (Glennie et al.,2016).
3)Full-field single-layer LIDAR is made of a single-layer LIDAR and a rotary motor. Single laser light source of the single-layer LIDAR can scan along a helix track which is generated by motor driving. Therefore, 3D point-cloud of surrounding could be reconstructed by the use of rotating 2D point-cloud plane of single-layer LIDAR. One of typical full-field single-layer LIDAR is RobotEye RE05 (Wood and Bishop, 2012) in Australia. Table 1. shows the comparison of typical 3D LIDAR products. Based on the mature single-layer LIDAR.
Better cost performance.
Scanning result 3 Application of 3D laser scanning in Yujialiang coal mine

Background
Yujialiang coal mine locates in Shenmu City, Shaanxi Province. The #43101 longwall was selected as experiment longwall. Longwall's length and advancing length are 350.98m and 1809.4m respectively. Coal-seam thickness is 1.0-1.7m, and average thickness is 1.47m. Dip angle is 1-3°. Designed mining height is 1.3-1.7m, and average height is 1.4m actually. Recoverable reserves are 1.141 million tons. The #43101 longwall was the first automatic thin coal seam longwall in Yujialiang coal mine. There are 206 hydraulic supports and the distance between neighbor supports is 1.75m. The scraper conveyer's middle section covers 197 supports(Gu and Zhao, 2020).
Based on transparent longwall self-adaptive mining solution of Yujialiang coal mine, it needed to compare high-precision 3D geological model and longwall's 3D scanning model, so as to calculate cutting template, which would be sent to shearer as reference to adjust drum height in feature. In particular, it was aim to compare roof curve in high-precision 3D geological model with intersection curve between longwall's coal wall and roof in longwall 3D scanning model. Therefore, it did not only need to capture point-cloud from longwall, but also to detect interaction curve between coal wall and roof from point-cloud and transform it to 3D geological model coordinate system to compare (Li et al., 2020).

Scanning device
In order to capture point-cloud from longwall, ExScan, which is made by CSIRO (Commonwealth Scientific and Industrial Research Organization) of Australia, was used in Yujialiang coal mine (Dunn et al., 2020). It is a full-field single-layer LIDAR, as shown in Fig.2. At present, it is one of few explosion-proof 3D laser scanners which could be used in underground directly (Ralston et al., 2014). ExScan's main parameters are shown in Table 2: There are two operation modes for ExScan:  ExScan mode -It is equivalent to a full-field single-layer LIDAR mounted at location. It could spin 360 degrees in explosion proof enclosure and capture panoramic point-cloud data. Fig.3 shows scanning result of ExScan mode from which unit was mounted on support. Patrol platform used by Yujialiang coal mine was a longwall patrol robot system which was independently researched and developed by Beijing Tiandi-Marco Electro-Hydraulic Control System Company (Li et al., 2020). This system consists of a guide-rail and a patrol robot running on. Guide-rail lays on scraper conveyer's cable trough. Patrol robot which carries trawl mode ExScan and LASC Lite, is able to complete patrolling and gain longwall 3D point-cloud within 10min. And maximum speed of patrol is 60m/s. Patrol robot is shown in Fig.5. And scanning result of trawl mode ExScan is shown in Fig.6.

Scanning solution on site
As mentioned above, it needed to arrange patrol robot and ExScans in Yujialiang coal mine to capture longwall's point-cloud, in order to detect intersecting curve between coal wall and roof from the point-cloud and to transform it to 3D geological model's coordinate system .
In order to achieve above objectives, it needed to arrange related devices in #43101 longwall of Yujiali-ang coal mine as Fig.7.

Fig.7 Devices arrangement in gateways and longwall
In longwall , patrol robot, including LASC Lite and ExScan, and guide-rail were installed to scan longwall and captured 3D point-cloud. Because of space blocking from reversed loader in main-gate and reversed loader in tail-gate, patrol robot could only scan from the 6th -200th support.
Once 3D point-cloud of longwall was captured, intersecting curve between coal wall and roof could be detected by curvature feature analysis. The detail is shown in Section 2.3.1.
At this stage, point-cloud from patrol robot(hereafter called patrol point-cloud) and detected intersecting curve were located in their local space coordinate system. It needed a simple and automatic method to import global coordinate information and transform the point-cloud and curve from their local coordinate system to 3D geological model's global space coordinate system . The method used in Yujialiang coal mine was to hang positioning sphere targets, whose global coordinate had been measured, in safe and convenient gateways. And then, transition matrix from local coordinate to 3D geological model's global coordinate could be calculated by mapping between targets' global positions and local positions in point-cloud, which also needed recognize targets in point-cloud. At last coordinate transformation was done based on this matrix. Sphere target recognition and position in point-cloud is shown in Section 2.3.2.
As mentioned above, due to limit of patrol range, patrol robot couldn't scan position sphere targets hung at main gate and tail gate. Therefore, it needed to extend scanning range. The solution in Yujialiang coal mine was to install 2 extra ExScans on 1th and 200th support. These 2 ExScans could not only cover positioning sphere targets in two gateways, but also had overlapping areas with patrol point-cloud. These 2 point-clouds from the extra ExScan are referred as main-gate point-cloud and tail-gate point-cloud. Based on mapping of sphere targets (Splicing sphere targets without measured global coordinate were used in Yujialiang coal mine) in the overlapping area from different point-clouds, main-gate point-cloud, tail-gate point-cloud and patrol point-cloud could be spliced together into an union point-cloud, including positioning sphere targets. At last, with the mentioned method, it was able to transform union point-cloud to 3D geological model's global space, including intersecting curve. The mentioned method is to calculate transition matrix from local coordinate to global coordinate by recognition and location location sphere targets. Point-cloud splicing is shown in Section 2.3.3.
Flow of 3D laser scanning and processing is shown is Fig.8.  Fig.9 shows the geometric relation of these variables. This method employs chord, neighbor normal vector and osculating circle, so we call this method as Chord And Normal vectors (CAN) method (Zhang et al., 2009). The advantage of this approach is that the normal vectors of neighboring points are used to estimate the main curvatures of a point. And then corresponding principal curvatures 1 k and 2 k can be deduced by the Euler formula and least square fit .Therefore the mean curvature H is given by (Santiesteban et al., 2014): Besides, in order to improve precision of detecting intersecting curve from point-cloud, normal vector angle between point and neighbor point is imported and analyzed. Because the normal vector angle between point and neighbor point is nearly 2 . Therefore, the noise could be removed by setting an optimum threshold.
Suppose the p point in longwall and let

Sphere target recognition and location (1)Selection and segmentation of suspected sphere targets in point-cloud
Sphere target is made by high strength PVC(Poly Vinyl Chloride) materials and is fully coated with special coating. Scanner would receive high intensity reflected signal from sphere target. Therefore, these points which do not belong to sphere targets could be filtered out from main-gate point-cloud, patrol point-cloud and tail-gate point-cloud, by setting an optimum threshold of intensity.
Principal curvatures 1 k and 2 k of each point in filtered point-cloud are calculated. If all points principal curvature in a certain surface S are 12 0 kk , the surface can be seen as a sphere surface. However due to some disturbing factors, such as measurement error of longwall's point-cloud, fitting error in detection of sphere target, principal curvature of reflected point from sphere target are not necessarily 12 0 kk actually. Therefore, suppose if difference between 1 k and 2 k is no more than a certain value , the point can be thought to be part of a sphere target.
Every suspected sphere target would be segmented by clustering algorithm, taking into account factors such as the center of suspected sphere target and average radius of curvature (Li et al., 2013).
(2)Location sphere centre In every space of suspected sphere target point-cloud, the sphere space equation is supposed as: Therefore, the question about locating sphere center is transformed to calculated the A , B , C , D . The objective function of the least square method is: Equation (9) can be written in the following matrix form: A B C D is the parameters F to be estimated . There are errors in the observation vector and coefficient matrix. Therefore, EIV(errors-in-variables) model is built as equation (11) shown: e is random error from observation vector.
E is random error from coefficient matrix. Therefore, the parameters F to be estimated are calculated under iteration by weighted least squares estimators (Mckeague, 1987;Guan et al., 2008). That is to say the center sphere targets are calculated.

Point cloud splicing
Main-gate, patrol and tail-gate point-clouds have their own coordinate systems. The essence of point-cloud splicing is to unify the coordinate systems of these point-clouds. Method used in Yujialiang coal mine was to transform main-gate and tail-gate point-cloud into patrol point-cloud's coordinate system.
Let's consider coordinate system of main-gate and patrol point-cloud as source coordinate system S C and target coordinate system T C . These splicing spheres are located at overlapping area between main-gate's and patrol's point-cloud. These sphere targets have two different kinds of local coordinate in main-gate and patrol point-cloud. Lets suppose these coordinates as ,,  H (Sun, 2006). Finally, all of points in main-gate point-cloud can be transformed to patrol point-cloud coordinate system (Thrun, 2002). Principle of point-cloud coordinate transformation is shown in Fig.10:   Fig.10 is transformed main-gate point-cloud's coordinate system which is in patrol point-cloud coordinate.
In a similar way, tail-gate point-cloud in its local coordinate system could be transformed to patrol point-cloud coordinate system by the splicing sphere targets between patrol and tail-gate point-cloud. In this way, main-gate, patrol and tail-gate point-clouds could be spliced into an union point-cloud which is located in patrol point-cloud's local coordinate system.

Experimental result
In order to analyze possibility and precision of 3D laser scanning application in Yujialiang coal mine, the underground test was carried out in #43101 longwall in July 2019 and acquired a good result. Fig.11 shows result of sphere targets recognition and location. Red curve in Fig.12 shows result of intersecting curve detection between coal wall and roof . These result tells that these algorithms, including sphere target recognition and location, detection of intersecting curve, are effective. The feasibility and precision of application 3D laser scanning in Yujialiang coal mine is known by comparing height information between ground-truth from geological surveyor and experiment-result from 3D laser scanning. As shown in Fig.13, horizontal axis is 34 sampling points from main-gate to tail-gate, and vertical axis is processed height information. Blue, red and green curves are ground-truth, experiment-result and difference respectively. Figure 13 shows that experiment-result of 3D laser scanning is near to ground-truth of geological surveyor, and the difference of them is less than 10 centimeters.
Result proves that 3D laser scanning technology could meet automatic digital mining of transparent longwall in Yujialiang coal mine requirements, and acquired rather good application effect actually.

Conclusion
With the rapid development of science and technology, the demand from country and people for finding a safe, efficient, green coal mining pattern is increasing. The development of coal mining in China is stepping into intelligent, unmanned mining stage from mechanized automatic mining stage. During intelligent mining exploration, research hotspot is changed from visual remote intervention to self-adaptive automation mining.
In 2019, a new self-adaptive digital mining technology based on the transparent longwall theory was proposed and implemented in Yujialiang coal mine. By comparing high-precision 3D geological model and 3D laser scanning model of longwall, it calculates shearer cutting template and sends the template to shear. Then shearer works by the template which is able to fit changing geological condition. 3D laser scanning technology played an important role in digitizing mined longwall space, and is one of important sensing technologies involved by self-adaptive automation mining. The practical application in Yujialiang coal mine proved that, by process of laser scanning data, including detection of intersecting curve between the coal wall and roof, point-cloud splicing and coordinate transformation, it is able to generate useful information for mining.
The experiment and practice in Yujialiang coal mine have shown that 3D laser scanning technology is beneficial to intelligent mining and is able to result good effect. It was a breakthroughing practice of 3D laser scanning application in Yujialiang coal mine. However, we have to realize that the road to truly self-adaptive automatic mining is still long. Lacking real time, reliable, high-precision sensing sensor is the main factor to limit its development. Although 3D laser scanning technology, even optical measurement, have showed their potential, there are still many problems to solve. The typical problems includes the performances of sensors and environmental factors, such as dust and light. Therefore, in order to apply optical measurement in coal mining to achieve reliable and precise sensing, which is the basic of intelligent automatic mining, full cooperation of scientific research and industry is still needed, also with R&D's(Research and Development) and engineering personnel 's efforts.