Image recognition of mine water inrush based on bilinear convolutional neural network with few-shot learning

doi:10.21203/rs.3.rs-2117040/v1

Download PDF

Research Article

Image recognition of mine water inrush based on bilinear convolutional neural network with few-shot learning

https://doi.org/10.21203/rs.3.rs-2117040/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

With the increasingly widespread application of deep-learning technology in the field of coal mines, the image recognition of mine water inrush has become a hot research topic. Underground environments are complex, and images have high noise and low brightness. Additionally mine water inrush is accidental, and few actual image samples are available. Therefore, this paper proposes an algorithm that recognizes mine water inrush images based on few-shot deep learning. According to the characteristics of images with coal wall water seepage, A bilinear neural network was used to extract the image features and enhance the network's fine-grained image recognition. First, features was extracted using a bilinear convolutional neural network. Second, the network was pre-trained based on cosine similarity. Finally, the network was fine-tuned on the predicted image. For single-line feature extraction, the method is compared with big data and few-shot learning. According to the experimental results, the recognition rate reaches 95.2% for few-shot learning based on bilinear neural network, thus demonstrating its effectiveness.

mine water inrush

few-shot learning

bilinear-CNN

image recognition

Mine water inrush is one of the main disasters in coal mines. In recent years, the major accidents in coal mines were all water disasters, causing heavy casualties and property losses. As computer technology has developed, deep learning(Jayapriya and Jacob et al., 2020; Byun and Kim et al., 2021) technology has gradually been applied to mine engineering (Sun and Nieto et al., 2010; Yang and Yue et al., 2018; H. and K., 2020) and water resource engineering(Hrnjica and Bonacci, 2019; Wang and Lin et al., 2019; Wu and Ma et al., 2021). Existing methods have mostly used traditional image recognition, which recognizes water inrush in only a particular scene, but the mine environment is harsh and the image scenes are diverse, so traditional image recognition has difficulty accurately recognizing complex scenes. Compared with traditional image recognition, image recognition algorithms based on deep learning have strong generalization ability and robustness, and they can adapt to image recognition in multiple scenes. However, these applications are based on deep learning with a large number of samples. Small sample sizes result in overfitting problems and low recognition rates. Most importantly, there is little probability of water inrush occurring in mines, so obtaining real samples of water inrush is difficult. To solve this problem, A deep-learning method was used to identify a limited number of water inrush samples using few-shot learning(Liu and Qiu et al., 2022). Few-shot learning refers to classifying new images with a small amount of labeled training data, comparing the predicted images with a small number of images already in the category, and comparing the similarity between them to determine the category of the image.

Among the methods currently in use, Liu et al. (Liu and Li et al., 2021) proposed a coal and coal gangue detection method based on YOLO v4, and Wang (Wang and Wang, 2014) proposed a new method of obtaining coal texture features using a co-occurrence matrix, which was input into the neural network as a feature vector. Zhang et al. (Zhang and Zhang, 2018) summarized the image recognition of coal and rock and pointed out the existing problems. Jing et al. (Li and Yong et al., 2019) proposed a new mine water source discrimination method based on a generative adversarial network using the fluorescence spectrum of water samples. Alfarzaeai et al. (Alfarzaeai and Niu et al., 2020) used a convolutional neural network (CNN) and thermal imaging to identify coal gangue. Lei et al. (Si and Xiong et al., 2020) proposed a coal rock–recognition algorithm based on a deep CNN, which solved the overfitting problem of the CNN and had a good recognition effect on coal rock images. These studies achieved good results in their respective fields, but few scholars have conducted in-depth research on the image recognition of mine water inrush. The main reason is that mine water inrush is extremely contingent, so line storage has very few samples, and large-scale learning is difficult to conduct. Therefore, image recognition based on small samples can well solve the problem of fewer samples.

The original method of few-shot learning uses generative models and data augmentation (A. and A., 2015). This method has also achieved certain results, but overfitting problems, among others, remain. Subsequently, meta-learning (Finn and Abbeel et al., 2017) has proposed a new idea for few-shot learning. Different from the traditional learning model, meta-learning guides learning based on the comparison method. For example, the Siamese neural network proposed by Koch et al. (Koch and Zemel et al., 2015) is a type of meta-learning. Providing an application case, Li et al. (Li and Eigen et al., 2019) proposed a feature-learning module based on the metric-learning algorithm, which specifically extracted "intra-class commonality" and "inter-class uniqueness". The most commonly used matching method in meta-learning is metric learning (Atkeson and Moore et al., 1997)(Atkeson and Moore et al., 1997), which classifies the extracted feature vectors using the distance measurement method. In the metric method, the cosine distance (Vinyals and Blundell et al., 2016) is often used to calculate the similarity of two feature vectors. Because underground conditions are dim, the feature extraction network has difficulty extracting the features of water seepage in the coal wall, and it cannot judge whether it has water inrush. Therefore, in image recognition, focus on the fine-grained classification of images.

Lin et al.(Lin and RoyChowdhury et al., 2015) proposed a bilinear convolutional neural network(Bilinear-CNN), which uses two feature extraction networks and intersects the feature maps in pairs to ensure that the image features are not lost. Considering the small number of mine water inrush samples and the problem of fine-grained image classification, A few-shot learning method was used based on metric classification to classify images and a bilinear neural network to extract feature maps to enhance the recognition rate of the fine-grained images.

This study used the training method of a matching network to obtain the feature vector of the image. then fine-tuned the support and target sets. And explain the fine-tuning effect through experimental comparison. Because all sudden mine water is water flow or water droplets attached to the coal wall, and the coal wall is dark in color and has an uneven surface, recognizing when water is seeping from the surface of the coal wall is difficult. For feature extraction, previous few-shot recognition algorithms have used monolinear neural networks, which are prone to losing many image details. Therefore, this problem was solved by classifying images with such features as fine grained (Gao and Beijbom et al., 2016; Cui and Zhou et al., 2017; Kong and Fowlkes, 2017; Sun and Wang et al., 2018; T. and A. et al., 2018; Wei and Zhang et al., 2018). Furthermore, bilinear neural networks was used for feature extraction during pre-training to enhance the feature channels of the images and recognize the hanging sweat feature of burst mine water. Finally, compared the feature extraction effects of bilinear convolutional neural networks based on resnet8, resnet50 and VGg16 respectively.

2.1 Network structure

The few-shot task divides the dataset into multiple meta-tasks in the training phase to learn the generalization ability of category changes. In the testing phase, a small number of samples in a new category were directly classified. In the training phase, the C categories was selected in the source dataset and a batch of data in each category for similarity pre-training. The target set is the dataset that needs to be identified. The target set is divided into C categories, and K data in each category are selected as the input of the model's support set. Then, a picture is randomly selected from the remaining target data as the prediction object of the model, called a query. This problem type is defined as the C way K shot.

The main process of identifying mine water inrush images in this paper is as follows. First, input the source dataset into the bilinear neural network to obtain the corresponding feature vectors. Divide the source data set into an anchor, a positive, and a negative image. Compare the cosine similarity of the positive and negative images with the anchor image. Use the mean squared error (MSE) loss to calculate the loss, and obtain the similarity weight of the image through backpropagation (Fig. 1). Then, on the target set, use the K shots and a query image as the model input, and load the pre-training weights to obtain the K shot and query feature vectors. Then, fine-tune the model by comparing the cosine similarity of the shot and query feature vectors (Fig. 2). Use cross-entropy to calculate the loss and backpropagation to update the K shot feature vectors parameters. Finally, select an image from the remaining target set samples as the predicted image, and generate the feature vector by loading and training the weighted bilinear neural network. Calculate the cosine similarity between the vector and the K shot feature vectors and take the highest score as the predicted category.

2.2 Bilinear feature extraction network

Most of the current few-shot feature extraction processes use simple neural networks, such as Conv-4 networks with four convolution kernels, VGG networks or ResNet networks. Considering that these simple CNN structures lead to the loss of some features, the hanging sweat feature cannot be detected, so in this few-shot learning process, A bilinear neural network was used to extract image features. The Bilinear-CNN consists of two identical CNNS, and Eq. 1 indicates that the Bilinear-CNN consists of a four-element tuple function:

${\text{b}}({f_A},{f_B},I,l)=f_{A}^{T}(l,I){f_B}(l,I)$ $\in {R^{M \times N}}$ (1)

where ${f_A},{f_B}$ is a feature extraction function based on a CNN, I is a iamge, and l is the pixel position of the image. Then add up all of the positions and convert to vector:

$\xi (I)=\sum\limits_{l}^{{}} {b({f_A},{f_B},I,l)}$ $\in {R^{M \times N}}$ (2)

$x=vec(\xi (I))$ $\in {R^{MN \times 1}}$ (3)

Then perform an x moment normalization and an L2 normalization on x to obtain bilinear eigenvector z₁:

$y=sign(x)\sqrt {\left| x \right|}$ $\in {R^{MN \times 1}}$ (4)

${z_1}=y/{\left\| y \right\|_2}$ $\in {R^{MN \times 1}}$ (5)

The bilinear neural network uses the matrix outer product of the last convolutional layer of the two neural networks to fuse the two sets of feature networks, which permits greater retention of image features, thereby increasing the fine-grained recognition of images. Therefore, seepage features can be well identified in mine water inrush situations.

Two ResNet18 networks were used on the two branches in the bilinear neural network, and each branch output 512 feature maps. According to Formula 1, the final output of the model was 512*2 feature maps. After running Eqs. 2–5, 512*2 parameters were obtained, which resulted in too many final output parameters. These are difficult to converge when using the MSE loss to calculate the loss function. Therefore, in this study, a fully connected layer (Eq. 6) was added after the output layer of the neural network, which not only ensured that the feature map would not be lost but also reduced the parameters so the model would converge during training.

2.3 Cosine similarity based on the bilinear neural network

The feature vector of each class is denoted as ${w_j}(j=1,2, \cdots ,N)$, where N is the number of categories of samples. The query image was defined to be compared as ${{\text{x}}_{ij}}(i=1,2, \cdots ,P)$, where is the number of test images of this category. According to the formula of cosine similarity, we define:

$$\begin{gathered} \cos ({{\text{x}}_{ij}},{w_j})=\frac{{{w_j}{x_{ij}}^{T}}}{{{{\left\| {{w_j}} \right\|}_2}{{\left\| {{x_{ij}}} \right\|}_2}}} \hfill \\ \hfill \\ \end{gathered}$$

2.4 Loss function calculation

Few-shot learning examines the similarity between two images and calculates the similarity loss function to update the network. Therefore, in this study, the similarity of the two images was obtained by calculating the cosine distance of the feature vectors of the two images. To make the loss function reflect the distance error, the MSE was used loss to calculate the loss function, as follows:

$$mse=\frac{1}{n}\sum\limits_{{i=1}}^{n} {({y_i} - {t_{ij}})}$$

where represents the total number of positive and negative samples, ${y_i}$ is the cosine distance value, and ${t_{ij}}$ is the target corresponding to the cosine value. Therefore, when the gradient is updated, the weight is updated in the direction of the positive sample, and the result is the similarity weight of the image.

According to a small number of water inrush samples and a certain number of non-water inrush samples collected, the datasets were divided into three categories: mine water inrush, water free, and industrial water (other water datasets). The sample was data augmented. Then, the data plan was divided into a training and a validation in an 8:2 ratio, performed similarity training on the training set, and performed model fine-tuning and data prediction on the validation set. The recognition rates were compared for the large- and few-shot learning methods, as were the recognition rates of the bidirectional and single-line neural networks. Finally, the importance of the fully connected layer in the bilinear neural network was obtained through experiments.

3.1 Dataset analysis

The analysis of this study is divided into three parts. First, the role of the fully connected layer in the bilinear neural network is determined through the loss function curve. Secondly, based on the three convolutional neural networks ResNet-18, ResNet-50 and Vgg16, the accuracy of small-sample learning and large-sample learning is compared to determine the accuracy of the model. Finally, the monolinear and bilinear feature extraction effects of the three convolutional neural networks are compared through the CAM(Hacıefendioğlu and Demir et al., 2021) diagram, and the results are explained.

3.1.1 Dataset division

During training and prediction, both the source and target datasets was divided into three categories: water inrush images, no water images and images with water but not water inrush (water spraying along the working face, water in the roadway, etc.). This method was used because water accumulates in the wells and water sprays along the working face in situations other than water inrush (Fig. 5), and such images are easily confused with situations of mine water inrush. Therefore, to improve the model’s ability to generalize and to verify the category recognition, these images was classified into categories.

In the few-shot recognition, when the target and source datasets were different data types, the final prediction results of the model vary greatly. Therefore, our source and target datasets used mine water inrush images to improve the model's performance.

3.1.2 Data augmentation

This study collected 30 videos of water inrush from coal mines, 76 videos of no water in mines, and a small amount of water accumulation in mines and videos of industrial water used in industrial production. The video files were parsed into video frame images, three key frames were extracted from each video, and each image frame was an RGB color image. The mine water inrush samples and other water samples were relatively few samples without water inrush. This situation causes the model to be biased toward the case of no water inrush during the training process, resulting in the misjudgment of the model and reducing its generalization ability.

To overcome these effects, the image was rotated and increased the image noise. Doing so expands the water inrush and other water datasets to improve the distinguishing ability of the model. The number of datasets ultimately constructed is shown in Table 1.

Table 1

Dataset distribution
Dataset	Before expansion	After expansion
No water	73	226
Mine water inrush	160	225
Other water	39	222

3.2 Evaluation results

In our experimental setup, choosed the commonly used feature extraction network model, and use Vgg16, ResNet-50 and ResNet-18 to perform feature extraction on images. Each model performs 4000 iterations, respectively, and fits the change curve of the loss value during the training process of each model. The results are shown in Fig. 6. For each model, the loss function curve was compare for bilinear and unilinear extraction methods. Since the bilinear neural network intersects the feature maps of the two branches in feature extraction, the number of parameters is larger than that of the single linear neural network. The loss of the single-linear model in stage I drops faster and the initial value of the loss function is smaller. The single-linear model in stage II reaches convergence very quickly. Since ResNet-50 has 49 convolutional layers, which is much larger than that of Vgg16 and ResNet-18, its loss function converges slowly.

Then compared on the validation set for different training methods. Table 2 summarizes its performance. It is divided into three groups. The first group compares bilinear (B) and monoilinear (M) learning results for each network based on the big data training method (BL). The second and third groups are based on unilinear few-shot learning (MF) and bilinear few-shot learning (BF) methods to compare the accuracy of the network, respectively. At the same time, the effect of finetuning on the recognition rate is compared.

As shown in Table 2, under the premise that the number of datasets is small, the performance of the model based on small sample learning is better than that of ordinary large sample learning. Under each training method, compare the performance of bilinear and monoilinear feature extraction, respectively. The bilinear was on average 3% higher than monolinear when using the same network model. When making fine-tuning comparison, fine-tuning has a great impact on the prediction results of the network. The data set is divided into three categories, and fine-tuning is performed using 3-way and 3-shot. It can be seen that the performance of the model is significantly improved after finetuning.

As shown in the table, among the three models of Vgg16, ResNet-18 and ResNet-50, the recognition rates of Vgg16 and ResNet-18 are very close, with little difference in performance. And ResNet-50 has the highest recognition rate, reaching 88.1%, 92.9% and 95.2% among the three training methods, respectively. According to the results in the table, the performance of the few-shot training method based on the bilinear ResNet-50 model is better than other methods.

Table 2

Accuracy of different training methods
Methods	Finetuning	Accuracy
BL + M-Resnet-18	None	81.8%
BL + M-Vgg16	None	84.8%
BL + M-Resnet-50	None	84.8%
BL + B-Resnet-18	None	87.9%
BL + B-Vgg16	None	87.9%
BL + B-Resnet-50	None	88.1%
MF + Resnet-18	No	70%
MF + Vgg16	No	70%
MF + Resnet-50	No	72%
MF + Resnet-18	Yes	90.5%
MF + Vgg16	Yes	90.9%
MF + Resnet-50	Yes	92.9%
BF + Resnet-18	No	71%
BF + Vgg16	No	73%
BF + Resnet-50	No	75%
BF + Resnet-18	Yes	90.5%
BF + Vgg16	Yes	92.2%
BF + Resnet-50	Yes	95.2%

3.3 Influence of the fully connected layer on the prediction results

Feature maps need to be intersected pairwise in bilinear neural networks, so the final output layer has ${C}^{2}$ feature maps, which leads to too many final parameters. Used the MSE loss function during training, and the loss function was difficult to converge when calculating the output of the bilinear neural network. To solve this problem, added a fully connected layer after the output layer of the bilinear neural network.

$z=fc({z_1})$ $\in {R^{C \times 1}}$ (8)

Doing so can not only ensure the intersection of the feature maps but also reduce the number of parameters when calculating the loss, which makes it easier to converge (Figs. 7).

3.4 Comparison of the two feature extraction methods for different CNN models

Before the mine water inrush, very few water droplets seep out. This situation is a precursor to the water inrush. This situation is small in grayscale, and the image detail is small. Such small features are easily lost when the image features are extracted. Bilinear neural networks can largely avoid this from happening. In order to better compare the feature extractor results of bilinear and unilinear neural networks, the CAM plots are used to visualize the extraction results.

As shown in Figs. 8, 9, and 10, compared the differences between bilinear and monolinear on the basis of the three models. The top image represents the original test image, and (b) and (c) represent the bilinear and unilinear CAM maps, respectively. In general, the darker the color on the activation map, the greater the weight of the model, the more the model pays attention to that position. Judging whether water inrush occurs according to image features. For example, the coal wall water inrush in the fourth test image, this image includes two features, namely water and coal wall. Therefore, it is necessary to judge whether water inrush occurs according to these two characteristics. For another example, the third test picture shows water spraying on the working face. If features only pay attention to the water and the coal wall and ignore the sprinkler, it is easy to judge that it is a mine water inrush situation. Therefore, not only need to let the model pay attention to the water and the coal wall, but also let the model pays attention to the sprinkler.

Figure 8 shows the activation map of ResNet-50. It can be seen that the monolinear network pays more attention to a certain position of the image. As mentioned earlier, it is easy to make misjudgments. The bilinear model not only focuses on water, but also on coal walls and sprinklers. These features will allow the model to determine that this image is water spraying on the face rather than mine water inrush. The ResNet-18 gap in Fig. 9 is not very obvious, but the region of interest of the bilinear network is more precise. In the Vgg16 network in Fig. 10, it can be clearly found that the activation map of the single linear neural network is not obvious, and it is difficult for the model to make judgments. Interestingly, the bilinear activation map shows that the model focuses on a wider area. This all shows that the bilinear neural network is more accurate in the representation of details. And ResNet-50 can more realistically characterize image features, resulting in a higher recognition rate.

For water flow image recognition, many traditional image recognition algorithms(Lin and Chang et al., 2013; Mettes and Tan et al., 2014) are used, and feature descriptors are used to extract image features through preprocessing, which is often difficult to capture high-level semantic information and complex features, resulting in low learning efficiency, and each step of detection is independent and lacks a global optimization scheme to control. When there are few training samples, deep learning algorithms with large datasets will lead to overfitting of the model at test time, resulting in poor generalization ability of the model. Although the convolutional neural network has the ability of representation learning, it can extract high-order features from the input information, and can ensure the translation invariance of the features. However, for images with complex textures, it is difficult to ensure that the features will not be lost, which requires complex preprocessing of the images.

From the collected samples, it can be clearly seen that the mine water inrush images have the characteristics of few samples, complex and dim environment and complex texture. Therefore, the few-shot learning algorithm and bilinear feature extraction algorithm in this study can solve such problems well. Although the bilinear neural network will increase the number of parameters, exporting the model will not affect the calculation speed when making predictions.

The small sample recognition algorithm is classified by the cosine similarity of the features of the two pictures. Its essence is to compare the similarity of the features of the two pictures, instead of actively judging which category it belongs to. Since the bilinear neural network has two feature extractors, the channel directions of the two feature maps are multiplied in pairs to ensure that the features will not be lost, and eventually the model will pay more attention to the seepage area, thereby improving the final recognition rate.

Mine water inrush images are very rare in engineering, and few image samples are available. Additionally, the images have some disturbances, such as low underground light, construction water, surface water reflection, and coal wall seepage. This paper proposes a few-shot recognition algorithm based on a bilinear neural network, which uses the high, fine-grained recognition characteristics of bilinear neural networks to recognize water inrush images in complex environments. The following main conclusions are drawn.

(1) The few-shot recognition algorithm compares images and can effectively solve the problem of insufficient training samples. The recognition rates were compared of large- and few-shot image recognition algorithms, and the efficiency of few-shot image recognition was verified.

(2) Using bilinear neural network feature map superposition, we extracted the image features. The heat map results indicate that the bilinear neural network can extract the features of water seepage under the well with good accuracy and improve the feature vector. Because of the MSE loss function used in this study, the fully connected layer affected the final convergence result.

(3) This study provides a new idea and method for identifying water inrush in mines. The method is based on deep learning, which can effectively identify water inrush underground and improve mine production intelligence. However, this study compares only the sudden and non-water inrush situations, and further research on graph water characteristics is needed.

Competing Interests: The authors declare that they have no competing interests

Funding: This work was financially supported by China National Natural Science Foundation (41877186、42027801), the Inner Mongolia Science and Technology Major Project (2020ZD0021)， Fundamental Research Funds for the Central Universities (2010YD02) and State Key Laboratory of Coal Resources and Safe Mining. Lastly, the authors would like to thank the editor and the reviewers for their constructive suggestions.

Author Contributions: All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Shuai Zhang, Qiang Wu, Hua Xu ,Yingwang Zhao and Yuanze Du. The first draft of the manuscript was written by Shuai Zhang and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Compliance with Ethical Standards:

The authors have no potential conflicts of interest.

Research do not involve Human Participants and/or Animals.

All authors have given informed consent.

All authors consent to participate.

All authors consent to publish.

The data and materials are available.

A., W. and Y. A.(2015). One Shot Learning via Compositions of Meaningful Patches. 2015 IEEE International Conference on Computer Vision (ICCV).
Alfarzaeai, M. S. and Q. Niu, et al. (2020). "Coal/Gangue Recognition Using Convolutional Neural Networks and Thermal Images." IEEE Access 8: 76780-76789.
Atkeson, C. G. and A. W. Moore, et al. (1997). "Locally Weighted Learning for Control." Artificial Intelligence Review 11(1-5): 75-113.
Byun, H. and J. Kim, et al. (2021). "A deep convolutional neural network for rock fracture image segmentation." Earth science informatics 14(4): 1937-1951.
Cui, Y. and F. Zhou, et al. (2017). Kernel pooling for convolutional neural networks. Proceedings of the IEEE conference on computer vision and pattern recognition.
Finn, C. and P. Abbeel, et al. (2017). "Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks.".
Gao, Y. and O. Beijbom, et al. (2016). Compact bilinear pooling. Proceedings of the IEEE conference on computer vision and pattern recognition.
H., S. and S. K.(2020). A Depthwise Separable Convolution Architecture for CNN Accelerator. 2020 IEEE Applied Signal Processing Conference (ASPCON).
Hacıefendioğlu, K. and G. Demir, et al. (2021). "Landslide detection using visualization techniques for deep convolutional neural network models." Natural Hazards 109(1): 329-350.
Hrnjica, B. and O. Bonacci(2019). "Lake Level Prediction using Feed Forward and Recurrent Neural Networks." Water resources management 33(7): 2471-2484.
Jayapriya, K. and I. J. Jacob, et al. (2020). "Hyperspectral image classification using multi-task feature leverage with multi-variant deep learning." Earth science informatics 13(4): 1093-1102.
Koch, G. and R. Zemel, et al. (2015). Siamese Neural Networks for One-shot Image Recognition. ICML.
Kong, S. and C. Fowlkes(2017). Low-rank bilinear pooling for fine-grained classification. Proceedings of the IEEE conference on computer vision and pattern recognition.
Li, H. and D. Eigen, et al. (2019). "Finding Task-Relevant Features for Few-Shot Learning by Category Traversal.".
Li, J. and Y. Yong, et al. (2019). "Generative adversarial nets in laserinduced fluorescence spectrum image__recognition of mine water inrush.".
Lin, F. and W. Chang, et al. (2013). Applications of image recognition for real-time water level and surface velocity. 2013 IEEE International Symposium on Multimedia, IEEE.
Lin, T. and A. RoyChowdhury, et al. (2015). Bilinear CNN Models for Fine-Grained Visual Recognition. ICCV, IEEE.
Liu, H. and Q. Qiu, et al. (2022). "Few-shot learning for name entity recognition in geological text based on GeoBERT." Earth Science Informatics: 1-13.
Liu, Q. and J. Li, et al. (2021). "Recognition Methods for Coal and Coal Gangue Based on Deep Learning." IEEE Access 9: 77599-77610.
Mettes, P. and R. T. Tan, et al. (2014). On the segmentation and classification of water in videos. 2014 International Conference on Computer Vision Theory and Applications (VISAPP), IEEE.
Si, L. and X. Xiong, et al. (2020). "A Deep Convolutional Neural Network Model for Intelligent Discrimination between Coal and Rocks in Coal Mining Face." Mathematical Problems in Engineering 2020: 1-12.
Sun, E. and A. Nieto, et al. (2010). "An integrated information technology assisted driving system to improve mine trucks-related safety." Safety science 48(10): 1490-1497.
Sun, Q. and Q. Wang, et al. (2018). "Hyperlayer bilinear pooling with application to fine-grained categorization and image retrieval." Neurocomputing 282: 174-183.
T., L. and R. A., et al. (2018). "Bilinear Convolutional Neural Networks for Fine-Grained Visual Recognition." IEEE Transactions on Pattern Analysis and Machine Intelligence 40(6): 1309-1322.
Vinyals, O. and C. Blundell, et al. (2016). "Matching Networks for One Shot Learning." Neural Inform.
Wang, J. and G. Lin, et al. (2019). "Real-Time Water-Level Forecasting Using Dilated Causal Convolutional Neural Networks." Water resources management 33(11): 3759-3780.
Wang, W. and Z. Wang(2014). "Characteristic analysis and recognition of coal and rock based on visual technology." Coal Technol 36(2): 272-274.
Wei, X. and Y. Zhang, et al. (2018). Grassmann pooling as compact homogeneous bilinear pooling for fine-grained visual classification. Proceedings of the European Conference on Computer Vision (ECCV).
Wu, Z. and B. Ma, et al. (2021). "Identification of sensitive parameters of urban flood model based on artificial neural network." Water Resources Management 35(7): 2115-2128.
Yang, Y. and J. H. Yue, et al. (2018). "Mine Water Inrush Sources Online Discrimination Model Using Fluorescence Spectrum and CNN." IEEE ACCESS 6: 47828-47835.
Zhang, S. and M. Zhang(2018). On Identification of Coal and Rock Images, IEEE.

Download PDF

Version 1

posted

You are reading this latest preprint version

Image recognition of mine water inrush based on bilinear convolutional neural network with few-shot learning

Status:

Version 1

Abstract

Figures

1. Introduction

2. Methods

2.1 Network structure

2.2 Bilinear feature extraction network

2.3 Cosine similarity based on the bilinear neural network

2.4 Loss function calculation

3. Experimental Analysis

3.1 Dataset analysis

3.1.1 Dataset division

3.1.2 Data augmentation

3.2 Evaluation results

3.3 Influence of the fully connected layer on the prediction results

3.4 Comparison of the two feature extraction methods for different CNN models

4 Discussion

5 Conclusion

Declarations

References

Status:

Version 1