Application of Hybrid CNN-Transformer for Classifying Major Coal Mine Accident Hazards

doi:10.21203/rs.3.rs-4617735/v1

Download PDF

Article

Application of Hybrid CNN-Transformer for Classifying Major Coal Mine Accident Hazards

https://doi.org/10.21203/rs.3.rs-4617735/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Most coal mining enterprises in China have established and use safety production information systems for hazard identification and management, but related accident hazard data have not been fully utilized. This study is based on the classification standards defined by the "Coal Mine Major Accident Hazard Determination Standards" implemented by the Ministry of Emergency Management in 2021. We constructed a classification system including 15 major hazard categories and 79 minor hazard categories, which served as sample labels for major coal mine accident hazards. The Hybrid CNN-Transformer model was used to perform hierarchical text classification on the coal mine major accident hazard data, with the BERT model used as a baseline for comparison. The results show that in the major hazard category classification experiments, the Hybrid CNN-Transformer model outperformed the BERT model by 3 percentage points in terms of accuracy, recall, and F1 score. In the minor hazard category classification experiments, the Hybrid CNN-Transformer model achieved a maximum classification performance of 98%, generally exceeding the BERT model. The coal mine accident hazard classification algorithm based on the Hybrid CNN-Transformer model demonstrates significant classification effectiveness, providing efficient and rapid input support for coal mine major accident hazard identification systems.

Physical sciences/Energy science and technology

Physical sciences/Engineering

Hybrid CNN-Transformer model

natural language processing

coal mine accident hazards

text classification

In recent years, China has continuously strengthened its coal mine safety system, leading to a sustained improvement in coal mine safety production conditions¹. However, coal mine accidents have not been fundamentally curbed and still occur occasionally, causing serious consequences. The safety production situation remains severe. With the nation's increasing attention to safety accidents, the mortality rate per million tons of coal has decreased from 3.08 in 2004 to 0.058 in 2020. However, this rate still lags behind that of developed countries such as the United States (0.028) and Australia (0.020). Ensuring safety production remains a top priority in the daily operations of coal mining enterprises².

The work of identifying major accident hazards in coal mines is a crucial measure for coal mining enterprises to implement the principle of prevention first, effectively preventing and reducing various safety accidents. Since the Ministry of Coal proposed the coal mine quality standardization in 1964, although it has been revised multiple times, hazard identification work was only officially included in the coal mine safety systematic governance framework in 2015. In recent years, with the development of information technology, coal mining enterprises have gradually introduced intelligent hazard identification systems. However, existing systems still suffer from insufficient classification accuracy. Therefore, researching an efficient hazard text classification model is of great significance. In 2016, the Office of the State Council's Work Safety Commission issued the "Opinions on Implementing Guidelines for Preventing Major Accidents and Establishing a Dual Prevention Mechanism," and in 2021, the new "Safety Production Law of the People's Republic of China" was implemented. Hazard identification has always occupied an important position in coal mine safety production³.

Most coal mining enterprises have now established hazard identification systems or risk control and hazard identification information systems, achieving basic management of major coal mine accident hazard identification. The application of these systems has improved the efficiency of hazard identification and management to a certain extent but also has some shortcomings. Regular safety inspection activities in coal mines generate a large amount of hazard text data, which often remains unused in the hazard identification information system after the hazard rectification is completed. By mining these unstructured text data, safety management personnel can not only grasp the distribution patterns of hazards but also guide the management of similar hazards.

In recent years, scholars at home and abroad have utilized deep learning, data mining, natural language processing, knowledge graphs, and other technologies to classify, mine, and analyze hazard text data. By employing ensemble learning, transfer learning, and reinforcement learning methods, the accuracy and effectiveness of hazard text data analysis have been improved. Text classification is a fundamental task in natural language processing (NLP), aiming to assign a piece of text to one or more predefined categories⁴. convolutional neural networks have limitations in extracting semantic features

With the rapid development of natural language processing technology, word vector-based deep learning models have been widely applied in short text classification across various fields, such as GloVe, FastText, ELMo⁵, and Transformer models. Maria Alejandra⁶ et al. proposed an ELMo-based model for text classification, which extracts contextual word vectors through the pre-trained ELMo model and extracts local features, finally making predictions through the classification layer. Experimental results show that this method significantly outperforms traditional word embedding methods in classification accuracy.

However, convolutional neural networks have limitations in extracting semantic features⁷, and many scholars have begun to try multi-model fusion methods for text classification. Jongga Lee⁸ investigated the impact of regularization on text classification models with limited labeled data. They compared a simple word embedding-based model with complex models (CNN and BiLSTM). Adversarial training improved supervised learning, while semi-supervised methods (Pi model, virtual adversarial training) enhanced performance with unlabeled data. Evaluating on four datasets (AG News, DBpedia, Yahoo! Answers, Yelp Polarity), they found that both simple and complex models benefit from regularization, with complex models showing significant improvements.

The Bidirectional Encoder Representations from Transformers (BERT) is a pre-training technique for natural language processing (NLP) proposed by Google in 2018⁹. The initial English BERT release provided two types of pre-trained models: BERTBASE and BERTLARGE¹⁰. The core part of BERT is a Transformer model, with variable numbers of encoding layers and self-attention heads. Hao Wang¹¹ et al. developed an efficient AI-generated text detection model based on the BERT algorithm, processing text with steps such as converting to lowercase, word splitting, and removing stop words. The model was trained and tested on a dataset split 60/40, showing an accuracy increase from 94.78–99.72% and a loss decrease from 0.261 to 0.021. The average training set loss was 0.0565, with a test set loss of 0.0917. The average accuracies were 98.1% for the training set and 97.71% for the test set, indicating good generalization. This BERT-based model demonstrates high accuracy and stability in detecting AI-generated text.

Although BERT performs excellently in text classification tasks, its high computational resource demand, long training time, large memory usage, slow inference speed, risk of overfitting, complex tuning, and poor interpretability need to be seriously considered in practical applications. This paper adopts a Hybrid CNN-Transformer model to classify coal mine accident hazard text data. The model uses CNN to extract local features and Transformer to capture global semantic information, thereby demonstrating outstanding performance in handling complex text classification tasks.

Major coal mine accident hazards are the direct causes of coal mine accidents and generally manifest as unsafe human behaviors, unsafe conditions of objects, adverse environments, and management deficiencies¹². Due to different research objectives, various classification methods for coal mine accident hazards have been summarized by academia and industry. From the perspective of accident causation, You Mengjie¹³ categorized coal mine accident hazards into four major categories and 45 subcategories based on human, machine, environment, and management factors. These categories include safety management (e.g., qualification certificates, organizational structures, mine rescue), personnel (e.g., qualifications, training, operational behavior), workplace (e.g., roof, ventilation, gas, hoisting and transportation), and equipment and facilities (e.g., mining, emergency evacuation, safety monitoring, positioning, blasting, protection). Meng Fanqiang¹⁴ divided coal mine accident hazards into eight categories based on the actual professional department settings of coal mines, including mining, tunneling, electromechanical, transportation, ventilation, geology, monitoring, and others. According to the difficulty of hazard rectification and impact scope, the "Interim Provisions on the Investigation and Management of Safety Production Accident Hazards" classifies accident hazards into general and major accident hazards. To deeply analyze the distribution characteristics of coal mine accident hazards, this paper defines a classification system based on the professional classification standards of the "Coal Mine Major Accident Hazard Determination Standards" implemented by the Ministry of Emergency Management in 2021. First, major coal mine accident hazards are divided into 15 primary hazard categories, including "exceeding capacity, intensity or personnel limits in production organization," "gas overrun," "outburst prevention measures," "gas extraction monitoring system," "ventilation," "water hazards," "cross-boundary mining," "rock burst," "spontaneous combustion," "eliminated equipment," "power supply," "over-scale mining," "illegal contracting," "unlicensed production," and "other major accidents." Then, corresponding secondary hazards under each primary hazard category are determined, totaling 79 secondary hazards. The detailed classification results of coal mine accident hazards are shown in Table 1.

Table 1

Classification System of Coal Mine Major Accident Hazards
Number	Primary Hazard	Secondary Hazards
1	Exceeding Capacity, Intensity, or Personnel Limits in Production Organization	Exceeding 10% of approved capacity, issuing illegal production plans, insufficient mining period, excessive mining faces, non-compliance with gas extraction standards, exceeding personnel limits by 20%
2	Gas Overrun	Illegal gas inspection, continued operation after gas overrun, gas accumulation not discharged
3	Outburst Prevention Measures	No related institutions or personnel, incomplete surface gas extraction system, no outburst prediction, no outburst prevention measures, no outburst verification, no safety measures, trolley wire motor
4	Gas Extraction Monitoring System	No system established or not functioning properly, equipment damage
5	Ventilation	Insufficient airflow in mining faces, ineffective ventilator, series ventilation, design defects in ventilation system, no dedicated return airway in special mining, short-circuiting in ventilation walls and doors, non-continuous intake and return airways, no electrical ventilation lockout, no dual-fan dual-power local ventilation, no full-pressure ventilation
6	Water Hazards	Inadequate exploration, lack of dedicated institutions, personnel, and equipment, non-compliance with water exploration regulations, mining water barrier coal pillars, failure to evacuate personnel upon signs of water inflow, failure to stop production after mine flooding, no permanent drainage, inadequate drainage capacity, failure to eliminate hazards in steeply inclined coal seams
7	Cross-Boundary Mining	Mining beyond stratigraphic elevation, mining beyond control limits, destruction of safety coal pillars
8	Rock Burst	No identification, no anti-burst institutions or personnel, no burst prediction, mining isolated coal pillars, no entry regulations
9	Spontaneous Combustion	No fire prevention plan, no measures for longwall mining, no measures for signs of fire, illegal reopening of sealed fire area
10	Eliminated Equipment	Use of prohibited equipment or processes, non-mining equipment, equipment unsuitable for underground conditions, materials unsuitable for underground conditions, mining face without two safety exits, forward mining in gas mines
11	Power Supply	Single circuit power supply, dual circuit wiring errors, no dual circuit in complex mines
12	Over-Scale Mining	No approval for commencement, coal mining during construction period, generation in expansion area, over-scale and over-capacity mining
13	Illegal Contracting	Separate contracting, no safety agreement, no safety production permit, illegal subcontracting, illegal split contracting
14	Unlicensed Production	Failure to implement safety production responsibilities, failure to implement safety production institutions and personnel, unlicensed production
15	Other Major Accidents	Unclear division of labor, unclear safety production fees, unclear gas classification, failure to identify outburst-prone mines, false personnel drawings, no monitoring system installed, no hoisting protection device, failure to inspect conveyor belt, incomplete equipment in special tunnels

4.1 Data Preparation

This paper utilizes 320,000 records of coal mine hazard identification collected from 2019 to 2022 for classification. These data were recorded by safety management personnel during coal mine inspections, and the hazard data had already been assigned relevant categories, meaning the data samples were already labeled.. Therefore, the main task during the data preparation stage was to organize the sample data according to the above-mentioned hazard classification system, forming a multi-class hazard sample dataset with 15 category labels. The samples were cleaned to remove punctuation marks, modal particles, spaces, and other words that did not contribute to the text features. The cleaned data were used for model training and validation.

4.2 Hybrid CNN-Transformer Model Structure¹⁵

4.2.1 Model Description

After preprocessing, the text needs to be vectorized for input into the classifier. The input text is first tokenized, and Word2Vec is used to convert words into word vectors, generating embedded representations of specific dimensions. These word vectors are input into the CNN layer and Transformer layer for feature extraction and classification. The CNN module is used to extract local features of the text, while the Transformer module captures long-distance dependencies and global semantic information through self-attention and multi-head attention mechanisms, thereby improving classification performance. As Fig. 1

4.2.2 CNN Module

The CNN module is used to extract local features of the text. The convolutional layer applies multiple filters to the input word vectors to perform convolution operations, extracting $n-gram$ features. The calculation process of the convolution operation is as follows:

$$Conv\left(\text{X}\right)=f(\text{W}\bullet \text{X}+b)$$

(1)

where $\text{X}$ is the input word vector, $\text{W}$ is the filter, $b$ is the bias, and $f$ is the activation function (e.g., ReLU). Each filter slides over the input matrix to extract local features. Subsequently, $MaxPooling$ layers are used to down sample the convolution results, retaining the most significant features. The $MaxPooling$ operation is as follows:

$$MaxPooling\left(\text{X}\right)=fmax\left(\text{X}\right)$$

(2)

The max-pooling layer reduces the size of the feature map by selecting the maximum value within the pooling window, thereby retaining important features and reducing computational complexity.

4.2.3Transformer Module

The Transformer module captures long-distance dependencies and global semantic information in the text through self-attention and multi-head attention mechanisms. The self-attention value of the input sequence is calculated as follows:

$$Attention\left(Q,K,V\right)=softmax\left(\frac{\text{Q}\text{K}{K}^{T}}{\sqrt{{d}_{k}}}\right)$$

(3)

where $Q,K,V$ are the query matrix, key matrix, and value matrix, respectively, and ${d}_{k}$ is the dimension of the key. The self-attention mechanism calculates the dot product of the query and key, divides by the square root of the key's dimension, and applies the $Softmax$ function to obtain the weight matrix used to weight the value matrix. The multi-head attention mechanism concatenates the results of multiple self-attention heads and generates the final feature representation through a linear transformation:

$$MultiHead\left(Q,K,V\right)=Concat\left({head}_{1},{head}_{2},\cdots {head}_{h}\right){W}^{o}$$

Each head's calculation process is:

$${head}_{i}= Attention\left(Q{W}_{i}^{Q},K{W}_{i}^{K},V{W}_{i}^{V}\right)$$

where$Q{W}_{i}^{Q},K{W}_{i}^{K},V{W}_{i}^{V}$ are the linear transformation matrices corresponding to each head.

4.2.4 Feature Fusion and Fully Connected Layer

After fusing the global features extracted by the Transformer module with the local features extracted by the CNN module, the fused feature vector is input into the fully connected layer to predict the hazard categories. The fully connected layer outputs the probability distribution of each category through the $Softmax$ function, achieving multi-class text classification. The specific calculation is as follows:

$$y=softmax(\text{W}\bullet \text{X}+b)$$

where $\text{X}$ is the fused feature representation, $\text{W}$ and $b$ are the weights and biases of the fully connected layer. The $Softmax$ function outputs the probability for each category, determining the final classification result.

4.2.5 Model Optimization and Training

During model training, the cross-entropy loss function is used as the optimization objective, and parameter updates are performed using the backpropagation algorithm. The cross-entropy loss function calculates the difference between the model's predictions and the actual labels, as follows:

$$L=-{\sum }_{i}{y}_{i}\text{log}\left({\widehat{y}}_{i}\right)$$

where ${y}_{i}$ is the actual label and ${\widehat{y}}_{i}$ is the predicted probability. To improve the model's generalization ability, dropout is used, randomly dropping some neurons during training to prevent overfitting. The effect of dropout is as follows:

$$Dropout\left(X\right)=\text{X}⨀Bernoulli\left(p\right)$$

where $⨀$ denotes element-wise multiplication, and $Bernoulli\left(p\right)$ is a Bernoulli distribution with probability p of retaining a neuron. Additionally, hyperparameters such as learning rate, batch size, filter size, number of Transformer layers, and number of attention heads were tuned to find the optimal training configuration.

This section may be divided by subheadings. It should provide a concise and precise description of the experimental results, their interpretation, as well as the experimental conclusions that can be drawn.

5.1 Model Parameters and Training Environment

Model training and validation were implemented based on the Pytorch environment. During training, the learning rate was set to 1${e}^{-4}$, the number of iterations was 20, and the Adam optimizer was used for model solving. The word vector training for the Hybrid CNN-Transformer model was implemented using the Jieba tokenization tool. To validate the classification effectiveness of the model, the BERT model was used as a comparison reference, specifically the bert-base-chinese pre-trained model from the Transformers library. Specific parameters are shown in Table 2:

Table 2

Model Hyper parameters
Parameter Name	Value
Word Vector Dimension	300 100
Number of Convolution Filters	300 100
Convolution Filter Sizes	3, 4, 5
Number of Transformer Layers	2
Number of Attention Heads	8
Hidden Layer Dimension	512
Dropout Rate	0.5
Batch Size	64

The dataset was divided into training, validation, and test sets in a 7:2:1 ratio. The training set was used for model training, the validation set was used for tuning hyperparameters and preventing overfitting, and the test set was used to evaluate the final performance of the model. The training environment was a Tesla V100 to accelerate the training process. During model training, the Hybrid CNN-Transformer model quickly converged after the first few iterations, and the training process was stable without significant overfitting. In contrast, although the BERT model performed similarly, it required a longer training time and exhibited greater fluctuations.

5.2 Evaluation Metrics

In text classification research, commonly used evaluation metrics include accuracy, precision, recall, and F1 score¹⁶. These metrics are calculated based on the confusion matrix. Considering the differences in the number of samples for each label, weighted averages are used for precision, recall, and F1 score. To comprehensively evaluate the classification performance of the model across different categories, both macro-averaged and micro-averaged approaches were used to weight the aforementioned metrics¹⁷. Macro-averaging averages the metrics for each category, while micro-averaging takes into account the sample size of each category and weights the metrics accordingly.

5.3 Primary Hazard Classification Results Analysis

During the experiment, the dataset was divided into training, validation, and test sets in a 7:2:1 ratio. The training set contained 190,094 data points, while the validation and test sets contained 54,313 and 27,156 data points, respectively. When training the model on the training set, the Hybrid CNN-Transformer model's training time was 26 minutes faster than the BERT model. The classification results on the test set are shown in Fig. 2. In terms of accuracy, precision, recall, and F1 score, the classification results of the Hybrid CNN-Transformer model exceeded 80%, which was 4 percentage points higher than the BERT model. This indicates that the Hybrid CNN-Transformer model performs better in extracting complex textual features of major coal mine safety hazards. By retaining historical information for a long time and combining the attention mechanism to automatically discover key words for classification tasks, the CNN ensures that important words receive higher weights while irrelevant words receive lower weights. This results in the model's output sequence better representing the semantic information of the hazard texts, thereby improving classification performance¹⁸.

When evaluating the performance of multi-class classification problems, the F1 score most comprehensively reflects classification performance. From the F1 scores shown in Table 3 for primary hazard classification results, it is evident that the F1 scores of the Hybrid CNN-Transformer model for several primary hazards exceed 80%, with some categories even exceeding 90%. In the categories of "outburst prevention measures," "gas extraction monitoring system," "water hazards," "cross-boundary mining," "rock burst," "eliminated equipment," "power supply," "unlicensed production," and "over-scale mining," the F1 scores of both the Hybrid CNN-Transformer model and the BERT model exceed 80%. The categories where the F1 scores of both models simultaneously exceed 70% reach 66.7%, and the categories where the F1 scores of both models simultaneously exceed 65% reach 80%, indicating significant classification performance. It can be seen that these categories have relatively more training samples, and the features learned by the models during training are more comprehensive, enhancing generalization ability.

The comparison results of the two models show that the classification results of the Hybrid CNN-Transformer model are generally better than those of the BERT model. In the aforementioned nine categories, the classification results of the Hybrid CNN-Transformer model are generally superior to the BERT model, with improvements ranging from 1–4%. For example, the "rock burst" and "ventilation" categories improved by 4%, the "over-scale mining," "gas overrun," and "water hazards" categories improved by 3%, and the "power supply," "eliminated equipment," "spontaneous combustion," and "cross-boundary mining" categories improved by 2%. It can be seen that the Hybrid CNN-Transformer model has slightly stronger capabilities in extracting semantic features of major coal mine safety hazards than the BERT model. However, in the categories of "illegal contracting" and "unlicensed production," the BERT model outperformed the Hybrid CNN-Transformer model, which is related to the smaller number of samples in these categories.

Table 3

Results of primary classification of hidden dangers
Number	Primary Hazard	BERT F1 Score	Hybrid CNN-Transformer F1 Score	Comparison	Sample Size
1	Exceeding Capacity, Intensity, or Personnel Limits in Production Organization组织生产	0.72	0.73	+ 0.01	852
2	Gas Overrun	0.73	0.76	+ 0.03	2245
3	Outburst Prevention Measures	0.84	0.86	+ 0.02	2393
4	Gas Extraction Monitoring System	0.81	0.83	+ 0.02	4763
5	Ventilation	0.67	0.71	+ 0.04	2897
6	Water Hazards	0.84	0.87	+ 0.03	471
7	Cross-Boundary Mining	0.85	0.87	+ 0.02	2013
8	Rock Burst	0.82	0.86	+ 0.04	1762
9	Spontaneous Combustion	0.80	0.82	+ 0.02	2341
10	Eliminated Equipment	0.89	0.91	+ 0.02	3875
11	Power Supply	0.83	0.85	+ 0.02	1989
12	Over-Scale Mining	0.84	0.87	+ 0.03	2359
13	Illegal Contracting	0.23	0.15	-0.08	93
14	Unlicensed Production	0.34	0.27	-0.07	118
15	Other Major Accidents	0.68	0.68	0	2691

The loss functions of the two models are shown in Fig. 3. From Fig. 3, it can be seen that:

Convergence Speed: Both curves show a decreasing trend in loss values as epochs increase, indicating that both models are gradually learning data features and converging.

Stability: The loss curve of the Hybrid CNN-Transformer model tends to stabilize in the later stages, while the loss curve of the BERT model fluctuates significantly. This indicates that the Hybrid CNN-Transformer model is more stable during training.

Performance Comparison: The overall loss value of the Hybrid CNN-Transformer model is lower than that of the BERT model, and it converges faster, indicating that the Hybrid CNN-Transformer model performs better in this experiment.

5.4 Secondary Hazard Classification Results Analysis

The secondary hazards under each primary hazard category were divided into training, validation, and test sets in a 7:2:1 ratio. Four primary hazards were randomly selected for secondary hazard classification calculation. The data sample situation is shown in Table 4. The F1 score calculation results of each dataset on the test set are shown in Table 5.

Table 4

Statistical Table of Sample Data for Four Class 1 Hazards
Primary Hazard	Number of Secondary Hazards	Total Samples	Training Set	Validation Set	Test Set
Spontaneous Combustion	4	12711	8897	2542	1271
Outburst Prevention Measures	7	39972	27980	7995	3997
Rock Burst	5	31932	22351	6388	3193
Eliminated Equipment	6	16223	11354	3244	1622

Table 5

Classification Results of Secondary Hazards under Four Primary Hazards
Primary Hazard	Secondary Hazard	BERT F1 Score	Hybrid CNN-Transformer F1 Score	Comparison	Sample Size
Spontaneous Combustion	No Fire Prevention Plan	0.97	0.98	+ 0.01	346
	No Measures for Longwall Mining	0.91	0.94	+ 0.03	135
	No Measures for Signs of Fire	0.81	0.82	+ 0.01	541
	Reopen Sealed Fire Area Illegally	0.00	0.39	+ 0.39	248
Outburst Prevention Measures	No Related Institution or Personnel	0.39	0.42	+ 0.03	547
	Incomplete Surface Gas Extraction System	0.69	0.69	0	1570
	No Outburst Prediction	0.90	0.90	0	983
	No Outburst Measures	0.00	0.00	0	657
	No Outburst Verification	0.00	0.00	0	87
	No Safety Measures	0.00	0.00	0	135
	Trolley Wire Motor	0.00	0.00	0	19
Rock Burst	No Burst Prediction	0.83	0.85	+ 0.02	1437
	No Anti-Burst Institution or Personnel	0.92	0.88	-0.04	337
	No Burst Prediction	0.96	0.96	0	1308
	Isolated Coal Pillar Mining	0.00	0.00	0	4
	No Entry Regulations	0.63	0.62	-0.01	104
Eliminated Equipment	Use of Prohibited Equipment	0.75	0.78	+ 0.03	347
	Non-Mining Equipment	0.94	0.94	0	232
	Equipment Not Suitable for Underground	0.61	0.65	+ 0.04	77
	Materials Not Suitable for Underground	0.95	0.95	0	908
	Mining Face Without Two Safety Exits	0.65	0.68	+ 0.03	41
	Forward Mining in Gas Mines	0.60	0.65	+ 0.05	17

From the calculation results in Table 5, it can be seen that the overall classification performance of the Hybrid CNN-Transformer model is also superior to that of the BERT model. However, the F1 scores vary significantly across different categories. The F1 scores for the labels "No Outburst Prediction," "No Fire Prevention Plan," "No Measures for Longwall Mining," "No Burst Prediction," "Non-Mining Equipment," and "Materials Not Suitable for Underground" all exceed 90%, indicating significant classification performance for both models in these six labels. At the same time, there are some categories with an F1 score of 0. This is because, in the highly specialized field of coal mine safety production, text representation tends to form a closed corpus environment. Coupled with the small sample size, the occurrence of low-frequency words leads to the inability to construct accurate word vectors, resulting in both the Hybrid CNN-Transformer model and the BERT model being unable to learn the semantic information of the text.

As the sample size increases, the F1 score continues to improve, and the classification accuracy also increases accordingly. By enhancing the professional capability of safety management personnel in recording hazard content and organizing more historical data information, the text corpus in the coal mine safety production field can be further optimized, thereby improving the effectiveness of automatic hazard classification.

6.1Model Construction and Effectiveness Verification

This paper constructs a classification system for major coal mine safety hazards based on the "Standards for Determination of Major Accident Hazards in Coal Mines" implemented by the Ministry of Emergency Management in 2021. The system includes 15 primary hazard categories and 79 secondary hazard categories. A deep learning model based on the Hybrid CNN-Transformer mechanism was used to conduct multi-classification research on coal mine safety accident hazards. Experimental results show that the Hybrid CNN-Transformer model outperforms the BERT model in terms of accuracy, precision, recall, and F1 score in both primary and secondary hazard classification tasks. Additionally, the Hybrid CNN-Transformer model demonstrates faster training speed and higher stability, showcasing better classification performance. Future work can further expand the dataset size, optimize the model structure, and apply the model to actual coal mine hazard identification systems to enhance the intelligence level of coal mine safety management.

6.2 Training Efficiency and Model Stability

On the same dataset, the training time of the Hybrid CNN-Transformer model is shorter than that of the BERT model. In comparison, the Hybrid CNN-Transformer model exhibits faster training speed and higher stability and convergence in multiple experiments.

6.3 Analysis of Secondary Hazard Classification

The experimental results of secondary hazard classification show that the F1 scores of the Hybrid CNN-Transformer model exceed 90% in several categories. However, due to the small number of samples in some categories, the classification performance of the model decreases. To further improve classification performance, the sample size can be increased, and the professionalism in recording hazard content can be optimized.

6.4 Future Research Directions

6.4.1Data Expansion and Optimization

Expand the dataset size, especially for categories with fewer samples, to further enhance the model's generalization ability.

6.4.2Model Improvement

Introduce more advanced deep learning models, such as Transformer variants or multi-model fusion methods, to further improve classification performance.

6.4.3 Application Promotion

Apply the improved model to actual coal mine major safety hazard identification systems, verify its effectiveness in real environments, and continuously optimize system functions to enhance the management level of coal mine safety production. Through the above improvements and optimizations, the application prospects of the Hybrid CNN-Transformer model in the classification of major coal mine safety hazards are broad, and it can provide more intelligent and efficient technical support for coal mine safety management.

Author Contribution

Q.T. and Z.L. conceptualized the study. Q.T. developed the methodology and performed validation with L.Y.Y.Z. curated the data. Q.T. wrote the original draft, and Z.L. reviewed and edited the manuscript.Z.L. supervised the project. All authors have read and agreed to the published version of the manuscript.

Data Availability

The datasets generated and/or analysed during the current study are not publicly available due to privacy and security concerns. However, the data are available from the corresponding author on reasonable request. The data were recorded using the coal mine hazard identification and rectification platform developed by the application team.

Wang K. Practice of Cloud Computing in Coal Mine Safety Production. Published online 2020. http://dx.doi.org/10.1088/1757-899x/750/1/012160
Wei L jiang, Hu J kun, Luo X rong, Liang W. Study and analyze the development of China coal mine safety management. Published online 2017. http://dx.doi.org/10.1108/ijesm-08-2015-0002
Long Y, Yang C, Li X, Lu W, Zhang Q, Gao J. Forecasting law enforcement frequency of internet + coal mine safety supervision. Int J Energy Sect Manag. 2023;18(4):789–811. doi:10.1108/IJESM-03-2023-0015
A VK, Aghila G. A Survey of Na\"ive Bayes Machine Learning approach in Text Document Classification. Published online March 9, 2010. http://arxiv.org/abs/1003.1795
Patel A, Sands A, Callison-Burch C, Apidianaki M. Magnitude: A Fast, Efficient Universal Vector Embedding Utility Package. Published online October 26, 2018. http://arxiv.org/abs/1810.11190
Ceron MAC. Using Word Embeddings to Analyze Protests News. Published online March 11, 2022. doi:10.48550/arXiv.2203.05875
Gao Y, Lu J, Li S, Li Y, Du S. Hypergraph-Based Multi-View Action Recognition Using Event Cameras. IEEE Trans Pattern Anal Mach Intell. Published online 2024:1–14. doi:10.1109/TPAMI.2024.3382117
Lee J, Yim J, Park S, Lim C. Comparing effectiveness of regularization methods on text classification: Simple and complex model in data shortage situation. Published online February 27, 2024. doi:10.48550/arXiv.2403.00825
Sonkiya P, Bajpai V, Bansal A. Stock price prediction using BERT and GAN. Published online July 18, 2021. doi:10.48550/arXiv.2107.09055
Kumar S, Pranesh RR. TweetBLM: A Hate Speech Dataset and Analysis of Black Lives Matter-related Microblogs on Twitter. Published online August 27, 2021. doi:10.48550/arXiv.2108.12521
Wang H, Li J, Li Z. AI-Generated Text Detection and Classification Based on BERT Deep Learning Algorithm. Published online May 26, 2024. Accessed May 29, 2024. http://arxiv.org/abs/2405.16422
Shi junwei. Research on Risk Evolution Assessment and Prevention and Control of Coal Mine Rock Burst Accidents. Published online 2023.
Mengjie Y, Shuang L, Dingwei L, Qing X. Study on the Influencing Factors of Miners’ Unsafe Behavior Propagation. Front Psychol. 2019;10. Accessed May 29, 2024.
Fanqiang M, Chunxia L. Safety warning of coal mining face based on big data association rule mining. J Comput Methods Sci Eng. 2022;22(4). Accessed May 29, 2024.
Zheng J, Zheng L. A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification. IEEE Access. 2019;7:106673–106685. doi:10.1109/ACCESS.2019.2932619
Dhulavvagol PM, Totad SG, Pratheek P, Ostwal R, Sudhanshu S, Veerabhadra MY. An Efficient Ensemble Based Model for Data Classification. In: 2022 IEEE 7th International Conference for Convergence in Technology (I2CT).; 2022:1–5. doi:10.1109/I2CT54291.2022.9824722
Liu Z, Yu W, Chen W, Wang S, Wu F. Short Text Feature Selection for Micro-Blog Mining. In: 2010 International Conference on Computational Intelligence and Software Engineering.; 2010:1–4. doi:10.1109/CISE.2010.5677015
Zhu B, Pan W. A Text Classification Model Based on BERT and Attention. In: 2023 4th International Conference on Computers and Artificial Intelligence Technology (CAIT).; 2023:90–95. doi:10.1109/CAIT59945.2023.10469363

No competing interests reported.

Download PDF

Reviewers invited by journal
13 Jul, 2024
Editor assigned by journal
13 Jul, 2024
Editor invited by journal
02 Jul, 2024
Submission checks completed at journal
28 Jun, 2024
First submitted to journal
21 Jun, 2024

You are reading this latest preprint version

Application of Hybrid CNN-Transformer for Classifying Major Coal Mine Accident Hazards

Status:

Version 1

Abstract

Figures

1. Introduction

2. Related Work

3. Definition of Major Coal Mine Accident Hazards

4. Construction of the Classification Model for Major Coal Mine Accident Hazards

4.1 Data Preparation

4.2 Hybrid CNN-Transformer Model Structure¹⁵

4.2.1 Model Description

4.2.2 CNN Module

4.2.3Transformer Module

4.2.4 Feature Fusion and Fully Connected Layer

4.2.5 Model Optimization and Training

5. Experimental Process and Results Analysis

5.1 Model Parameters and Training Environment

5.2 Evaluation Metrics

5.3 Primary Hazard Classification Results Analysis

5.4 Secondary Hazard Classification Results Analysis

6. Conclusion and Future Work

6.1Model Construction and Effectiveness Verification

6.2 Training Efficiency and Model Stability

6.3 Analysis of Secondary Hazard Classification

6.4 Future Research Directions

6.4.1Data Expansion and Optimization

6.4.2Model Improvement

6.4.3 Application Promotion

Declarations

Author Contribution

Data Availability

References

Additional Declarations

Status:

Version 1

Application of Hybrid CNN-Transformer for Classifying Major Coal Mine Accident Hazards

Status:

Version 1

Abstract

Figures

1. Introduction

2. Related Work

3. Definition of Major Coal Mine Accident Hazards

4. Construction of the Classification Model for Major Coal Mine Accident Hazards

4.1 Data Preparation

4.2 Hybrid CNN-Transformer Model Structure15

4.2.1 Model Description

4.2.2 CNN Module

4.2.3Transformer Module

4.2.4 Feature Fusion and Fully Connected Layer

4.2.5 Model Optimization and Training

5. Experimental Process and Results Analysis

5.1 Model Parameters and Training Environment

5.2 Evaluation Metrics

5.3 Primary Hazard Classification Results Analysis

5.4 Secondary Hazard Classification Results Analysis

6. Conclusion and Future Work

6.1Model Construction and Effectiveness Verification

6.2 Training Efficiency and Model Stability

6.3 Analysis of Secondary Hazard Classification

6.4 Future Research Directions

6.4.1Data Expansion and Optimization

6.4.2Model Improvement

6.4.3 Application Promotion

Declarations

Author Contribution

Data Availability

References

Additional Declarations

Status:

Version 1

4.2 Hybrid CNN-Transformer Model Structure¹⁵