In order to establish a deep learning algorithm that can classify water seepage images of asphalt pavement and extract and calculate water seepage characteristics, this paper uses two models of MobileNet network and EfficientNet network to learn water seepage characteristics of asphalt pavement. The algorithm design is based on python language and keras development framework.
3.1 Data collection and processing
The model designed in this paper is mainly used to classify and identify the four types of water seepage characteristics of tunnel asphalt pavement: point seepage, line seepage, surface seepage, and no seepage, as shown in Fig. 2. Using the mobile phone, 200 pictures of each of the 4 types of water seepage characteristics of asphalt pavement in different tunnels were collected. The four types of features are labeled as A, B, C, and D respectively, and they are stored in a csv format file. Since deep learning models training require a large amount of data, and considering the time cost of collecting data in real scenes and the influence of various environmental factors, in order to ensure the diversity of data, the collected data is expanded and enhanced.
As shown in Table 1, the data set is expanded by means of flipping up and down, mirroring, changing the brightness, and Gaussian blur. The four types of pavement water seepage characteristic images were increased to 600 respectively, which basically met the requirements of model training. Not only can it effectively prevent the model from overfitting, but it can also improve the model's recognition rate of pavement seepage.
3.3 EfficientNet network model construction
The EfficientNet network model established in this paper is divided into 9 parts, a total of 18 layers of neural networks. The first part is an ordinary convolutional layer with a convolution kernel of 3*3 and a step size of 2, which contains BN and Swish. The second part is the MBConv structures with the 3*3 convolution kernel. Part 3~8 are the MBConv structures that expand the channel of the input feature matrix to 6 times the original, and the convolution kernels are 3*3, 5*5, 3*3, 5*5, 5*5, 3*3 respectively. The 9th part consists of a 1×1 convolutional layer (including BN and Swish), an average pooling layer and a fully connected layer. The network structure is shown in Fig. 3.
As shown in Fig. 4, the MBConv structure is mainly composed of a 1*1 ordinary convolution for dimension upgrade, a deep convolution with a convolution kernel size of 3*3 or 5*5, an SE module, and a 1*1 ordinary convolution for dimensionality reduction and a dropout layer. In the SE module, the number of nodes in the first fully connected layer is 1/4 of the channels input to the MBConv feature matrix, and the Swish activation function is used. The number of nodes in the second fully connected layer is the same as the number of feature matrix channels output by the deep convolutional layer, and the Sigmoid activation function is used.
3.4 Comparison of model results
The test environment is: Windows10 operating system, Intel i7-4720HQ processor, NVIDIA GeForce RTX2060, python3.8, keras2.2.5, tensorflow1.14.0. The 2400 images after image enhancement are used as the training data set, 70% of which are used as the training set, 20% are used as the test set, and 10% are used as the validation set for testing. To ensure that the two models are performed in the same environment, the batch size is set to 32, and the number of training epochs is set to 60.
The training results of the MobileNet model and the EfficientNet model are shown in Fig. 5.
As shown in Fig. 5, the accuracy of the first epoch training set is 83.84%, and the loss value is 0.3929. After 60 epochs of training, the accuracy of the model in the training set is 97%, and the loss value is 3.4×10-4. The accuracy rate of the validation set increased from 70.07–96.77%, an increase of 26.7%, and the loss value was reduced from 0.8786 to 0.0527, a decrease of 0.8259. In the whole training process, the accuracy rate increases with the increase of the training, and the loss decreases with the increase of the training, which meets the training requirements as a whole. However, the fluctuations in the accuracy and loss of the verification set are significantly larger than that of the training set. This is due to the fact that the amount of data in the verification set is less than that of the training set, and does not affect the judgment of the result.
After 60 epochs of training, the accuracy of the EfficientNet model training set has been increased from 90.88–99.85%, and the loss has been reduced from 0.2318 to 1.67×10-5. The accuracy of the validation set has increased from 93.83–97.53%, an increase of 3.7%, and the loss has been reduced from 0.2976 to 0.1023, which is a decrease of 0.1973. Compared with the MobileNet model, the EfficientNet model validation set has a smaller range of accuracy and loss, and the model has a better recognition effect for images.
In order to further compare the accuracy of the two models for the prediction of water seepage feature images, the F1 Score index is introduced for evaluation. F1 Score takes into account the precision and recall of the classification model, and is the harmonic average of the accuracy and recall of the model. The equation is as follows:
\(Pr=\frac{TP}{TP+FP}\) (3)
In the Eq. (3): Pr is the accuracy rate; TP is the true positive prediction; FP is the false positive prediction.
\(Re=\frac{TP}{TP+FN}\) (4)
In the Eq. (4): Re is the recall rate; FN is the false negative prediction.
\(F1 Score=\frac{2·Pr·Re}{Pr+Re}\) (5)
Randomly select 200 images in the data set for prediction. Among them, there are 34 point seepage images, 42 line seepage images, 64 surface seepage images, and 60 no seepage images. Evaluate the results of each type of prediction and use the weighted average to obtain the average F1 Score of the model. The results are shown in Table 2.
It can be seen from the table that the prediction accuracy of the MobileNet network model for point seepage is 95.5%, the prediction accuracy for line seepage is 94.0%, the prediction accuracy for surface seepage is 91.5%, and the prediction accuracy for no seepage is 99.0%. After training, the MobileNet network model has an average prediction accuracy of 95.0% for the four types of water seepage characteristics. The EfficientNet network model has a prediction accuracy of 98.0% for point seepage, 97.5% for line seepage, 97.0% for surface seepage, and 99.5% for no seepage. In the prediction of the four types of water seepage characteristics, the prediction accuracy of the EfficientNet network model is higher than that of the MobileNet network model, and the average prediction accuracy of the EfficientNet network model is 98.0%, which is 3% higher than that of the MobileNet network model.
The average F1 Score of the two models reached more than 90%, and both models can identify pavement water seepage characteristics very well. The EfficientNet network model predicts four types of water seepage characteristics not only in accuracy, but also in precision, recall and F1 Score higher than MobileNet. And the average F1 Score is 5.98% higher than MobileNet. It shows that the EfficientNet network model is more accurate for pavement water seepage recognition and has better performance.