Segmentation and recognition of filed sweet pepper based on improved self-attention convolutional neural networks

Automatic and accurate recognition of the parts to be picked is the key to realize the intelligent picking of sweet pepper. However, pepper fruits are always covered by other organs, and small objects such as stems and shoots are difficult to be recognized by machines or cameras under certain extreme conditions. To accurately segment and recognize all kinds of objects in sweet pepper images captured at night, three experiments were performed in this paper, and an enhanced model based on convolutional neural networks was eventually achieved. In experiment I, several semantic segmentation networks were trained on a small data set, and the full-resolution residual network (FRRN) was taken as a primary network. Then, the impact of resolution of input images on the segmentation effect was investigated in experiment II. To strengthen the feature presentation of inconspicuous objects, the position attention module was appended on top of the FRRN in experiment III. This architecture was further trained to provide more precise segmentation results. The experimental result shows that the mean intersection over union is 78.88%, which is at least 1.94% points higher than other models, and the mean pixel accuracy is 97.94% on the test set. The proposed method has higher generalization performance when facing unforeseen picking situations; meanwhile, it is generic and can be applied to other fruits and vegetables.


Introduction
Sweet pepper is one of the most popular vegetables in the world, with rich nutritional value and edible value [1]. According to Internet data, the global annual output of sweet pepper exceeds 10million tons [2]. Such a huge output requires considerable effort and energy from growers, which not only affects the production efficiency of sweet pepper, but also wastes valuable labor resources [3]. In addition, in actual production, sweet pepper is mostly cultivated in the greenhouse. The high temperature and humidity in the greenhouse make it difficult for growers to work continuously in the greenhouse. Therefore, it is urgent to find a technology that can replace people to do this work [4,5]. In recent years, with the development of robot technology and modern precision agriculture, it is of great significance to use picking robots to replace human beings in picking work [6][7][8]. Among them, image recognition technology plays a vital role in robot picking. How to accurately and quickly identify sweet peppers and stems to be picked is a difficult problem to improve mechanical efficiency.
At present, researchers have made numerous researches on image recognition of fruits and vegetables. The main methods are based on the combination of machine learning algorithms and machine vision techniques. Wang et al. [9] used an improved Otsu segmentation algorithm to locate the cotton peach region and sampled RGB values of the pixels in different regions. Thus, the problem of image segmentation was transformed into the problem of pixel classification. As a result, they trained the ELM classification model and realized accurate segmentation of the cotton peach images. In the work of Dhingra et al. [10], the recognition of leaf diseases was studied. An extended segmentation technique based on neutron logic was used to extract the region of interest (ROI) of leaves. Then several features, including texture, color, and histogram, were employed to detect diseased leaves. Nine different classifiers were constructed, and the best classification accuracy of 98.4% was eventually obtained. To recognize and locate cucumber accurately, Bao et al. [11] proposed a multi-template matching method and established a multi-template matching library containing 65 cucumber images. The robot vision system can utilize the matching library to calculate the normalized correlation coefficient matrix of the target image one by one, and judge whether there is a target cucumber in the image. The aforementioned researches prove that image recognition technology has been widely used in the agricultural field to identify and locate vegetables accurately. However, most of the above methods adopt traditional machine learning, which mainly depends on multiple feature selection and application under specific background. It is difficult to play a role in the complex environment of the greenhouse.
With the development of high-performance computing systems and the sharing of massive data sets, deep learning has gradually become one of the most important technologies in the field of intelligent agriculture. The basic deep learning tool used in this work is convolutional neural networks (CNNs), which have a strong feature extraction capability and are suitable for processing a large amount of input images in parallel. Therefore, CNN is one of the most powerful technologies in the field of image recognition, which is widely used in image classification [12,13], object detection [14,15] and image segmentation [16,17]. In recent years, CNN technology has been introduced into the field of fruit and vegetable image recognition. Zhao et al. [18] proposed an apple location method based on YOLOv3, which realized apple detection in a complex environment. This work provided a theoretical basis for the research and development of apple harvesting robot. Fu et al. [19] studied the problem of fast identification of multiple clusters of Kiwifruit in the field. They trained LeNet as a classification model to identify occluded fruits, overlapping fruits, adjacent fruits, and independent fruits, and finally achieved an average recognition rate of more than 85%. In the work presented by Yang et al. [20] based on the Mask-RCNN, Resnet-50 was adopted as a backbone network. On this basis, the feature pyramid network (FPN) architecture was integrated to construct the feature extraction network. Their method accurately located strawberry picking points, which was helpful to improve the working performance of strawberry picking robot. Barth et al. [21] used VGG16 as a feature extraction network to conduct five experiments on real sweet pepper data sets and synthetic sweet pepper data sets. The final test results show that the model with the addition àtrous spatial pyramid pooling achieves the best performance, Mean IoU exceeded 0.53. Chen et al. [22] proposed a novel positioning approach based on instance segmentation using a monocular RGB camera to solve the problem of how to accurately pick sweet pepper. They used CNN to derive binary segmentation map and embedded feature map, redesigned the coding part of the network based on vgg16, and finally realized the example segmentation of sweet pepper outline. Lehnert et al. [23] realized the segmentation of sweet pepper by introducing a novel peduncle segmentation system. This system used an efficient deep convolution neural network, in conjunction with three-dimensional postfiltering to detect the critical cutting positions. The aforementioned literatures demonstrate that it is feasible to recognize fruit and vegetable by convolutional neural networks instead of traditional machine learning.
However, there are also some problems in the above literatures. For example, they mainly focus on the location of fruits. In the actual picking work, to ensure the integrity of fruits, we need to pick the stems and vines of fruits. To solve this problem, this paper designed and verified an improved CNN structure based on FRRN, and applied it to the segmentation of sweet pepper images collected in the greenhouse at night. This method can not only accurately identify the position of sweet pepper fruit, but also display the shape of stem and vine, which is very convenient for picking. Specifically, this paper took FRRN as the main network to segment the fruit, stem and vine of sweet pepper crops in the greenhouse at night. In addition, aiming at the problems of complex background and many interference items, this paper added PAM on the basis of FRRN to improve the feature representation, which helped the whole network focus on the main segmentation objects, such as fruits and stems, so as to obtain more accurate segmentation results. Finally, we also discussed the influence of image resolution on the segmentation results, and obtained a segmentation model suitable for sweet pepper picking.
The rest of this paper is organized as follows. The data set and experiment arrangements were introduced in Sect.

Data set description
The sweet pepper data set, used in this paper, was obtained from 4TU. 10 500 greenhouse sweet pepper images and their corresponding ground truth labels can be obtained at ResearchData (https:// data. 4tu. nl). These synthetic images were rendered through Blender based on 21 empirically measured plant properties. Considering that the sweet pepper harvesting robot will acquire images from different angles, they acquire sweet pepper images from different angles to simulate the picking habits of the robot [24]. We prepared three experiments. In Experiment I, 400 images were used for training, and 100 images were used for verification and test, respectively. In Experiment II, 6300 images were used for training and 2100 images were used for verification and test, respectively. Besides, the sweet pepper data set also contains 50 empirical images of the crop obtained from a high-tech commercial greenhouse. These empirical images were used in experiment III, these images were mainly used to test the generalization performance of the model. The sweet pepper data set was publicly released with the intention of comparing the performance of agricultural computer vision methods. There are 8 classes of segmentation objects in the picture, including background, leaves, peppers, peduncles, stems, shoots, wires, and cuts for picking pepper. In the labeled images, 8 classes were annotated on a per-pixel level, corresponding to the color of black, blue, yellow, red, pink, green, white, and light blue. In Fig. 1, examples of synthetic images are shown.

Overview of experiment arrangement
To obtain the optimal segmentation model of the sweet pepper image, three main experiments were designed and carried out in this study. Overview of performed main experiments I through III are shown in Table 1.
Experiments I: A suitable CNN architecture was picked out for sweet pepper segmentation. Since the same network structure may perform differently on different data sets, in a specific task, it is necessary to select the appropriate network according to the data set. Existing semantic segmentation network models are mostly applied to segmentation of street scenes, and rarely used in agricultural scene. In this experiment, several model structures were applied on the same sweet pepper data set for training and testing, and the differences in segmentation performance of different models were analyzed. Each network structure is verified with the same number of images, 400 images were used as training set, and 100 images were used as validation set and test set, respectively. Uniformly set the size of the input image to 384 × 384, so as to test the training time and performance of different models on a fair basis.
Experiments II: The effect of image resolution on segmentation results was investigated. After experiment I, a suitable model architecture for the sweet pepper segmentation task was obtained. Then the impact of the input image size on the segmentation performance of the model was further explored. Images with different sizes were input into the model for training. Unlike experiment I, experiment II was  performed on a larger data set. In this experiment, 10,500 sweet pepper images were divided into training set, validation set and test set according to 3:1:1, specifically, 6300 images in training set, 2100 images in validation set and test set, respectively. Experiments III: The self-attention mechanism was introduced into the network to improve the segmentation performance. The self-attention mechanism can effectively establish dependencies between remote features, and make the model focus on extracting some key features. In our work, the PAM was used to capture global dependencies in the spatial dimensions. It was added on top of the FRR. Afterwards, the enhanced model was trained and tested on our data set. The size of the input image and the distribution of the data set in experiment III were the same as that in experiment II. Finally, the model with the best segmentation performance was further evaluated on empirical (real) images.

Semantic segmentation network architecture
Semantic segmentation method was used to segment and recognize sweet pepper in this paper. The goal of semantic segmentation is to segment and parse an image into different image regions associated with semantic categories (e.g., leaf, sweet pepper, shoot). Semantic segmentation models based on convolutional neural network have attracted extensive attention, since FCNs [25] were proposed, and have made great progress. The existing depth-based semantic segmentation methods are mainly divided into two kinds: one is based on regional classification represented by Mask R-CNN [26], the other is based on pixel classification represented by FCNs. The former method is based on target detection. First, the image is divided into different image patches, then each pixel in the image block is semantically classified. Finally, semantics segmentation is realized. This kind of method first appeared in R-CNN and gradually evolved into improved algorithms, such as Faster R-CNN [27] and Mask R-CNN.
The advantage of this method is that it can accomplish both tasks of target detection and semantic segmentation at the same time. However, due to the lack of global information of the image, this method is not effective for small-scale objects and small areas. Therefore, this method was not considered in our experiments. The semantic segmentation method based on pixel classification is favored by many researchers and has been widely used. In particular, dilation convolution, as well as atrous spatial pyramid pooling was adopted in the Deeplab series [28,29] to embed contextual features of different scales. Moreover, the encoder-decoder structures can effectively fuse mid-level and high-level semantic features. In experiment I, since the methods based on pixel classification performed better in fine segmentation task, we mainly adopted them.

Full-resolution residual networks
To obtain high-level semantic information and broader receptive fields, feature maps generally need to go through multi-layer pooling operations. Although the pooling operation can highly describe the objects in the picture, it will cause a huge loss of positioning accuracy (that is, the spatial details of some targets will be lost, resulting in the unsatisfactory segmentation effect of the target boundary). Obviously, if it is not improved, it will inevitably affect the final segmentation result of sweet pepper. FRRN refers to the residual structure of ResNet [30], and divides the original single data flow into two branches, as shown in Fig. 2. One stream is a pooling stream: high-level semantic information is captured through a series of convolution and pooling operations to identify pixel categories. The other stream is residual flow: it carries the characteristics of full resolution and is used to provide accurate boundary information. This branch structure not only obtains the necessary high-level semantic features, but also transmits the feature map with full resolution information to the whole network. In short, the impact of pooling operation on FRRN structure will be reduced. Meanwhile, FRRN consists of a series of full resolution residual units (FRRUs). The structure of FRRU is given in Fig. 3. Each unit includes two inputs and outputs, because they are applied to both streams at the same time. If z n−1 is the residual input of the nth FRRU and y n−1 is pooling input. Then the outputs are computed as where H(·) and G(·) represent residual calculation and pooling operation, respectively. W n is weight matrix.

Position attention module
The attention mechanism in deep learning is generated by imitating human selective visual attention mechanism. It has been widely used in image processing [31], speech recognition [32] and natural language [33] in recent years. The essence of attention mechanism lies in the automatic and efficient allocation of attention resources, which contributes to the acquisition of the most critical information to solve the current task. There are many variants of the attention model, including the soft attention model, the global attention model [34], and the key-value attention model [35]. PAM can adaptively aggregate long-range context information to improve the feature representation of semantic segmentation. The implementation details of PAM are shown in (1) z n = z n−1 + H y n−1 , z n−1 ;W n , (2) y n = G y n−1 , z n−1 ;W n , Fig. 4. A represents a local feature, where A ∈ R C×H×W . It is first fed into a convolution layer to generate two new feature maps B and C, respectively, where {B, C} ∈ R C×H×W . Then they are reshaped to R C×N , where N = H × W is the number of pixels. After that, a matrix multiplication is performed between transposed B and C, and a softmax layer is applied to calculate the spatial attention map M ∈ R N×N : where M ji measures i th position's impact on j th position. The more similar feature representations of two positions contribute to a greater correlation between them.
Meanwhile, feature A is also fed into a convolution layer to generate a new feature map D ∈ R C×H×W and is reshaped to R C×N . Then a matrix multiplication is performed between D and M, and the result is reshaped to R C×H×W . Finally, it is multiplied by a scale parameter , and an elementwise sum operation is performed between it and features A to obtain the final output Q ∈ R C×H×W as follows: where is initialized as 0 and gradually assigned more weight.

Attention module embedding with full-resolution residual networks
As a tool for extracting image object features, CNN enhances the expression ability of features through the connection between layers. Because it shares convolution kernel parameters, redundant computation is avoided and computational efficiency is improved. However, due to the "moving window" attribute of convolution kernel, CNN still has some limitations in capturing global features, which is not conducive to the model to capture complete context information. It can be inferred from Eq. 4 that the resulting feature Q at each position is a weighted sum of the features across all positions and original features. Therefore, PAM can not only reflect the complex spatial transformation, but also establish the dependence between long-distance features, so as to obtain the global feature representation. In the sweet pepper images, some 'stem' and 'shoot' are inconspicuous or incomplete objects because of the influence of lighting and view. Those dominated salient objects (e.g., leaf, sweet pepper) would harm those inconspicuous object labeling.
Since the PAM can selectively aggregate the similar features of inconspicuous objects to highlight their feature representations and avoid the influence of salient objects, this study proposed a sweet pepper segmentation model combining FRRN and PAM. The ability of PAM to capture remote feature dependencies makes up for the shortcomings of FRRN. The proposed FRRN-based network structure for semantic segmentation of the sweet pepper image is shown in Fig. 5. As illustrated in Fig. 5, FRRN was employed as the backbone. The whole network structure mainly refers to encoder-decoder [36] structure. When the network starts working, the input feature maps are down-sampled through four pooling layers to extract the deep features in the images. After that, 4 up-sample layers are used to restore the size of the feature maps. In addition, the mapping relationship between the original image and the label is obtained. Then the features from the FRRN would be fed into the PAM. Specifically, PAM is cascaded behind the concatenate (fusion) layer. The output features of the attention module are transformed by the convolution layer and input into the last convolution layer to generate the final prediction map. Besides, the notations RU and FRRU in Fig. 5 refer to the residual unit and the full-resolution residual unit, and the numbers on the lower right corner represent the number of convolution kernels in the unit, respectively. In addition, the number of convolution kernels of each convolution layer in Fig. 5 is 48, 32, and 8 (number of classes to predict). The sizes of convolution kernels are 5 × 5, 1 × 1, and 1 × 1, respectively. The model architecture is proposed to contribute to the field of agricultural segmentation task.

Experiment platform
The experiments in this work were conducted on the Ubuntu 16.10 LTS 64bit system. All the methods were implemented based on Tensorflow, and the TF-Slim library provided main functions for building model framework. The computer is equipped with 16 GB RAM and Intel Core i7-7700 K CPU. Meanwhile, NVIDIA GTX1080Ti GPU acceleration technology was applied to improve the training speed.

Implementation details
The RMSProp optimization algorithm was adopted during the process of training models. For the experiment I, the base learning rate was set to 1e−4, and the weight decay coefficient was set to 0.995. For the remaining experiments, the base learning rate and attenuation coefficient were set to 1e−5 and 0.995, respectively. Batch size was only set to 1 for all data sets because of the limitation of GPU memory and computing power. The number of iterations was set to 300 epochs for the experiment I and 200 epochs for the other experiments. The loss function used in our study was the softmax cross-entropy function. Besides, the data augmentation technology was not adopted in the work of this paper. The mean IoU is the most important segmentation evaluation metric, which is computed as follows: where k represents different classes, p ii is the number of pixels correctly predicted, k ∑ j=0 p ij is the number of pixels in which class i is predicted to be class j.
The precision, recall, and F1 score were introduced as additional evaluation indexes to evaluate the segmentation effect of sweet pepper images more objectively. In general, the higher and closer the precision and recall, the better the classification recognition effect. The F1 score is computed based on the mean per-class precision/recall results as follows:

Evaluation of experiment I
In experiment I, several existing semantic segmentation methods were evaluated and compared on a small sweet pepper data set. The mean pixel accuracy (PA) and mean intersection over union (IoU) of the trained models on the test set are shown in Table 2. These model architectures achieved good segmentation results when used for sweet pepper segmentation in this paper. It is worth noting that most of these methods were training from scratch, rather than based on the pre-trained models. The approach based on pre-trained models was also adopted in our experiments. The results showed that although the training time of this method was less than other means, the segmentation effect was far inferior to those methods training from scratch. Consequently, the results of approach based on pre-trained models were not shown and compared in this paper. As shown in Table 2, the mean PA and mean IoU of FRRN are 94.3% and 69.18%, respectively. In particular, its recognition accuracy for sweet peppers reached 93.2%. The segmentation and recognition effect of FRRN is better than that of other methods. From the perspective of pixel classification, the mean PAs of the five methods are similar, and all of them exceed 90%. It illustrates that the five networks have achieved good performance in pixel classification, especially for salient objects (e.g., leaf, sweet pepper). Meanwhile, the mean PAs of sub-experiments I-A, I-B, and I-E are slightly higher than that of the other two methods. From Table 2, it can be seen that the main difference is caused by the small objects, including peduncles, stems, shoots, and wires. The PA of wires in experiment I-C was 6.30%, while that of peduncle in experiment I-D was only 6.00%. FRRN performs better than other methods, which verifies that it is more suitable for small-scale object recognition and fine segmentation. Moreover, the effects of several methods can be visualized in Fig. 6. These visualizations show that the FRRN achieved better segmentation results, both numerically and qualitatively. However, the segmentation effect of FRRN for sweet pepper needs to be improved, especially for the pixel-level recognition of inconspicuous objects. Hence, we took experiment II and experiment III.

Evaluation of experiment II
Semantic segmentation belongs to pixel-level tasks, which need to provide pixel by pixel output. Consequently, semantic segmentation requires higher resolution and more detailed information. It is supposed that the higher resolution of the input images, the richer feature information that the convolutional neural network could capture. Highresolution images help to improve the recognition accuracy and segmentation performance of the model. Especially for fine agriculture segmentation, higher image resolution is beneficial. However, if the resolution is extremely high, maybe it causes the problem of slow training speed. In this study, the resolution of the original images in the data set is 800 × 600, but the general model cannot directly perform  1 3 feature analysis and extraction on such images. To balance the relationships between segmentation effect and computational complexity, the original image was cropped to 384 × 384 and 512 × 512 in experiment II, and train FRRN on images of different sizes. To explore the influence of different image resolutions on the segmentation results, we sent images with different resolutions into five different models, and obtained the data shown in Table 3. By observing the data in Table 3, we can see that the same model achieves higher segmentation performance after inputting high-resolution images than inputting low-resolution images. Among them, FRRN obtained the highest mean PA and mean IoU. Specifically, when the input size is 384 × 384, FRRN obtained 73.67% in mean IoU, however, when the input image size changed to 512 × 512, the mean IoU increased to 76.81%, an increase of 3.14%. This shows that the improvement of resolution is helpful to refine the segmentation results. In addition, when the size of the input image is 512 × 512, the recall and F1 score obtained by FRRN were further improved to 95.47%  and 95.57%, respectively. Although the improvement effect of pixel classification is not significant, the accuracy and recall rate have been maintained at a high level and close to each other, which shows that the classification effect of the model has been improved. The results show that within a certain range, the higher the resolution of the input image, the better the segmentation performance of FRRN.

Evaluation of experiment III
The PAMs were employed on top of the FRRN to capture long-distance dependencies for better feature presentation.
To verify the improvement effect of PAM on the model, we used images with different resolutions as input to train three models. The various evaluation indicators implemented by these three models on the test set are shown in Table 3. It can be observed that the performance of the three models has improved after PAM is embedded. Where, when the input image resolution is 512 × 512, FRRN + PAM achieved the best performance. Compared with the baseline FRRN, the employment of PAM yielded a result of 78.88% in mean IoU, which brought an improvement of 2.07% points. In addition, mean PA increased by 2.47% points. Meanwhile, the precision, recall, and F1 score have also improved. In conclusion, results show that attention modules bring benefits to sweet pepper segmentation. Quantitative evaluation is not enough to verify the importance of PAM, thus the effects of PAMs were visualized in Fig. 7. As expected in Sect. 3.4, the PAM helps to strengthen the feature presentation of some inconspicuous objects. It can be observed from Fig. 7 that the employment of PAM makes some segmentation details and object boundaries clearer. As shown by the red circles in the first row, it is challenging to segment small-scale objects in detail, but under the action of PAM, the segmentation effect of the third column is significantly better than that of the second column. Specifically, in the second row, the FRRN model with PAM is better than the original FRRN model for the segmentation of small objects, such as leaf stems. Therefore, Fig. 7 demonstrates that selective fusion over local features enhances the discrimination of details and refines the segmentation results. Besides, compared with the research of Barth et al. [21], the accuracy of pepper segmentation has been improved obviously.
To explore the generalization performance exhibited by a synthetically trained model when faced with similar data in the same domain, the best performing model (model of experiment III-B) was evaluated on 20 empirical (real) images. The segmentation results of the proposed model on empirical data are shown in Fig. 8. The model failed to discriminate and segment all the classes properly. This result confirms our hypothesis that the synthetically bootstrapped model could not generalize accurately to empirical data without fine-tuning. Note that these empirical images were not used for training or validation. To prove our hypothesis, 30 empirical images were used to fine-tune the proposed model, and the generalization performance of the model was tested on 20 empirical photos. As illustrated in Table 4, this practice resulted in increased performance, with a mean IoU of 59.81%. The result implies fine-tuning on a synthetically trained network can generalize to similar data in the same domain (empirical images).
Note that, for the empirical data set, due to the influence of dark regions, some objects that were not manually annotated in the ground truth would be detected in the images and have a certain impact on the segmented images. Hence, although these parts were true positive, they were still evaluated as false positives. This annotation bias resulted in a lower segmentation performance of empirical images. In short, the results of experiment III show that the proposed

Results compared with other models
We further compared our method with existing methods on the sweet pepper testing set. The results are shown in Table 5. The FRRN + PAM model performs better than other segmentation methods. It is supposed that these networks are designed for street scene segmentation tasks, so they maybe not perform well in agricultural segmentation tasks. Among these methods, to achieve real-time performance, only ENet refrained from using a pre-trained network when designing the structure, so it did not get high scores. Specifically, a pretrained network was abandoned by our network either, but it outperformed ENet by a large margin. Moreover, it also surpassed DANet, which uses a complex backbone ResNet-101.

Conclusions
In this paper, CNN was applied to the semantic segmentation of the sweet pepper images. The FRRN model was improved by combining with attention mechanism. Specifically, PAMs were embedded at the top of the network to capture global dependencies in the spatial dimensions. This new architecture is clean, requires no additional postprocessing, and can be trained from scratch. The proposed network (FRRN + PAM) was trained to segment sweet pepper images captured at night. Relevant experimental results show that PAMs contribute to providing more precise segmentation results. Meanwhile, our experiments indicate that proper image resolution benefits for segmentation performance. Besides, the synthetically trained model combined with fine-tuning can also be well-generalized to the similar data in the same domain (empirical data set). Our method achieves good performance on the test set of sweet pepper, with a mean IoU of 78.88%. The proposed method is generic and can be applied for other fruits and vegetables. It is hoped that this study can provide a theoretical basis for the development of image recognition in precision agriculture. The improved model proposed in this paper has achieved relatively good results, but there are also some shortcomings.
For example, compared with other models, the improved model has improved the segmentation effect of some small targets, but the overall accuracy is not high. There are many reasons for this problem: (1) Poor drawing of label map; (2) The target object is too complex and the number of targets is huge; (3) The model structure itself has room for optimization. Therefore, in the follow-up research, we will explore a more accurate and efficient sweet pepper segmentation method based on this research. Mean IoU/% ENet [39] 65.78 BiseNet [38] 69.75 DANet [40] 76.94 Our method (FRRN + PAM) 78.88