The adjustment and performance of the CNN model could be visualised by graphs of training and validation losses. Figure 1.a depicts these metrics, both the validation and the training losses decrease and stabilise, reaching its minimum after 80 until 100 epochs (complete passes of the training data set through the algorithm). Our model shows an optimal generalisation to validation data (optimal fit) and then is not under- or over-fitted because the curves of the training and validation losses overlap. The accuracy of the model is depicted in the Fig. 1.b. The plot illustrates that both the training and the validation accuracies are linearly increasing until they reach 92%. This behaviour is attained in the model without dropout or weight regularisations.
Our mode had a hold-out of 80 − 20 (80% of data for training and 20% of the data for validating). The performance of the training model was tested on the validation dataset, which consisted of 200 images of FD curves of each class. Hence the final model used 2400 images of FD curves (800 images for each of the 3 classes) for training and 600 images for validating the performance of the model (200 images per class). The classification capability of the model is then appreciated in a confusion matrix (Fig. 1.c).
The Fig. 1.c illustrates that the true positives (TP) are high for the three predicted classes on the validation dataset (it can be seen on the diagonal going from up-left corner to the bottom-right corner). On 200 FD curves belonging to the domains denoted as Inter, the algorithm correctly classifies 174 curves as Inter and misclassify 4 curves as Nano and 22 as Out. On 200 FD curves belonging to the domains denoted as Nano, the algorithm correctly classifies 199 curves as Nano and misclassify 1 curve as Inter and 0 as Out. On 200 FD curves belonging to the domains denoted as Out, the algorithm correctly classifies 182 curves as Out and misclassify 18 curves as Inter and 0 as Nano. From the confusion matrix, we can make a classification report that includes the commonly useful metric parameters as follows. The precision for Inter, Nano and Out classes are 0.89, 0.99 and 0.88, respectively (Fig. 1.d). The recall (sensitivity or true positive rate) for Inter, Nano and Out classes are 0.86, 0.97 and 0.92 respectively. The F1-score, which combines the precision and recall scores, for Inter, Nano and Out classes are 0.88, 0.98 and 0.89, respectively. This last parameter is used to obtain 0.92 of macro-averaged F1 score and 0.92 for the weighted average, finally the general accuracy was of 92%.
The trained model predicts each FD curve in a few milliseconds giving probabilities displayed following the order in which the model classify the FD curves (see Fig. 3b), i.e. the first number corresponds to the Inter class of the FD curve, the second to the Nano class and the third to the Out class. Then by feeding a random FD curve image, the algorithm recognises that it belongs to an intermediate part of the cell (Fig. 2.a) with a probability of 99.99% (displayed as .9999 considering 1 as 100% of probability), it also gives a probability of 5.61 x 10− 5 that it belongs to a nanodomain and a probability of 3.96 x 10− 8 that it belongs to an external part of the membrane cell, interpreting these as 0% probability, for the latter 2. Figure 2.b also shows the recognition of an FD curve corresponding to part of a nanodomain, in this case only with the retract curve, with 100% confidence. It also includes the probability value corresponding to an FD curve of an Inter class as small as 6.5 x 10− 25 and 5.3 x 10− 27 belonging to the Out class, which can be considered as a 0% probability. The model can display a list of all predicted probabilities.
The objective of our classification algorithm is to generalise on unseen data coming from other C. albicans cells. To validate this objective, we worked on a different JPK quantitative imaging data with also 128 X 128 nanoindentations (16,384 FD curves) corresponding to another C. albicans cell (Fig. 3.a). We then processed in 2 steps: first, 300 FD curves from each class (Inter, Nano and Out) were selected and exported by using the free ROI. Figure 3.b depicts the prediction result on the testing dataset, it gives the classification of the 3 classes in just 5 seconds using GPU once the trained mode was loaded. The prediction was 298, 304 and 298 for Inter, Nano and Out, respectively, which represents more than 99% confidence. In a second step, the algorithm was fed with the full matrix made of 16,384 FD curves. In less than 90 seconds, it provides a classification that is presented in Fig. 3.c. Further tests were performed to corroborate the performance of our model. A test dataset was prepared as the previous one, but in this case, each image of the FD curves included only the retract curve, allowing us to corroborate that the main feature the CNN is taking into consideration to learn and predict is the adhesion force of the curve.
The above tests have allowed us to be fully confident that our model of extension .h5 (available together with the code in GitHub) is powerful and can predict on any dataset or any group of FD curves exported as images without needing to be labelled. Simply exporting this file from an AFM adhesion map of a C. albicans cell makes the prediction over 99% reliable, giving the amount of nanodomain FD curves correlated to a degree of adhesion.