Fast and Accurate Automated Recognition of the Dominant Cells From Fecal Images Based on Faster R-CNN

doi:10.21203/rs.3.rs-133739/v1

PDF

Research Article

Fast and Accurate Automated Recognition of the Dominant Cells From Fecal Images Based on Faster R-CNN

https://doi.org/10.21203/rs.3.rs-133739/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Fecal samples can easily be collected and are representative of a person’s current health state; therefore, the demand for routine fecal examination has increased sharply. However, manual operation may pollute the samples, and low efficiency limits the general examination speed; therefore, automatic analysis is needed. Nevertheless, recognition exhaust time and accuracy remain major challenges in automatic testing. Here, we introduce a fast and efficient cell detection algorithm based on the Faster-R-CNN technique and is the Resnet-152 convolutional neural network architecture. Additionally, a region proposal network and a network combined with principal component analysis were proposed for cell location and recognition in microscopic images. Our algorithm achieved a mean average precision of 84% and a 723 ms detection time per sample for 40560 fecal images. Thus, this approach may provide a solid theoretical basis for real-time detection in routine clinical examinations while accelerating the process to satisfy increasing demand.

General Cell Biology & Physiology

Health Policy

Health Economics & Outcomes Research

object detection

deep learning

cell detection

convolutional neural network

From a biological perspective, the metabolic process is an important bridge between biological function and structure (Langemann and Rehberg, 2010). In the human digestive process, food or water enters the oral cavity first; after a series of chewing cycles, the content flows through the esophagus into the stomach. Gastric acid and enzymes digest the contents under gastric motility (Zorn, 2017). Several hours later, the contents are delivered through the duodenum to the small intestine and large intestine (Friedman, 1961). Therefore, fecal matter clearly contains abundant biological information (Obokhare, 2012).

The total worldwide population is close to 7.8 billion, and the male to female ratio is approximately 1.02. According to the WHO disease report, the incidence of digestive disease is 20%-40% in per and the incidence of gynecological disease is 24.94% (Dossett et al., 2017), (Ji et al., 2018). Clearly, there is abundant demand for routine clinical examination of feces. Furthermore, these kinds of biological samples are widely accepted in diagnosis due to their characteristics, including being noninvasive (Gerber and Opriessnig, 2015), (Rezasoltani et al., 2019) and representative (Martinez-Guryn et al., 2019) and providing disease-related information (Kim et al., 2018). However, another challenge is becoming clear: how to overcome the limitations of manual operation, such as bad odor and aseptic, inefficient and tedious operation (Abraham, 2018). Solutions to these problems have become increasingly urgent for routine clinical examination.

At present, the automatic recognition of tangible components such as cells under the microscope applies mainly to machine vision. However, the traditional machine vision method requires the design of complex feature extractors (such as morphological features and texture features), and many images need to be preprocessed before training (Manik et al., 2016), (Ghosh et al., 2016). In addition, the training process is inadequate and complex.

The lack of an automatic recognition algorithm for organic components under the microscope seriously restricts the automation of routine stool analysis. Recently, deep learning technology has been successfully used in image classification, object detection and other computer vision tasks (Afridi et al., 2017), (Zhang et al., 2017). Compared with traditional machine learning methods, convolutional neural networks automatically extract image features, simplify and avoid unnecessary image preprocessing, and improve the validity and accuracy of detection (Simonyan and Zisserman, 2014; Szegedy et al., 2015). Therefore, we introduce an automated cell detection approach based on a faster region-based convolutional neural network (Faster R-CNN) (Ren et al., 2017), which we termed principal component analysis (PCA)-based (Jolliffe, 1986) Faster R-CNN (PCA-Faster RCNN).

Ethics approval and consent to participate:

The Institute of Institutional Review Board and Ethics Committee of the Fourth Affiliated Hospital of Nanchang University approved this study (SFYLL-PJ-2015-001). Written informed consent was provided by all participants. All biological samples were anonymized. All methods were carried out in accordance with relevant guidelines and regulations.

Fecal sample collection

In total, 676 positive samples were collected from the Fourth Affiliated Hospital of Nanchang University. These samples were diluted, stirred, allowed to stand and finally sent to a flow cell. To observe a clear sample image, an OLYMPUS CX31 was used in the optical system as the basic optical structure with a 40× objective lens (numerical aperture (NA): 0.65, material distance: 0.6 mm). An EXCCD01400KMA CCD camera was used to capture images with 6.45 µm resolution, and a standard halogen lamp was chosen for illumination.

The size of the collected images was 1600×1200. Annotation of each image was conducted manually as the ground truth. The location and size of RBCs, WBCs, molds and pyocytes were recorded according to the image analysis. Only the standard cell structure was annotated from the images, and the defocused image was not marked to reduce false detection of impurities. A total of 8,785 images with stylized components were collected. Training a on a small number of images can affect the test performance of a model. Therefore, to reduce the effect of overfitting, data argumentation was performed using random vertical and horizontal flipping and random contrast and saturation adjustments.

Proposal

Four main elements must be identified during routine fecal examination: red blood cells (RBCs), white blood cells (WBCs), pyocytes (PYOs), and mildews (Mids). Other components, such as calcium oxalate crystals, starch granules, pollen, plant cells, plant fibers and food residues, are classified as impurities with less clinical significance. For detail, please see Fig. 1 (a)~(h).

Faster R-CNN (Ren et al., 2017) consists of three main parts: (1) a feature extraction layer, (2) a region proposal network (RPN), and (3) a classification and regression network; see Fig. 2 for a detailed model schematic diagram. Among them, the RPN and classification and regression network share the previous feature extraction layer, as shown in Fig. 2(a). The feature extraction layer is composed of a series of convolutional neural networks composed of a convolutional layer, pooling layer, and activation layer. According to the feature map generated by the feature extraction layer, the RPN can generate anchors of different sizes and aspects, which are then used to generate the region proposal. The proposed region generated by RPN is input into the classification and regression network for the type recognition and box accurate regression. Because the scale of the feature map layer corresponding to different foreground regions is inconsistent, Fast R-CNN adopts an ROI pooling strategy to unify the dimensions. Although the calculation is simplified, some features are lost; therefore, we propose PCA dimension reduction to normalize the dimensions of the features.

The feature extraction layers use Resnet (Ren et al., 2017), a 152-layer network composed of 4 residual network blocks: the first three residual network blocks are selected as feature extractors (see Fig. 2(b)).

The RPN network was used to generate a batch of proposals, similar to the selective search used in R-CNN (Girshick et al., 2014) and Fast R-CNN (Girshick, 2015). The network structure is consistent with the RPN used in Faster R-CNN: a 256-channel output is generated by a 3 × 3 convolutional layer after the feature map layer (conv4b_35), which is used to fuse the information around the features and to fuse information across channels. Meanwhile, the fused layer is connected by two branches, termed the SoftMax classification head and box location regression head; for details, see Fig. 3(a). Different from the RPN in Faster R-CNN, whose box dimensions are hand-selected, the generated anchors are based on the average size of the foreground target, which allows the regression network to run smoothly to learn and predict good locations; for details, see Fig. 3(b).

In the training process, the RPN module is trained jointly, rather than alternately, with the object recognition network. Since the structure of the Faster R-CNN is end-to-end, both the RPN and the object recognition network can provide feedback on the feature extraction layer. During backpropagation, the loss functions from both the RPN and the fast R-CNN were combined and calculated together. Moreover, we introduced the PCA strategy in the classification and regression component of Faster R-CNN that should be trained separately. The original Faster R-CNN model, denoted as M0, can improve the RPN network (3.1.2) and the ROI pooling strategy. PCA-based Faster R-CNN is denoted as M1. The training process is shown in Fig. 4.

Experimental setup

All experiments were conducted using models developed based on TensorFlow, which provides libraries to build the main structure of deep learning models. The experiments were executed on a Windows system with an Intel Core i7-5960X CPU @ 3.0 GHz×8, an NVIDIA GeForce GTX 1080 Ti GPU and 32 GB RAM. The microscopy process involved taking 5 images with different focal lengths and recording 12 fields of view by means of a movable platform.

In total, 676 biological samples were obtained from the Fourth Affiliated Hospital of Nanchang University. Therefore, 40560 fecal images were used to develop the detection algorithm based on Faster R-CNN. All images were collected independently from the microscopic imaging system. The best resolutions of the twelve images were collected for each sample. To further validate the algorithm, experienced laboratory experts annotated the cells of all the images in the development dataset with the different colors of rectangular boxes as the ground truth. For more details, please see S1. Detailed fecal sample information and the dataset split are summarized in Table 1.

Table 1

Overview of the dataset
Contents	Dataset A: Training	Dataset B: Validation	Dataset C: Testing
# images	6150	880	1755
Cells	12348	388	2628
RBCs	5588	1	1572
WBCs	682	5	454
pyocyte	70	0	54
Molds	6008	382	548
Footnote: The data set is divided randomly. As some samples are negative, they contain fewer cells

After training, the network was tested. The WBCs are marked with blue squares and percentages (Fig. 5a, 5b, 5c), while the RBCs are marked with green squares and percentages (Fig. 5a). PYOs are marked with light blue squares and percentages (Fig. 5a). Furthermore, the remaining components, MEDEW, are marked with gray squares and percentages (Fig. 5b&5d); for details, please see Fig. 5.

AP and mAP were used to detect the cells and identify their location from the microscopic images. Due to the insufficient sample size during training, the detection recognition rate was low. For example, for RBCs, WBCs and mildews, the detection results reflect the performance of the model, and the mAP value was 84%. Two established classes of methods are used for object detection in images: one based on morphology segmentation or selective search, which is used in R-CNN and Fast R-CNN, and the other based on region proposal classification. We compared the proposed model of PCA-based Faster R-CNN with R-CNN, Fast R-CNN, Faster R-CNN and R-FCN (Dai et al., 2016). The mAP of our method was the highest (0.84). Moreover, the time consumed per image (723 ms) was significantly shorter than that of R-CNN and Fast R-CNN, whereas no significant difference was observed with respect to Faster R-CNN and R-FCN. Specifically, the AP values for RBCs, WBCs, Pyocytes and Mildews were 0.92, 0.85, 0.81, and 0.75, respectively. The AP was 0.84; moreover, the AP values for the four types of cells obtained with our proposed method were higher than those of the other four methods (see Table 2).

Table 2

Comparison of 5 cell detection algorithms
MODEL	mAP	dur/image	AP
MODEL	mAP	dur/image	RBC	WBC	Mildew	PYO
R-CNN	0.64	14.9 s	0.80	0.74	0.77	0.26
Fast R-CNN	0.66	4.2 s	0.81	0.77	0.79	0.28
Faster R-CNN	0.80	517 ms	0.89	0.81	0.78	0.72
R-FCN	0.81	468 ms	0.90	0.80	0.80	0.73
PCA-Faster-RCNN	0.84	723 ms	0.92	0.85	0.81	0.78
Footnote: dataset C was used to validate the average precision.

Clearly, the selective search segmentation method used by R-CNN and Fast R-CNN consumed substantial amounts of time. With the introduction of PCA into the feature extraction layer, the features were assigned the main component during the classification and regression process, and the features of Faster R-CNN and R-FCN were filtered out through the pooling strategy. These results also indicate that the Faster R-CNN method based on PCA had the highest overall recognition rate.

The large number of impurities in the fecal samples made the background of the images complex. Inevitably, the pattern components in the images were difficult to address. However, our algorithm can effectively distinguish the adhesive type components. Unfortunately, the morphological or selective search method cannot accomplish this task. For instance, when an RBC and mold in the image were stuck together, our algorithm could distinguish the two components (see Fig. 6).

In summary, 676 fecal samples and more than 40560 microscopic images were prepared for algorithm development. Our algorithm presented good performance in identifying four kinds of cells and their locations in microscopic images. The algorithm has two major advantages, the average time required to analyze a sample and accuracy.

Clearly, our algorithm consumes significantly less time than R-CNN and Fast R-CNN, which may be due to the introduction of RPN. The R-CNN and Fast R-CNN models use selective search in the segmentation of foreground objects, which requires considerable running time. Each foreground target unit propagates forward to extract features in R-CNN post segmentation (Girshick et al., 2014), while Fast R-CNN shares the convolutional layer, which can extract features by propagating forward once (Girshick, 2015). However, no significant exhaustion time difference was found between Faster R-CNN and R-FCN. R-FCN uses the position-sensitive map method to avoid the fully connected layer and simplify the training parameters; consequently, the time consumption is slightly lower than that of Faster R-CNN. The time consumption of PCA-Faster-RCNN is slightly higher, mainly because of the introduction of the PCA strategy after feature extraction.

With respect to the AP performance for four kinds of cells from a single image, the AP of RBCs was the best (0.92), which we believe to be a result of the obvious characteristics of RBCs and the fact that there are no significant morphological changes for different RBCs. The number of RBCs in the collected data set is large, and data enhancement is adopted to improve the training of RBCs.

The AP values of WBCs and Mildew were 0.85 and 0.81, respectively. This reduced performance may be caused by the specific characteristics of cells in different views. In different samples, leukocytes may be round and influenced by osmotic pressure or be shaped as irregular ellipses. Similarly, different mildews have different spore numbers, sizes and shapes after budding, so the recognition rate is lower than that of RBCs. Meanwhile, due to the sample size, the accuracy of mildews is slightly better than that of WBCs. Furthermore, the AP of PYO was 0.78, likely a result of the small sample size and insufficient training. PYOs are usually composed of many WBCs with a large irregular shapes. Due to the small sample size, the training model suffered from a certain degree of overfitting.

Excitingly, our algorithm presented better mean average precision (mAP = 0.84) than the other methods. The results indicate that PCA plays an important role in feature selection. After introducing PCA into our algorithm model, we proposed a model training method that did not follow the end-to-end architecture of the original Faster R-CNN. The disadvantage is that the model does not represent imbalanced samples well. For example, the number of PYOs is small, and the average precision is relatively low compared with that of other types of cells. The PCA-Faster-RCNN model can be used in other fields of recognition of components in microscopic images, such as target detection in leucorrhea, type component detection in urine, and cell counting in blood.

A deep learning model for cell detection is proposed for locating and identifying objects from microscopy images. The algorithm achieves the highest mean average precision (mAP) and has the ability to detect and locate red blood cells (RBCs), white blood cells (WBCs), mildews, and pyocytes rapidly. The mAP is approximately 84%, and the detection time is 723 ms per image (1600*1200 resolution).

Due to the small sample size in the collected data set, fat globules are not considered in this analysis. When the number of samples belonging to a certain category is small, for example, pyocytes, as training proceeds, the model can easily suffer from overfitting. Artificial adhesion of leukocytes can be used to expand the number of samples via data enhancement.

Acknowledgements

We would like to express our thanks to professor Yutang Ye and all the staff in the MOEMIL laboratory who collected and counted the cells used in this study.

Funding

This research was supported partly by the National Natural Science Foundation of China (No. 61905036) and the Fundamental Research Funds for the Central Universities (University of Electronic Science and Technology of China) (No. ZYGX2019J053).

Authors’ contribution

ZJ and LY constructed the concept, DXH, WZX, NGM performed the data collection and image analysis, LJX, HRQ and LL conducted the image visualization, XF proofread and ensure the general quality of manuscript.

Competing interest: No

Tensorflow https://tensorflow.google.cn/. (03-20- 2020)
World population, https://countrymeters.info/en/World. (06-19-2020)
Abraham, B. P. Fecal Lactoferrin Testing. Gastroenterol Hepatol (N Y). 14, 713–716 (2018).
Afridi, M. J. et al. Intelligent and Automatic In Vivo Detection and Quantification of Transplanted Cells in MRI. Magn Reson Med. 78, 1991–2002 (2017).
Dai, J., Li, Y., He, K. & Sun, J. 2016. R-FCN: object detection via region-based fully convolutional networks, Proceedings of the 30th International Conference on Neural Information Processing Systems. Curran Associates Inc., Barcelona, Spain, pp. 379–387.
Dossett, M. L., Cohen, E. M. & Cohen, J. Integrative Medicine for Gastrointestinal Disease. Prim Care. 44, 265–280 (2017).
Friedman, J. 1961. The normal physiology of the digestive system. Heidelberg, Berlin.
Gerber, P. F. & Opriessnig, T. Detection of immunoglobulin (Ig) A antibodies against porcine epidemic diarrhea virus (PEDV) in fecal and serum samples. MethodsX. 2, 368–373 (2015).
Ghosh, P., Bhattacharjee, D. & Nasipuri, M. Blood smear analyzer for white blood cell counting: A hybrid microscopic image analyzing technique. Appl Soft Comput. 46, 629–638 (2016).
Girshick, R. & Fast, R-C-N-N. 2015. 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1440–1448.
Girshick, R., Donahue, J., Darrell, T. & Malik, J. 2014. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587.
Ji, N. et al. Disease burden for gynecological disease in China. Zhonghua Fu Chan Ke Za Zhi. 53, 313–318 (2018).
Jolliffe, I. T. 1986. Principal Component Analysis, 1986 ed. Springer-Verlag, pp. 1-487.
Kim, H. K., Kostidis, S. & Choi, Y. H. NMR Analysis of Fecal Samples. Methods Mol Biol. 1730, 317–328 (2018).
Langemann, D. & Rehberg, M. Unbuffered and buffered supply chains in human metabolism. J Biol Phys. 36, 227–244 (2010).
Manik, S., Saini, L. M. & Vadera, N. 2016. Counting and classification of white blood cell using Artificial Neural Network (ANN), 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), pp. 1–5.
Martinez-Guryn, K., Leone, V. & Chang, E. B. Regional Diversity of the Gastrointestinal Microbiome. Cell Host Microbe. 26, 314–324 (2019).
Obokhare, I. Fecal impaction: a cause for concern?. Clin Colon Rectal Surg. 25, 53–58 (2012).
Ren, S., He, K., Girshick, R. & Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence. 39, 1137–1149 (2017).
Rezasoltani, S. et al. The gut microflora assay in patients with colorectal cancer: in feces or tissue samples?. Iran J Microbiol. 11, 1–6 (2019).
Simonyan, K. & Zisserman, A. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 1409.1556.
Szegedy, C. et al. 2015. Going deeper with convolutions, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9.
Zhang, J. et al. Computerized detection of leukocytes in microscopic leukorrhea images. Med Phys. 44, 4620–4629 (2017).
Zorn, A. M. Development of the digestive system. Semin Cell Dev Biol. 66, 1–2 (2017).

Supplementarydocument20200618.pdf

PDF

Editorial decision: Major revision
15 Mar, 2021
Reviews received at journal
07 Mar, 2021
Reviewers agreed at journal
01 Mar, 2021
Reviewers agreed at journal
16 Feb, 2021
Reviewers invited by journal
13 Feb, 2021
Editor assigned by journal
12 Feb, 2021
Editor invited by journal
08 Jan, 2021
Submission checks completed at journal
06 Jan, 2021
First submitted to journal
21 Dec, 2020

You are reading this latest preprint version

Fast and Accurate Automated Recognition of the Dominant Cells From Fecal Images Based on Faster R-CNN

Status:

Version 1

Abstract

Figures

Introduction

Method

Ethics approval and consent to participate:

Fecal sample collection

Proposal

Experimental setup

Results

Discussion

Conclusion

Limitation

Declarations

References

Supplementary Files

Status:

Version 1