COVID- 19 Screening by RT-PCR: An Epidemiological Modelling

DOI: https://doi.org/10.21203/rs.3.rs-35816/v1

Abstract

Background: Given the attention on COVID-19 testing and its role in helping to halt the spread of COVID-19 Pandemic, wider testing is urgently needed for successful pandemic control. The level of the test’s performance is also important for effective management of the different stages of the pandemic.

Objectives: To study the impact of RT-PCR testing in control of COVID-19 Pandemic and validity of RT-PCR as a predictor for COVID- 19 disease

Methods: The data was collected essentially by using secondary data. All cases and deaths in WHO Situation Reports and total tests in Worldometer were included in the study. Wolfram Player 12 software was used for the Susceptible Infected Recovered (SIR) epidemic dynamics of COVID-19. Survival analysis was carried out to determine the cumulative proportional survival of COVID-19 in Egypt. Six studies discussing the validity of RT-PCR was also reanalyzed. Receiver Operating Characteristic (ROC) curve analysis was used to study the diagnostic performance of RT-PCR.

Results: There was a negative correlation between both case fatality rate of COVID-19 and reproductive rate with RT-PCR tests performed . This difference is significant (r = - 0.307 and. – 0.361) respectively. RT-PCR had a sensitivity of 61.19%, Specificity 94.75%% and an accuracy of 76.72%. The area under the ROC (AUC) for RT-PCR was 0.780.

Conclusion and recommendation: RT-PCR testing will continue to be needed. It reduced the case fatality and reproduction rates of COVID-19 Pandemic. The AUC for RT-PCR is less than optimal. The combination of clinical symptoms, exposure history and CT must be considered to identify COVID-19 with higher sensitivity.

Introduction

For centuries, humanity has been marked by adversity in the pursuit of survival. Wars, famine and the climate are some of main challenges for humanity’s progress and survival. However, no other factor brings so much fear to society as epidemics. (1) The Coronavirus pandemic is the defining global health crisis of our time. Since its emergence in late 2019, the virus has spread to - continent Efforts to completely contain the COVID –19 the pandemic responsible for infecting more than six millions with the disease and more than three and half hundred thousand deaths - have failed.(2, 3)

Testing for the presence of or past exposure to the SARS-CoV-2 virus is an essential aspect of combatting the COVID-19 outbreak and the associated public health crisis. Wider testing is urgently needed for successful pandemic control. For effective management of the different stages of the pandemic, it is vital to understand first what information different tests can deliver, i.e. what is the intended purpose of a given test, and second the level of a test’s performance, i.e. how well it is able to achieve that purpose.

Evaluation of diagnostic tests is a matter of concern in modern medicine not only for confirming the presence of disease but also to rule out the disease in healthy subjects. Given the attention on COVID-19 testing and its role in helping to halt the spread of COVID-19 pandemic, public heath personnel always want to know they can trust the accuracy of lab test results, this has never been truer than it is now. (4, 5)    Laboratory tests are characterized by their ability to detect a positive case (sensitivity) and their ability to determine a negative case (specificity). So a sensitive test is less likely to provide a false-negative result and a specific test is less likely to provide a falsepositive result. As COVID-19 testing becomes more widely available, it's vital that health care providers and public health officials understand its limits and the impact false results can have on efforts to curb the pandemic. (6) It is important to note the sensitivity of reverse-transcription polymerase chain reaction test (RT-PCR) testing as described, is less than optimal. It is the most common diagnostic test used to identify people currently infected with SARS-CoV-2 and  works by detecting viral RNA in a person’s cells – most often collected from their nose ( 7 ) . A single negative RT-PCR should not exclude COVID-19, especially if clinical suspicion is high. (8)

As there is no consensus on how accurate our testing is, and given the potential for asymptomatic carriage and prolonged viral shedding post-infection, no test is perfect, and we are all learning together on this one. In addition since the test is new, its performance needs to be compared to the performance of a current “gold-standard” test, also known as the “reference standard  .There currently is no gold-standard diagnostic test for SARS-CoV-2 since the virus is new to us. We are moving between two stones missing cases by RT-PCR testing to be treated or putting the sensitivity value of CT chest in the isolation room and look beyond the numbers. (9, 10) 

The study was carried out with the aim to study the impact of RT-PCR testing in control of COVID-19 Pandemic and validity of RT-PCR as a predictor for COVID- 19 disease.

Material

The data was collected essentially by using secondary data. All cases and deaths in the situation reports of WHO (6)   and the total tests   and tests per million in Worldometer on 27th of May 2020 were also included in the study (18).  The case fatality rate in this study was calculated as deaths among COVID19 cases divided by the total number of cases with defined outcome (either died or cured). Survival   analysis was carried out to determine the cumulative proportional survival of COVID-19.  Wolfram Player 12 software was used for the Susceptible Infected Recovered (SIR) epidemic dynamics of COVID-19. (10)

The following SIR model was considered: where S is the fraction of susceptible individuals, I is the fraction of infectious individuals, and R is the fraction of recovered individuals, β is the transmission rate per infectious individual, and γ is the recovery rate. β was calculated by the equation: C= βSI – Iγ. Where C is the incidence of cases in the second day, S is the susceptible population, I is the number of infectious cases and γ is the recovery rate. The recovery rate was calculated by dividing the recovered cases by the reported cases. Note that the basic reproduction number (R0) is R =β/γ. The infectious period is exponentially distributed with a mean 1/γ.  

In addition reanalysis of data collected from six studies discussing the validity of RT-PCR in and outside China (11, 12, 13, and 14). The   study analyzed pooled 2418 hospitalized patients with suspected COVID-19 patients undergone both serial RT-PCR testing and chest CT. Every case was represented by a row. Positive cases in RT-PCR testing and chest CT was coded as 1 and negatives by 0. IBM SPSS Statistics, 25th edition was used for entering, merging and analysis of data (15). MedCalc was also used to calculate sensitivity, specificity and accuracy of the diagnostic tests.(16)

The Receiver Operating Characteristic (ROC) curve analysis was used to study the diagnostic performance of RT-PCR as a predictor of COVID -19 and discriminate positive cases from negative ones A discriminatory power of 0.9 of 0 or more is considered an excellent performance, while 0.80-0.89 is considered as a good performance and 0.70-0.79 is considered as fair discriminatory performance. (Zweig and Campbell, 1993). (17)  

Results

The total number of RT-PCR tests performed globally until 27th May was 76813201.0 with a mean of 26189.32 per million. There was a negative correlation between case fatality rate of COVID-19 and reproductive rate with tests performed   per million population. This difference is significant (r = - 0.307 and – 0.361) respectively, Table (1). The same pattern was observed in studying the rates in USA, Brazil, Russia, Germany, Italy, Egypt, Saudi Arabia and UAE with no significant difference (Spearman Correlation coefficient: TPM with CFR = - 0.167, TPM with RR = - 0.419), Table 2 and Figure (1).

 

As for Egypt, the total number of tests was 135000 with 1321 test per million. No mass testing was applied in Egypt. The total number of cases by 27 th May in Egypt was 910 with 19 deaths For calculating the case fatality rate, survival analysis was used where new cases were added, deaths and recovered were subtracted. Table (3) Shows that the cumulative proportional survival of COVID-19 was 0.88. This means that 88% of the active cases of COVID -19 at 27th May in Egypt will survive by end of the epidemic. Figure (2) shows the SIR Epidemic Dynamics of COVID-19 in 27 th May.

Validity of RT-PCR Test

Table (4) shows that out of the 2418 suspected cases of COVID-19, were true positive cases were 790, 58 false positive, 505 false negative and 1065 cases were true negatives. As there is currently no gold-standard diagnostic test for SARS-CoV-2,   CT  was considered as reference test in this study, Table (5) shows the summary characteristics of RT-PCR where sensitivity was 61.19%, Specificity 94.75%% and accuracy 76.72%.

Figures (3) show that the Area under the ROC curve (AUC) of RT-PCR as a predictor of COVID -19 of the studied group. It appears from the figure that the AUC was 0.780 (0.760- 0.799). The area under the ROC curve was significantly different from 0.5 (null hypothesis area) and that therefore there is evidence that RT-PCR has a fair ability to distinguish between the two positive and negative groups (P (Area=0.5 was<0.000).

Discussion

COVID-19 disease is associated with three major patterns of clinical course of infection; mild illness with upper respiratory tract presenting symptoms, non-life-threatening pneumonia and severe pneumonia with acute respiratory distress syndrome (ARDS). Given that the manifestation of COVID-19 infection is highly non-specific, diagnostic tests specific to this infection are crucial and urgently needed to confirm suspected cases, screen patients, and conduct virus surveillance. (18)

RT-PCR is accepted by scientists and medical staff as a robust and well documented technique. With RT-PCR being so common in research and medicine, the technology is already in place to test for COVID-19. RT-PCR can detect current infections of disease, allowing medical staff to determine who is currently infected and who is not. But it should be remembered that RT-PCR relies on capturing and detecting the virus and so it is possible to miss patients who have cleared virus and recovered from disease. In addition the distribution of virus across the respiratory tract varies between patients, so even if a person is infected, the virus may only be detectable in sputum or nasopharyngeal swab but not necessarily at both locations at the same time. RT-PCR for COVID-19 can only tell if a person is currently infected with this coronavirus. It can’t provide information on other diseases or symptoms particular. (19)

Many countries in the world   did not underestimate the problem since the first reported cases, knowing well that COVID-19 is inevitable and started to deal early, to limit transmission and increase recovery. Social distancing was paired with the basic epidemiology that’s needed. Contact tracing — the practice of identifying and testing every person that an infected person came into contact with after they themselves contracted the virus — has been prioritized. Almost all efforts to develop the infrastructure for quarantining the exposed or isolating the infected persons (20). Epidemiological testing — where the contacts of infected people are identified, tested in turn and isolated as needed — is the only way to fully break the chains of transmission. Without it, the virus will come roaring back as soon as social distancing guidelines are relaxed. The timing of conducting the tests however is crucial to avoid false-negative results. Both positive and negative results must be utilized in conjunction with clinical observations, patient history, and epidemiological information

ROC analysis has become a popular method for evaluating the accuracy of medical diagnostic systems. It is used in clinical epidemiology to quantify how accurately medical diagnostic tests (or systems) can discriminate between two patient states, typically referred to as "diseased" and "no diseased" (5, 7, 20, and 21). The most desirable property of ROC analysis is that the accuracy indices derived from this technique are not distorted by fluctuations caused by the use of arbitrarily chosen decision criteria or cut-offs. In other words, the indices of accuracy are not influenced by the decision criterion (i.e. the tendency of a reader or observer to choose a specific threshold on the separator variable) and/or to consider the prior probability of the "signal" (5). The derived summary measure of accuracy, such as the area under the curve (AUC) determines the inherent ability of the test to discriminate between the diseased and healthy populations (7). Using this as a measure of a diagnostic performance, one can compare individual tests or judge whether the various combination of tests can improve diagnostic accuracy. In addition one can easily obtain the sensitivity at specific FPF by visualizing the curve and   optimal cut- off value can be determined using ROC curve analysis (7).

In summary, despite the fantastic feature of ROC analysis in diagnostic test evaluation and the meaningful interpretation of AUC and its asymptotic properties, a proper design with broad spectrum of case and control and avoidance of bias and control for confounding are necessary for a valid and reliable conclusion in the assessment of performance of diagnostic tests. Spectrum and bias should be considered with careful consideration in study design while confounding can be controlled in analysis as well. While the adjustment of confounding is widely used in etiologic studies in epidemiology, a little attention has been focused for the control of confounding in ROC analysis of medical published diagnostic studies. (5)

Regarding performance of tests in the context of population testing, there are drawbacks both from insufficient diagnostic sensitivity (e.g. leading to missing infected individuals) and insufficient diagnostic specificity (e.g. imposing confinement measures on individuals who are not true positives). This needs to be taken into account along with the stage of the pandemic in a particular population. For example, in the control stage it may be particularly important to identify positive cases with a high level of specificity (i.e. distinguishing COVID-19 from other similar but less dangerous diseases) to avoid unnecessary burden on the healthcare system. In contrast, in the deescalation stage, sensitivity (detecting all remaining infected individuals) could be more important than specificity to make sure the disease is indeed contained. It is also important to take account of the features of the population in which the test is intended to be used, for example whether the prevalence of infection is expected to be low or high, or whether there are local virus variants. Scarcity of reference methods and materials poses difficulties for these validation studies, and also for the evaluation of device performance by manufacturers.

Conclusion and recommendation

PCR testing will continue to be needed. It is important to note that the sensitivity of RT-PCR testing, is less than optimal. A single negative RT-PCR should not exclude COVID-19, especially if clinical suspicion is high. The combination of clinical symptoms, exposure history and CT must be considered to identify COVID-19 with higher sensitivity.

Declarations

Competing interests: The authors declare no competing interests.

References

  1. Erivelton G. N., Denise F R and Márcio J L. A Survey of the Individual-Based Model applied in Biomedical and Epidemiology. J Biomed Res Rev 2018. Volume 1:
  2. Wang C, Hornby PW, Hayden FG, Gao GF. A novel coronavirus outbreak of global health concern. Lancet. 2020; 395(10223):470-3. Doi: 10.1016/S0140-6736(20)30185-
  3. COVID-19 coronavirus pandemic. United States: Dadax; 2008 [updated 2020; cited 2020 May 15]. Available from: https://www.worldometers.info/ coronavirus/.
  4. Torn Walley. Evaluating laboratory diagnostic tests BMJ. 2008 Mar 15; 336(7644): 569–570
  1. Karimollah Hajian-Tilaki. Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation Caspian Journal of Internal Medicine.Vols. 2 to 11; 2011 to 2020 
  2. Swets JA. Indices of discrimination or diagnostic accuracy: their ROCs and implied models. Psychol Bull. 1986; 99:100–17. [PubMed] [Google Scholar]
  3. Carver C, Jon. es N. Comparative accuracy of oropharyngeal and nasopharyngeal swabs for diagnosis of COVID-19. Oxford COVID-19 evidence service team centre for evidence based medicine. 2020.,
  4. Metz CE. Basic principles of ROC analysis. Semin Nucl Med. 1978; 8:283–98. [PubMed] [Google Scholar]
  5. COVID-19 and chest CT: do not put the sensitivity value in the isolation room and look beyond the numbers.  https://pubs.rsna.org/author/Kwee%2C+Robert+M
  6. American Society for Microbiology False Negatives and Reinfections: the Challenges of SARS-CoV-2 RT-PCR Testing. April 27, 2020. corona%20virus/Falsenegative%20COVID19%20test%20results%20may%20lead%20to%20false% 20sense%20of%20security%20--%20 ScienceDaily.html 
  1. Tao Ai*, Zhenlu Yang*, Hongyan Hou, Chenao Zhan,  Chong Chen,  Wenzhi Lv,  Qian Tao, Ziyong Sun,  Liming XiaCorrelation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases.https://pubs.rsna.org/doi/10.1148/radiol.2020200642
  1. COVID-19 and chest CT: do not put the sensitivity value in the isolation room and look beyond the numbers.  https://pubs.rsna.org/author/Kwee%2C+Robert+M
  2. He JL, Luo L, Luo ZD, Lyu JX, Ng MY, Shen XP, Wen Z Diagnostic performance between CT and initial real-time RT-PCR for clinically suspected 2019 coronavirus disease (COVID-19) patients outside Wuhan, China .Respir Med. 2020 Apr 21;168:105980. doi: 10.1016/j.rmed.2020.105980
  1. Chuanhong Wanga, Bingliang Zenga, Zicong Lia, Xiaofen Lia, Honglu Lia.Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT? .European Journal of Radiology.16 (2020)
  2. IBM Corp. Released 2017. IBM SPSS Statistics for Windows VA, NY: IBM Corp.
  3. MedCalc’s Diagnostic test evaluation calculator https://www.medcalc.org/calc/diagnostic_test.php
  4. Zweig MH1, Campbell G.Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993 Apr; 39(4):561-77.
  5. GENOAMP® REAL-TIME PCR TESTS FORDETECTION OF COVID-19 http://ww.moh.gov.my/moh/resources/penerbitan/mymahtas/MaHTAS%20COVID19%20Rapid%20Evidence
  6. Mandic S1, Go C, Aggarwal I, Myers J, Froelicher VF. Relationship of predictive modeling to receiver operating characteristics.
  7. Kile Green, Amanda Winter, Rachel Dickinson, Sara Graziadio, Robert Wolff, Susan Mallett and A Joy Alle. What tests could potentially be used for the screening, diagnosis and monitoring of COVID-19 and what are their advantages and disadvantages. April 1920
  8. Mayo Clinic. False-negative COVID-19 test results may lead to false sense of security April 9, 2020. corona%20virus/False-negative%20COVID19%20test%20results%20may%20lead%20to%20false%20sense%20of%20security%20--%20ScienceDaily.html