Viral RNA load quanti � cation as helpful tool to arbitrate SARS – CoV-2 detection results in respiratory samples

Flora Marzia Liotti Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy Giulia Menchinelli Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy Simona Marchetti Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy Grazia Angela Morandotti Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy Maurizio Sanguinetti (  maurizio.sanguinetti@unicatt.it ) Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy https://orcid.org/0000-0002-9780-7059 Brunella Posteraro Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy Paola Cattani Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy


Introduction
Since rst isolation on December 2019 [1], the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) -initially called 2019-nCoV-which causes the illness referred to as coronavirus disease 2019 (COVID-19) has increasingly spread worldwide. By 29 April 2020, the number of con rmed cases reported by the World Health Organization (WHO) had reached 3,023,788 (https://covid19.who.int/), hence representing an unprecedented viral pandemic. To prevent virus transmission and/or ensure appropriate management of COVID-19 patients [2], clinical microbiology laboratories are constantly requested to implement relatively quick and sensitive diagnostic assays for SARS-CoV-2 RNA detection in clinical samples [3].
Nowadays, real-time reverse-transcription-polymerase chain reaction (RT-PCR)-based assays performed on upper respiratory tract (URT) samples (e.g., nasopharyngeal and/or oropharyngeal swabs) is the current diagnostic strategy to con rm COVID-19 cases [4], regardless of clinical disease manifestation [5]. In general, diagnosis relies upon the in vitro ampli cation of one or more molecular targets within the positive-sense, single-stranded SARS-CoV-2 RNA, including the envelope (E), RNA-dependent RNA polymerase (RdRP), and nucleocapsid (N) genes among others [6,7]. In particular, the assay developed by the Centers for Disease Control and Prevention (CDC)-the most widely used in the United States-utilizes two N gene regions (N1 and N2) as targets [4].
As soon as the WHO published protocols for RT-PCR assays [8], Seegene launched the Allplex™ 2019-nCoV assay-approved for emergency use authorization (EUA) from U.S. Food and Drug Administration (FDA) on 21 April 2020. This single-tube assay identi es E, RdRP, and N genes, as established by the WHO (https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/laboratory-guidance). Later, DiaSorin Molecular developed the Simplexa™ COVID-19 Direct assay, for which the FDA granted a EUA on 19 March 2020 [4]. The assay targets two regions within the SARS-CoV-2 genome, one encoding the spike (S) protein (i.e., the S gene) and the other well-conserved non-structural proteins (i.e., the open reading frames ORF1a and ORF1b) of SARS-CoV-2. Remarkably, both assays received CE (Conformité Européenne) marking. Compared to the Allplex™ 2019-nCoV assay, the Simplexa™ COVID-19 Direct assay has the advantage of quicker turnaround test results (75 minutes vs 4 hours, respectively). However, the true sensitivity of currently available assays is unknown [9]. In particular, few studies so far have compared the results obtained with different commercial assays in routine laboratory practice [10][11][12].

Study design and samples
This retrospective study was performed on NOS samples collected from patients admitted to the Fondazione Policlinico Universitario A. Gemelli (FPG) IRCCS hospital's emergency department with COVID-19 suspicion during a two-week period in May 2020. NOS samples were collected together within a single tube of universal transport medium (UTMâ; Copan Italia S.p.A., Brescia, Italy) to prevent viral RNA degradation and/or bacterial/fungal overgrowth. We considered all samples tested for SARS-CoV-2 RNA by the Allplex™ 2019-nCoV assay (see below) eligible for inclusion. Among SARS-CoV-2 positive samples, we randomly selected samples that were representative of differing target(s) positive levels, as assessed by their cycle threshold (C T ) values (see below). We also selected negative samples to reach a number of 125 samples in total. Aliquots of primary samples were immediately frozen and kept at -70°C until further analysis. Before testing, aliquots were thawed at room temperature and brie y vortexed.

SARS-CoV-2 molecular detection
Testing of NOS sample aliquots using SARS-CoV-2 molecular assays were performed in accordance with the manufacturer's instructions.
Allplex™ 2019-nCoV assay Brie y, 200 μl of sample was processed with a Seegene Nimbus automated system (Arrow Diagnostics), which performs both RNA extraction-using STARMag Universal Cartridge kit-and PCR assay setup. A reaction microplate with therein-extracted RNA was loaded onto a real-time PCR CFX96 Touch™ system (Bio-Rad Laboratories, Hercules, CA, USA). Positive and negative controls were included in each run. After assay's completion, the Seegene Viewer 2019-nCoV software allowed automated analysis and interpretation of results. A positive result (i.e., a C T less than 40) for at least one of two viral targets (i.e., RdRP and N genes) or for the E gene alone indicates, respectively, the certain or presumptive presence of SARS-CoV-2 RNA in the patient sample. An invalid result (e.g., due to internal control failure) indicates inconclusive determination of the SARS-CoV-2 RNA presence or absence in the patient sample, thus requiring sample retesting.
Simplexa™ COVID-19 Direct assay Brie y, 50 µl of sample and 50 µl of reaction mixture were separately loaded into Direct real-time PCR ampli cation-disc wells and onto a LIAISON® MDX instrument (DiaSorin Molecular) and allowed to react for a 75 min run. Positive and negative controls were included in each run. After assay's completion, the instrument's Studio software automatically calculated and displayed results. A positive result (i.e., a C T less than 40) for at least one of two viral targets (i.e., S and ORF1ab genes) indicates the presence of SARS-CoV-2 RNA in the patient sample. As with the Allplex™ 2019-nCoV assay, an invalid result requires sample retesting.
Quanty COVID-19 assay Brie y, separate wells on each real-time PCR microplate was lled with 5-µl derivative sample (i.e., derived from the Nimbus RNA extraction step), positive control, negative control, and standards in. For SARS-CoV-2 RNA qualitative detection, the instrument's software automatically analyzed and interpreted the results. A positive result (i.e., a C T less than 40) for all three viral targets (N1, N2, and N3 genes) indicates the presence of SARS-CoV-2 RNA in the patient sample. Otherwise, the software de nes the result as inconclusive, requiring sample retesting. For SARS-CoV-2 RNA quantitative detection, the software built a standard curve with the C T values obtained following ampli cation of the aforementioned standards (which contain 10 1 , 10 2 , 10 3 , 10 4 , and 10 5 copies/μl of synthetic viral N1-encoding RNA, respectively). This allowed calculating the viral load in the patient sample by interpolation of the corresponding C T value with the standard curve. Then, the actual viral load of the sample (expressed in copies/ml) was determined multiplying the calculated number of viral copies by 1000/V e and E v /E a ratios, where V e is the extracted sample volume (200 µl), Ev is the eluted sample volume during the extraction step (100 µl), and Ea is the extracted sample volume used for ampli cation (5 µl). To validate the manufacturer's standards, we generated a standard curve using the Quantitative Synthetic SARS-CoV2 RNA: ORF, E, N (ATCC® VR3276SD™), which was diluted at the same concentrations as the standards used in the Quanty COVID-19 assay. In preliminary experiments, each of the ATCC® VR3276SD™ RNA samples was quanti ed in triplicate with the Quanty COVID-19 assay, and results were in the expected C T values ranges (data not shown).

Data analysis
No sample retesting was performed due to the absence of invalid results; consequently, we analyzed the rst testing results for all study samples. We calculated sensitivity, speci city, and positive and negative predictive values, together with their respective con dence intervals (CIs), for the Allplex™ 2019-nCoV assay and Simplexa™ COVID-19 Direct assay using the Quanty COVID-19 assay as an arbitrator. Analysis was performed with Stata software version 11.1 (StataCorp, College Station, TX, USA). Differences between the C T values in sample groups were assessed using the Student's t-test. Two-sided P values of <0.05 were considered statistically signi cant. We used Cohen's kappa to assess the strength of agreement between the assays [13]. Values greater than zero indicated none to slight (0.01-0.20), fair (0.21-0.40), moderate (0.41-0.60), substantial (0.61-0.80), or almost perfect (0.81-1.00) levels of agreement, and values lower than/equal to zero indicated the absence of agreement. To assess the relationship between the viral load levels determined by the Quanty COVID-19 assay and the C T values determined by the Allplex™ 2019-nCoV or Simplexa™ COVID-19 Direct assays, we performed a Spearman correlation on all samples where the concentration of the SARS-CoV-2 N1 gene was within a range of 10 1 to 10 7 copies per ml.

Results
To evaluate the performance of the Simplexa™ COVID-19 Direct assay-a quasi-point-of-care SARS-CoV-2 detection assay-we compared the results from 125 cases that rst tested either positive (n = 54) or negative (n = 71) with the Allplex™ 2019-nCoV assay-the rst implemented SARS-CoV-2 detection assay in our laboratory.
As shown in Table 1 (for details, see Table S1 in  The agreement between the Allplex™ 2019-nCoV assay and the Simplexa™ COVID-19 Direct assay results was 93.6% (Table 1). Of eight samples with discordant results, seven samples tested positive with the Allplex™ 2019-nCoV assay (the N gene was detected alone or in combination with E and/or RdRP genes) but negative with the Simplexa™ COVID-19 Direct assay. The remaining one sample tested negative with the Allplex™ 2019-nCoV assay but positive with the Simplexa™ COVID-19 Direct assay (both S and ORF1ab genes were detected). As detailed in Table S1, the mean (± SD) C T value of the N gene in the seven samples with discordant results was 34.7 ± 5.9, and this value differed from that of the 47 remaining Allplex™ 2019-nCoV positive samples (31.2 ± 5.0; P = 0.09). Table 1 Overall results of 125 NOS samples tested by two molecular SARS-CoV-2 detection assays. a Value for the following assays expressed as number (C T range):

Analytic performance of molecular assays
We used the Quanty COVID-19 assay to arbitrate the results of the two molecular assays under comparison. Table S1 provides an overview of testing results. Fifty-ve samples, including Allplex™ 2019-nCoV (n =54) and Simplexa™ COVID-19 (n = 48) positive samples, tested positive and the remaining 70 samples tested negative for all the N gene regions targeted by Quanty COVID-19 assay. In summary, the Simplexa™ COVID-19 Direct assay failed to detect seven positive samples and the Allplex™ 2019-nCoV assay failed for one positive sample.
Interestingly, this one sample tested positive also with the Simplexa™ COVID-19 Direct assay. Tables 2 and 3, sensitivity and negative predictive value (NPV) of the Allplex™ 2019-nCoV assay were 98.2% and 97.2%, respectively, and those of the Simplexa™ COVID-19 Direct assay were 87.3% and 90.9%, respectively. When analyzing the results according to single assay's targets, we found lower sensitivities and NPVs for RdRP (76.4% and 84.3%, respectively) and E (61.8% and 76.9%, respectively) genes in one assay ( Table 2) and for both S and ORF1ab (80.0% and 86.4%, respectively) genes in the other assay (Table 3).  To determine if there was relationship between viral load and C T value, we performed a Spearman's correlation analysis. Before that, samples with C T values ≥40 by the Allplex™ 2019-nCoV assay or the Simplexa™ COVID-19

Relationship between samples' C T values and viral loads
Direct assay were assigned a value of 40. Analyzing all 55 samples that tested positive or negative by the assays, we found a strong (negative) association between the C T values of N (Spearman's ρ = −0.92; P <0.001) and RdRP (ρ = −0.91; P <0.001) genes-detected by the Allplex™ 2019-nCoV assay-and viral loads (Fig. 1).

Discussion
The current speed with which the laboratory-based diagnostic landscape for COVID-19 is changing [3] creates an impelling necessity to assess rigorously the diagnostic accuracy of newly introduced SARS-CoV-2 assays. The DiaSorin Molecular Simplexa™ COVID-19 Direct assay is one of 28 commercially available assays that was EUA granted from the FDA as of 4 April 2020 [4]. One study compared the DiaSorin Molecular assay with the Abbott ID Now assay, using a modi ed CDC assay as the reference standard [10]. Another study compared the DiaSorin Molecular assay with a modi ed CDC Diagnostic Panel, the Diagnostics GenMark ePlex SARS-CoV-2 assay, and the Hologic Panther Fusion SARS-CoV-2 assay [11]. In the latter study [11], the authors used a "consensus result", namely a result obtained by at least three out of four evaluated assays, to establish the reference standard. Both the studies tested URT samples (n = 96 [10] and n = 104 [11], respectively). Independently (albeit concurrently), we assessed the performance of the Simplexa™ COVID-19 Direct assay in comparison with that of the Allplex™ 2019-nCoV assay-one of the rst commercialized assays since SARS-CoV-2 had been isolated for the rst time [1]. Notably, we determined the agreement of the two assays after arbitrating the discordant results with the Quanty COVID-19 assay, the reference assay in our study. Additionally, we used the Quanty COVID-19 assay to quantitate the SARS-CoV-2 RNA (i.e., the N1 gene) in the 125 NOS samples (Table S1) under consideration.
Our ndings show that the Allplex™ 2019-nCoV and the Simplexa™ COVID-19 Direct assays yielded comparable results (Cohen's kappa value, 0.86). However, while there was 100% agreement between the assays for negative samples (speci city was 100% for both assays), discordant results were found in eight positive samples, i.e., one false negative by the Allplex™ 2019-nCoV assay and seven false negative by the Simplexa™ COVID-19 Direct assay (sensitivity was 98.2% and 87.3%, respectively). Consistently, agreement rates of the Allplex™ Unfortunately, we did not perform viral sequencing to clarify this issue [14].
Ideally, these targets should include at least one conserved (e.g., the ORF1ab) and one speci c (e.g., the S gene) region within the viral genome. The Simplexa™ COVID-19 Direct assay as well as the Allplex™ 2019-nCoV assay fully satis es these requirements. In particular, the Seegene assay detects the RdRP (non-structural, conserved) together with N and E (structural, speci c) genes of SARS-CoV-2. This was in line with the WHO recommending a rst-line sample screening with the E gene followed by con rmation with the RdRP gene [8]. From the Allplex™ 2019-nCoV assay's implementation [15] to current use in our laboratory, Seegene modi ed the interpretative criteria, so that positivity for one of three assay targets is now su cient to adjudicate a sample as positive for SARS-CoV-2 RNA. Excluding one sample (negative for all three targets), it is remarkable that in all 54 Allplex™ 2019-nCoV positive samples the N gene was detected. Thus, we are not surprised that the US CDC recommended the N gene as a SARS-CoV-2 assay target alone [16].
As viral dynamics in COVID-19 cases is not fully understood [17], SARS-CoV-2 loads determined by RT PCR assays may not be useful to indicate disease severity [18][19][20]. However, the viral load in a clinical (primarily URT) sample may be an indication of pathogen transmissibility [21], thereby accounting for the widespread person-to-person transmission that also occurs in patients with mild, limited, or no symptoms [2]. Taking advantage of studies showing that lower C T values are inversely related to higher viral copy numbers [19,20,22], we explored the relationship between qualitative and quantitative detection results obtained for the 125 study samples. As expected, we found that viral loads were negatively associated with the C T values of RT PCRs performed with either the Allplex™ 2019-nCoV assay or the Simplexa™ COVID-19 Direct assay. However, we noted a slight difference in the strength of this association between assays, which was in favor of the Allplex™ 2019-nCoV assay.
We and other laboratory scientists acknowledge the importance of comparative evaluation studies to guide implementation and/or correct interpretation of SARS-CoV-2 RNA detection assays. While con rming previously published results (albeit restricted to the Simplexa™ COVID-19 Direct assay) [10,11], we unlike others [11] did not investigate the clinical sensitivity of these assays, but we expanded the general knowledge about performance features of molecular SARS-CoV-2 detection assays, including sample-to-answer platforms [23][24][25], commercially available to date. As assay target genes are present in equal copy numbers in the SARS-CoV-2 genome, assay performance is usually not dependent on the target itself [9]. However, the nding that one molecular target would work better than the other is helpful in redesigning such assays (e.g., shifting from multiple targets to a single target) to enhance reagent utilization [3]. Meanwhile, showing the equivalence of assays may aid to promptly redirect our laboratory choice of RNA-based diagnostic assays towards those with less supply chain trouble at that time [3].
In conclusion, the study showed that the Allplex™ 2019-nCoV assay is equivalent to the Simplexa™ COVID-19 Direct assay for the laboratory-con rmed diagnosis of COVID-19, whereas the Quanty COVID-19 assay allows to maximize diagnosis. Additionally, the Quanty COVID-19 assay providing quantitative data may be useful for SARS-CoV-2 infection monitoring purposes. However, further studies are warranted to de ne the role these assays might play in future clinical practice. Certainly, as testing for COVID-19 increases, these assays or their re nements will contribute to improve the laboratory capacity to identify patients with SARS-CoV-2 infection.
Declarations Figure 1 Correlation between the viral load levels quanti ed by the Quanty COVID-19 assay and the CT values obtained with the Allplex™ 2019-nCoV assay. Values are shown for each SARS-CoV-2 gene (E, RdRP, or N) detected by the assay.