Assessment of 12 Quantitative RT-PCR Commercial Kits for the Detection of SARS-CoV-2

The emergence of the severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) in the nal months of 2019 had the health ocials declare a public emergency raising a global response. In response to the burden of the current pandemic, strict measurements were globally implemented to stop further transmission of the virus. These Measurements rely on an accurate diagnosis of individuals infected with the virus by using real-time reverse transcriptase PCR (RT-PCR). The goal of our study is to relate the fundamental clinical and analytical performance of chosen kits of RT-PCR from distinct manufacturers. Methods A total of 94 nasopharyngeal and oropharyngeal clinical samples were selected randomly, these samples were previously conrmed as 64 positives, and 30 negatives for SARS-COV-2. Generally, 400 µl of each respiratory specimen was subjected to extraction using ExiPrep 96 Viral RNA Kit with the ExiPrep 96 Lite Automated NA Purication System. All kits master mix preparation, cycling protocol, and results interpretation were carried out according to the manufactures’ instructions of use and recommendations. In our study, we were able to evaluate the performance of 12 commercial kits in detecting SARS-COV-2 using 5 different targets. The performance of the kits was comparable except for LYRA kit as it was less sensitive (F=67, P <0.001). Particularly, the performance of RealStar, BGI, DiaPlexQ, Genesig, LightMix, TaqPath, and RADI kits were the most sensitive. Moreover, the performance of these commercial kits by gene target showed no signicant change in Ct values which indicates that kits disparities are mainly linked to the choice of the gene target (F=0.49, P=0.73). We believe that most of the commercially available RT-PCR kits included in this study can be used for routine diagnosis of SARS-COV-2 patients. Moreover, we recommend that regardless of the laboratory choice of diagnostic commercial kit for the clinical detection of COVID-19 patients the need a for good plan for validation and collaboration with exterior laboratories is essential in order to monitor the virus changes overtime, procedures, technicians, and the different kits performances. our chosen kits were CE-IVD certied and can be produced in large amounts. A concise panel of ninety-four clinical samples was used to evaluate the performance of these commercial kits. The goal of our study was to relate the fundamental clinical and analytical performance of chosen kits of RT-PCR from distinct manufacturers. The manufacturers enrolled in our assessment were TIB MOLBIOL, Altona Diagnostics, Thermo Fisher Scientic, Solgent, Quidel, BGI, OPTOLANE Technology, Kogene Biotech, Sansure Biotech, Novacyt/Primer design, GeneReach Biotechnology, and KH Medical. The 12 commercial kits amplied 5 unique targets in SARS-CoV-2 genome, including: N, E, S, RdRp, and ORF1ab/ PP 1ab genes. This study will provide an abundant assessment on the detection of SARS-COV-2 by using different kits and targets to show the importance of accurate testing during these dicult times.


Introduction
The emergence of a novel coronavirus, the severe acute respiratory syndrome coronavirus 2 (SARS-COV-2) in the nal months of 2019 had the health o cials declare a public health emergency raising a global response [1]. The new virus (SARS-COV-2) has been identi ed as a new strain of Betacoronvirus from group 2B with approximately 70% genetic similarity to the severe acute respiratory syndrome coronavirus (SARS-COV) [2]. Coronaviruses are positive-sense RNA viruses that express their replication, transcription, and their RNA-dependent RNA polymerase (RdRp) through single, large open reading frame (ORF1ab) [3]. Several structural proteins are found within the virus such as envelope (E), nucleocapsid (N), and spike (S) proteins. In response to the burden of the current pandemic, strict measurements were globally implemented to stop further transmission of the virus. These Measurements rely on an accurate diagnosis of individuals infected with the virus by using real-time reverse transcriptase PCR (RT-PCR) [4]. The most common targets for detecting SARS-COV-2 by (RT-PCR) assays in diagnostic laboratories are the ORF1ab gene, the RdRp gene, the E gene, the N gene, or the S gene [5].
The SARS-CoV-2 pandemic caused a huge burden on the societal, nancial, and healthcare systems in the sphere, and various measures are developed to control its spread. Most of the control measures mainly depend on the precise testing of the individuals infected by the virus. Real-time reverse transcription-polymerase chain (RT-PCR) method of detection is the most common and reliable test in detecting viral genome, therefore the world health organization (WHO) has recommended the use of this method as the gold standard during the current time [6]. Many SARS-CoV-2 RT-PCR diagnostic kits are commonly available in the market, however, independent examination of these products is not available in the public sector and is required to give directions of implementation to accurately test [7].
In the Saudi Center for Disease Prevention and Control (SCDC) Laboratories, we have performed an evaluation of twelve RT-PCR SARS-CoV-2 commercial kits from various manufacturers. Most of our chosen kits were CE-IVD certi ed and can be produced in large amounts. A concise panel of ninety-four clinical samples was used to evaluate the performance of these commercial kits. The goal of our study was to relate the fundamental clinical and analytical performance of chosen kits of RT-PCR from distinct manufacturers. The manufacturers enrolled in our assessment were TIB MOLBIOL, Altona Diagnostics, Thermo Fisher Scienti c, Solgent, Quidel, BGI, OPTOLANE Technology, Kogene Biotech, Sansure Biotech, Novacyt/Primer design, GeneReach Biotechnology, and KH Medical. The 12 commercial kits ampli ed 5 unique targets in SARS-CoV-2 genome, including: N, E, S, RdRp, and ORF1ab/ PP 1ab genes. This study will provide an abundant assessment on the detection of SARS-COV-2 by using different kits and targets to show the importance of accurate testing during these di cult times.

Samples selection
This study encompasses 12 commercial RT-PCR COVID-19 detection kits available in the market, sent by the manufacturers free of charge to the Saudi Center for Disease Control and Prevention (SCDC) for evaluation. None of the manufacturers were involved in the assessment or data analysis. A total of 94 nasopharyngeal and oropharyngeal clinical samples were selected randomly, these samples included 64 previously con rmed samples and 30 samples con rmed as negatives for SARS-COV-2. The ethical approval was obtained from our institution the Saudi CDC Number: (SCDC-IRB-A012-2020).

Samples preparation and RT-PCR procedure
Generally, 400 µl of each respiratory specimen was subjected to extraction using ExiPrep 96 Viral RNA Kit, and ExiPrep 96 Lite Automated NA Puri cation System (Bioneer, Korea). The nucleic acid (RNA) extraction process has been performed twice (with 100 µl elution volume for each) then the extracted RNA was pooled and stored at -80 C until use. This extraction instrument was chosen due to its ability to provide su cient elution volume to process 10 different assays and its availability in all Saudi's regional laboratories. No exogenous internal controls were added to the extraction or the master mix. All kits master mix preparation, cycling protocol and results interpretation were carried out according to the manufactures' instructions of use and recommendations (Details of the compared kits and instruments used are summarized in Table 1). When the result was inconclusive or invalid, as per the manufactures' results interpretation, retesting was performed, and no PCR inhibition noted spotted.

Data analysis
For Statistical analysis, Data was collected and analyzed using GraphPad Prism (San Diageo California, USA), version 8.4. Descriptive analysis was done on the reported Cycling Threshold (Ct) and results were compared by the commercial RT-PCR kits and targets. ANOVA test was used to detect the signi cance of the Ct values reported by the commercial RT-PCR kits and targets. Boxplots and bar graphs were used to show the distribution of CT values and detection results by the different commercial kits. All P values reported are 2-sided and were considered to be statistically signi cant at alpha < 0.05.

The summary of the SARS-COV-2 Detection by multiple kits
The 94 samples were all processed, and the results were interpreted as recommended by each of the kits' manufacture recommendations. Kits with single targets were reported positive if their cycling threshold (CT) were below the cutoff value. All samples were tested by all kits except for the RADI, we only had enough for 55 samples, therefore we selected the enrolled samples randomly. The summary of the detection results is shown in

Quantitative analysis of the positive samples by Targets
By target, commercial kits showed similar performance and reported comparable CT values. The commercial kits had 5 different target genes which were N, E, S, RdRp, and ORF1ab/ PP1ab. The results of the ANOVA test showed that by target the CT values of the commercial kits were similar (F = 1.1, P < .05). Figure 3 shows the distribution of CT values by the target. Overall, the lowest CT values were mostly reported with kits targeting the N gene. The highest CT values were mostly reported with kits targeting the RdRp gene. Table 2 shows the overall CT summary reported by kits targeting the ORF1ab gene which were BGI, Sansure, IQ REAL, DiaPlexQ, LYRA, and TaqPath. For the ORF1ab gene, the quantitative analysis showed a similar result across the different kits (ANOVA, F = 1, P value = 0.39). Table 3 shows the overall CT summary reported by kits targeting the RdRp gene which were KAIRA, LightMix, Genesig, RADI, and PowerCheck. For the RdRp gene, the kits showed a signi cant difference in CT values reported by the kits (ANOVA F = 2, P value = 0.03). The change was detected with the Genesig kit which reported distinctive CT values compared to KAIRA, LightMix, and PowerCheck. Table 4 shows the overall CT summary reported by kits targeting the E gene which were reported with KAIRA, LightMix, RealStar, and PowerCheck. For the E gene results, the kits showed a signi cant difference in CT values reported by the kits (ANOVA, F = 20, P value < 0.001). The change was detected with KAIRA and RealStar CT values which were different compared to LightMix, PowerCheck, and each other. Table 5 shows the overall CT value summary by kits targeting the N gene which were reported with TaqPath, DiaPlexQ, and Sansure. For the N gene CT values' quantitative analysis, the kits showed no signi cant difference in CT values reported by the kits (ANOVA F = 1.27, P value = 0.27). Similar results were reported with the S gene as shown in Table 6, no signi cance was detected (ANOVA, F = 1.75, P value = 0.18).

Discussion
During the current stage of the COVID-19 pandemic, many testing kits were developed and available commercially, in our study we were able to evaluate the performance of 12 commercial kits in detecting the SARS-COV-2 virus. The performance of the kits was comparable except for LYRA kit as it was less sensitive. Particularly, the performance of RealStar, BGI, DiaPlexQ, Genesig, LightMix, TaqPath, and RADI kits were the most sensitive.
These kits had more than one target except for Genesig and BGI. This nding indicates the importance of having a con rmatory gene to ensure the speci city of diagnostic testing. Moreover, the performance of these commercial kits by gene target showed no signi cant change in Ct values which indicates that kits disparities are mainly linked to the choice of the gene target. From the 5 targets, ORF1ab/PP1ab, S, and N genes reported similar results by the different kits, however, RdRp and E genes showed signi cant differences by the reported Ct values. In consideration of time, the shortest tests were KAIRA, LightMix, and RADI with 60 minutes, however, these kits did miss few positive samples.
Indeed, other studies evaluating the performance of RT-PCR kits are in concordance with our results [8]. Comparing commercial kits showed similar results in detection, however, different targets did show variation in CT values. Moreover, kits with multiple targets such as Realstar, Taqpath, LightMix, and Sansure in another evaluation study did show higher sensitivity and specify than with other kits detecting a single target [9]. In our study, we didn't evaluate the limit of detection (LOD) which can play a big rule, as with several of the commercial kits approved for the pandemic situation many were approved without evaluation with appropriate numbers of samples [10]. Another important variation source in many PCR kits results that we didn't manage to evaluate is variation in instrument and technicians, in a multiple center euro surveillance study, the same commercial kits showed variation in results between different centers, many of these centers had different instruments and technicians [11].
One of the biggest challenges in the diagnosis of SARS-COV-2 in clinical settings is the high rate of false-negative cases. As known with RNA viruses the mutation rate is high compared with other pathogens. Moreover, the nucleotide mutation rate reported for SARS-COV-2 was 8E-04 substitution per site per year [12,13]. Many studies have already shown the high evolution rate of SAR-COV-2 in many cases and linked its evolution with its origin [5]. Detection methods using RT-PCR are based on a xed target, however, as the pandemic proceed the virus is changing and many cases are missed. In one study, evaluating multiple RT-PCR targets has shown a high loss of sensitivity, with a total of 11,627 cases missed due to variations in genetic code [14]. In one of the biggest studies worldwide tracking the mutations of SARS-COV-2, one mutation D614 was decreasing and another mutation G614 mutation was growing, the new mutation has lower Ct values which indicate a change in virus virulence [15]. In a genomic surveillance study, a rise in mutation located in the ORF gene was linked to the Middle East SARS-COV-2 cases [16]. Overall these studies indicate the rise of genetic mutations in SARS-COV-2 globally. This rise will cause many positive cases to be missed in the RT-PCR, which will cause a challenge with using these xed commercial RT-PCR kits. In one case for a systematic SARS-COV-2 case, the patient was negative in many RT-PCR tests, however he was only positive by the antibody test, moreover, the case was later investigated with sequencing and the virus infecting the patient had two major mutations one located at the NP genes and another located at the ORF gene [17 ].The rise of these different mutations in SARS-COV-2 at multiple geographical locations indicates the need for genetic screening periodically in each country to count for these changes as they may signi cantly play a role in choosing the commercial diagnostic kit used at the testing centers.
We are aware that our study may have several limitations. The rst one is that we were not able to study the cross-reactivity of these commercial kits with other viruses which can signi cantly alter the results of the detection. Even though our conducted study implemented a comparative evaluation it is still not a comprehensive study as we could not include all RT-PCR diagnostic kits available at the market. In our future work we hope to evaluate genetic changes in the virus and its effect on detecting the virus by the available commercial kits.

Conclusion
In conclusion, we believe that most of the commercially available RT-PCR kits included in this study can be used for routine diagnosis of SARS-COV-2 patients. Most of the kits succeeded in detecting the virus, however few distinctions were found with speci c kits and targets. Kits with multiple targets have shown to be the more sensitive and speci c as they counted for the target variations. Moreover, we recommend that regardless of the laboratory choice of diagnostic commercial kit for the clinical detection of COVID-19 patients the need a for good plan for validation and collaboration with exterior laboratories is essential in order to monitor the changes in the virus, procedures, technicians, and the different kits

Consent to publication
All authors have reviewed the nal version of the manuscript and approve it for publication.

Availability of data and materials
Data available on request from the authors.

Declaration of Competing Interest
Not Applicable.

Funding statement
This work was funded by Saudi CDC.The RT-PCR kits included in this study were provided free of charge. Table 1 Overview of the SARS-COV-2 detection kits encompasses in this study.