Here, we report on the performance evaluation of a new and faster in vitro diagnostic method in patients of a general hospital following a nosocomial SARS-Cov-2 outbreak. This outbreak was detected in March 2020 at a normal care unit of the department of trauma surgery and geriatrics. COVID-19 was confirmed by RT-PCR assays performed by an external institute and latency of results was up to 9 days at that time. For efficient SARS-CoV-2 screening of every person in our hospital, i.e. all patients and the whole staff, as well as for realtime results we decided to use a point-of-cate device based on LAMP-technology as recommended by others [6, 12].
The potential of LAMP-technology has been demonstrated impressively. Synthesized RNA of SARS-CoV-2 could be amplified to detectable levels in dilutions as low as 2 – 100 copies per reaction [12, 13]. Sensitivity of LAMP in detecting intact viral RNA, that was extracted from cell culture supernatants of isolates from COVID-19 patients, has been reported to be ten-fold lower than that of qRT-PCR (10-7 versus 10-8 dilutions), while specifity was high against all viruses tested [10]. In clinical specimens sensitivity and specifity of LAMP after RNA extraction was 100% and 98.7% – 100%, respectively [10, 14]. In general, nucleic acid-based methods are thought to be sensitive but prone to false positive [15].
After introduction of the variplex test system end of March 2020 in our hospital we rarely observed discrepant results compared to RT-PCR suggestive of false positive. However, negative variplex test results in patients with CT-scans typical for COVID-19 as well as with positive RT-PCR results were noticed. Therefore, since 15th of April we increased sample volume (SLSolution) from 75 µL to 100 µL according to the manufacturers advice to increase sensitivity. When observations of false negative variplex test results persisted, we performed an interim analysis. Endpoint of the interim analysis was the false negative rate of variplex test system compared to RT-PCR in all patients admitted in April 2020.
Assuming a sensitivity of 60% poor and one of 75% good, corresponding false negative rates would be 40% and 25%, respectively. The false negative rate of the variplex SARS-CoV-2 test system compared to RT-PCR in all patients admitted to our general hospital in April 2020, in whom simultaneous swabs could be obtained (n = 109), was 83% and sensitivity was 8/47 = 17% (95%CI 7.6% – 30.8%). As a consequence, we discontinued variplex testing without RNA extraction by routine and initiated further analysis of all data recorded. In addition to the false negative rate compared to RT-PCR as primary endpoint, we were interested in a comparison to a clinically based COVID-19 diagnosis as secondary endpoint and some retrospective sample size calculations for self control.
Sample size calculations were based on the optimal two stage designs by R. Simon [16]. In addition to the assumptions mentioned above (sensitivity of 60% being poor, of 75% good), a type I error of 10% and a power of 90% would result in a first stage sample size of 43 RT-PCR positive patients (MinMax Design). Further assuming a sensitivity of RT-PCR of 70% in throat swabs in COVID-19 patients and a prevalence of 60% of COVID-19 in all patients admitted to our hospital, a total of 102 patients had to be screened simultaneously for SARS-CoV-2. Upper limit for first stage rejection of variplex test in this setting is 25, hence much more than the 8 positive variplex tests out of 47 RT-PCR positive patients observed. Ignoring the first stage stopping rule due to the significant delay of RT-PCR results and in regard to modification of our throat swab sample volume from 75µL to 100 µL, a sample size of 64 PCR positive patients, corresponding a total of 152 patients admitted to our hospital had to be screened. This is in line with the total of 153 patients screened, thereof 109 patients with simultaneous swabs for variplex test and RT-PCR assay, reported in this study. Upper limit for 2nd stage rejection in this setting is 43 variplex test positive out of 64 RT-PCR positive patients.
SARS-CoV-2 viral load in upper respiratory specimens of infected patients decreases in the course of the disease [1]. This temporal dynamics in viral shedding [17] could be the reason for negative results of variplex test as well as of RT-PCR assay and raises the need for a clinical suspected diagnosis independent of viral detection. Clinical diagnosis of COVID-19 pneumonia is based on CT scan predominantly. A strong correlation of increasing levels of suspicion of pulmonary involvement with the positive rate of PCR assays has been demonstrated recently and ranged from 6% for CO-RADS 1 up to 93% for CO-RADS 5 [11].
Co-RADS 3 implies equivocal findings for pulmonary involvement of COVID-19 based on CT features that can also be found in other viral pneumonias or non-infectious etiologies and CO-RADS 6, was introduced to indicate proven COVID-19 as signified by a positive RT-PCR test for virus-specific nucleic acid [11]. Based on the experience of our interim analysis we developed a clinically based COVID-19 rating and diagnosis system (clinCO-RADS). We adopted the levels of CO-RADS except for level 3 and level 6. We decided to split CO-RADS 3 by means of the whole clinical information such as history, examination and other findings and classified in a first step to a four point scale (clinCO-RADS 1, 2, 4, 5). In a second step, patients classified clinCO-RADS 1 or 2 were classified clinCO-RADS 3, if SARS-CoV-2 was detected by PCR.
Sensitivity of RT-PCR in detecting clinCO-RADS levels 3 – 5 was almost 80%, hence in the expected range, resulting in a false negative rate of about 20%. We could demonstrate a increasing positive rate of RT-PCR assays with higher clinCO-RADS levels (figure 3) similar to the findings of Prokop et al. [11]. Furthermore, number and rate of non-survivors correlated with clinCO-RADS levels (figure 1), consistent with the association of radiologic findings with mortality of COVID-19 patients [18]. In contrast to the expected results of RT-PCR assays, the sensitivity of variplex test in detecting clinCO-RADS levels 3 – 5 was 15% and the false negative rate was 85% (figure 2).
In view of this disappointing results we analysed the whole process from the technique of throat swabs to the release of variplex test results. Detection of SARS-CoV-2 depends on type of clinical specimen. While positive rates in bronchoalveolar lavage fluid is highest with 93%, positive rates with nasal swabs are 63% and with pharyngeal swabs 32%, respectively. However, the number of specimens analysed were different and partly very low (BAL n = 15, nasal swabs n = 8, pharyngeal swabs n = 398) [19]. In our hospital, simultaneous swabs were taken by well trained nurses, both oropharyngeal or nasopharygeal, with identical swabs. Diagnostic yield depends on sampling and therefore on swabs. Swabs with short fiber strands such as FLOQSwabs (Copan) may be superior compared to standard swabs with rayon flocking. However, identical swabs were used for variplex test as well as for RT-PCR assay. Furthermore, due to pandemic caused scarcity of resources, only standard swabs were available in our hospital.