The Ct values (Table 1) presented to the author for the data analysis and retrospective review were generated in the past several years by following certain algorithms as described below (Fig. 1). Each sample was initially screened in the real-time PCR assay. If a sample was determined to be positive in the initial screening, it would be re-tested in duplicate for confirmation. (a) In the initial screening stage, each sample was tested in the real-time PCR assay, and both the Ct value and the copy number were used to determine whether a sample was positive or negative. A sample with a Ct value less than 37.0 and a copy number equal to or greater than 10 based on the calibration curve in the initial screening was considered negative and no confirmatory retests would be needed for this sample. A sample with a Ct value of less than 37.0 but with a copy number less than 10 based on the calibration curve would undergo an initial test again. Finally, a sample with a Ct value equal to or greater than 37.0 in the initial screening would be presumed positive. (b) The initially presumed positive sample was further confirmed via the same real-time PCR assay in duplicate. A sample with a Ct value equal to or greater than 37.7 in the confirmatory tests was presumptive positive in the laboratory. Also, a sample with a Ct value less than 37.7 but a copy number less than 10 based on the calibration curve in the confirmatory test would be presumptive positive as well. (c) The presumptive positive sample in the laboratory would be further diagnosed by other methods to definitively confirm whether they were true positives.
Table 1
Example of Ct value data used in this study
Sample
|
Ct
|
Quantity
|
Y-Intercept
|
R2
|
Slope
|
1
|
36.563
|
3.394
|
38.132
|
0.986
|
-2.957
|
2
|
36.073
|
4.969
|
38.132
|
0.986
|
-2.957
|
3
|
37.161
|
5.9
|
39.744
|
0.986
|
-3.352
|
4
|
36.033
|
5.127
|
38.132
|
0.986
|
-2.957
|
5
|
36.328
|
4.074
|
38.132
|
0.986
|
-2.957
|
…
|
…
|
…
|
…
|
…
|
…
|
…
|
…
|
…
|
…
|
…
|
…
|
718931
|
36.184
|
4.556
|
38.132
|
0.986
|
-2.957
|
718932
|
31.595
|
643.25
|
40.62
|
0.959
|
-3.214
|
718933
|
32.1
|
589.403
|
40.604
|
0.957
|
-3.07
|
718934
|
32.533
|
328.474
|
40.62
|
0.959
|
-3.214
|
718935
|
31.186
|
1,169.16
|
40.604
|
0.957
|
-3.07
|
Table 2
Presumptive positives and their further assessment with other diagnostic method
Cases
|
Current
|
Strategy I
|
Strategy II
|
Strategy III
|
Presumptive Positives
|
189
|
161
|
121
|
162
|
Further Diagnosis
|
|
|
|
|
|
9
|
9
|
9
|
9
|
|
36
|
35
|
29
|
36
|
|
144
|
117
|
83
|
117
|
Current Data Summary
A total of 3084 successful real-time PCR assays were included in this evaluation. All the data of Ct values from these assays were combined into a .csv file for analysis with the Pandas data analysis package (version 1.2.5) in Python (version 3.9.0). The combined data were then split into two categories or subfiles: a file containing all the single entries and another one containing all the duplicate entries. Single entries represent samples that underwent only one test in the initial screening stage and no confirmatory retest was needed. Duplicate entries are samples that had been tested multiple times either in the initial or confirmatory stage. Description of the data in Pandas summarized that a total of 718,935 samples had been tested either once or multiple times in the real-time PCR assays. These samples were initially screened and 221 of them were finally presented as presumptive positives in the laboratory. Of these 221 samples, 189 had diagnostic results but the rest were not diagnosed due to other reasons. The diagnosis through other methods confirmed that 9 out of these 189 samples were true positives and the other samples were either negative or irrelevant cases (Fig. 2). Ct values for these presumptive positives varied from 35.604–40 (Fig. 3). While the cases for the true positives had Ct values all above 39.0, other irrelevant diagnoses showed relatively larger variation in their Ct values from 36.604–40.
Furthermore, in the 221 presumptive positives, there were 48 samples with Ct values straddling cut-off across multiple tests. In these 48 samples, 9 were not diagnosed due to other reasons, 3 were diagnosed as irrelevant, and the rest as negative. In addition, 5 samples had Ct values all below 37.7. They were considered presumptive positive in the laboratory due to having copy numbers less than 10 inferred from the calibration curves in the real-time PCR assays. For these 5 samples, one was not diagnosed due to certain reasons, but the other four samples were all diagnosed as negative. Below, all these real-time PCR assays were retrospectively reviewed to evaluate how the calibration curves affect presentation and identification of presumptive positives in the laboratory. The 32 presumptive positives that were not assessed through other diagnostic methods because of certain reasons are not included in the review process due to the lack of final diagnostic results.
Calibration Curves in Presumption of Positives
The data analysis revealed that a total of 2474 out of 718,935 samples were further repeatedly tested after initial screening. Out of the 2474 samples, 221 were presented as presumptive positives for further assessment as described above. The major question is how calibration curves included in the real-time PCR assays for inferring the target copy numbers have contributed to the identification of these presumptive positives. A total of 1450 samples had undergone initial tests multiple times solely due to their copy numbers being presumed to be less than 10 in one of the initial tests based on the calibration curves (Fig. 4).
This demonstrates that the rate of undergoing multiple initial tests attributed to calibration curves was relatively high at 58.6% (1450/2474). However, the actual result shown by the lab was the opposite. Out of the 1450 repeated samples, only 28 were identified in the lab as presumptive positives requesting for further assessment. This means that the rate for identification of presumptive positives contributed by calibration curves is as low as 2% (28/1450) and the rate of retest attributed to calibration curves is over 98% ((1450-28) / 1450). Further, all these 28 presumptive positives do not have the true positive phenotype according to further assessment, in which 26 were diagnosed negative, and 2 were diagnosed as irrelevant. The root cause for the high rate of false retest is the variation of the standards themselves. Each standard varied on the Ct values about almost 2 cycles, e.g., standard 1 (S1) from 35–37 and standard 3 (S3) from 32–34 (Fig. 5), which is flawed for accurately inferring reliable copy numbers. In contrast, the other 193 presumptive positives (221 − 28) were presented after repeated testing of 1024 samples (2474 − 1450) solely based on their Ct values. The rate for identification of presumptive positives is 19% (193/1024), much higher than 2% in the retest based on standards.
Calibration Curves in Presumption of Positives
All the initial and confirmatory tests were virtually replayed retrospectively to evaluate how the omission of calibration curves in real-time PCR assays would affect the data presentation. Three replaying strategies were presented in this study to cover the possible impacts and solutions after omission of calibration curves in the real-time PCR assays. Also, in the replay process, the possible impact on data presentation after omitting calibration curves was maximized by using the following criteria.
(1) Any samples with a Ct value less than the screening cut-off (37.0) in the initial assay or confirmatory retest would be categorized as "negative" and no confirmatory retest would be required. It should be noted that among these "negative" samples, a very small portion had been identified as "presumptive positives" in the lab either due to their target copy numbers (< 10 derived by the calibration curves) or Ct values (≥ 37.7 in some of the repeated tests). The samples that would be categorized as "negative" in the replay process but were identified as presumptive positive in the lab would be queried with the other diagnosis results to determine if they were false negatives.
(2) Any samples with all Ct values ≥ 37 and at least one Ct value ≥ 37.7 in the initial screening or confirmatory retest would be categorized as presumptive positive according to the current testing and follow-up procedure. The other diagnosis results would then be queried to determine if there were any false positives.
(3) Samples that all had Ct values between 37–37.7 in the initial screening and confirmatory retest and were presented as presumptive positives due to the derived target copy numbers (< 10) would be categorized as "negative" in the replay process. These samples would be also checked with the other diagnosis results to determine if there were any false negatives.
Strategy I
In this strategy, the data replay would follow the actual order of the lab testing for each sample, which means that the initial screening would be considered as the first run and the retests would be repeated runs. The omission of calibration curves in the initial screening would impact 109 samples. These 109 samples had Ct values below 37.0 in the initial screening but were retested because of their target copy numbers being less than 10 based on the calibration curves. The retests determined that 106 of them were negative and 3 of them were presumptive positive. The further assessment confirmed that these 3 samples were also negative. In the scenario of this strategy replaying data without considering calibration curves, these 109 samples would have been reported as negative in the initial screening and thus not trigger the retest. This would result in reducing 4% (109 / 2474) of samples that had required retests due to calibration curves.
As described earlier, 1450 samples had been repeatedly tested at least once due to one of the calibration curves showing their target copy numbers to be less than 10. Therefore, aside from the above 109 samples affected in the initial tests, the other 1341 samples (1450 − 109) would be affected in the confirmatory retests with omission of calibration curves. With calibration curves in the real-time PCR assays, 25 presumptive positives were identified from these 1341 samples after one or more times of retests. The query with further assessment results confirmed that 23 out of these 25 presumptive positives were negatives, and that 2 were false positives with other irrelevant diagnosis results. In other words, if calibration curves were omitted in the real-time PCR assays, these 1341 samples would be retested only once and then reported as negative – a significant improvement.
Strategy II
In this strategy, the data replay would not consider whether Ct values for a sample were from the initial screening or confirmatory retest. For example, if a sample had an initial Ct value of 37.875 and confirmatory Ct values of 37.825 and 36.538, it would just be reported as negative and would not require a confirmatory retest. This is because, when replaying the data for this sample, the author would consider the possibility that the Ct value 36.538 could be from the initial screening. In this way, it would be able to maximize the possibility of influence on the data presentation if the calibration curves would be removed.
Compared to the current data presentation with calibration curves, a total of 2335 samples would have been possibly screened as negative, avoiding the confirmatory retest in this replay strategy because at least one of their Ct values was below 37.0; therefore, they might have been screened as negative in the initial test without triggering the request for confirmatory retest. This replay strategy would also possibly lead 68 presumptive positives to be reported as negative without requesting the further assessment because one of their Ct values was below 37.0. Indeed, these 68 samples were all false positives according to the query with further assessment results. They included 7 samples which were diagnosed as irrelevant and 61 samples which were diagnosed as negative (Figs. 6 & 7).
Strategy III
As discussed above in strategy II, 68 presumptive positives would be possibly identified as negative without requesting further assessment because there was at least one Ct value below 37.0. In these 68 samples, 61 were negative and 7 were diagnosed as irrelevant. These 7 irrelevant samples had Ct values above 36.0 in all of the tests. Therefore, in the principle of not missing other irrelevant cases, strategy III would lower the initial screening cut-off to 36.0 as compared to the replay in strategy II. If the Ct value 36.0 was used as the initial cut-off, the retest rate would be 0.33% ((2474-89) / 718935) based on the available data in this study. Compared to the current data presentation in the lab, a total of 2138 samples would be possibly screened as negative, avoiding confirmatory retests in strategy III, because at least one of their Ct values was below 36.0; therefore, they might be screened as negative in the initial test without prompting a confirmatory retest. Strategy III would also possibly lead 27 presumptive positives to be reported as negative without requesting clinical assessment because one of their Ct values is below 36.0. These 27 samples had been presented as presumptive positives in the lab (due to straddling cut-off across multiple assays), and the query with further assessment results confirmed that they were negative.