From October 2020 through mid-February 2022, over 240,000 saliva samples were screened using the MP4 assay (Supplementary Fig. 7). There are 3 key features of the MP4 assay, which collectively, distinguish it from other published assays. First, the assay bypasses an RNA extraction step, instead relying on pretreatment of saliva with Proteinase K (PK). This strategy has been used successfully by other groups, although we optimized the MP4 assay to use less PK relative to other published assays to reduce reagent usage and cost (e.g. ~3-fold less than SalivaDirect)23,24. Second, we created two paired pools of 8 samples each such that each sample is represented in 2 unique pools. Third, we used ddPCR rather than qPCR as the platform for amplification and detection. Many studies have shown that RT-ddPCR is more sensitive than qPCR for detection of SARS-CoV-2 25,26,27,28,29, which we reasoned would help offset the reduction in sensitivity with our pooling strategy. ddPCR also is less sensitive to inhibitors due to fractionation of the reaction mixture prior to the RT and amplification steps30, which we also reasoned would be advantageous for an assay using minimally processed saliva samples. Relative to the commercially available Bio-Rad SARS-CoV-2 ddPCR kit, we reduced the concentrations of primers and probes to approximately half of what was recommended to conserve on reagents and reduce costs. Since RT-ddPCR was used for COVID screening far less than RT-qPCR, we experienced no supply chain issues or competition for reagents related to the more conventional RT-qPCR tests. Although ddPCR reactions take longer to run than qPCR assays, modification of BioRad’s recommended thermocycler cycling program allowed us to shorten this step from 3.5 to 1.5 hours. (Fig. 1, Supplementary Fig. 3). Taken together, these modifications and pooling enabled us to report results the next day for > 95% of samples.
The dataset presented in Fig. 3 highlighted the power of our paired pool approach, by presenting an experiment in which one pool failed to generate a positive call and outlining the resulting steps leading to the successful identification of the positive sample. Pools containing one or more positive samples can result in negative calls for a variety of reasons including low viral loads, the presence of inhibitors, or technical errors such as pipetting. In our data from 2021, 5.5% (n=130) of our positive samples were identified from a single positive pool, suggesting that while this was a low occurrence, running paired pools increased the detection rate of our assay relative to assays relying on single pools. Most of these samples (88.5%) had <10 N1 droplets in the corresponding pool and therefore were most likely missed due to low viral concentration. The remaining 11.5% had >10 N1 droplets in the corresponding pool; technical errors such as pipetting or the presence of inhibitors most likely account for these results.
Overall, our data from the Fall semester of 2021, which included 386 positive samples, showed how well our pooling strategy performed (Fig. 6). The high concordance between the letter and number pools indicated a high degree of technical reproducibility that is remarkable considering this step was performed by >10 lab personnel within this timeframe. Furthermore, the high degree of correlation between pools and individual samples (~7-fold difference) is very close to the predicted 8-fold difference based on our pooling strategy as well as the 6-fold difference in LOD concentrations observed between individual and pooled samples (2 and 12 copies per µl, respectively). We attribute part of the success of our pooling strategy, as well as the MP4 screening assay, to our reliance on manual processing of saliva samples, avoiding the use of automated liquid handling instruments in favor of manual inspection and pipetting, which allowed greater oversight over all steps of the assay. We and others experienced problems with automated pipetting of saliva due to viscosity and other issues17.
Because of the high degree of concordance between the letter and number pools and the quantitative nature of RT-ddPCR, we could often infer which samples were positive in plates with more than one positive sample by “matching” concentrations in the pools. This can be seen in Fig. 5, in which the droplet numbers from the SP5 pools indicated that the positive samples were most likely A5 and D2 because of the similarity in pool droplet numbers (10 and 4 for pools A and 5, respectively and 37 and 35 for pools D and 2, respectively), which was ultimately confirmed in the re-run data. This reveals a further advantage of ddPCR and the MP4 assay that could be exploited to provide enriched, more informative data. Since ddPCR is a quantitative assay that does not require a standard curve, viral loads determined in each reaction can be used to identify potential super shedders or spreaders for prioritization of mitigation efforts to prevent significant community spread. Curiously, we did observe a slightly virus titer in number pools. One explanation for these results is that per our SOP, number pools were created after the letter pools and the reduced volume in the well may have resulted in withdrawal of more of the precipitate in the bottom of the tube. In one study, centrifugation of samples led to a 10-fold reduction in LOD, suggesting that the precipitate may harbor a greater concentration of viral material31.
Unlike most universities or other entities employing large-scale COVID testing and/or screening, we tested every saliva sample at least in duplicate. Positive samples were confirmed in a third replicate reaction. Thus, the MP4 assay not only saves reagents and costs less than $10 per reaction, but also provides greater assurance that the results are reproducible and reliable.
Although pooling samples has many advantages, especially for screening assays such as ours, one of the biggest limitations is that invalid samples, defined as samples that do not amplify any N1 or RP droplets when run individually, would be called negative in our assay. The threshold for calling a sample negative according to BioRad guidelines is <2 N1 droplets and ≥4 RP droplets, with the presence of RP droplets serving as a necessary internal control to distinguish SARS-CoV-2 negative samples from invalid samples (ex: water). While we monitored RP amplification in our pooled samples, we found it difficult to establish a threshold because of wide variation in RP levels from sample to sample. Furthermore, one or more invalid sample(s) in a pool of 8 would be masked by RP signal from the other samples present in the pool; thus, resulting in those samples being called negative rather than invalid. Because we manually pipetted each saliva sample, we were able to identify potentially problematic samples and flag them for individual testing. However, samples with high levels of inhibitors (tobacco or mouthwash, for instance) may not be distinguishable by eye from valid samples32 and could potentially limit the detection of positive samples when present in pools. Our data, however, suggests that pooling acts to dilute inhibitors in samples, as we observed many paired pools that skewed above the trendline shown in Fig. 6 (indicated by green circles), indicating higher than average amplification in pools as compared to the individual sample. Although we believe that this is most likely due to the presence of inhibitors in the individual sample that were diluted during pooling, we also note that there are other explanations for lower-than-expected concentrations in the individual samples such as pipetting errors. Other disadvantages of our assay, such as cost, time, and adherence to a specific platform, all pertain to the RT-ddPCR technology and have been previously reviewed in detail28.
Over the course of 311 days of screening (October 2020 to mid-February 2022), we experienced 63 days with positivity rates >2% (Supplementary Fig. 7). The bulk of these days occurred at the end of 2020 (October-December) and during the Omicron wave at the end of December of 2021 through January of 2022. Although we did not change our pooling strategy during these peak prevalence periods, our modeling data suggests that either shifting to 4-sample pools or including a third pool would have been reasonable strategies (Figs. 7 & 10). 3D pooling would result in fewer pooled samples to test (32 pooled samples per plate for 2D pools/4 samples per pool versus 24 pooled samples for 3D pools/8-samples per pool). In general, any pooling will increase the efficiency of an assay (a measure of the reduction of the number of tests run as compared to running samples individually) (Supplementary Table 2) but will lower the sensitivity (i.e. increase the number of false negatives). As shown here and by others, both smaller pool sizes and 3D pooling are less susceptible to changes in positivity rates20. The optimal pooling strategy should be one that is flexible and can be adjusted according to changes in prevalence and viral loads within a population to maximize efficiency and sensitivity.
It is difficult to compare the LOD of assays conducted in different labs using different reference materials, especially given the problems noted by others regarding available reference materials for SARS-CoV-2 assay validation 22, 33,34,35. The LOD of the MP4 assay established here (2 copies per µl) is higher than that reported for the BioRad SARS-CoV-2 ddPCR kit (0.15 copies per µL)36. This is partly explained by the ~4-fold increase in volume of starting sample in the BioRad protocol. Our LOD is lower than that reported for SalivaDirect (6–12 SARS-CoV-2 copies per µL)24, which has a similar sample workflow, but relies on the less sensitive RT-qPCR platform37. Most other screening assays report LODs within this range as well (0.5- 10.6 copies per µl) 12,13,14,15,16,17,18. For pooled samples our LOD was 12 copies per µl, which is in line with at least one other report16 and considered an acceptable trade-off for screening assays in which frequency and turnaround time rather than sensitivity are more important for mitigation of viral spread38. We propose that individuals with low-titer positive samples likely shed significantly less virus than those with much higher titer samples, whom the MP4 assay easily identifies for isolation before significant spreading of COVID-19 occurs.
The concordance data showed that our assay had very high agreement with an FDA approved ANS test, as we were able to identify all ANS positive individuals as positive with our MP4 assay. We even identified 2 additional individuals as positive that had tested negative via ANS. There are several explanations that could account for the discrepancy between these two tests. First, it is well established that viral loads can differ between sample types (saliva vs ANS) during infection, with viral loads in saliva often detected before those in nares during the early stage of infection39,40,41,42. Second, inadequate self-collection by ANS could result in a false negative, an issue that is far less encountered with saliva sampling. Third, variant-specific differences related to tissue tropism, as exemplified by the more recent Omicron variants which colonize upper airways and saliva more than lower respiratory and nasal cavities43,44.
In our data from the Fall semester of 2021, we noted that the majority of our positive saliva samples had viral loads at the low end of the observed range. This skewed distribution of observed viral loads may be due to our screening of asymptomatic individuals likely in the initial stages of infection during which viral loads and associated shedding are relatively low45. This trend has also been observed in other large-scale screening assays15,8. These data show that it is possible to detect positive individuals with low viral loads from an asymptomatic population.
Although the MP4 assay was developed in response to the SARS-CoV-2 pandemic, it can be easily adapted for the future. Screening can be easily expanded to other targets of interest and higher order multiplexing for more than 2 targets per reaction can be performed using amplitude and probe-based strategies or with newer ddPCR instrumentation capable of reading more than 2 fluorescent channels46. This strategy could prove useful to monitor both COVID-19 and Influenza outbreaks. And although saliva has many advantages as a sample type47, we have successfully used the MP4 assay with nasal swabs and non-invasive polyvinyl alcohol mask strips48. And depending on the needs of the assay, a wide variety of pooling strategies can be employed to balance sensitivity with efficiency.