Citywide serosurveillance of the initial SARS-CoV-2 outbreak in San Francisco

Serosurveillance provides a unique opportunity to quantify the proportion of the population that has been exposed to pathogens. Here, we developed and piloted Serosurveillance for Continuous, ActionabLe Epidemiologic Intelligence of Transmission (SCALE-IT), a platform through which we systematically tested remnant samples from routine blood draws in two major hospital networks in San Francisco for SARS-CoV-2 antibodies during the early months of the pandemic. Importantly, SCALE-IT allows for algorithmic sample selection and rich data on covariates by leveraging electronic medical record data. We estimated overall seroprevalence at 4.2%, corresponding to a case ascertainment rate of only 4.9%, and identified important heterogeneities by neighborhood, homelessness status, and race/ethnicity. Neighborhood seroprevalence estimates from SCALE-IT were comparable to local community-based surveys, while providing results encompassing the entire city that have been previously unavailable. Leveraging this hybrid serosurveillance approach has strong potential for application beyond this local context and for diseases other than SARS-CoV-2.


Estimating test performance and positivity cutoffs for the serological assays
Selecting samples for confirmatory testing: All 5,244 SCALE-IT laboratory samples (corresponding to 4,735 unique patients) were first screened on the ELISA platform. In addition, 117 positive control samples from the LIINC cohort and 93 negative control samples were tested on this platform. The antibody concentration of each sample was calculated from the ELISA OD value using a plate-specific standard curve from serial dilutions of a pool of positive control samples. Based on the distributions of concentration values among these control samples, SCALE-IT samples with a concentration value above 0.049 were selected for confirmatory testing, corresponding to test performance characteristics of 98.3% sensitivity and 97.8% specificity.
Determining seropositivity of SCALE-IT samples: Based on the above, 653 SCALE-IT samples were selected for confirmatory testing on the Luminex platform, on which we included three SARS-CoV-2 antigens (one preparation each of the S, RBD, and N proteins). In addition, 260 positive control samples from the LIINC cohort and 114 negative control samples were tested on this platform. The antibody concentration of each antigen of each sample was calculated from the Luminex MFI value using a plate-specific standard curve from serial dilutions of a pool of positive control samples.
We then fit a multiple logistic regression model to the control samples and their Luminex concentration values for the three antigens. We used this model to classify each SCALE-IT sample as seropositive or seronegative; samples with a predicted probability value which corresponded to a specificity of 100.0% and sensitivity of 95.8% (AUC: 0.983) were classified as seropositive. The five-fold cross-validated sensitivity of this algorithm, fixing specificity at 100.0%, was estimated to be 95.4%. Given the relatively low expected seropositivity in the population, we chose to maximize the specificity of this classifier.
Determining the test performance characteristics of the two-assay procedure: The test performance characteristics of a single assay (i.e., sensitivity and specificity) can be determined from a 2x2 table of positive/negative control samples and their binary classification on that assay using a binomial model 1 . For a two-assay scenario, the binomial model can be extended to a multinomial framework where each control sample has two test results: their binary classification on each of the two assays 2 . Importantly, there may be conditional dependence between assays, where conditional on the true disease status of a given sample, the test performance of one assay may vary depending on the result on the other assay. The magnitude of this conditional dependence between two assays can be directly estimated based on the results of control samples that have been tested on both assays.
Here We employed a modeling framework that jointly estimates assay-specific sensitivities (Se_ELISA and Se_Luminex), assay-specific specificities (Sp_ELISA and Sp_Luminex), correlation between sensitivities (covariance_Se), correlation between specificities (covariance_Sp), and seroprevalence. We allowed control samples that were tested only on one assay to contribute to the estimation of that assay's performance characteristics using the standard binomial model.
As the SCALE-IT samples were tested in a serial procedure that required a sample to be positive on the two assays to be classified as seropositive, we estimated the overall sensitivity of the approach as: Se_overall = Se_ELISA * Se_Luminex -covariance_Se, and the overall specificity of the approach as: Sp_overall = 1-(1-Sp_ELISA)*(1-Sp_Luminex) -covariance_Sp. Using these estimates of overall sensitivity and specificity, we obtained adjusted estimates of seroprevalence as: adjusted prevalence = (raw prevalence + Sp_overall -1) / (Se_overall + Sp_overall -1) 1 . The posterior estimates of these parameters are provided in Supplementary Table 1. The code to implement this model is included in our GitHub repository.