Model Structure
For each affiliate subpopulation (in-state residential student, out-of-state residential student, non-residential student, faculty, staff, community), individuals are assigned into an immunity (or protection) level: no immunity, previous SARS-CoV-2 infection only, full vaccination, boosted, full vaccination with previous infection, boosted with previous infection (additional detail provided in Methods and Supplementary Text). Within each affiliate/immunity level subpopulation, individuals are placed in one of the compartments detailed in Fig. 1 (additional detail provided in Methods and Supplementary Text). Details on statistical models used to estimate protection against SARS-CoV-2 infection by immunity level are provided in Methods and Supplementary Text. Additional detail on estimation of protection parameters, disease transmission and transition parameters, including those derived from scientific literature or institutional protocol, is provided in Supplementary Text. Initial compartment states and disease transmission/transition parameters are then inserted as input parameters into the compartment-based modeling (CBM) framework. The CBM provides predictions of the weekly number of cases and infection rates, the daily number of isolated individuals, and the daily number of isolated and quarantined individuals (by affiliate subpopulation). In addition, the toolkit also displays a summary of the initial states and the estimated disease transmission dynamics. A step-by-step tutorial of this publicly available toolkit is included as a supplement to this article.
Main analysis - Clemson University Analysis (Spring 2022)
There were 27,516 individuals in the main-campus population, including 22,634 students (4,853 in-state residential students, 2,265 out-of-state residential students, 15,516 non-residential students) and 4,882 employees (1,611 faculty, 3,271 staff). Also included were 17,681 from the local community.29 The residential population was split into in-state and out-of-state, since out-of-state residential students were more likely to use university-provided housing (if SARS-CoV-2 positive) due to travel restrictions. Students and employees were subject to mandatory arrival testing and weekly surveillance testing during in-person instruction. Initial values for students and employees in each compartment are based on empirical data with adjustments for underreporting (Table S1) at the start of the prediction period (January 10, 2022). During this period, the omicron BA.1 variant accounted for 99.2% of SARS-CoV-2 cases in South Carolina.9
Estimated student and employee disease prevalence at baseline (January 6th through 9th ) was 15.1% and 4.8%, respectively. The number of individuals in each immunity level, along with estimated protection by immunity level, is provided in Table S9. The disease reproductive number for each subpopulation was validated using empirical data from the Spring and Fall 2021 semesters and published literature (Methods and Supplementary Appendix 1). Predicted SARS-CoV-2 cases under weekly surveillance testing for students and employees during the 5-week follow-up period (January 10 – February 13, 2022) are provided in Fig. 2. Observed cases represent the total number of tests with positive results during the indicated prediction period. Predicted cases represent the total number of students and employees tested positive during the indicated prediction period. Total predicted student and employee cases (%) during this 5-week period was 4,947 (21.9%) and 891 (19.2%). Total observed cases (%) for these populations were 4,876 (21.5%) and 876 (17.9%), respectively.
Further, the percent-agreement for total detected cases was 98.6% for students and 93.2% for employees. In addition, the percent agreement for the peak number of weekly detected cases is 81.9% for students (observed N = 2,035; predicted N = 1,667) and 79.5% for employees (observed N = 308; predicted N = 245). The predicted peak for students concurred with the observed peak at Week 1 (Jan. 10–16), but the predicted peak for employees occurred a week later than the observed peak.
Observed and predicted students in isolation over the 5-week prediction period are presented in Fig. 3. Clemson University’s Isolation and Quarantine (I/Q) policies were based on latest CDC guidelines.30 We are interested in the maximum number of students in isolation, since this is directly linked to procurement of rooms. Predicted and observed peak isolation count was 1,710 and 1,881, respectively, corresponding to an agreement of 91.8%. Of particular interest is the residential student population, since this population lives in congregated housing and therefore cannot isolate/quarantine in place. Among residential students, predicted and observed peak isolation count was 673 and 649 (% agreement: 96.3%). In addition, among out-of-state residential students, predicted and observed peak isolation capacity was 264 and 194 (% agreement: 73.5%).
There was some daily variation in observed peak isolation (relative to predicted). Of note is the discrepancy between peak capacity towards the end of week 2 (predicted peak: 1,086, observed peak: 1,515; agreement: 72%). This was primarily due to daily fluctuation in student testing schedules and limited weekend testing, which was not incorporated into the modeling framework.
Prior to the start of each semester, we were tasked with evaluating the impact of testing strategies on mitigating disease spread. This has been extensively studied for previous variants (prior to omicron), which have concluded that testing at least once per week is sufficient for mitigating disease spread.12,13 Here we compare the projected cases during the five-week projection period under four different testing strategies: weekly, bi-weekly, monthly, and voluntary testing. We consider two time periods: Spring 2022 semester (omicron BA.1 variant) and Fall 2022 semester (omicron BA.5 variant).
For voluntary testing, we estimated that only 10% of total SARS-CoV-2 infections would be detected for students and 15% for employees. Results for the Spring 2022 semester presented in Fig. 4. Weekly testing led to 1.10, 1.50, and 2.57 times more detected student cases compared to bi-weekly, monthly, and voluntary testing (weekly: 4,947, bi-weekly: 4,492, monthly: 3,293, voluntary: 1,928) and 1.02, 1.30, and 1.92 times more detected employee cases compared to bi-weekly, monthly, and voluntary testing (weekly: 891, bi-weekly: 871, monthly: 688, voluntary: 463). The opposite was true for total cases (both symptomatic and asymptomatic). Here, voluntary testing led to 1.65, 1.19, and 1.06 times more total student cases compared to weekly, bi-weekly, and monthly testing (weekly: 5,669, bi-weekly: 7,859, monthly: 8,851, voluntary: 9,379) and 1.79, 1.29, and 1.10 times more total employee cases compared to weekly, bi-weekly, and monthly testing (weekly: 1,206, bi-weekly: 1,671, monthly: 1,954, voluntary: 2,153). Based on these findings, Clemson University continued with weekly testing during the first half of the Spring 2022 semester. While similar (relative) trends were observed when comparing testing strategies prior to the Fall 2022 semester (Fig. S1), overall predicted cases were lower under the four testing strategies. This is primarily due to the substantial increase in population immunity from the Omicron BA.1 variant, which resulted in a lower susceptible population.9,31
Extension to other institutions and time periods
We generalize the modeling framework above to obtain predictions in three other settings. The first two projections were conducted for the University of Georgia (UGA) and Pennsylvania State University (PSU) during the Spring ’22 semester. These institutions were natural choices for external validation, as both are land-grant universities with publicly accessible data on weekly Covid-19 cases. Because institutional vaccination data was unavailable, we used literature-based estimates of vaccine protection for these populations (Table S7). The third set of projections utilized the generalized modeling framework for predictions at Clemson University during the Fall 2022 semester (omicron BA.5 variant).
Table 1
Comparison of observed and predicted cases (detected) at UGA and PSU during first 5 weeks of Spring ’22. The % Agreement is calculated as min(Oi,Pi)/max(Oi,Pi), where Oi and Pi are the observed and predicted Covid-19 cases in week i.
|
Observed cases
|
Predicted cases
|
% Agreement
|
University of Georgia
|
|
|
|
1/10 − 1/16
|
1,003
|
529
|
|
1/17 − 1/23
|
929
|
656
|
|
1/24 − 1/30
|
363
|
526
|
|
1/31 − 2/6
|
166
|
326
|
|
2/7 − 2/13
|
89
|
329
|
|
Total
|
2,550
|
2,467
|
96.7%
|
Pennsylvania State University
|
|
|
|
1/10 − 1/16
|
539
|
286
|
|
1/17 − 1/23
|
631
|
455
|
|
1/24 − 1/30
|
340
|
477
|
|
1/31 − 2/6
|
128
|
436
|
|
2/7 − 2/13
|
70
|
329
|
|
Total
|
1,708
|
1,983
|
89.5%
|
For UGA and PSU, we obtained the total number of students and employees in each university and the number of infections during the week prior to the prediction start (January 10th, 2022) from institutional websites and Covid-19 dashboards.32,33 Because UGA and PSU did not implement mandatory surveillance testing, reported Covid-19 cases are from voluntary testing and therefore overall case prevalence is underreported. We adjust these estimates by an (estimated) constant to obtain the asymptomatic/ undetected infection rate at baseline (see Methods and Supplementary Appendix 1). Due to lack of information on vaccination and previous infection rates, we estimate these quantities using a combination of Clemson institutional data and data from the Centers for Disease Control and Prevention (CDC).34 The calculation of subpopulation sizes and other details are provided in Supplementary Appendix 1.
We used our toolkit to predict the number of weekly cases and the maximum number of weekly cases for university students and employees at UGA and PSU over the 5-week period (January 10 to February 13, 2022). The results are provided in Table 1. Additional information on the initial values, estimated individuals in each protection level, and model input parameters is given in the Supplementary Materials (Table S3-4, S7, S10-11). The percent agreement for the total detected cases over the prediction period was 96.7% for UGA (observed N = 2,550; predicted N = 2,467) and 89.5% for PSU (observed N = 1,708; predicted N = 1,983). In addition, we examined the peak number of cases during the five weeks, as this informs decisions on health resources (isolation beds, meals, medical staff, contact tracers, etc.). The percent agreement for peak weekly cases was 65.4% (observed N = 1,003; predicted N = 656) for UGA and 75.6% (observed N = 631; predicted N = 477) for PSU. In both scenarios, the predicted peak occurred one week after the observed peak.
Clemson University Analysis (Fall 2022)
In the third extension, we use the model to project the number of cases and number in isolation for the beginning of the Fall ’22 semester (August 24 – September 27, 2022) at Clemson University, where the BA.5 omicron variant was the dominant SARS-CoV-2 in the population.35 The notable difference, compared to the main analysis, is that the University implemented a voluntary/symptomatic testing strategy mid-way through the Spring ’22 semester. Consequently, many infections between this period and the Fall ’22 semester went unreported. We therefore imputed estimates of unreported infections during periods of voluntary testing (December 12, 2021 – January 2, 2022 and April 2 – May 22, 2022) into the previously infected compartments. Estimated unreported infections occurring in the 90-day window between May 23, 2022 and the start of follow-up are imputed into the recovered compartment. Estimated unreported infections during the 90-day window prior to start of the Fall 2022 semester (May 23 – August 21, 2022) were added to the recovered compartment. Details on the estimation procedures are provided in Methods and Supplementary Text. Due to lack of mandatory pre-arrival or arrival testing which resulting in small sample sizes at the semester start, these predictions no longer utilize statistical models to estimate protection from vaccine or previous infection. Rather, the protection parameter for each protection level was set according to existing literature.36 Full details on initial values and model input parameters for this analysis are provided in Supplementary Materials (Table S5 and S8, respectively).
There were 24,264 individuals in the main-campus population, including 19,082 students (4,670 in-state residential students, 2,323 out-of-state residential students, 12,089 non-residential students) and 5,183 employees (1,754 faculty, 3,429 staff). Estimated student and employee disease prevalence at baseline was 29.3% and 14.1%, respectively. The number of individuals in each immunity level, along with estimated protection by immunity level, is provided in Table S12. Predicted Covid-19 symptomatic infections for students and employees during the follow-up period are provided in Fig. 5.
Predicted student and employee symptomatic infections (% of population) during this 5-week period was 644 (3.4%) and 183 (3.6%). Total observed cases (% of population) for these populations was 636 (3.3%) and 118 (2.2%), respectively. Figure 5 provide a weekly comparison between the projected and observed number of detected cases during the five-week prediction period. The percent agreement for total detected cases was 98.8% for students and 64.5% for employees. In addition, the percent agreement for the peak number of weekly detected cases is 61.0% for students (observed N = 254; predicted N = 155) and 40.7% for employees (observed N = 33; predicted N = 81). The predicted peak for students occurred two weeks later than the observed peak, and for employees one week prior to the observed peak.
Input parameter sensitivity
Sensitivity of predictions to model input parameters have been extensively studied for Covid-19.12,13,37,38 In this section, we explore sensitivity to some of the parameters unique to our modeling framework. One novel feature is accounting for protection from previous infection. We conduct a sensitivity analysis ignoring this assumption by assuming no protection from previous infection. In all settings, cases were substantially overestimated (range: 5.7–62.7%, see Table S13-S15). At Clemson University, ignoring this assumption would have led to an estimated increase in necessary I/Q capacity of 137.7% during the Fall 2022 semester, but is estimated to have had no impact on I/Q during the Spring 2022 (which is expected, since previous infection offered little protection against the omicron BA.1 variant).
In addition, there are many individuals whose infection history is unknown. We overcome this limitation by estimating the number of individuals who were previously infected by omicron but not recorded in institutional databases. If we ignore this assumption and assume that no previously infected individuals were missed, this leads to substantial overestimation in the number of predicted cases (range across scenarios: 64.2–343.0%, see Table S13-S15). At Clemson University, ignoring this assumption would have led to an estimated increase in necessary I/Q capacity of 39.8% (Spring 2022) and 96.5% (Fall 2022).
The proportion of individuals who voluntarily seek a Covid-19 test when infected is an important assumption in prediction modeling. Increasing the proportion of infectious individuals who seek a Covid-19 test from our assumption of 10–20% for students and from 15–30% for employees, the predicted number of cases in Spring 2022, when mandatory weekly testing was implemented, increased by 0.3% for students and 2.2% for employees. This result is expected, as increasing voluntary testing rates under mandatory weekly testing would only impact how soon symptomatic individuals would seek a test after infection, but would not impact their decision to obtain a test. In Fall 2022, when mandatory testing was no longer in place, doubling the proportion of infectious individuals who seek a Covid-19 test would have led to an estimated 77.0% increase in detected cases among students and a 69.4% increase among employees.
At multiple periods throughout the pandemic, this toolkit was used to inform the removal of mitigation measures, including social distancing requirements, mask mandates, and mandatory testing. Because it is difficult to model the precise impact of a masking or social distancing mandate, we instead compared predicted cases under two scenarios: strong effect of the mitigation measure versus no effect of the mitigation measure. For example, our team was tasked with evaluating the impact of the classroom mask mandate mid-way through the Spring 2022 semester (after the omicron BA.1 wave had resided). To evaluate sensitivity of model predictions to changes in mitigation measures, we incorporated six daily time steps (4 hours each) into our model. Under the reference setting (corresponding to 4 weekday time steps), which was assumed to represent non work or school hours, we assumed minimal contact between students and employees or community members.13 During class hours (1 weekday time step) and work/study hours (1 weekday time step), we assumed increased contact between students and faculty, but decreased rates of transmission. Weekend time steps assumed increased transmission rates and higher contact rates between students and employees with community members. Transmission rates across time steps were calibrated to correspond to reference transmission levels (on average). Full detail on the contact network matrix and transmission rates by time step are provided in Supplementary Appendix 1.
Assuming masks decrease disease transmission by 50%,39 we conservatively assume an absence of a mask mandate will double transmission during the classroom time step. During the first 5 weeks of the Spring 2022 semester, removing the mask mandate would have led to an estimated increase of 171 student cases and 119 employee cases. During the first 5 weeks of the Fall 2022 semester, implementing the mask mandate would have led to a decrease of 15 student cases and 9 employee cases. Negligible differences in Fall 2022 are not surprising given that the majority of high-density social interactions occur outside of the classroom. Since Covid-19 prevalence was relatively low compared to previous states of the pandemic and a high-majority of the population had protection from previous infection or vaccination, a mask mandate implemented during a period of the day in which social contact was reduced would have minimal impact on overall disease spread.
Our results were not overly sensitive to the choice of contact network structure. To assess sensitivity to assumptions of contact network, we increased contact rates between students and employees/community members by 25%. This led to a decrease of 21 student cases and an increase of 13 employee cases in Spring 2022 and a decrease of 6 student cases and an increase of 3 employee cases in Fall 2022.