Towards Health System Resiliency: An Agile Systems Modelling Framework for Bed Resource Planning During COVID-19

Background: We describe the development of a dynamic simulation modelling framework to support agile resource planning during the COVID-19 pandemic. The framework takes into consideration the dynamic evolution of the pandemic and the rapidly evolving policies and processes to deal with the ever-changing outbreak scenarios. Methods: A specific use case based on short-term bed resource planning is described within the proposed framework. The simulation model was calibrated against historical data for the Singapore COVID-19 situation. The time period for model calibration was from 1st April till 30th April 2020. The model was used to project for bed resource needs over the period from 1st May 2020 till 31st May 2020. Multivariate sensitivity analysis was also conducted for ICU and general isolation bed demand, length-of-stay (LOS), and age-adjusted conversion rates across different care needs. The unmet needs under various scenarios were also evaluated for planning purposes. Results: Several variants of the agile resource planning model were developed to adapt to the fast-changing COVID-19 situation in Singapore. The use case demonstrated an agile adaptation of the model to account for previously unexpected scenarios. The rapid evolution of the pandemic locally revealed streams of new infections that arose from two distinct sources. The model projections were calibrated with the latest data for short-term projections. The agility in flexing plans and collaborative management structures to rapidly deploy human and capital resources to surge the level of care during the COVID-19 pandemic have proven utility in guiding the allocation of scarce healthcare resources and helped system resiliency. Conclusions: The rapidly evolving COVID-19 pandemic in Singapore has necessitated the development of an agile and adaptable modelling framework that can be quickly calibrated to changes both from demand and supply. The modelling framework is able to deploy systems modelling concepts in a holistic manner. This facilitates the evaluation of complex cause-and-effect relationships. A robust collaborative framework, coupled with the availability of in-depth domain knowledge and accurate and updated data availability ensures a model is realistic, timely and useful.


Introduction
With more than 21 million cases worldwide in August 2020 (1), the COVID-19 pandemic caused by the severe acute respiratory syndrome Coronavirus-2 virus (SARS CoV-2) is leading to substantial healthcare, economic, social and psychological impacts. Even though the first cases of COVID-19 were confirmed in December 2019, the scientific world has just begun to understand better the health problems caused by this virus (2). Apart from the obvious respiratory issues, the virus is known to attack other organ systems, and in some cases resulting in catastrophic damage. The many unknowns in such emerging infectious diseases would mean that governments and health systems around the world have to react rapidly in the face of new information.
The sufficiency of healthcare capacity is the cornerstone for access to care and outcomes for COVID-19. Many healthcare facilities around the world are seeing a surge in demand for hospital and intensive care unit (ICU) beds due to the pandemic. The case fatality rates (CFR) of COVID patients has shown to be significantly worse in overstretched health systems (3). Apart from containment and mitigation strategies that are important for "flattening" the infection curve, there is a critical need to ensure a resilient health system that can withstand unpredictable shocks resulting from the pandemic. Well-coordinated public and private sector policies and initiatives are essential to maintain high-quality care outcomes for the population (4). The United States and several European as well as Asian countries have all reported commendable efforts in planning and implementing surge capacities to cope with the on-going COVID-19 pandemic (5-7).
Singapore has proactively ramped up bed capacities for intensive care units (ICUs), isolation and quarantine beds for suspected and confirmed cases within the health system to prepare for a surge in COVID-19 infections. Local health authorities have moved rapidly since the start of the pandemic to ensure that there were sufficient and sustainable healthcare resources to deal with the fast-growing surge in bed resource needs within a short timeframe (24). A wide-variety of programs and purpose-built facilities were established within a relatively short time frame. The purpose-built in-hospital and out of hospital (ex-hospital) COVID-care facilities included in the model were deployed in phases as the demand increased since the first large clusters were detected on 30 th March 2020. The public healthcare system has managed to pivot nimbly to deal with a fast-evolving situation as the pandemic evolved, in part due to an ability to leverage on accurate ground data to develop agile planning models. Despite the unique epidemiological evolution of the disease in Singapore, the CFR for COVID-19 in Singapore has been one of the lowest in the world (8).
Most of the traditional epidemiological models developed for COVID-19 have focused on the demand side (or flattening of the epidemic curve) (9). Such epidemiological models may not fully capture nuanced disease outbreak scenarios together with high-resolution policies and processes that are important to consider during pandemics (10). The projections from traditional epidemiological models are fraught with uncertainties during the initial phases of a pandemic. This has shown to be the case when potential hotspots cannot be identified a priori to guide the best resource modelling efforts (11). Such was the case in Singapore as most of the earlier demand projections were based on importation and secondary local transmission models in the community and were not able to predict the massive clusters of positive cases that were discovered in migrant workers' dormitories (12). On the other hand, systems modelling techniques (e.g., systems dynamics, or SD, and discrete events simulation, or DES, models) (13,14) have seen many applications for both predictable demand patterns (15)(16)(17) and situations where the surge in demand is less predictable (e.g., disaster planning and pandemics due to emerging infectious diseases) (18,19). These simulation methods have the ability to capture both the detailed behaviour of the system (dynamic complexity) and structure (causal relationship) to provide a risk-free virtualized experimentation platform to evaluate strategies in scenarios that are subjected to significant uncertainties (20). Such platforms allow for the rapid adaptation of new scientific information, structural and behavioural assumptions on both the demand and supply sides within the model, and dynamically adapts to the fast-evolving outbreak scenarios (21).
We describe the development of a dynamic simulation modelling framework to support agile resource planning efforts during the COVID-19 pandemic. This modelling framework can efficiently take into consideration the dynamic evolution of the pandemic, which are much less predictable in the initial phases (22), and the rapidly evolving resource management policies and processes to take care of unexpected scenarios. To realize the effective development of the models, and deploy rapid adaptation capabilities, a robust and effective data capture infrastructure within health systems, effective data governance policies that can ensure the efficiency of data utilization for modelling and experienced modelling expertise embedded within the health systems with deep domain knowledge are among the critical ingredients. The development process leveraged on the data science and health services research expertise that has been embedded within our health system since prior to the pandemic. The dynamic simulation model that was developed by the team was able to support the indexed hospital (IH) with improved resource management strategies. The model was further expanded to incorporate the national response which included rapid flexing of bed capacity beyond the IH and informing the other resource planning needs for the IH and national healthcare authorities.

Data and Methods
The development of the modelling framework leverages on the Singapore COVID-19 scenario. Singapore is a city-state with a population size of approximately 5.7 million (of which 4 million are citizens and permanent residents) (23) with a land area of approximately 724 square km (24). Relevant to the context of this study, Singapore has approximately 300,000 migrant workers working in the construction sector at the time of the study (30). In addition, Singapore had about 12,000 acute care beds in public and private hospitals (25) and approximately 400 ICU beds. In 2018, Singapore had a trained doctor-to-patient ratio of 2.4 doctors per 1,000 patients (26).
The Academic Medical Centre (AMC) in which the study was conducted is made up of the Duke-NUS Medical School and the Singapore Health Services (SingHealth). SingHealth is one of the three public healthcare clusters in Singapore with the IH being the largest and oldest comprehensive hospital in Singapore. A core resource of the AMC that developed and implemented the modelling framework is a small embedded data science unit in the Health Services Research Centre. The health services research and data science resources are strategic resources established since 2015 within the AMC. Together with these resources, the modelling team comprises domain experts in infectious disease, intensive care, emergency medicine, anaesthesiology and surgical specialties. The bed management unit was also involved in the development of the model.
The modelling framework was conceived to provide estimates of mortality, morbidity and the impact on waiting times to specialist appointments and elective surgeries, length of stay (LOS) in the emergency department (ED), bed -demands and inpatient LOS and the utilization of critical hospital facilities (e.g., operating theatres, hospital and intensive care beds, isolation wards, diagnostic equipment, laboratory services). The systems modelling methodology based on system dynamics (SD) simulation [17], [18] was used to develop the inpatient bed resource management module. Where more detailed modelling was required-for instance, the Emergency Department (ED) and surgical resource planning, discrete events simulation (DES) [19] and agent-based modelling (ABM) [20] approaches were used. The entire modelling framework has segments that are structurally linked: infectious disease, primary care, ED, hospital inpatient care (including isolation wards, intensive care), surgical resources and specialist outpatient clinics. The high-level schematic of the systems modelling framework is shown in Figure  1, which describes how the models derive real-world data from the existing operational and clinical available through the enterprise data warehouse, electronic medical records (EMRs) and other sources. For the use case, the primary healthcare resource that we considered was the number of beds. Key outcome measures were: (1) Number of beds required for in-hospital and ex-hospital demands (demands that can be addressed by facilities outside of hospitals); (2) unmet needs considering in-hospital and ex-hospital capacities, and; (3) overall in-hospital and ex-hospital mortality rates. The types of beds that we considered were: (1) beds in isolation facilities for confirmed cases; (2) beds in quarantine facilities for suspect cases, and; (3) beds in the ICUs. ICU beds are in-hospital critical care facilities, whereas isolation and quarantine facilities can be set up outside of the hospital. The model was calibrated against historical data. The time period for calibration was from 1 st April till 30 th April 2020. The model was then used to project the demand for the period from 1 st May 2020 till 31 st May 2020.
The COVID-19 pandemic in Singapore has turned out to be a perfect use case for the development of an agile and adaptable systems modelling framework that can be quickly calibrated to changes both from the demand and supply side. On the demand side, the dominant stream of dormitory infection from day 74 (since the 1 st case was detected on 23 January 2020) grew rapidly. Fortunately, the massive foreign worker dormitory clusters was contained and separated from the larger community, thereby preventing widespread community transmission. There have been several dynamic adjustments made in terms of the detection, diagnosis and disposition policies for COVID-19. These include the controlled rates of swabbing in the dormitories and community, heightened surveillance and targeted testing for vulnerable groups (27) and the agile flexing of in-hospital and the swift rampup of external isolation facilities (28). The various structural interventions and purpose-built facilities are summarized as follows.
(1) Fever Screening Area (FSA) dedicated to the screening, triaging and disposition of COVID related cases were established within the hospital compound. These facilities were planned and went operational on 20 th March 2020 in the IH. FSA provides much needed screening services for those who meet suspect case definition but not yet require the acute and resuscitation care facilities available in hospital ED. The redirection of suspect cases to FSAs within the IH effectively kept all suspect cases within reach of acute care facilities and additional interventions would be readily available should the health conditions of suspect cases deteriorate. In addition, the dedicated FSA reduced the risk of cross-contamination of positive patients to uninfected patients who require acute or emergency care at the ED. FSA also played a key role in the implementation of the Swab and Discharge Programme.
(2) Swab and Discharge Programme (SDP). The hospital's first points of contact (ED and FSA) take in patients with Covid-19 symptoms and those who met the suspect case definition. SDP allows some of the suspect cases who are at lower risks to be sent home for quarantine after the respiratory swab for SARS CoV-2 PCR (polymerase chain reaction) test has been performed, while awaiting result confirmation.
(3) Types of beds for suspected and confirmed COVID-19 patients. The types of beds that are designated for the care of suspected/confirmed COVID-19 patients are the Acute Respiratory Infection (ARI) wards, the Isolation Wards (ISO) and the ICUs. ARI beds are dedicated for suspect cases as well as other acute respiratory conditions. Some of these beds were reconfigured from wards with 6 or 8 beds to house fewer beds per room (3 to 4 beds) to ensure there is sufficient distancing between beds. Policies requiring mandatory use of surgical masks in the wards are also instituted. For patients without other respiratory illnesses, and tested negative, they can be moved out of these ARI beds. ISO beds are isolation care facilities for confirmed cases. Patients with acute respiratory illnesses but are not infected remain in the ARI beds, whereas positive patients are moved to ISO wards. Both ISO and ARI beds have external community facilities for the decanting of patients to ensure hospital facilities are not overstretched.
(4) Other surge capacities. In the hospital, the flexing of bed capacity and the corresponding reduction of the non-COVID care loads were activated from 7 February 2020 when the national outbreak alert system level was raised to "Orange". ARI beds that were deployed to admit suspects and pneumonia patients while waiting for lab results were consolidated from non-respiratory wards and ring-fenced. With the rapid case rise from community and migrant outbreaks, the isolation capacity was also expanded rapidly within the IH. Outside the hospital, the partnerships with private hospitals and external large-scale facility operators, such as exhibition centres, military camps and port facilities, accelerated the significant expansion of community isolation facilities (CIF) from 500 to more than 40,000 beds progressively in phases. Plans were put in place to deploy Community Recovery Facilities (CRF) for the further stepdown care of patients who are well and asymptomatic, but remained SARS-CoV2 PCR positive.
Historical data from the IH and the national health authorities used for estimating the model parameters largely consisted of 3 main sources: (1) daily hospital reports from the IH about cumulative numbers of hospitalized, discharged, transferred to external facilities, death and the daily census of admissions; (2) situation reports and data consolidated by the disease outbreak task forces and institutional command centres, and; (3) the public domain data released by the Ministry of Health. These open and transparent statistics provided valuable data related to suspect and confirmed cases, test result and molecular lab workload to inform the modelling efforts. Data with missing records and incomplete statistics were excluded from the analysis. While the first case of migrant worker infection was reported in early February, it is worth noting that the large scale of the COVID-19 outbreak was not apparent until around the end of March. As such, the tracking of the dormitory flows only started from 6 th April 2020. Records containing data entry errors or duplicates were removed. All data were derived from government agency and public healthcare institutions for reporting purposes in management platforms and further data veracity checks revealed no outliers in both the raw and aggregate data. The multiple layers of data cleaning and veracity checks to ensure that data were sufficiently clean and robust for development of the models.
For resource planning purposes, projections of daily cases were made for Best-Case, Base-Case and Worst-Case scenarios. The underlying infection dynamics governing the rise in cases were assumed to be the same across all the cases, except that the key differentiating factor amongst the three cases is the time when the apex of the infection curve is reached. The various scenarios were estimated from a compartmental model that included the migrant dormitories (29) with further assumptions of a finite population in the migrant dormitories approximately 300,000 migrant workers at the time of the study (30). For the Best-Case scenario, the apex was assumed to have been reached at the third week of April 2020. For the Base Case, the apex was projected to be reached in the start of May and for the Worst Case, the apex was estimated to be towards the end of May 2020. The projections started when the surge in infections occurred at the end of April 2020 (See Figure 2 for both the projected number of daily cases and the cumulative number of cases across the 3 scenarios) and have been constantly updated with available data as the pandemic evolved over time. Given the evolving pandemic, changing ground operations and dynamic policies in case detection and surveillance, the model projections can be regularly calibrated for short to medium term projections of what would be required for the next week or month ahead. Having both the Base and the Worst cases offers alternative scenarios to develop robust strategies that can deal with unpredictable surges in confirmed or suspect cases. To adapt to the dynamic COVID-19 situation over time, several variants of the fast response resource planning models were developed. Three of these variants that evolved across the initial phases of the pandemic are described as follows: • Variant 1: This variant captured all the hospital flows of COVID patients across the multiple inpatient ward classes and external decanting facilities. However, this model did not take into account the different flows of incoming patients from dormitories and community given that the tracking of these two flows only started on 6 April 2020. The variant considered the presence of both in-hospital care facilities across ARI, ISO and ICU, as well as external care facilities with the same capabilities. External surge ARI and ISO facilities were specially setup in dedicated facilities to care for COVID cases and suspects. External ICU facilities refer to the pool of ICU facilities available in both public and private hospitals in Singapore. • Variant 2: Variant 1 was extended to differentiate different incoming streams from the foreign worker dormitories and community. The model explicitly accounts for the fact that patients with mild symptoms and less risk, no existing comorbidities and patients who are younger than 40 years old will be decanted to external ISO facilities. On the other hand, patients who do not satisfy these conditions will be cared for in the ISO wards in the hospital. Consequently, the transition rates to more severe cases were assumed to be less for cases transferred to external ISO facilities (see Table 1). To better reflect the situation on the ground, we allowed for the transition of care from external ISO facilities to inpatient ISO and ICU wards (See Figure 3). This essentially accounted for the mild cases that were earlier decanted to return to the hospital for higher levels of care. Consequently, there could be recirculatory flows between the hospital and external isolation facilities. • Variant 3: Variant 2 was extended to consider the different age segments presented by the incoming dormitory flows. The consideration of age segments allows for the age-stratified risks of mild and asymptomatic cases turning symptomatic and severe for the dormitory cases, thus providing a highresolution representation of the actual risks of over-stretching the ICU capacities in the hospital.
Data was not yet readily available at the time of model building in Variant 1 and Variant 2 to confidently estimate parameters such as the Length of Stay (LOS) statistics, ICU admission rate and the CFR. Regardless, the model was quickly built with available data and statistics found in peer-reviewed journal publications complementing the limited local data availability. Based on these aforementioned limitations when Variant 1 was developed, we assumed that out of all infected patients, 5% of patients would be critically ill, 15% would be moderately ill and the remaining 80% to be mildly ill (31,32). Furthermore, prevailing research has also established that the majority of cases would be asymptomatic (33). These assumptions were also incorporated in the model for the community cases for Variant 2. Given the unique situation that Singapore is facing with a surge primarily due to the migrant worker population in the foreign workers' dormitories, the risk factors had to be adjusted accordingly by age groups. The age-adjusted risks of ICU admissions for the dormitory population and community were then separately estimated based on the age profiles of these groups in Variant 3. These age-adjusted risks were accounted for in the demand projections across the facilities for in-hospital and ex-hospital ARI, ISO and ICU beds in Variant 3.
The dual input streams from migrant dormitories and the community resulted in a bi-agent model that was deployed for Variant 2. For conciseness, the detailed model structure for Variant 2 is presented in Figure 3. This variant demonstrates the differentiated inflows for patients from the community and the migrant dormitories and captures the structural flows between various inpatient facilities as well as external decanting facilities. Variant 3 considered the following sub-groups according to the age distribution of patients from the migrant dormitories and community: (1) less than 45 years old; (2) 45-49 years old; (3) 50-54 years old; (4) 55-59 years old, and; (5) 50 years old and above. For Variant 3, the age distribution for the confirmed cases was estimated from historical data. Other parameters were estimated from the literature and available data and information provided by the IH (see Table 1).  (2) median in-hospital ISO LOS of 10-21 days, and; (3) median LOS of inhospital ISO between 10-21 days and EISO of 14-24 days. In consideration of the interventions, the unmet needs under various scenarios were evaluated and dynamic perturbations to the resource capacities were evaluated through sensitivity analysis. Sensitivity analysis were also conducted on key parameters that would have a significant impact on the capacity projections -the average length of stay (ALOS) for COVID patients in the ICU and ISO wards and the proportion of the daily national COVID demands that come to the IH. Validation of the results was conducted with the management representatives who are involved in the COVID planning and operations command in both the IH and the national health authorities.

Results
As of 21 April 2020, there were 3,566 COVID-19 patients in nationwide hospital isolation wards and 27 in intensive care, translating to about 31.7% of inpatient hospital beds used for COVID-19 patients. The IH has around 1,785 beds and 18 ICU beds in 2019. However, the bed occupancy rate (BOR) at the IH, was at over 70% occupancy rate most of the time. As the surge of the number of cases from the dormitories evolved to be the dominant stream of confirmed cases, tracking for dormitory and community cases was started only from Day 74 (see Figure 4). The number of cases from the dormitories were observed to increase steadily since the first cases were tracked from 6 April 2020 and peaked at 1,426 cases on 20 th April 2020, declining thereafter (35). This stream of cases remained stable due to controlled active surveillance policies deployed for the dormitories, frontline workers and at-risk populations in the community.
The use case demonstrated the rapid evolution of the model to account for previously unexpected scenarios and new policies. The unfolding of the pandemic revealed the streams of new infections arose from two distinct sources: migrant dormitory population and community/imported cases. Given the dominant streams of positive cases from the dormitories and the distinct demographic characteristics between these two streams, a bi-agent model was deemed necessary. The later variant of the model also took into account the rapidly changing resource management policies both for in-hospital and external isolation facilities, swab and discharge policies. Within the hospital, the operations in a satellite FSA located separately from the ED were ramped up. The bi-agent model incorporated the two entry points into the IHthe ED and FSA. Decisions were then made through these areas for either hospitalization or swab and discharge (SDC) under the Swab and Discharge Programme (SDP). ED attendees were referred to FSA if they had mild conditions and required simple examination and consultation while FSA would send patients with more severe symptoms to the ED. During the period from 1 st till 20 th April 2020, 1,589 SDCs were carried out from the total of 2,189 suspect assessments. Out of the 600 cases admitted in the IH, 404 (37.10%) and 196 (17.82%) admission decisions were made at ED and FSA respectively. During the same period, it was noted that the higher positive swab rates corresponded with a higher number of hospitalizations.
In the data collection period, 242 cases of confirmed Covid-19 were detected from the 600 hospitalized subjects. The differences in the positive swabs across the two premises revealed the severity of cases seen at each facility.
As of 12 May 2020, approximately 3,900 tests per hundred thousand people in Singapore were conducted (26).
Based on an analysis of 766 COVID-19 cases, it was determined that the duration of viral shedding through polymerase chain reaction (PCR) tests via nasopharyngeal swabs could be longer than 33 days for approximately 5% of all confirmed cases (36). Further external evidence has also shown that viable viral replication drops rapidly after 7-10 days from the onset of symptoms (37). Consequently, it was determined that deisolation and discharge policies should not depend solely on the viral ribonucleic acid (RNA) detection via PCR tests (36). More aggressive discharge of patients based on the evidence presented on the time of course of infectiousness and other clinical parameters, rather than PCR results, were subsequently instituted. This led to a higher discharge rate of patients from Day 110 as shown in Figure 4, and better resource focus on patients with early presentations and those with acute respiratory symptoms which would have positively impacted timelier public health intervention and containment (36). Age-dependent risk factors for severe COVID-19 have been reported in existing research (14,31,38). An unpublished study by the National Centre for Infectious Diseases (NCID) in Singapore estimated the age-adjusted risks of step-up care for Singapore COVID-19 patients (from non-ICU to ICU beds and from EISO to ISO) based on data of 1,481 patients. The risk of patients requiring ICU care in the limited empirical study on Singapore's COVID-19 patient profile ranged from 0% for the population below 30 years old to 19.45% for patients who are 65 years old and above. The age-adjusted ICU conversion rate was determined to be 1.55% for the dormitory and 4.95% for the community stream. The age profile of dormitory cases is also different from the age profile of community cases (see Figure 5). The bi-agent model was re-calibrated with the new age-adjusted risk assumptions. The number of national ICU and non-ICU caseloads was validated to track the epidemiological and interventional changes to support policy decisions on the need to either expand or reduce the number of ICU, in-hospital and external isolation beds.
For the use case presented, model calibration was made against historical data from 1 April till 30 April 2020. The model was then used to project the demand for the period from 1 May 2020 till 31 May 2020. The calibration curves closely matched the historical trends and tracked the rapidly evolving dynamics, from stabilising community infection to the surge predominantly arising within the dormitories. More than 90% of the cases were projected and observed to be from the migrant dormitories. The projections based on the bi-agent model (including the time to peak for ICU and ISO bed needs for the IH and national demands) with percentiles of the sensitivity analyses for the baseline, best, base and worst cases are listed in Table 2. Sensitivity analysis results for the bed demands, assuming 4-8% and median ICU LOS of 7-18 days, are shown in Figure 6 for the best and base case parameters for IH ICU beds requirements.

Discussion
During the early stages of the pandemic, the lack of scientific knowledge regarding the SARS-CoV2 betacoronavirus resulted in the difficulties of accurate predictions and the necessary precautionary measures to take (e.g., the wearing of face masks). Due to the inherent uncertainties, any predictive models must be able to learn from new data, and the health system must have the capability to assimilate the new knowledge, plan and respond effectively. Health system resilience, which is the capacity of health actors, institutions, and populations to prepare for and effectively respond to crises whilst maintaining core functions when a crisis hits (39), is crucial in the sustainable delivery of care by the health system during such a novel pandemic. This study is a first step towards realizing a framework to achieve the goals of a resilient healthcare system, with the use case that demonstrated the ability of the health system to prepare, respond and strengthen the health services delivery system during the COVID-19 pandemic (41,42).
To achieve the goals of a resilient system, the main challenges that the proposed modelling framework has been able to deal with are: (1) demand side uncertainties related to the uncertain pandemic scenarios, evolving detection and disposition policies (43); (2) supply side uncertainties related to the dynamic resource management policies including the bed capacities of in-hospital and external isolation facilities; (3) parametric uncertainties related to the lack of precise and accurate estimation of key parameters required for the modelling framework, and; (4) a collaborative framework that facilitates the development of the model and assimilation of modelling insights by key stakeholders. The proposed modelling framework is agile and adaptable, and is supported by a strong ecosystem to facilitate the assimilation of new data and knowledge for evidence-based decision support. The presence of robust communication channels throughout the health system further ensured the timely and accurate dissemination of modelling insights. With the ability to continuously learn and calibrate the responses with new data and knowledge, the framework facilitated the rapid reorganization and adaptation to achieve a resilient health system (39,40).
With the dynamic evolution of the pandemic in Singapore, the model had been rapidly adapted to deal with demand side uncertainties brought about by new outbreak scenarios and evolving response strategies.  (Day) dormitory cases. This happened in the initial stages of heightened pandemic preparedness period during national disease outbreak level of "Orange" (44). The dominant stream of infections in migrant dormitories from day 74 (since the 1 st case was detected on 23 rd January 2020) began to surge rapidly despite the additional measures introduced in the heightened preparedness period. Fortunately, the massive dormitory cluster was contained and separated from the community, thereby preventing widespread community transmission. After Day 74, the model quickly evolved to consider a bi-agent model to take into consideration the stream of infections from the dormitory clusters, separated from the community cluster. With the updated model, we were able to differentiate the unique stream from the dormitories which grew from 6 th April till 6 th May 2020 to stabilise at an average of 98.8% (IQR: 0.11%) (from 7 th till 31 st May 2020). There had also been a number of dynamic adjustments made in terms of the detection, diagnosis and disposition policies for COVID-19. These included the controlled rates of swabbing in the dormitories and community, heightened surveillance and targeted testing for vulnerable groups (27), the agile flexing of in-hospital capacities and the swift ramp-up of external isolation facilities (28). The agile modelling framework allowed for these adaptations, and the incorporation of age-adjusted risk modifiers based on the most updated datasets.
On the supply side, the modelling framework supported high-resolution resource planning decisions over the next month. In anticipation of the surge in demand prior to the migrant worker outbreak, the IH had progressively increased its bed capacities and healthcare resources. The IH had surge plans for her ICU capacity from a pre-COVID level of around 40 beds to 200 beds. Across the various phases of surge, the additional capacities for ICU require setup times ranging from 1 day to 15 days to ramp up. Consequently, plans have to be established at least two weeks in advance to prepare for any potential surge in infections. By working in tandem with external isolation facilities run by the government and private hospitals, the modelling results showed that the swift decanting of Covid-19 patients with mild symptoms to external CIFs and CRFs had further prevented the over-congestion of hospital capacities as seen in other jurisdictions. These external isolation facilities are manned by trained medical and nursing manpower that can ensure all patients are provided with adequate care and health monitoring services. Model results showed that the planning capacities for the IH and national ICU beds was well-prepared to deal with the worst-case demand projections at the 97.5 th percentile level against the uncertain parameters. For the non-ICU beds, the planning capacity appeared to be sufficient to deal with 97.5% for the best-case scenario in the IH bed demands, and the base case scenario for the national ISO bed demands (which included the external isolation facilities). By maintaining the sufficiency of healthcare capacity with a respectable safety margin, Singapore was able to keep the CFR to be amongst the lowest in the world (8).
Data transparency and information sharing are important to develop robust policies under parametric uncertainties. The database architecture within the AMC was well-developed prior to the pandemic. The core data science expertise embedded in the health system have the necessary domain knowledge to pool together the set of realworld data from the various source systems and the enterprise data warehouse (EDW) that is necessary for the model to be updated with the latest projections. Based on patient demographics, comorbidities, laboratory and radiological test results, the bed types were generalized to consider high needs patients (ICU), patients with minor symptoms but require isolation in external isolation facilities and those who require isolation and higher levels of in-hospital care. Clinical risk factors that have been considered in patient disposition included age, chronic comorbidities (diabetes mellitus, heart, lung and kidney diseases), supplementary oxygen needs, clinical features (e.g., dyspnoea, respiratory rates and SpO2 levels), chest X-rays and laboratory results. The data showed limited risks of patients turning severe to require higher levels of care (e.g., in-hospital beds and ICUs). To achieve more robust insights, sensitivity analyses were performed to evaluate the resource needs (ICU and external vs inhospital conversion rates and, the LOS in these different facilities) and the best, base and worst-case scenarios in the pandemic. The projections and sensitivity analysis given the uncertainties in the risks of ICU conversion and LOS provided useful inputs for the policy makers and have been utilized to guide the evaluation and improvement of the pandemic preparedness plans.
Rapid health systems modelling during pandemics requires a strong collaborative effort amongst a wide variety of stakeholders (45,46). The modelling team in this study comprised of senior clinicians from the emergency medicine, critical care medicine, infectious disease, epidemiological and health services research domains. The modelling expertise came from disciplines ranging from industrial engineering, computer scientists and biostatisticians within the AMC. The health services research and data science team within the AMC supported a robust collaborative infrastructure that coordinates key technical, clinical and operational partners is in place. The team was able to quickly adapt the model with access to high resolution clinical and operational data. The formal organization structures established since the beginning of the outbreak (e.g., health system's disease outbreak task force, the critical care and ICU planning team, the operating theatre and bed management unit) facilitated the rapid dissemination of study results. Clear channels of communication with healthcare policy-makers and stakeholders, together with ready availability of robust and credible data sources, ensured that the models are realistic and credible for the use by decision-makers. All these factors are critical considerations for the realization of a resilient healthcare system during pandemics (41).
The main limitations of the modelling framework are that the model was built in consideration of the bed management policies in a public hospital that was demonstrated in a use case related to a sudden surge in infections from a well-defined source (migrant workers' infection in the dormitories). Nonetheless, the modelling framework can be customized to consider different types of bed resources, patient demands, and process flows. Despite the ability of Singapore's health system to adapt and respond swiftly to the migrant workers' clusters, there is a need for constant vigilance. Robust data sharing and collaborative mechanisms must be maintained to effectively manage the bed capacities given the constantly evolving situation. Moving forward, the dynamic hypotheses captured by the current models have to continually evolve over time. Even as we see declining cases in Singapore, sporadic numbers of community and imported cases have been detected through active surveillance and the screening of targeted groups (26). As the countries start to gradually exit from lockdowns in phases, and transnational travel are revived, allowing for the cross-border flows of people (48), the risk of new waves of infections is a realistic concern (49). Given that Singapore is an open economy and has proceeded to open up the economy in phases, future resurgence of the pandemic may be possible until effective vaccines or drugs can be developed.

Conclusion
This study showcases a modelling framework that was successfully deployed in a use case in Singapore for healthcare resource planning during the COVID-19 pandemic. The rapidly evolving pandemic and growing clinical and scientific knowledge of the disease necessitated the development of an agile modelling framework. The framework provides a platform for decision-makers to quickly evaluate complex cause-and-effect relationships, internal feedback and delays to support resource planning in a holistic manner. The study has also shown that the tightly integrated nature of the Singapore healthcare system is important to enable close coordination and timely information sharing across diverse groups of stakeholders and decision makers.