Development of a Social Distancing Monitoring System in Korea: Results of a Modified Delphi Process

Abstract

Background: Since the COVID-19 pandemic is an ongoing situation in most countries worldwide, a “social distancing” policy as a non-pharmaceutical intervention has been implemented for several months in many countries including Korea. Social distancing policies work in different ways and at different levels. In addition, various forms of surveillance systems have been implemented in different countries. However, there is an almost complete lack of specific surveillance system in Korea to effectively monitor social distancing policy. This study aims to develop a monitoring system for social distancing measures in Korea to evaluate and improve the implemented policy.

Methods: A draft monitoring system was developed after reviewing Korea's social distancing measures (central and local government briefings) and checking available data for applications. The modified Delphi process was used to evaluate the draft of the monitoring system. In total, 27 experts participated in the evaluation. The round 1 evaluation includes (1) commenting on the composition of the monitoring fields (open response), (2) monitoring indicators for each monitoring field (10-point Likert scale), and (3) commenting on the source of data used to develop the monitoring system (open response). In the round 2 evaluation, 55 indicators, excepting open responses, were re-evaluated.

Results: The response rate for the Delphi survey was 100% in both the first and second rounds. Of the 55 indicators, 14 were excluded according to experts’ open response comments, as these indicators did not satisfy the quantitative criteria. Finally, 41 indicators were included with 12 available data sources. The monitoring system domain was divided into input, process/output, and result.

Conclusions: This study is significant in that it is the first in Korea to develop a comprehensive monitoring system for social distancing policy, and is applicable to estimates utilizing data that are immediately available for each indicator. Furthermore, the developed monitoring system could be a reference for other countries that require the development of such systems to monitor social distancing measures.

Background

On March 11, 2020, the World Health Organization (WHO) declared the COVID-19 pandemic. In total, more than 118,300 people had been infected, and 4,291 had died in 114 countries [1]. To overcome the situation, several pharmaceutical companies had been working to develop vaccines and treatments, vaccination is taking place worldwide. However, the effects of vaccination for preventing the spread of viruses will take some time and Covid-19 virus variants continue to be reported [2]; thus, public health measures such as wearing masks and maintaining social distance would likely remain important [3, 4].

Social distancing is a public health measure that reduces the spread of viruses by reducing physical contact between people [5]. Social distancing measures are defined and implemented in various ways at different levels depending on the country [6]. The WHO has delineated guidelines for social distancing measures according to situations and locations (e.g., workplaces, schools, group gatherings, public places, and transportation) [7]. The European Centre for Display Prevention and Control (ECDC) defined and delineated social distancing according to the individual and group level. Social distancing at the individual level includes isolating cases, isolating contacts, and the recommendation to stay indoors. Social distancing at the group level includes educational institution closures, cancellation of rallies/demonstrations, restrictions on movements between buildings or residential areas, and mandatory quarantine [8].

As social distancing policies vary, various monitoring systems are available [9–11]. Social distancing worldwide is mainly monitored by non-profit organizations and understood as the presence or degree of implementation of the policy in different countries. In addition, various private organizations such as Google provide data to identify social distancing levels [12–14].

In Korea, social distancing is analyzed and presented in various ways at the institutional level [15, 16]. The National Disaster and Safety Status Control Center has announced the results of analyzing mobile phone traffic, credit card use data, and public transportation utilization in the Seoul metropolitan area. Private research companies also reported indicators based on their regular COVID-19 risk perception survey results such as individuals’ social distancing practice rate, and restrictions on outdoor activities or eating out. However, social distancing has not yet been sufficiently evaluated with such information from a one-time and individual-level survey. Therefore, no comprehensive monitoring system is in place.

Under unforeseeable circumstances due to infectious diseases, it is very difficult for policymakers to determine when, how, and to what degree policy interventions should be implemented. The social impact of infectious diseases has spread beyond the public health and medical domains. It is difficult to measure this changing environment due to infections, and there is a limitation to monitoring with the current surveillance system. Simply put, the present public health care monitoring systems alone are not sufficient to accurately assess the current COVID-19 pandemic situation. Therefore, it is necessary to develop various indicators and establish a better monitoring system to assess the effectiveness of the social distancing policy, along with understanding the COVID-19 pandemic situation.

Therefore, this study aims to identify the status of social distancing measures in Korea, develop an appropriate monitoring system based on available data sources, and utilize them for policy evaluation and improvement in the future.

Methods

Study Design

The modified Delphi process [17], consisting of two surveys and a three-step process, was used to reach consensus on the overall draft of the developed monitoring system for social distancing domains and indicators. The Delphi method is used to achieve consensus on a particular topic by having experts vote through multiple meetings and provide feedback [18, 19]. The difference between the modified and conventional Delphi process is that it can reduce the number of surveys from rounds 3 to 2 using structured questionnaires from the beginning, improving the convenience and efficiency of the survey. In addition, it was carried out online to increase the participation of experts from various fields and overcome the shortcoming of converging expert opinions according to their reputations and authorities. After evaluating the primary draft (checking scores and open responses) in the first round, we obtained consensus on open responses through adding and deleting indicators. In the second round, we reassessed the drafts (score checking only) to establish a final monitoring system. Figure 1 shows a flowchart of the study method.

Participant Recruitment

Since social distancing is a policy [20] to prevent the spread of COVID-19, 27 experts were recruited based on two major conditions. They were (1) clinical healthcare professionals related to COVID-19, and (2) healthcare system development and analysis specialists. The composition of experts included not only those knowledgeable in the COVID-19 virus, but also other infectious disease specialists, nursing professionals who work with routine infection control and patient management in hospitals, and public health and preventive medicine specialists with experience and knowledge in public health measures and policies. Experts were invited to participate in the modified Delphi process via email. Those who agreed to participate were sent an email with an online link to conduct the first survey.

Initial Dimensions and Criteria of Monitoring System

Only a few studies have investigated social distancing policies related to COVID-19 [21–23]. In addition, while various forms of social distancing policy monitoring systems are being established in other countries, there are several limitations to the use of a monitoring system in Korea. First, since different countries have implemented social distancing policy in various ways, it is not appropriate to include their policy indicators (e.g., complete lockdowns, restrictions on movement between cities) that have not been considered in Korea. Second, even though similar policies may have been implemented in Korea, these are still difficult to apply because the recommended level or detailed methods may differ. Because of these problems, we believed that a systematic review of the methodology and monitoring systems in other countries would not be suitable for this study. Therefore, this study reviewed social distancing policies at the national and local government levels to develop a monitoring system draft based on available data sources.

First, 207 regular COVID-19 infection briefings (from January 31 to October 7, 2020) by the National Disaster and Safety Status Control Center were reviewed to understand the overall social distancing policy trend in Korea. In addition, 1,101 regular press briefings and press releases issued (from March 4 to October 7, 2020) by the Gyeonggi-do provincial government were reviewed to determine whether a special form of social distancing policy was implemented [24, 25]. Domestically available data sources included both public and private data; thus, 12 data sources including secondary data from surveys were selected. Furthermore, 55 indicators were selected according to available data sources in Korea, and the monitoring system domain was divided into five categories based on two levels (Table 1).

Table 1

Results of the assessments of indicators in the Delphi process.
Domain 1	Domain 2	Round 1			Round 2
		Mean	Median	SD	Mean	Median	SD
Direct-Policy implementation level
	1. Level of social distancing policy	9.3	10	1.61	9.5	10	1.07
	2. School closures	8.85	9	1.7	8.85	9	1.19
	3. Workplace closures	8.11	9	2.06	8.35	9	1.13
	4. Cancellation of events	8.33	9	1.71	8.54	8.5	1.07
	5. Restrictions on rallies/demonstrations	8.41	9	1.67	8.69	9	0.88
	6. Restrictions on the use of transportation	6.93	8	2.6	7.04	7.5	1.48
	7. Staying indoors (at home)	7.56	8	1.95	7.5	8	0.99
	8. Restrictions on domestic travel	7.11	7	2.19	6.92	7	1.2
	9. Control on international travel	8.52	9	1.72	8.77	9	0.86
Direct-Community level
	10. Rate of implementation of telecommuting	8.41	8	1.39	8.23	8	0.91
	11. Experience rate of working even when sick	8.59	9	1.47	8.81	9	0.9
	12. Facility measures performance rate	8.15	9	2.14	8.35	8	1.09
	13. Floating population change	8.41	9	1.97	8.54	9	1.17
Direct-individual level
	14. Facial mask wearing rate	9.52	10	0.8	9.69	10	0.55
	15. Indoor facial mask wearing rate	9.15	10	1.38	9.54	10	0.76
	16. Hand washing practice rate	9.26	10	1.1	9.54	10	0.71
	17. Use rate of hand sanitizers or soap	8.3	9	1.81	8.65	9	1.38
	18. Degree of daily life change	8.11	8	1.6	8	8	0.94
	19. Residential mobility rate	6.89	7	1.74	6.85	7	1.32
	20. Experience rate of the cancellation of hospital visits	7.59	8	1.87	7.27	7	1.19
	21. Movement rate in retail stores and leisure facilities	8.41	8	1.45	7.96	8	1.11
	22. Movement rate in grocery stores and pharmacies	7.96	8	1.68	7.62	8	1.13
	23. Movement rate in parks	7.37	7	1.9	7.12	7	1.42
	24. Restriction rate of using multi-use facilities	8.44	9	1.5	8.31	8	1.05
	25. Restriction rate of outdoor activities	7.93	8	1.52	7.77	8	0.99
	26. Restriction rate of eating out	7.74	8	1.4	7.62	8	0.98
	27. Experience rate of the cancellation of group gatherings (dinner)	7.67	8	1.44	7.69	8	0.84
	28. Restriction rate on the use of public transportation	7.67	8	1.69	7.77	8	0.76
	29. Change rate of movement at public transportation stops	7.37	8	1.94	7.46	8	1.39
	30. Change rate of bus use	7.89	8	1.74	7.92	8	1.02
	31. Change rate of subway use	7.89	8	1.74	7.81	8	1.2
	32. Change rate of taxi use	7.59	8	1.5	7.62	8	1.1
	33. Change rate of car use	7.96	8	1.4	8	8	0.94
	34. Change rate of railway use	7.44	8	1.8	7.65	8	1.2
	35. Purpose-specific transportation utilization rate	7.04	7	1.74	7	7	1.1
	36. Change rate of domestic flight use	7.63	8	1.84	7.73	8	1.28
	37. Experience of domestic travel	7.81	8	1.64	7.58	8	1.03
	38. Change rate of tourist visits at hotspots	8.15	8	1.54	7.92	8	0.98
	39. Change rate of international flight use	7.93	8	1.9	7.85	8	1.19
	40. Change rate of foreigner arrivals	8.22	9	1.86	8.19	8	1.27
	41. Change rate of resident (Korean) departures	8.19	8	1.8	8	8	1.3
	42. Rate of controlling working hours	7.85	8	1.58	7.69	8	1.01
	43. Commuting rate	7.07	7	1.74	6.88	7	1.24
Indirect level
	44. Change rate of non-contact purchases	8.3	8	1.37	8.27	8	0.92
	45. Change rate of food delivery	8.41	8	0.92	8.27	8	0.53
	46. Change rate of TV home shopping, and Internet shopping purchases	8.11	8	1.58	7.96	8	0.77
	47. Experience rate of purchasing necessities in bulk	6.85	7	1.63	6.77	7	1.31
	48. Change rate in the number of restaurants/entertainment sales	8.04	8	1.32	7.88	8	0.91
	49. Change rate of private academy sales	7.33	8	1.69	7.27	8	1.31
	50. Change rate in the number of sports/culture/leisure facility sales	7.74	8	1.61	7.54	8	1.36
	51. Change rate in the number of travel/transportation sales	8.04	8	1.45	7.88	8	1.14
Others
	Practice rate of social distancing of 2 meters between individuals	7.48	8	1.97	7.62	8	1.53
	Non-contact meeting implementation rate	8	8	1.66	8.19	8	1.13
	Monitoring rate of facility workers or visitors	7.3	8	1.82	7.73	8	1.19
	Closing rate of public areas	7.22	8	1.79	7.54	8	1.14

Data Analysis

Round 1

The first round of the modified Delphi process was conducted for six days from July 9–14, 2020. In total, 27 experts from various fields received e-mails with information on the overall research, purpose of the modified Delphi process, detailed monitoring domains and indicators, and survey links for the appropriateness of the data source assessment. The survey consisted of (1) feedback on monitoring domains (open response), (2) evaluation of indicators by monitoring domain (10-point Likert scale: from “1 = no need at all” to “10 = absolutely needed” and Open Responses), and (3) opinions regarding the data sources used (Open Response). Based on SMART criteria [26], each indicator was evaluated on a 10-point Likert scale so that quantitative results could be estimated, and the open responses were utilized to provide various opinions for the overall evaluation parts. The 'SMART' criteria(Specific, Measurable, Attributable, Reliable, and Timely) is often used in developing indicators for monitoring and evaluation. Those who had not yet completed the survey were reminded to do so three days before the deadline.

Between Rounds

Data from round 1 were collected and analyzed. For the evaluation of the indicators, the mean, median, and standard deviations were calculated. SAS 9.3 was used for the statistical analysis. The second round of the survey was conducted for indicators with an average score of 7 or higher. All open responses were submitted to the entire research team prior to the initiation of the second round of the study, including recommendations for changes to other indicators or comments regarding the addition or exclusion of indicators, as well as to the purposes and scope of the monitoring domains or opinions about data sources. In response, our research team members discussed whether to reflect on the study based on accessibility, feasibility, timeliness, etc. Thus, except for the open response questions, the second round of evaluation for indicators in each monitoring domain was conducted using a 10-point Likert scale. In the second round, the results from the first round (average and median) were shared with the experts by providing the survey link via e-mail.

Round 2

The second round of the modified Delphi process lasted six days from July 17–22, 2020. The survey was written using Google Forms and distributed to the same experts (n = 27). An assessment of the same 55 indicators by monitoring domain as in the first round was performed in the second round. The mean and median values were provided for each indicator and evaluated on a 10-point Likert scale using the same SMART criteria as in the first round. Those who did not complete the survey were reminded to do so one day before the deadline.

Priority

The most important principles for developing a monitoring system were (1) policy linkage and (2) measurability (application in reality), which were reflected in the open response comments. In addition, the indicators were chosen based on an average score of 7 or higher and included any other indicators from the discussion among research team members, which may have been greatly affected by the social distancing policy.

Results

Round 1 – Based on Monitoring Indicator Evaluations

In total, 55 potential indicators were included in the first round of voting (Table 1), and all indicators excepting “public transportation restrictions” (6.93 points), “residential mobility rate” (6.89 points), and “high expenses for daily necessities experience rate” (6.85 points) were evaluated as higher than 7 points. Thus, all indicators were evaluated in the second round. However, apart from the calculated scores, the exclusion, segmentation, and addition of indicators were suggested in the experts’ open responses. Additional discussions to reflect on their opinions were conducted. Open responses will be addressed in the discussion on these later in the paper.

Round 2 – Based on Monitoring Indicator Evaluations

Similar to the first round of evaluation, an assessment of the same 55 indicators by monitoring domain was conducted in the second round (Table 1). For each indicator, the mean and median scores were obtained, which made a difference from the previous round. As with the results of the first round, two indicators were considered less important: “residential mobility rate” (6.85 points) and “high expenses for daily necessities experience rate” (6.77 points), which scored less than 7 points on average. However, in the open responses, only “residential mobility rate” was excluded from the final monitoring system since the importance of “high expenses for daily necessities experience rate” was emphasized. In addition, “restriction of public transportation” scored 7.04 points in the second round, highlighting its importance. Although the “restriction on domestic mobility” indicator was considered important in the first round (7.11 points), it was evaluated as 6.92 points in the second round. Both indicators were included in the direct policy level domain, which was excluded from the finalized monitoring system because the monitoring domain was modified based on the discussion of open responses. A detailed explanation is provided in the section discussing the open responses.

Discussion about Open-Ended Responses – Monitoring Domain

Most experts stated that the monitoring system domain was not clearly classified. The issue was raised that if monitoring system indicators are distinguished as being direct or indirect, some may overlap. Thus, it was suggested that the monitoring domains be delineated as input, process/output, and result domains. Therefore, the research team removed the direct/indirect level of domain 1. The research team revised the policy level of domain 2 to the “Input” domain, and the community/individual/indirect level of domain 2 to the “Process/output” domain. In addition, as the consequences of social distancing policy, the indicators for results were added. As such, the indicators were categorized as input, process/output, and result domains (Fig. 2). The indicators related to national social distancing policies were included in the input domain, and those affected by social distancing measures in action were included in the process/output domain. Furthermore, the indicators related to the consequences of the social distancing policy and evidence to control the level of measures were included in the result domain.

Discussion about Open-Ended Responses – Indicators

In addition to the index-specific evaluation scores, the necessary indicators were added and conventional indicators subdivided based on the experts’ feedback. Some indicators were combined or excluded according to the revised monitoring domains.

Direct indicators: Policy level → Input domain

Nine monitoring indicators for direct indicators in the policy level domain were suggested by the team referring to the social distancing policies implemented worldwide. The experts provided their opinions about the nine monitoring indicators. Since the chosen indicators were based on the contexts of other countries, they have a limitation in that it is difficult to adequately monitor social distancing measures in Korea. Thus, indicators were changed to precondition the establishment of social distancing measures in Korea. Therefore, measures such as the prohibition of events (e.g., general community meetings, sports events) and closure of institutions (e.g., multi-facilities, schools, large companies), and continuously emphasized personal quarantine guidelines were included as indicators in this domain.

Direct indicators: Community level → Process/output domain

Among the direct indicators presented by the researchers, four indicators were selected, displaying the degree of social distancing measures at the community (group) level. The experts’ opinions were reflected as follows: Most experts suggested adding more indicators. “Whether to participate in online church worship services” and “the rate of conducting online lectures” (schools, private academic classes, etc.) were added based on representativeness and validation. There were recommendations to add "monitoring rate for employees or facility visitors,” "recording of guest book or QR code utilization rate,” and "temperature measurement rate of visitors,” which replaced the "facility measures performance rate" presented in the early version. Some suggested adding an “non-contact meeting conducting rate” indicator, but this was not reflected in the monitoring system since it was not possible to obtain data sources for the measurement.

Direct indicators: Individual level → Process/output domain

Among the direct indicators presented by the researchers, 38 displayed the degree of social distancing policy at the individual level. The experts’ opinions were reflected as follows: Because of the number of indicators in the domain, it was suggested that a representative indicator be selected. Overall, the initial versions of the monitoring indicators and additional indicators based on the experts’ comments were collected to determine representative indicators. Specifically, indicators such as “residential mobility rate” and “purpose-specific transportation utilization rate” were excluded as monitoring indicators since they were not clearly defined because of data source limitations. In particular, the “residential mobility rate” was scored below 7 points in the evaluation.

For additional indicators, considering the definition and purpose of the process/output domain and indicator validity, indicators such as “rate of commuting with own car” and “frequency of daily ventilation at home” were added. There were suggestions to add the measurement indicator "use of CCTV images and infrared sensors," but this was difficult to reflect in the monitoring system because there are currently no available data sources.

Indirect indicators: Indirect level → Process/output domain

The indirect level of monitoring indicators suggested by the researchers displayed eight indicators that could not be considered directly affected by social distancing measures, but could be indirectly impacted. However, because of the vague domain definition of indirect effects, there were more opinions on domain modifications than opinions on indicators within the domain. In conclusion, all indicators within the indirect level were included in the process/output domain.

Discussion about Open-Ended Responses – Data Source

There were many negative opinions regarding the data sources. Most opinions revealed that currently available data sources have problems of timeliness, feasibility, and representativeness. However, these data sources were not excluded, as they were considered meaningful and could be immediately applicable to monitoring social distance policy. Thus, the monitoring system developed in this study was based on (1) the available data source at this stage and (2) data source accessible at the metropolitan level.

Finally, 12 data sources were included: 3 data sources with characteristics of primary data (Saramin survey, COVID-19 perception survey by Korea Research, Reports (current situation) by the National Disaster and Safety Status Control Center), and 9 data sources with secondary data (Google Mobility, Floating populations and phone calls provided by SKT in Tmap data, Floating populations data provided by KT, Credit card Sales data provided by Shinhan Card, Credit card Sales data provided by BC Card, Transportation data provided by Gyonggido Traffic control center, Korean Travel Survey, Korea Tourism Statistics, Air transport statistics). Table 2 provides details on each data source.

Table 2

Data source explanations.
Types of data source	Name of data source	Details (explanations)
Primary data sources	Saramin survey	- Variables included: Type of business companies (large, mid-sized companies, small, etc.), type of industry (financial/insurance, information and IT), telecommuting or not, telecommuting reasons, stress due to social distancing, areas of stress due to social distancing, degree of feeling stress due to social distancing, need for social distancing, types of social distancing practice in daily life, perceptions regarding taking “3–4 days off if feeling sick,” reasons, applicability of measure, reasons why it cannot be implemented, experience commuting even when feeling sick, etc. - Data period: Once
	COVID-19 perception survey by Korea Research	- Variables included: Monthly, whether or not the following measures are in practice in regions (Seoul, Incheon/Gyeonggi, Daejeon/Sejong/Chungcheong, etc.): mask wearing, indoor mask wearing, use of hand sanitizer, daily life changes, restrictions on use of public transportation, restrictions on visits to multi-facilities, refraining from going or eating out, cancellation of gatherings (group dinner), control of work hours, cancellation or delay of hospital visits, etc. - Data period: Irregular
	Reports (current situation) by National Disaster and Safety Status Control Center	- Variables included: Number of business facilities such as Karaoke bars, Internet cafes, nightclubs, private educational institutes are under infectious disease preventive control management - Data period: Irregular
Secondary data sources
	Google Mobility	- Variables included: Nationality, dates movement data for retail stores, leisure facilities, grocery stores, pharmacies, parks, public transportation stops, and working from home arrangements - Data period: Daily
	Floating population and phone calls provided by SKT (SK Telecom) in Tmap data	-Variables included: dates, Si-Gun-Gu, time (hour unit), sex, age (10-year), number of floating population, type of business classified as large scale (amenity, restaurant, medical facility), type of business classified as small scale, rate of making phone calls, search location ranks, etc. - Data period: Monthly
	Floating population data provided by KT	-Variables included: dates, Si-Gun-Gu, time (hour unit), sex, age (10-year), number of floating population - Data period: Monthly
	Credit card sales data provided by Shinhan Card	- Variables included: dates, Si-Gun-Gu, Eup-Myeon-Dong, type of business classified as large scale (restaurant, entertainment, travel, transportation), type of business classified as medium scale, type of business classified as small scale, sex, age, location, income level, credit card sales amount, number of credit card sales - Data period: Daily
	Credit card sales data provided by BC Card	- Variables included: dates, Si-Gun-Gu, Eup-Myeon-Dong, type of business classified as large scale (restaurant, entertainment, travel, transportation), type of business classified as middle scale, type of business classified as small scale, sex, age, location, income level, credit card sales amount, number of credit card sales - Data period: Daily
	Transportation data provided by Gyonggido Traffic control center	- Variables included: Si-Gun-Gu, month, day, number of passengers in express city bus/ regular bus/town bus, name of subway station, number of passengers using subway station, number of passengers in personal/corporate taxi, purpose-specific transportation use (commuting, going to school from home, working, shopping), sex, Si-Do, Si-Gun-Gu, purpose-specific transportation use by distance (units of 5 km) (commuting, going to school, shopping, leisure activities), type of transportation (car, bus, subway, taxi, bike, other) - Data period: Monthly/Annual
	Korean Travel Survey	- Variables included: Resident sex, age, occupation, education, number of people in household, household income, experience of domestic travel / resident or foreigner, number of visitors in major travel hotspot areas domestically, etc. - Data period: Monthly - Available data in 2018
	Korea Tourism Statistics	- Variables included: Purpose (tourism, commercial, public, study abroad, etc.), nationality of people arriving in Korea, number of departures of Korean residents, destination country, etc. - Data period: Monthly - Data source: Korea Tourism Organization
	Air transport statistics	- Variables included: International flights by region (Asia, Europe, North America, etc.), number of passengers, cargo weight in tons, number of domestic flights, number of passengers, cargo weight in tons, etc. - Data period: Monthly - Data source: Korea Airports Corporation, Incheon International Airport

Final Monitoring System

The final monitoring system was delineated as the input, process/output, and result domains, and included 41 indicators (Fig. 3). Detailed definitions are provided in Additional file 1.

The input domain is related to domestic social distancing policies and measures. According to the social distancing stage in Korea, the measures taken for events (general gatherings, meetings, social events, and sports events) and in institutions (multi-facilities/schools/private companies), and continuously emphasized personal preventive practices were included as indicators.

A process/output domain is created during the process influenced by social distancing measures (input domain). It was divided into eight domains including daily movement, domestic and international movement, personal hygiene and quarantine, school and workplace, personal daily life, consumption, risk perceptions, and facility measures, and included relevant indicators for each domain. In the daily movement domain, the rates for the change in floating population, restrictions on public transportation, change in types of transportation use, commuting, retail store and leisure facility movement, grocery store and pharmacy movement, movement in parks, and commuting with own car were included. In the national domestic and international movement domain, the change rates of the use of domestic flights, use of international flights, outbound travel by locals, arrival of foreigners, and average travel distance were included. In the personal hygiene and quarantine domain, the rates of hand washing practice, wearing of face masks, frequency of ventilation in the home, and practicing social distancing of 2 meters between individuals were included. The school and workplace domain included the rates of implementation of telecommuting, experience of working even when sick, working hours adjustment, conducting online meetings, and online lectures. The personal and daily life domain included the rates of controlling one’s going out, cancellation of meetings, cancellation of hospital visits, restrictions on using multi-use facilities, domestic travel experience, changes in tourist visits to hotspots, and participation in online church worship services. The consumption domain included the change rates of private academy sales, number of sports/culture/leisure facilities sales, number of travel/transportation sales, number of restaurant/entertainment sales, non-contact purchases (home shopping, Internet shopping, etc.), food delivery, experience of purchasing necessities in bulk, and total sales of personal hygiene products (face masks, hand sanitizer). The (risk) perception domain included the degree of change in daily life and degree of perception of social distancing measures. Measures for facilities included the rates of implementation of these measures and closure of public areas.

The results domain is the consequence of social distancing measures and related to controlling the stages of these measures. Indicators such as number of daily confirmed cases, rate of unknown infection routes, status or number of mass outbreaks, and rate of management in quarantine areas were included. In addition, the indicator “number of new self-quarantines” was included to identify contacts.

Discussion

Currently, different monitoring systems exist for various social distancing measures worldwide. While some institutions presented “blocking and closing” and “healthcare systems” on a 100-point scale based on policy intensity [10], others evaluated whether implementing such affective measures would impact the socioeconomic environment, healthcare system capabilities, and government intervention [9]. Studies have proven the efficacy of social distancing policy in preventing the spread of COVID-19 beyond monitoring [5, 6, 27].

This study reviewed social distancing measures at the national and metropolitan levels of Korea and developed a social distancing monitoring system based on data sources available in the country. The modified Delphi process was used to elicit the participation of experts from various fields such as infectious diseases, nursing, and health sciences. In total, 41 indicators were included in the input, process/output, and result domains.

First, domain-specific suggestions of the monitoring system presented in this study are as follows: Currently, the input domain simply contains the indicators of each type of social distancing measure. In the future, a policy evaluation study is needed to evaluate the feasibility and appropriateness of domestic social distancing stages. Furthermore, it will be necessary to develop tools to compare the levels of social distancing policies in Korea and other countries. For the process/output domain, a large number of indicators were provided by the surveys. However, various national monitoring methods are still needed to measure practice rates in a “realistic” or “objective” way, such as using CCTV to monitor hand washing and the wearing of masks, attaching infrared sensors on faucets, conducting observation surveys, and using video conference equipment in public conference rooms. In addition, it is necessary to develop and support measurement methods for indicators that cannot be added because of the lack of proper data sources. The result domain should clarify the definition and formula of the result indicators of COVID-19, which is intermittently produced by the National Disaster and Safety Status Control Center, so that the actual effectiveness of social distancing policy can be accurately determined.

Various types of environmental and institutional support must be presented for the monitoring systems developed in this study to work realistically. First, a national social distancing policy monitoring system should be established that combines all policies from each government department. Above all, it is necessary to design systematic surveys to assess the effectiveness of social distancing policies in a more objective way, rather than over the long term. Second, social distancing policies and social consensus should be defined. The current social distancing policies in Korea include national/institutional-level measures such as business restrictions on high-risk facilities, and school regulations and compliance with individual-level quarantine guidelines. In addition, confusion in the community continues in line with changing guidelines, as the definition for each stage of social distancing policy is not clear. Therefore, it is necessary to draw social consensus on the definition of social distancing policies and each level of measures, and share them in a more consistent way. Finally, social distancing policies that reflect group/organizational vulnerabilities should be developed. A social distancing policy should be prepared, and detailed guidelines introduced considering the characteristics of each place such as the working environment (small workplaces) and traditional markets, where applying conventional social distancing policies is difficult. It is also necessary to consider differences in social distancing measures specific to social/medical vulnerabilities and to provide active policy interventions that can support them.

Although the vaccine for COVID-19 has been developed, it has been announced that it will still take a long time to achieve the desired expectations [3, 4]. Social distancing measures to reduce the transmission of COVID-19 are being implemented worldwide, but research is lacking on how the policy affects the prevalence and mortality of COVID-19 [27]. It is very important to assess and monitor the effectiveness of social distancing policy not only for COVID-19 but also for an effective response to possible new infectious diseases in the future [27].

Strengths and Weaknesses

The limitations of this study are as follows: First, this study was conducted to establish a monitoring system for the social distancing measures implemented in Korea, and the definition thereof or consensus were not elaborated. Second, this study developed monitoring indicators based on currently available data sources, excluding infinite measurement methods and indicators that can measure social distancing policies.

Nevertheless, this study is meaningful in that it developed a comprehensive monitoring system for social distancing policy for the first time in Korea and provided available data sources by indicator. In addition, based on the social and economic impact created by social distancing policy [6], we developed indicators that distinguish domains by field and quantitatively understand the degree of social distancing practice in each. Since the COVID-19 pandemic is prolonged, not only in Korea but also in other countries, we can refer to our developed monitoring system for decision-making regarding the timing and intensity of various infectious disease response policies.

Conclusion

For the first time in Korea, this study developed a comprehensive and realistic monitoring system for social distancing policy by providing data sources that can be immediately identified by indicators. This study will be available not only in Korea but also in other countries that need to develop monitoring systems for social distancing policies. However, the results of this study should be applied in accordance with each country’s measures and contexts (environments), and the status continuously reported, analyzed, and updated, even after the monitoring system is established. Subsequently, efforts are required to continuously monitor the status of the system and utilize it to improve social distancing measures.

In addition, more research is needed to analyze the preventive effect of COVID-19 according to the scope and intensity of the social distancing measures taken in various countries. It is also suggested that a future study determine how each social distancing measure (e.g., school closures, suspension of public transport use, etc.) actually affects the prevention of COVID-19 by applying various mathematical prediction models.

Abbreviations

WHO: World Health Organization

ECDC: European Centre for Display Prevention and Control

Declarations

Acknowledgments

Not Applicable

Author’s contributions

SI, JO, SO, HY, designed the study. SI, JO, SO contributed to the data acquisition and analysis. All authors contributed to the interpretation of the analysis results. SI, JO drafted the publication, and managed subsequent revisions. All authors reviewed and approved the final manuscript. HY led the entire study process.

Funding

The work leading to these results was part of the study “Development and analysis of a Social Distancing Monitoring System”. Local government (Gyeonggi-do Provincial Government) budget was used to conduct the research. The funders had no role in the design of the study and collection, management, analysis, and interpretation of the data and in writing the manuscript. The authors and their contributions to the manuscript are independent of the funder.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Ethics approval and consent to participate

Ethical approval for the project was provided by Seoul National University Bundang Hospital Institutional Review Board (IRB No: n-2106688604). The need for consent was waived by the IRB at Seoul National University Bundang Hospital Institutional Review Board because the study was deemed not human subjects research as defined by HRPP SOP ver 3.9. The study was considered public health practice. Scoring and discussions during the Delphi process was anonymous.

Consent for publication

Not Applicable

Competing Interests

The authors declare that they have no competing interests.

References

World Health Organization (WHO). WHO Director-General's opening remarks at the media briefing on COVID-19-11 March 2020. In: Geneva, Switzerland; 2020. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020. [cited 18 Jun 2021].
World Health Organization (WHO). The effects of virus variants on COVID-19 vaccines. 2021. Available from: https://www.who.int/news-room/feature-stories/detail/the-effects-of-virus-variants-on-covid-19-vaccines?gclid=CjwKCAjwiLGGBhAqEiwAgq3q_q7rWj6iCvZ_vrGOGBAvMM1YRPYRRoUY-4eFjYO4ahL29Ix8ivz1yhoCH1oQAvD_BwE. [cited 18 Jun 2021].
Patel MD, Rosenstrom E, Ivy JS, et al. Association of Simulated COVID-19 Vaccination and Nonpharmaceutical Interventions With Infections, Hospitalizations, and Mortality. JAMA Netw Open. 2021;4(6):e2110782.
World Health Organization (WHO). Behavioural considerations for acceptance and uptake of COVID-19 vaccines: WHO technical advisory group on behavioural insights and sciences for health. 2020. Available from: https://www.who.int/publications/i/item/9789240016927. [cited 18 Jun 2021].
Ahmed F, Zviedrite N, Uzicanin A. Effectiveness of workplace social distancing measures in reducing influenza transmission: a systematic review. BMC Public Health. 2018;18(1):518.
Teslya A, Pham TM, Godijk NG, Kretzschmar ME, Bootsma MCJ, Rozhnova G. Impact of self-imposed prevention measures and short-term government-imposed social distancing on mitigating and delaying a COVID-19 epidemic: A modelling study. PLoS Med. 2020;17(7):e1003166.
World Health Organization (WHO). Overview of public health and social measures in the context of COVID-19: interim guidance. 2020. Available from: https://www.who.int/publications/i/item/overview-of-public-health-and-social-measures-in-the-context-of-covid-19. [cited 18 Jun 2021].
European Centre for Disease Prevention and Control (ECDC). Considerations relating to social distancing measures in response to COVID-19 – second update. 2020. Available from: https://www.ecdc.europa.eu/en/publications-data/considerations-relating-social-distancing-measures-response-covid-19-second. [cited 18 Jun 2021].
ACAPS. COVID-19 – Government measures. 2020. Available from: https://www.acaps.org/projects/covid19/data. [cited 18 Jun 2021].
Blavatnik School of Government(BSG). Coronavirus Government Response Tracker. 2020. Available from: https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker. [cited 18 Jun 2021].
PoliMap. A global repository of policy responses to tackle COVID-19. 2020. Available from: https://polimap.org/. [cited 18 Jun 2021].
Google. COVID-19 Community Mobility Reports - Google. 2020. Available from: https://www.google.com/covid19/mobility/. [cited 18 Jun 2021].
Deloitte. Predicting the post-pandemic rebound. 2020. Available from: https://www2.deloitte.com/ca/en/pages/about-deloitte/articles/covid-dashboard.html. [cited 18 Jun 2021].
Maryland Transportation Institute. COVID-19 Impact Analysis Platform. 2020. Available from: https://data.covid.umd.edu/. [cited 18 Jun 2021].
Central Disaster Management Headquarters. Regular COVID-19 infection briefings-5 Oct 2020. In: Seoul, South Korea; 2020. Available from: http://www.mohw.go.kr/upload/viewer/skin/doc.html?fn=1601863697249_20201005110817.hwp&rs=/upload/viewer/result/202102/. [cited 18 Jun 2021].
Hankook Research. Dashboard on regular COVID-19 risk perception survey results. 2020. Available from: https://public.tableau.com/profile/lee.donghan#!/vizhome/HRC_covid19_survey_dashboard/Intro. [cited 18 Jun 2021].
Murry Jr JW, Hammons JO. Delphi: A versatile methodology for conducting qualitative research. The review of higher education. 1995;18(4):423–436.
Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74(9):979–983.
de Meyrick J. The Delphi method and health research. Health education. 2003; 103(1):7–16.
Kissler SM, Tedijanto C, Lipsitch M, Grad Y. Social distancing strategies for curbing the COVID-19 epidemic. medRxiv. 2020. Available from: https://www.medrxiv.org/content/medrxiv/early/2020/03/24/2020.03.22.20041079.full.pdf.
Chen SL, Yen AM, Lai CC, Hsu CY, Chan CC, Chen TH. An Index for Lifting Social Distancing During the COVID-19 Pandemic: Algorithm Recommendation for Lifting Social Distancing. J Med Internet Res. 2020;22(9):e22469.
Gibson L, Rush D. Novel Coronavirus in Cape Town Informal Settlements: Feasibility of Using Informal Dwelling Outlines to Identify High Risk Areas for COVID-19 Transmission From A Social Distancing Perspective. JMIR Public Health Surveill. 2020;6(2):e18844.
Nilsen P, Seing I, Ericsson C, et al. Implementing social distancing policy measures in the battle against the coronavirus: protocol of a comparative study of Denmark and Sweden. Implement Sci Commun. 2020;1:77.
Central Disaster Management Headquarters. Regular COVID-19 infection briefings. In: Seoul, South Korea; 2020. Available from: http://www.cdc.go.kr/board/board.es?mid=a20501000000&bid=0015. [cited 18 Jun 2021].
Gyeonggi-do Provincial Government. Regular COVID-19 infection briefings. In: Suwon, South Korea; 2020. Available from: https://www.gg.go.kr/bbs/board.do?bsIdx=724&menuId=2910. [cited 18 Jun 2021].
Doran GT. There’s a SMART way to write management’s goals and objectives. Management review. 1981;70(11):35–36.
VoPham T, Weaver MD, Hart JE, Ton M, White E, Newcomb PA. Effect of social distancing on COVID-19 incidence and mortality in the US. medRxiv. 2020. Available from: https://www.medrxiv.org/content/10.1101/2020.06.10.20127589v1.full.pdf