Quantitative Data Collection Approaches in Subject-Reported Oral Health Research: A Scoping Review

doi:10.21203/rs.3.rs-1220476/v1

Background

The era of modern information technology enables multiple methods and modes of collecting self-reported oral health information. The scope of the use of these various modalities has not been well classified or summarized, therefore it is necessary to study the degree to which various technologies were used to collect oral health outcomes and related data. This paper reports on a scoping review of the range of data collection methods in quantitative research to measure self-reported oral health status. All methods of data collection were assessed; papers whose methodologies employed newer technologies were identified.

Method

The review was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) with the search on four online bibliographic databases. We analyzed peer-reviewed articles and reviews published between 2011 and September 2021 to determine data collection methods. Eligible publications were required to use quantitative methods that have a principal objective of measuring self-assessed oral health status, and/or oral health-related quality of life.

Result

We examined the past decade-long trends and found that there has been an increased interest in assessing self-reported oral health and oral health related quality of life data, a dominance of impact measures used, and the application of technologies to the methods. We found that 50 percent of all papers focused mainly on an impact measure. Technologies used included: computer-generated sample populations through polling techniques and crowd-sourcing software; together with tablets and computers to assist in data collection.

Conclusion

The study provides an overview of methods and modes to collect self-reported oral health data, especially with emerging technologies. Contemporary web-based approaches offer enhanced opportunities for survey research to boost recruitment, participation of study subjects with wide and diverse backgrounds from almost unlimited geographic areas, and efforts to improve quality by assuring increased response rates and reducing bias, especially in longitudinal studies. Several studies with strong standard methods of conducting oral health studies that have been successfully collecting these data for years were often not likely to adopt newer technologies because computer access is not available to their populations.

Quantitative Research

Patient-reported Outcomes

Dental Disease Experience

Oral Health Related Quality of Life

Data Collection

RG Watt et al[1] state that “a fundamentally different approach is now needed” to address the oral health of populations worldwide. For this to occur, it will be necessary to understand the perceptions of these populations, so that clinically determining approaches and remedies can be considered, while accounting for their clinical needs. Surveys of self-reported oral health can be used to bridge clinical needs and effective demand for oral health. After conducting several studies assessing oral health status in children ages 8-17 and their parents [2, 3], we became interested in the various methodologies employed in collecting self-reported oral health data, and whether we could identify new technologies used in survey research. Since this topic is broad, we decided to conduct a scoping review to understand what methods were being employed, and the extent to which new technologies were being incorporated in the research methodologies.

A scoping review provides a broad perspective of main research concepts and evidence of professional practices inclusive of many different study designs [4–6]. Compared to the systematic review, scoping review is a relatively new approach with the focus to identify knowledge gaps, possible areas for innovation, and to investigate research conduct [4, 7]. In assessing these scoping reviews’ limitations, we needed to better understand the range of methodologies used to collect subject-reported oral health status data and newer technologies devised to survey population-based samples. In the past decade, nine oral health and dental-related scoping review have been published. The focus of these reviews were: specific populations [8, 9], specific illnesses or conditions [10–14], disparities[15, 16], and treatment [17]. Although a few studied oral health perceptions, notably in relation to mediating and moderating factors in the social environment. None of these scoping reviews examined the use of oral health related quality of life (OHRQoL) and oral health status (OHS), and the methods employed in this literature.

The objective of this paper is to perform a scoping review that answers the following research question: “What data collection measures, methods, approaches, and/or modes have been used in conducting subject-reported surveys to measure oral health status?”

This work was implemented following the framework of scoping reviews [4, 5, 18] and presented according to the recommendations of the Preferred Reporting of Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR), as listed in Appendix A. [19].

2.1. Search Strategy and Data Sources

The research team with a health science librarian developed the search strategy to identify articles dealing with oral health subject-reported surveys. Survey search terms consist of three broad categories, including survey modes, subject-reported outcomes, and oral health and disease (See Appendix B for the full list of search strings). The search consisted of peer-reviewed, journal articles, conference proceedings and review papers with at least one keyword from each of three aspects. Four online databases: Ovid Medline, Embase, Web of Science, and Cochrane Reviews and Trials were used. In addition, we searched manually with similar keywords for the gray literature achieved on MedRxiv. The search focused on peer-reviewed and achieved papers written in English and published between 2011 to September 2021. We focused on the publications in the last decade to investigate the extent to which different modalities are being used and the trends that are occurring in this period. Final search was completed on September 29, 2021. Using Mendeley, all references were imported for review and appraisal. The duplicates have been identified in Mendeley and manually verified. After removing the duplicates, we tabulated data in Microsoft Excel for recording screening results and data charting.

2.2. Study Inclusion Criteria

Studies that did not answer the research question were excluded using a screening tool (Appendix C). First, a review of the title-and-abstract screening selected studies that conducted quantitative surveys to assess subject-reported OHS as their primary objective. Only surveys with more than three items that related to OHS were considered. Literature reviews focus groups and other qualitative studies were excluded, as were papers whose objectives were to validate measures or predict specific oral disease entities such as caries or gingival bleeding rather than over OHS. Studies primarily focusing on general health status or other systemic diseases instead of OHS were eliminated. Case-control studies that tested an active agent (e.g., therapy, experiment, and medicine) were excluded because the main research purpose was comparison treatment rather than an assessment of subject-reported OHS. Also, works using secondary analysis and/or validating well-known legacy measures of oral health were rejected since the intent is not to assess the OHS of a population under study.

The research team performed the secondary screening through a full-text review using the same set of inclusion criteria aforementioned and further excluded papers missing full-text, not in English, and without information about data collection methods.

2.3. Selection Strategies

Figure 1 outlines the review process utilizing the PRISMA-ScR framework. The title-and-abstract screening was completed by a researcher (D.X.) against the inclusion criteria using a screening tool (Appendix C). To check for reliability and consistency, one of the researchers (L.Z.) randomly screened 10% of articles independently and compared the inclusion decisions. Given the result of title-and-abstract screening, two researchers (L.Z. and Y.H.) verified the eligibility of the remaining articles independently through full-text review. Inclusion discrepancies were resolved by an additional researcher (D.X.).

2.4. Data Extraction

The data charting form consisted of quantitative and qualitative variables related to the issues raised by the research question and ancillary issues. Data were tabulated in the spreadsheet including the following variables: year of data collection, sample size, response rate, survey population, country of origin, survey outcome of interest, recruit method, data collection procedures, assistive devices and data sources, and study design. The form has been pre-tested by two project staff (C.M., M.M.) before being utilized (Appendix D). Two researchers (Y.H. and L.Z.) extracted data using the form. Two project staff (C.M., M.M.) collaborated to review the charted study characteristics and the discrepancies have been addressed through discussion.

2.5. Data Synthesis and Analysis

The scoping review synthesizes the research findings based on dimensions and attributes of major oral health survey data collection methods using descriptive and content analyses. It provided an overview of various related data collection methods in the recent literature and the trends in using these new technologies. Through the literature review, we found four major types of data collection methods: in-person, Internet-based, telephone-based, and mail-in based methods.

3.1. Screening and Study Selection

After removing duplicates, the initial search revealed 2,981 articles from four online databases for title-and-abstract screening, 2,503 of which were excluded after being examined against the inclusion criteria. The interrater reliability of screening was measured by Kappa agreement as 0.94 (95% confidence interval [0.89, 0.99]) for title-and-abstract screening, which implies almost perfect agreement [20]. After full-text reviewing and excluding 269 articles, we summarized and categorized the remaining 239 studies based on the pre-tested data charting form. In addition, we identified 12 studies that used technology to collect data.

3.2. General Characteristics of Included Studies

Table 1 presents a total of 239 articles published from 2011 to September 2021 have been charted and their characteristics have been summarized in Table 1. Relevant research has been published stably increase in the past 10 years, except during 2020 and 2021. Historically, 88.6% and 55.2% of the studies utilized in-person survey methods and direct recruitment. Half of the works (55.2%) were published in dental journals, with around 40% of the relevant articles from Asia/ Pacific region (APAC), 25% from Europe (EMEA), and 23.4% from Latin America (LATAM). The majority of studies (68.2%) focused on the healthy population, and the rest of them were sampled from a population with medical (19.7%) or dental problems (12.1%). 193 (80.8%) studies investigated the self-reported outcomes only and additional 27 studies recorded both self- and proxy-reported information. Supplemental data types have been used in addition to the survey, including clinical examination, medical records, and public data sources. More than half (61%) of the studies are implemented together with a clinical examination. The data charting details are listed in Appendix D.

Table 1

Characteristics of Studies Analyzed
	Number of Studies (n)	Percentage (%)
Type of Study
Peer-reviewed articles	238	99.6%
Achieved Papers	1	0.4%
Conference Proceedings	0	0.0%
Publication Year
2011 - 2012	32	13.4%
2013 - 2014	42	17.6%
2015 - 2016	39	16.3%
2017 - 2018	52	21.8%
2019 - 2020	50	20.9%
2021 (Up to September)	24	10.0%
Journal Type
Dental Journal	132	55.2%
Non-Dental Journal	107	44.8%
Region
APAC: Asia/Pacific (including the Middle East)	94	39.3%
EMEA: Europe	59	24.7%
LATAM: Latin America	56	23.4%
NA: North America	20	8.4%
AFR: Africa	10	4.2%
Missing	0	0.0%
Data Collection Start Year
Before-2005	3	1.3%
2006 – 2010	55	23.0%
2011 – 2015	62	25.9%
2016 – 2020	50	20.9%
Missing	69	28.9%
Survey Modes & Methods
In-Person Inclusive	206	86.2%
Mail-in Inclusive	15	6.3%
Internet-based Inclusive	6	2.5%
Telephone Inclusive	3	1.3%
Mixed Modes	9	3.8%
State of Health
Healthy	163	68.2%
Medical Problem	47	19.7%
Dental Problem	29	12.1%
Missing	0	0.0%
Recruitment Method
Direct	132	55.2%
Random sampling	57	23.8%
Database	17	7.1%
Hardcopy	5	2.1%
Web-based	1	0.4%
Other	8	3.3%
Mixed	19	7.9%
Report Respondents
Self-reported Only	193	80.8%
Proxy Only	19	7.9%
Both	27	11.3%
Use of Supplemental Data Type
Clinical Exam / Dental Exam only	129	54.0%
Medical / Dental Records only	9	3.8%
Public database/ National surveys only	3	1.3%
Mixed exams and records	12	5.0%
Mix exams and public database	2	0.8%
Other exams	8	3.3%
None	76	31.8%

3.3. Dimensions and Attributes of Data Collection Methods

There are four main data collection methods, including in-person (N = 206, 86.2%), mail-in (N = 15, 6.3%), Internet-based (N = 6, 2.5%), and telephone-based methods (N = 3, 1.3%). We ignored the additional 9 studies that used more than one of the four methods, as they all shared similar characteristics with studies using the four main methods. The dimensions and attributes of the data collection methods have been summarized in Table 2. In addition to the categories identified in these research papers, we also list several approaches that are common recruitment methods that have been used in both practices and industry. While not mentioned in the publications, these methods may provide some insights for future applications.

The majority of the studies using in-person surveys have a high response rate, which averaged 90.6%. Among studies using in-person survey modes, more than half (55.8%) of the studies employed face-to-face interviews, and 35.4% used the paper-and-pencil approach. Participants for 58.7% of the studies are recruited through direct recruitment from clinics [21], hospitals [22], and community care centers [23]. To increase the prediction accuracy, now that many records are electronic, visual components (e.g., X-ray [24], visual examinations [25]) and medical records [22, 26, 27], have been incorporated with such survey mode. Meanwhile, different qualitative assessments (e.g., Malocclusion Assessment [21] and Masticatory Performance Test [23, 28]) can be found in patient progress notes.

The mail-in survey strategy used by 15 studies is more cost-effective than the in-person one through two main sources, via post (80%) and by carriers (20%). It shared a relatively high response rate (72%) on average, especially when children or other respondents bring surveys back home to complete. Similar to in-person surveys, mail-in surveys can incorporate additional resources, such as photographs in the explanation sheet [29], and medical records [30].

Only six studies were identified as using Internet-based technology in their methods, mainly through computer-assisted web interviews (4 studies), and email (2 studies). Three applied direct recruitment and another three recruited participants through websites and databases. The averaged response rate is as low as 36.7% for this strategy and the study size is normally small with median as 259 participants.

The other three studies used telephone survey methods covering much larger populations compared to other strategies. Two of these studies recruited participants through an existing database, and all were administered as interviews. We did not find the commonly used computer-assisted telephone interviews (CATI)[31] and Voice Response System [32]. in the studies.

Table 2

Dimensions of Various Data Collection Strategies
Survey Strategies (N, %)	Approaches¹	Common Recruit Methods¹	Duration Length of Data Collection, Median (Min, Max)	Number of Responders, Median	Response Rate (Unadjusted), Median	Special Devices/Tools
In-Person (N = 206, 86.2%)	- Face-to-face interview (55.8%) - Paper and Pencil (35.4%) - Mixed (6.4%) - Computer-Assisted Personal Interview (CAPI) and Electronic Survey (1.9%) - Uncertain (0.5%) - Video Survey (0.0%)	- Direct recruitment (58.7%) - Random sampling (25.3%) - Database (6.4%) - Mixed (5.9%) - Other (2.0%) - Hard-copy advertisement (1.9%) - Web-based (0.0%)	1 (0, 12) in years	321	90.6%	- Medical records - Clinical exam - Salivary sampling - Oral diadochokinesis - Clinical examination and ball-ended probes - Clinical exam with mirrors - X-ray - Malocclusion assessment - Visual examination - Radiographic examinations - Masticatory performance test, color changing chewing gum - National survey
Mail-in (N = 15, 6.3%)	- Via Post (80.0%) - By Carrier (e.g., sent home with the child) (20.0%) - By Drop-off (Leave at door) (0%) - Uncertain (0%)	- Direct recruitment (20.0%) - Random sampling (33.3%) - Mixed (20.0%) - Other (13.3%) - Database (6.7%) - Hard-copy advertisement (6.7%) - Web-based (0.0%)	0 (0, 1) in years	879	72%	- Clinical exam with mirrors - Photographs were added in the form of an accompanying explanation sheet. - Dental examination - Medical record
Internet-based (N = 6, 2.5%)	- Computer-Assisted Web Interview (CAWI) and Online Survey (66.7%) - Email (33,3%) - Uncertain (0%)	- Direct recruitment (33.3%) - Other (33.3%) - Database (16.7%) - Web-based (16.7%) - hard-copy advertisement (0.0%) - Random sampling (0.0%)	0 (0, 1) in years	259	36.7%	None
Telephone (N = 3, 1.3%)	- Interview (100%) - Computer-Assisted Telephone Interview (CATI) (0%) - Voice Response System (0%) - Uncertain (0%)	- Database (66.7%) - Direct recruitment (33.3%)	0.5 (0, 1) in years	1500	55.5%	None

¹ Re-ordered by the percentage of studies in each category.

In addition to the data collection strategies, we further categorized the measures used in the 239 articles. Table 3 presents the frequencies and percentages of the various OHRQoL measures and self- or proxy ratings of OHS. The three basic approaches were: impact, functions, and self- or proxy-ratings of OHS. These were used as single measures or in combinations. The Oral Health Impact Profile-14 (OHIP-14) was the most prevalent single measure with 69 of the papers and 29% overall, while the Geriatric Oral Health Assessment Index (GOHAI), a functional measure, was second with 21 papers and 9% overall. It was first among the studies of the elderly. The self- or proxy-rating of OHS had 18 single measure papers representing 8%. There was a total of 63 papers using more than one type of measure. Either combining functional and impact measures (36 and 15%) or self-rating OHS and one or more of the other measures (27 and 11%). The group of single impact measures was 50% of the overall and were also represented where two or more measures were used. The single measure, GOHAI, based on function was only 9% of all measures but also played a role in combination with other measures. Finally, the self-reported OHS as a single measure represents 8% of the total. Its role was mainly in combination with other measures and represented another 15%.

Currently, the use of many advanced technologies has emerged in the field of survey research to improve the quality and quantity of data collection. After reviewing and charting all qualified 239 articles, twelve studies that employed technological approaches are summarized in Table 4.

Table 3

Number and Percent of OHRQoL and Self-Rating of Oral Health Status by Type, Group. and Combinations
Major Characteristic	Description	Measures	Number of Studies	Percent by Group	Percent Overall
Impact Measured	Impacts caused by Functional limitations, Physical, Psychological and Social	Oral Health Impact Profile 14 (OHIP-14) [33]	69	58%	29%
		Oral Impacts on Daily Performances (OIDP) [34]	20	17%	8%
		Early Childhood Oral Health Impact Scale (ECOHIS) [35]	15	13%	6%
		Child Oral Impacts on Daily Performance (C-OIDP) [36]	7	6%	3%
		Oral Health Impact Profile (OHIP-49) [37]	3	3%	1%
		Child Oral Health Impact Profile (COHIP) [38]	3	3%	1%
		Other Impact Measures	3	3%	1%
		Total	120	100%	50%
Functioning Measures	Assesses Physical, Social and Psychosocial Functions of the Elderly	Geriatric Oral Health Assessment Index (GOHAI) [39]	21	91%	9%
	Physical, Social, Role, Emotional, and Oral Problems	Teen Oral Health-related Quality of Life (TOQOL) [40]	2	9%	1%
		Total	23	100%	10%
Perceived OHS and Functioning	Measures perceived oral health status score, functional limitations, emotional and social well-being	Child Perceptions Questionnaire 8-10 (CPQ8-10) [41]	1	7%	0%
		Child Perceptions Questionnaire 11-14 (CPQ11-14) [42]	11	73%	5%
		Multiple CPQ	3	20%	1%
		Total	15	100%	6%
Self-Rating of OHS (only)		Self-Rating of OHS (only)	18	100%	8%
		Total	18	100%	8%
Using more than one measure		More than one measure is used (excluding Self Rated Scales)	36	100%	15%
		Combination of Self Rated scale and oral health quality of life	27	100%	11%
	Total		239	100%	100%

Table 4

Summary of the Studies that Use Technological Approaches (N = 12)
Study (Author-Year-Country)	Study Population & Setting	Themes	Recruitment Method	Survey Mode and Approach	Survey Description	Sample Size (Response Rate, %)
Broughton et al-2012-New Zealand [43]	Healthy Māori teenagers (16-18) in New Zealand	self-reported oral health and use of oral health services self-reported oral health and use of oral health services self-reported oral health and use of oral health services self-reported oral health and use of oral health services	A convenient sample of Rangatahi as undertaken in the Rohe of Tainui. Participants completed a computer-based questionnaire under their supervision. No reward was mentioned.	In-Person / Group: Electronic/Computer-based Survey	OHIP-14. self-reported	238 (NA)
Kotzer et al-2012-Canada [44]	Canadian healthy pre-seniors and seniors	OHRQoL	Facility-based as well as random sampling of pre-seniors and seniors: Random digit dialing telephone survey for community residents or face-to-face interview for LTC residents. Reward: placed in contention to win one of two $250 prizes by means of a lottery	Telephone interview	OHIP-14 (English or French), Oral health questions, General health questions, medication use. Self-reported.	1461 (NA)
Nam et al-2017-Korea [45]	Healthy university students in Korea (age >= 20)	Quality of life related to dental health	Students in 3 majors at Kangwon University Dogye Campus answered a self-administered Google-based survey on the quality of life related to dental health No rewards were mentioned.	Internet-based online survey	Quality of life, dental health status, dental health education Self-administered.	130 (NA)
Hakeberg and Wide-2018-Sweden [46]	Swedish Healthy Adult Residents (age >19)	dental anxiety and health-related quality of life (HRQL) among adults.	A telemarketing company (TNS SIFO) selected participants from Swedish Personal Address Registry using a simple random selection procedure. Participants were invited to a telephone interview containing 38 questions pertaining to dental utilization. No rewards.	Telephone interview	dental anxiety, HRQL [Oral Health Impact Profile-5 (OHIP-5) and EuroQol-5D (EQ-5D)] and several socioeconomic variables. Self-reported.	3500 (49.7%)
Hanisch et al-2018-Germany [47]	German patients affected by a rare disease (age >16)	Oral health-related quality of life (OHRQoL), satisfaction with dental care	An electronic questionnaire was sent digitally to all 125 German member associations of the umbrella organization of self-help groups, the Alliance of Chronic Rare Diseases. The Snowball method was used for survey distribution. Rewards not mentioned.	Internet-based online survey	The standardized version of the OHIP-14, free text questions addressing participants' satisfaction with the dental treatment and the health care system in Germany. Self-reported.	451 (NA)
Liu et al-2018-US [2]	Patients (age 8–17) without orthodontic appliances, and their parents or guardians, reflecting the general US population	development of two multi-item self-report scales for use in assessing the oral health status of children and adolescents	Clinic-based recruitment of patients and their parents/guardians at dental clinics located in LA county. The oral health items were administered to children and parents by computer using Questionnaire Development System™ (QDS™). Rewards not mentioned.	In-person / Group: CAPI -- Computer Assisted Personal Interviewing	Questions related to oral health for both children and proxy.	334 (NA)
Morgan et al-2018-Rwanda [48]	Healthy individuals representing population of Rwanda	Assess the oral disease burden and inform oral health promotion strategies	Random sampling using Pathfinder methodology. The study was promoted and organized through local community leaders at the district No rewards.	Computer-Assisted Personal Instrument (CAPI) was developed to administer the study instrument.	Oral health practices and behaviors, oral health related quality of life. Self-reported	2097 (NA)
Mortimer-Jones et al-2018-Australia [49]	Australian healthy nursing students across all year levels	Association between anxiety and temporomandibular-related symptoms in nursing students, and the effect on quality of life.	Study info was circulated by staff members and publicized during lectures, also send to all nursing students in email form. The announcements and emails provided a link to the online survey (Survey Monkey) No rewards.	Internet-based online survey	OHIP-TMD, PROMIS short form. Self-reported	281 (25%)
Echeverria et al-2020-Brazil [50]	Healthy college students in Brazil	Verify the prevalence and factors associated with regular use of dental services in university students	School-based recruitment: the study is part of a census conducted at UFPel that sought information on health/behaviors of university students. Standardized questionnaires using RedCap software installed on tablets were answered in classrooms or other environments within the university. No rewards.	In-Person / Group: Electronic/Computer-based Survey	Rating of oral health Excel., Very Good, & Moderate, Poor Self-reported.	1865 (69%)
Mohamad Fuad et al – 2020 – Malaysia [51]	Older persons in Malaysia (age >60)	OHRQoL among older persons in Malaysia and its associations with sociodemographic and self-perception towards general health as well as oral health.	Random sampling: This study was part of the NHMS 2018: Elderly Health. Stratified cluster sampling strategy was used: primary stratum constitutes the states and federal LQ selections, followed by the secondary stratum (urban and rural areas). Participants completed a face-to-face interview using tablets. No rewards mentioned	face-to-face interview using tablets	GOHAI Malay version and oral health scale (Very healthy to Very unhealthy) Self-reported	3867 (97.2%)
Liu-2021-China [52]	Chinese Healthy Kids (3-6)	Oral Health Perception	Online recruitment with a recruitment link or quick response (QR) code to be distributed to groups in their WeChat. People can also spread the link to others. No rewards.	Internet-based survey platform	Oral health status and care behavior, caregivers’ attitudes, Proxy	4495 (NA)
Makizodila-2021-Netherlands [53]	Motor Neuron Disease (MND) patients in Netherland	Needs and barriers of Motor Neuron Disease (MND) and their caregivers in performing oral hygiene	Recruit from da database: all registered MND patients in the Netherlands. Potential participants who were registered in the Prospective ALS study Netherlands Database were informed via an email newsletter of the Dutch ALS Centre. Patients were asked to forward the newsletter email to their caregivers. When considering participation, they could click on a link to receive information about the study. No rewards.	Internet-based email	GOHAI, Self and caregiver reported	259 (36.7%)

We asked the following question at the start of our research: “What data collection methods, approaches, and/or modes and measures have been used in conducting subject-reported surveys to measure oral health status?” To answer the question, we conducted a scoping review of methodologies used in oral health and disease self-reported surveys of children, parents, and adults. The focus is on different methods of capturing respondent perceptions such as paper and pencil, interviews online or in-person; mail-in surveys, computer-assisted with research staff present, non-assisted media-based surveys using tablets, mobile phones, and websites; and respondent feedback that includes self-pictures of the face and teeth. These technologies are not necessarily innovative, nor are they used in research. They are however applications of technology that may lead to substantial changes in future research methodologies. These approaches, moreover, will be especially important for oral health researchers’ ability to access patients and conduct instrument-based surveys due to the health concerns emerging with the COVID-19 pandemic.

4.1. Overview of Data Collection Methods in Studies Included

We systematically identify different survey modes, methods, and assisting devices for collecting patient and subject self-reported dental and oral health data. We summarize different data collection methods with respect to multiple dimensions and associated attributes, such as the use of the assistant device in surveys, age group limitation, response rate, reliability rate, completion rate, feasibility, sample selection bias, group and individual interview settings, and other dimensions. The scoping review also sets out the facilitators and barriers to remote technology and Internet-based approaches. The paper concludes with future research applications and identifies new directions and approaches to improve quality and reduce biases in data collection.

For this scoping review, the work of Gupta (2018) and Yang (2020) guided our approach [54, 55]. Gupta analyzed the use of mixed methods in population oral health research, synthesizing the findings of quantitative and qualitative studies to examine oral health outcomes. Yang’s review appraised the quality of children’s OHRQoL data to improve standards for patient-reported outcome measures. We broadened these perspectives by including all age groups and all measures of oral health and specifically the diverse methods and modes of collecting self-reported OHS and OHRQoL data. This view is important as we are in a changing world of available remote and two-way technologies that may open doors to both researchers and the various populations that use or need oral health care.

Locker points out that OHRQoL is commonly used as an outcome measure of oral health conditions [56]. Originally, they were used as “socio-dental indicators or subjective oral health indicators and are now more usually referred to as measures of OHRQoL.” He concludes that “the claim that oral disorders affect the quality of life has yet to be established”. Perhaps that is why we found that many different measures address themselves as OHRQoL and that researchers felt they needed to use more than one measure to assess the oral quality of life. A rationale for using the GOHAI and an impact measure (OIDP) is proposed by MT John et al. which advocated for the integration of the GOHAI and the OIDP to produce a model that consists of oral function, orofacial pain, orofacial appearance, and psychosocial impact to produce a short but “clinically meaningful” measure of OHRQoL [57].

In the scoping review, we found a high percentage of papers that used measures that were reported to be OHRQoL. Although OHIP-14 was the most utilized single measure, many papers used portions of this measure, while adding other items that may have measured need, satisfaction, OHS, etc. This resulted in a dizzying number of approaches, which did not answer the criticism regarding whether they are measuring OHRQoL. The measures using self-reported OHS were consistent in what they were measuring only varying in the number of Likert Scales used, they varied from 3 to 10 responses. It seems that some combination of impact (e.g., OHIP), function (e.g., GOHAI) and self-reported OHS might better reflect the quality of life than any of these measures alone.

4.2. Technological Data Collection Methods to Measure Oral Health Conditions

In the main, our review has identified research that employs time-honored methods. The face-to-face interview and the pencil and paper format are conventionally used in many studies along with a clinical dental exam. Countries, such as Brazil, have relied on these techniques to develop national epidemiological oral health surveys that have been used for years [58–60]. These surveys are very well-organized and are established throughout the country. We did not see the introduction of newer technologies into their time-honored approach. Understandably, using their present model is effective, since hiring fewer people to gain efficiency would not be popular. Moreover, a large proportion of the population is poor and may not have access to web-based technologies or be facile with computers. We need to consider the realities on the ground that different oral health researchers face in implementing studies. With this in mind, two technological approaches to survey research methodology emerged from our review, notably in the areas of 1) sample selection, and 2) mode of interaction with the subject.

With respect to sample selection, North American researchers are finding different platforms to access subjects for their studies. Canadian studies have used random digit dialing to recruit and conduct computer-assisted interviews [61]. In the United States, researchers accessed existing polling populations or used Amazon’s MTurk platform of “workers,” who are paid small amounts for each survey they respond to [62]. In each of these cases, the representativeness of the study population is a concern.

The second approach is the use of computers to collect self-reported data. The basic surveying technique is the Computer-Assisted Personal Interviewing (CAPI) with interviewer directly entering the data into a database. There is also Computer-Assisted Telephone Interviewing (CATI), a survey technique, where the interviewer follows a scripted interview guided by a questionnaire that appears on the screen. A third Internet-based survey technique, the Computer-Assisted Web Interviewing (CAWI), requires no live interviewer. Instead, the respondent follows a script made in a program for designing web interviews that may include images, audio and video clips, and web-based information.

An innovative technological method worth noting is the use of OralCam to perform self-examination using a smartphone camera [63]. This paper was not included in our scoping search, because its primary objective is not related to subject-reported OHS. The study applied research used in medicine to detect liver problems from face photos[64] as well as other diseases. The paper describes the use of a smartphone camera to interact with a computer using diagnostic algorithms, such as the deep convolutional neural network-based multitask learning approach. Based on over three thousand intraoral photos, the system has learned to analyze teeth and gingiva. The smartphone camera takes a picture using a mouth opener. The computer’s algorithms analyze the captured picture, along with survey data, to diagnose several dental conditions including caries, chronic gingival inflammation, and dental calculus. This use of multitask learning technology, with the extensive availability of cell phones, may revolutionize oral health research and care.

As we move on in the field of self-reported data collection, in those settings where it makes sense to use computer-based technologies to select samples and carry out interviews, we are likely to see more interaction between smartphones and algorithms that utilize machine learning. However, in settings where Internet-based approaches are not practical, longstanding and effective conventional oral health data collection techniques in research will continue.

4.3. Strengths and Limitations

This scoping review is limited to oral health survey-based studies in peer-reviewed dental and public health journals, which have published at least ten or more papers on self-reported oral health over the last decade; we did not review the grey literature. Further, many of the reviewed papers did not adequately describe the methods they used to collect data.

The study provides an overview of data collection methods and modes for self-reported oral health data, especially with newly emerging remote and two-way technologies. Although the in-person strategy provides unique flexibility and is easier to administer, these are more labor-intensive and normally take more time compared to other modes. Contemporary web-based approaches, therefore, offer enhanced opportunities for survey research to boost recruitment, participation of study subjects with wide and diverse backgrounds from almost unlimited geographic areas, and to improve quality of data collected by assuring increased response rates and reducing bias, especially in longitudinal studies.

PRISMA-ScR

Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews

APAC

Asia/Pacific (including the Middle East)

EMEA

Europe

LATAM

Latin America

NA

North America

AFR

Africa

CAPI

Computer-Assisted Personal Interviewing

CAWI

Computer-Assisted Web Interviewing

CATI

computer-assisted telephone interview

OHRQoL

Oral Health Related Quality of Life

OHIP-14

Oral Health Impact Profile-14

GOHAI

Geriatric Oral Health Assessment Index

OHS

Oral Health Status.

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Availability of data and materials

All data generated or analysed during this study are included in this published article and its supplementary information files.

Competing interests

The authors declare that they have no competing interests.

Funding

This research was supported by an NIDCR/NIH grant to the University of California, Los Angeles (UCLA) (U01DE029491). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Authors' contributions

C.M., D.X., M.M. and H.L. conceptualized the study and designed the data collection form and established the data analysis plan. A.O. developed search strategies and carried out searching on multiple databases. D.X., Y.L., Y.H., J.S. and Y.L. performed additional searching and tested the data charting form. D.X., Y.L., and Y.H. helped to screen studies for relevance and data charting. C.M. and M.M. reviewed full-text papers and verify the data charting results. C.M., D.X., and M.M. drafted the original manuscript. D.X. and L.Z. prepared tables 1, 2, and figure 1. C.M., D.X., M.M., and L.Z prepared tables 3 and 4. All authors read and provided substantial comments/edits on the manuscript and approved the final version.

Acknowledgments

Not applicable

Watt RG, Daly B, Allison P, Macpherson LMD, Venturelli R, Listl S, et al. Ending the neglect of global oral health: time for radical action. Lancet (London, England) 2019;394:261–72. https://doi.org/10.1016/S0140-6736(19)31133-X.
Liu H, Hays R, Wang Y, Marcus M, Maida C, Shen J, et al. Short form development for oral health patient-reported outcome evaluation in children and adolescents. Qual Life Res 2018;27:1599–611. https://doi.org/10.1007/s11136-018-1820-9.
Liu H, Hays R, Wang Y, Marcus M, Maida C, Shen J, et al. Short form development for oral health patient-reported outcome evaluation in children and adolescents. Quality of Life Research 2018;27:1599–611. https://doi.org/10.1007/s11136-018-1820-9.
Arksey H, O’Malley L. Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology: Theory and Practice 2005;8:19–32. https://doi.org/10.1080/1364557032000119616.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implementation Science 2010;5:69. https://doi.org/10.1186/1748-5908-5-69.
Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, et al. A scoping review on the conduct and reporting of scoping reviews. BMC Medical Research Methodology 2016;16:15. https://doi.org/10.1186/s12874-016-0116-4.
Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Medical Research Methodology 2018 18:1 2018;18:1–7. https://doi.org/10.1186/S12874-018-0611-X.
Keboa MT, Hiles N, Macdonald ME. The oral health of refugees and asylum seekers: A scoping review. Globalization and Health 2016;12:1–11. https://doi.org/10.1186/S12992-016-0200-X/TABLES/2.
Beaton L, Humphris G, Rodriguez A, Freeman R. Community-based oral health interventions for people experiencing homelessness: a scoping review. Community Dental Health 2020;37:150–60. https://doi.org/10.1922/CDH_00014BEATON11.
Wilson NJ, Lin Z, Villarosa A, Lewis P, Philip P, Sumar B, et al. Countering the poor oral health of people with intellectual and developmental disability: A scoping literature review. BMC Public Health 2019;19:1–16. https://doi.org/10.1186/S12889-019-7863-1/TABLES/1.
Ajwani S, Jayanti S, Burkolter N, Anderson C, Bhole S, Itaoui R, et al. Integrated oral health care for stroke patients - a scoping review. Journal of Clinical Nursing 2017;26:891–901. https://doi.org/10.1111/JOCN.13520.
Shrestha AD, Vedsted P, Kallestrup P, Neupane D. Prevalence and incidence of oral cancer in low- and middle-income countries: A scoping review. European Journal of Cancer Care 2020;29. https://doi.org/10.1111/ECC.13207.
Patterson-Norrie T, Ramjan L, Sousa MS, Sank L, George A. Eating disorders and oral health: A scoping review on the role of dietitians. Journal of Eating Disorders 2020;8:1–21. https://doi.org/10.1186/S40337-020-00325-0/TABLES/1.
Lansdown K, Smithers-Sheedy H, Mathieu Coulton K, Irving M. Oral health outcomes for people with cerebral palsy: a scoping review protocol. JBI Database of Systematic Reviews and Implementation Reports 2019;17:2551–8. https://doi.org/10.11124/JBISRIR-2017-004037.
Marquillier T, Lombrail P, Azogui-Lévy S. [Social inequalities in oral health and early childhood caries: How can they be effectively prevented? A scoping review of disease predictors]. Revue d’epidemiologie et de Sante Publique 2020;68:201–14. https://doi.org/10.1016/J.RESPE.2020.06.004.
Como DH, Duker LIS, Polido JC, Cermak SA. The Persistence of Oral Health Disparities for African American Children: A Scoping Review. International Journal of Environmental Research and Public Health 2019;16. https://doi.org/10.3390/IJERPH16050710.
Stein K, Farmer J, Singhal S, Marra F, Sutherland S, Quiñonez C. The use and misuse of antibiotics in dentistry: A scoping review. Journal of the American Dental Association (1939) 2018;149:869-884.e5. https://doi.org/10.1016/J.ADAJ.2018.05.034.
Paré G, Trudel M-C, Jaana M, Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews 2015. https://doi.org/10.1016/j.im.2014.08.008.
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Https://DoiOrg/107326/M18-0850 2018;169:467–73. https://doi.org/10.7326/M18-0850.
Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977;33:159. https://doi.org/10.2307/2529310.
Masood M, Masood Y, Newton T, Lahti S. Development of a conceptual model of oral health for malocclusion patients. The Angle Orthodontist 2015;85:1057–63. https://doi.org/10.2319/081514-575.1.
Massarente DB, Domaneschi C, Marques HHS, Andrade SB, Goursand D, Antunes JLF. Oral health-related quality of life of paediatric patients with AIDS. BMC Oral Health 2011;11:2. https://doi.org/10.1186/1472-6831-11-2.
Lu TY, Chen JH, Du JK, Lin YC, Ho PS, Lee CH, et al. Dysphagia and masticatory performance as a mediator of the xerostomia to quality of life relation in the older population. BMC Geriatrics 2020;20. https://doi.org/10.1186/S12877-020-01901-4.
Strömberg E, Holmèn A, Hagman-Gustafsson ML, Gabre P, Wardh I. Oral health-related quality-of-life in homebound elderly dependent on moderate and substantial supportive care for daily living. Acta Odontologica Scandinavica 2013;71:771–7. https://doi.org/10.3109/00016357.2012.734398.
Morgan JP, Isyagi M, Ntaganira J, Gatarayiha A, Pagni SE, Roomian TC, et al. Building oral health research infrastructure: the first national oral health survey of Rwanda. Global Health Action 2018;11. https://doi.org/10.1080/16549716.2018.1477249.
Preciado A, del Río J, Suárez-García MJ, Montero J, Lynch CD, Castillo-Oyagüe R. Differences in impact of patient and prosthetic characteristics on oral health-related quality of life among implant-retained overdenture wearers. Journal of Dentistry 2012;40:857–65. https://doi.org/10.1016/J.JDENT.2012.07.006.
de Quadros Coelho M, Cordeiro JM, Vargas AMD, de Barros Lima Martins AME, de Almeida Santa Rosa TT, Senna MIB, et al. Functional and psychosocial impact of oral disorders and quality of life of people living with HIV/AIDS. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation 2015;24:503–11. https://doi.org/10.1007/S11136-014-0778-5.
Said M, Otomaru T, Aimaijiang Y, Li N, Taniguchi H. Association Between Masticatory Function and Oral Health-Related Quality of Life in Partial Maxillectomy Patients. The International Journal of Prosthodontics 2016;29:561–4. https://doi.org/10.11607/IJP.4852.
Owens J, Jones K, Marshman Z. The oral health of people with learning disabilities - a user-friendly questionnaire survey. Community Dental Health 2017;34:4–7. https://doi.org/10.1922/CDH_3867OWENS04.
Abuzar MA, Kahwagi E, Yamakawa T. Investigating oral health-related quality of life and self-perceived satisfaction with partial dentures. Journal of Investigative and Clinical Dentistry 2012;3:109–17. https://doi.org/10.1111/J.2041-1626.2012.00111.X.
Wilson D, Taylor A, Chittleborough C. The Second Computer Assisted Telephone Interview (CATI) Forum: The state of play of CATI survey methods in Australia. Australian and New Zealand Journal of Public Health 2001;25:272–4. https://doi.org/10.1111/J.1467-842X.2001.TB00576.X.
Lee H, Friedman ME, Cukor P, Ahern D. Interactive voice response system (IVRS) in health care services. Nursing Outlook 2003;51:277–83. https://doi.org/10.1016/S0029-6554(03)00161-1.
Slade GD. Derivation and validation of a short-form oral health impact profile. Community Dentistry and Oral Epidemiology 1997;25:284–90. https://doi.org/10.1111/J.1600-0528.1997.TB00941.X.
Adulyanon S, Vourapukjaru J, Sheiham A. Oral impacts affecting daily performance in a low dental disease Thai population. Community Dentistry and Oral Epidemiology 1996;24:385–9. https://doi.org/10.1111/J.1600-0528.1996.TB00884.X.
Pahel BT, Rozier RG, Slade GD. Parental perceptions of children’s oral health: The Early Childhood Oral Health Impact Scale (ECOHIS). Health and Quality of Life Outcomes 2007;5:6. https://doi.org/10.1186/1477-7525-5-6.
Gherunpong S, Tsakos G, Sheiham A. Developing and evaluating an oral health-related quality of life index for children; the CHILD-OIDP. Community Dent Health 2004;21:161–9.
Slade GD, Spencer AJ. Development and evaluation of the Oral Health Impact Profile. Community Dental Health 1994;11:3–11.
Broder HL, Wilson-Genderson M. Reliability and convergent and discriminant validity of the Child Oral Health Impact Profile (COHIP Child’s version). Community Dentistry and Oral Epidemiology 2007;35 Suppl 1:20–31. https://doi.org/10.1111/J.1600-0528.2007.0002.X.
Atieh MA. Arabic version of the geriatric oral health assessment Index. Gerodontology 2008;25:34–41. https://doi.org/10.1111/j.1741-2358.2007.00195.x.
Wright WG, Spiro A, Jones JA, Rich SE, Garcia RI. Development of the Teen Oral Health-Related Quality of Life Instrument. Journal of Public Health Dentistry 2017;77:115–24. https://doi.org/10.1111/JPHD.12181.
Jokovic A, Locker D, Tompson B, Guyatt G. Questionnaire for measuring oral health-related quality of life in eight- to ten-year-old children. Pediatric Dentistry 2004;26:512–8.
Jokovic A, Locker D, Stephens M, Kenny D, Tompson B, Guyatt G. Validity and reliability of a questionnaire for measuring child oral-health-related quality of life. Journal of Dental Research 2002;81:459–63. https://doi.org/10.1177/154405910208100705.
Broughton JR, TeH Maipi J, Person M, Randall A, Thomson WM. Self-reported oral health and dental service-use of rangatahi within the rohe of Tainui. New Zealand Dental Journal 2012;108:90–4.
Kotzer RD, Lawrence HP, Clovis JB, Matthews DC. Oral health-related quality of life in an aging Canadian population. Health and Quality of Life Outcomes 2012;10:50. https://doi.org/10.1186/1477-7525-10-50.
Seoul-Hee Nam H-YKD-IC. Influential factors on the quality of life and dental health of university students in a specific area n.d.
Hakeberg M, Wide U. General and oral health problems among adults with focus on dentally anxious individuals. International Dental Journal 2018;68:405–10. https://doi.org/10.1111/IDJ.12400.
Hanisch M, Wiemann S, Bohner L, Kleinheinz J, Susanne SJ. Association between Oral Health-Related Quality of Life in People with Rare Diseases and Their Satisfaction with Dental Care in the Health System of the Federal Republic of Germany. International Journal of Environmental Research and Public Health 2018;15. https://doi.org/10.3390/IJERPH15081732.
Monaghan N, Karki A, Playle R, Johnson I, Morgan M. Measuring oral health impact among care home residents in Wales. Community Dental Health 2017;34:14–8. https://doi.org/10.1922/CDH_3950MORGAN05.
Mortimer-Jones S, Stomski N, Cope V, Maurice L, Théroux J. Association between temporomandibular symptoms, anxiety and quality of life among nursing students. Collegian 2019;26:373–7. https://doi.org/10.1016/J.COLEGN.2018.10.003.
Echeverria MS, Silva AER, Agostini BA, Schuch HS, Demarco FF. Regular use of dental services among university students in southern Brazil. Revista de Saude Publica 2020;54:85. https://doi.org/10.11606/S1518-8787.2020054001935.
Mohamad Fuad MA, Yacob H, Mohamed N, Wong NI. Association of sociodemographic factors and self-perception of health status on oral health-related quality of life among the older persons in Malaysia. Geriatrics & Gerontology International 2020;20 Suppl 2:57–62. https://doi.org/10.1111/GGI.13969.
Liu C, Zhang S, Zhang C, Tai B, Jiang H, Du M. The impact of coronavirus lockdown on oral healthcare and its associated issues of pre-schoolers in China: an online cross-sectional survey. BMC Oral Health 2021;21. https://doi.org/10.1186/s12903-021-01410-9.
Makizodila BAM, van de Wijdeven JHE, de Soet JJ, van Selms MKA, Volgenant CMC. Oral hygiene in patients with motor neuron disease requires attention: A cross-sectional survey study. Special Care in Dentistry: Official Publication of the American Association of Hospital Dentists, the Academy of Dentistry for the Handicapped, and the American Society for Geriatric Dentistry 2021. https://doi.org/10.1111/SCD.12636.
Gupta M, Bosma H, Angeli F, Kaur M, Chakrapani V, Rana M, et al. A mixed methods study on evaluating the performance of a multi-strategy national health program to reduce maternal and child health disparities in Haryana, India. BMC Public Health 2017;17:698. https://doi.org/10.1186/s12889-017-4706-9.
Yang C, Crystal YO, Ruff RR, Veitz-Keenan A, McGowan RC, Niederman R. Quality Appraisal of Child Oral Health–Related Quality of Life Measures: A Scoping Review. JDR Clinical and Translational Research 2020;5:109–17. https://doi.org/10.1177/2380084419855636.
Locker D, Allen F. What do measures of “oral health-related quality of life” measure? Community Dentistry and Oral Epidemiology 2007;35:401–11. https://doi.org/10.1111/J.1600-0528.2007.00418.X.
John MT, Reissmann DR, Čelebić A, Baba K, Kende D, Larsson P, et al. Integration of oral health-related quality of life instruments. Journal of Dentistry 2016;53:38–43. https://doi.org/10.1016/J.JDENT.2016.06.006.
Saintrain MVDL, de Souza EHA. Impact of tooth loss on the quality of life. Gerodontology 2012;29. https://doi.org/10.1111/J.1741-2358.2011.00535.X.
de Quadros Coelho M, Cordeiro JM, Vargas AMD, de Barros Lima Martins AME, de Almeida Santa Rosa TT, Senna MIB, et al. Functional and psychosocial impact of oral disorders and quality of life of people living with HIV/AIDS. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation 2015;24:503–11. https://doi.org/10.1007/S11136-014-0778-5.
Grando LJ, Mello ALSF, Salvato L, Brancher AP, del Moral JAG, Steffenello-Durigon G. Impact of leukemia and lymphoma chemotherapy on oral cavity and quality of life. Special Care in Dentistry: Official Publication of the American Association of Hospital Dentists, the Academy of Dentistry for the Handicapped, and the American Society for Geriatric Dentistry 2015;35:236–42. https://doi.org/10.1111/SCD.12113.
Kotzer RD, Lawrence HP, Clovis JB, Matthews DC. Oral health-related quality of life in an aging Canadian population. Health and Quality of Life Outcomes 2012;10:50. https://doi.org/10.1186/1477-7525-10-50.
Burnham MJ, Le YK, Piedmont RL. Who is Mturk? Personal characteristics and sample consistency of these online workers. Https://DoiOrg/101080/1367467620181486394 2018;21:934–44. https://doi.org/10.1080/13674676.2018.1486394.
Liang Y, Fan HW, Fang Z, Miao L, Li W, Zhang X, et al. OralCam: Enabling self-examination and awareness of oral health using a smartphone camera. Conference on Human Factors in Computing Systems - Proceedings, vol. 20, New York, NY, USA: Association for Computing Machinery; 2020, p. 1–13. https://doi.org/10.1145/3313831.3376238.
Ding X, Jiang Y, Qin X, Chen Y, Zhang W, Qi L. Reading face, reading health: Exploring face reading technologies for everyday health. Conference on Human Factors in Computing Systems - Proceedings, New York, NY, USA: Association for Computing Machinery; 2019, p. 1–13. https://doi.org/10.1145/3290605.3300435.

No competing interests reported.

Quantitative Data Collection Approaches in Subject-Reported Oral Health Research: A Scoping Review

Status:

Version 1

Abstract

Background

Method

Result

Conclusion

Figures

1. Background

2. Methods

2.1. Search Strategy and Data Sources

2.2. Study Inclusion Criteria

2.3. Selection Strategies

2.4. Data Extraction

2.5. Data Synthesis and Analysis

3. Results

3.1. Screening and Study Selection

3.2. General Characteristics of Included Studies

3.3. Dimensions and Attributes of Data Collection Methods

4. Discussion

4.1. Overview of Data Collection Methods in Studies Included

4.2. Technological Data Collection Methods to Measure Oral Health Conditions

4.3. Strengths and Limitations

5. Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1