This scoping review protocol was developed using the methodological framework for scoping studies developed by Arksey and O’Malley [13] and incorporates the updated frameworks suggested by Levac, Colquhoun and O'Brien [14] and Peters, Marnie and Tricco et al [15]. In addition, reporting will follow the PRISMA extension for Scoping Reviews [16] (see the checklist in Additional file 1). The framework consists of six phases namely: (1) Identification of the research question; (2) Identification of the relevant studies; (3) Careful selection of the studies to be included using a consultative approach: (4) Charting of the data in both tabular and narrative format; (5) Collation, summarising and reporting of the results for identification of the implications related to the findings linked to practice and research; (6) Consultation with various stakeholders, to validate the findings.
This protocol is registered as a priori in the Open Science Framework on https://osf.io/7yfbq.
Stage one: Identify the research question(s) and objectives
The aim of this scoping review will be to systematically map the available literature on the use of GRS and checklists within clinical simulation-based assessments and to potentially identify any existing gaps. We used the Population-Concept-Context (PCC) framework [15]to formulate our research question and this scoping review therefore asks: “What is known from the existing literature about the use of GRS and checklists (concept) within clinical simulation-based (context) assessment practices (population)?” The sub-questions for this scoping review is:
- What definitions for GRS and checklists exist within clinical simulation-based assessment assessments/practises?.
- What are the perceived advantages of GRS and checklists within clinical simulation-based assessments?
- What are the perceived disadvantages of GRS and checklists within clinical simulation-based assessments?
- What different types of GRS and/or checklists are used within clinical simulation-based assessments?
- How are GRS and checklists used in formative and/or summative assessment?
- What are the characteristics of the raters used to assess clinical simulation using GRS and/or checklists?
Within the context of the above questions we will be exploring the range and extent of research conducted in clinical simulation assessment where GRS and/or checklists are used.
Stage two: Identification of relevant studies
To ensure this study is as comprehensive as possible, grey literature and indexed databases related to GRS and simulation-based assessments will be searched for possible studies that will allow us to answer the research question. We will adopt the approach of Arksey and O’Mally as well as that of Sucharew and Macaluso [13,17] by searching for evidence via different sources that include, but are not necessarily limited to electronic searching for primary studies on electronic databases, hand-searching key journals, scrutinising reference lists of articles that form part of the review and searching networks, organisations and conferences.
Information sources
Both researchers and a qualified research librarian will develop a search strategy for this scoping review to discover relevant sources of evidence. The primary sources of literature will be a structured search of major electronic databased that includes PubMed, CINAHL, Education Resources Information Center (ERIC), Cochrane Library, Scopus, EBSCO, ScienceDirect, Web of Science and the Directory of open Access Journals (DOAJ). We will also use the Rayyan screening tool as an adjunct to assist in identification and selection of eligible studies [18].
The secondary sources will include Google (Google books and Google scholar), ProQuest, OpenGrey (www.opengray.eu), worldwidescience.org and various university dissertation and thesis repositories [19]. Additionally, hand-searching of key journals and relevant references of included studies will be done to identify any potential sources that might have been missed in the initial searches of journals and reviews. Lastly, pertinent organisations and individuals involved in the relevant projects will be contacted to obtain information on unpublished and/or ongoing studies that were identified.
A draft search strategy developed by the research librarian for MEDLINE is presented in Table 1. This will be pilot tested and refined as needed for subsequent database searches.
Table 1 Pilot search in MEDLINE electronic database
Date
|
Keywords
|
Search results
|
2021/11/26
|
"(((Global Rating Scale) OR (Checklist)) AND (clinical simulation assessment)) AND ((""2010/01/01""[Date - Publication] : ""2021/12/31""[Date - Publication]))",,,"(((""global""[All Fields] OR ""globalism""[All Fields] OR ""globalize""[All Fields] OR ""globalized""[All Fields] OR ""globalizes""[All Fields] OR ""globalizing""[All Fields] OR ""globally""[All Fields] OR ""globals""[All Fields] OR ""internationality""[MeSH Terms] OR ""internationality""[All Fields] OR ""globalization""[All Fields]) AND (""rated""[All Fields] OR ""ratee""[All Fields] OR ""ratees""[All Fields] OR ""rating""[All Fields] OR ""ratings""[All Fields]) AND (""scale s""[All Fields] OR ""scaled""[All Fields] OR ""scaling""[All Fields] OR ""scalings""[All Fields] OR ""weights and measures""[MeSH Terms] OR (""weights""[All Fields] AND ""measures""[All Fields]) OR ""weights and measures""[All Fields] OR ""scale""[All Fields] OR ""scales""[All Fields])) OR (""checklist""[MeSH Terms] OR ""checklist""[All Fields] OR ""checklists""[All Fields] OR ""checklist s""[All Fields])) AND ((""ambulatory care facilities""[MeSH Terms] OR (""ambulatory""[All Fields] AND ""care""[All Fields] AND ""facilities""[All Fields]) OR ""ambulatory care facilities""[All Fields] OR ""clinic""[All Fields] OR ""clinic s""[All Fields] OR ""clinical""[All Fields] OR ""clinically""[All Fields] OR ""clinicals""[All Fields] OR ""clinics""[All Fields]) AND (""computer simulation""[MeSH Terms] OR (""computer""[All Fields] AND ""simulation""[All Fields]) OR ""computer simulation""[All Fields] OR ""simulation""[All Fields] OR ""simul""[All Fields] OR ""simulate""[All Fields] OR ""simulated""[All Fields] OR ""simulates""[All Fields] OR ""simulating""[All Fields] OR ""simulation s""[All Fields] OR ""simulational""[All Fields] OR ""simulations""[All Fields] OR ""simulative""[All Fields] OR ""simulator""[All Fields] OR ""simulator s""[All Fields] OR ""simulators""[All Fields]) AND (""assess""[All Fields] OR ""assessed""[All Fields] OR ""assessement""[All Fields] OR ""assesses""[All Fields] OR ""assessing""[All Fields] OR ""assessment""[All Fields] OR ""assessment s""[All Fields] OR ""assessments""[All Fields])) AND 2010/01/01:2021/12/31[Date - Publication]"
|
1013
|
Eligibility Criteria
Our review will include publications that relate to the use of global rating scales and/or checklists in simulation-based clinical assessments. We will consider all peer-reviewed publications and grey literature for inclusion, including theoretical, conceptual or empirical designs, that focus on use of global rating scales and/or checklists in simulation-based clinical assessments. Non-research study designs will be excluded, including editorials, discussion papers, opinion papers, letters, guidelines and non-systematic reviews. All publications considered for inclusion must have an abstract and an aim or aims that are clearly stated. We will only consider sources published after 1 January 2010. All potential sources will be scrutinised for ethical approval and where any ethical risks are identified within a study, these sources will be excluded. Sources not available in English will be excluded. The detailed eligibility and inclusion criteria have been specified in Figure 1 and depict the relationship between the research objectives, questions and eligibility criteria. The population, concept and context inclusion criteria recommended by the updated JBI Reviewer's Manual have been included within the PCC Framework of Figure 1 [15].
Stage three: Study selection
The selection of sources of evidence will be based on the eligibility criteria, mentioned above, and will be conducted by 2 academics (AM and KH) in the research team. The results of the Rayyan and librarian-developed search strategy will form the base of the selection process. The initial selection of sources will be a screening exercise based on the title of the artifact. The abstracts of sources that passed the screening phase will be read and compared with the inclusion and exclusion criteria of this scoping review. Sources that will be included in the study will be downloaded in full and again measured against the primary inclusion and exclusion criteria. This process will be carried out independently by the two review authors. This process will be tested using a pilot sample of twenty titles and abstracts from the literature search to ensure consistency and reliability prior to starting the formal screening process.
Stage four: Charting and appraising the data
For the development of our data extraction forms we adopted the strategy of the Joanna Briggs Institute [15]. The data extraction phase of this scoping review will be competed for each eligible source and data will be recorded as depicted in Appendix 1. The data extraction form will be piloted with 10 articles to determine its appropriateness and the research team will use an iterative process to make any changes deemed necessary. The charting procedure will be conducted independently by the same 2 academics that selected the sources. This process will involve reading and extracting the data from each study that is included and then comparing the data extraction forms for congruency. Data in both extraction forms will periodically be compared by AM and KH. Disagreement will be resolved by rereading the papers that they disagree upon, followed by a consensus discussion. To add further rigour to the review, we will perform a critical appraisal of the methodological quality of the evidence sources using the mixed methods appraisal tool (MMAT) [20]. To reduce bias, AM and KH will independently conduct the MMAT appraisal and then compare and discuss the results.
Stage five: Collating, summarising and reporting of the results
This scoping will present a tabular and narrative account of the available literature on the use of GRS and checklists for clinical simulation-based assessments. We will use two methods to present the data; a quantitative data presentation and a content analysis data presentation. The quantitative data presentation will summarise the data as it relates to the extent, nature and distribution of the studies identified. This numerical data will provide an overview of the identified information relating to the aim and objectives and will provide an overview of potential knowledge gaps in the literature. The second data presentation strategy will utilise thematic analysis to map the use of GRS and checklists within clinical simulation assessment. This strategy will include information related to the use of GRS and checklists within the context of clinical simulation-based assessment strategies, learning outcomes and any identified learning theories. Data from the narrative will be imported into Atlas.ti (version 8, ATLAS.ti Scientific Software Development GmbH, Berlin, Germany) and thematic analysis used to identify themes that will then be summarised and discussed in relation to the study aims and objectives [21].
Step Six: Consultation exercise
Although optional, to add validity to this scoping review, the authors will review the identified literature and will be afforded the opportunity to provide their own insights where relevant to inform and validate the findings. During this phase the following questions will be asked [22]; have all the elements of the review questions and objectives been addressed accurately and has this scoping review been adequately situated within the context of the relevant field of literature, practice or policy?