Needs assessment is an essential step for the implementation of any simulation curriculum. Suppose we select Obstetrics/Gynecology specialty as an example to explain the process. After reviewing the curriculum, their champions listed 49 skills and behaviors, then they added five skills and behaviors not mentioned in the curriculum after survey analysis. When champions conducted focus group discussion (Phase 3), the participants listed an additional 30 skills and behavior after the first round table discussion and came up with 84 as a combined list. After the third-round table brainstorming, participants agreed to have ten skills and behaviors above the agreed imaginary threshold line and will be concluded and worth funding. The champions used Integration Simulation Sheet and divided the final ranked list into 3 themes as step 4. See Table 9 to compare other specialties.
Moreover, the panel developed the specially designed train the trainers (TOT) guidelines on conducting the simulation process. These guidelines include many areas related to the instructor's qualifications (see Table 11 for TOT guidelines), such as developing the course objectives, design, and facility. The TOT simulation education program aims to create a safe learning environment for residents that will help them understand essential concepts and gain the knowledge and on-the-job training experience to help them increase positive patient outcomes. Trainers who participate in this program must have a healthcare background, be registered with the SCFHS, and hold a qualification in healthcare simulation. More importantly, they must prove they have experience in the Train the Trainers (TOT) simulation course [23-27].
Many works of literature described using multi-sources to collect training gaps' information to facilitate defining more accurate training gaps. Mann and Chaytor described conducting a large-scale survey and then focus group discussion as a needs assessment methodology to assess a specific group of physicians [28-29]. The authors used the chronological sequence of reviewing the curriculum, conducting the survey, and then the focus group discussion.
We found in the sequence of targeted audience simulation needs assessment that the curriculum might not cover some skills and behaviors that are sometimes taught individually in scattered activities. On the other hand, sending a simple targeted audience survey through emails could identify critical learning gaps, and the focus group of stakeholders' interviews added additional learning gaps not addressed by earlier methods. Interestingly, several skills and behaviors showed similar trends in most of the nine specialties in all phases: total combined list and final ranked list (Table 9).
Salzman et al. and his group used a six-step approach to establish a standardized proposal submission and evaluation mechanism. They targeted 190 training programs and 1136 trainees. Their approach was based on recruiting simulation experts to build a structural format to review curricula. Then they recognized those curricula might benefit from simulation experts mentoring and supporting by making proper decisions regarding funds [30].
Moreover, Wehbi et al. published a needs assessment methodology for simulation-based training focused on a single-specialty (i.e., emergency medical practice) to build customized curricula targeting on-site training using mobile units in rural areas to cover training gaps [31].
The methodology sequence started with a focus group, conducted surveys, and analyzed results. As a comparison, the needs assessment gap in our work started with a curriculum review of ongoing structural training programs, followed by an open-ended question survey, then a focus on specialized group workshops. It would be worth comparing emergency medicine themes from our work with Wehbi et al. work to find overlapping areas which is out of this paper's scope.
A previous work of Arab A et al. was found using other approaches in assessing simulation in anesthesia residency training programs in Saudi Arabia [32]. The author set an implementation plan to be staged gradually between 2014-2018. It started by implementing mandatory standardized workshops that matched specialty curriculum and ongoing training activities [33]. Because of that implementation plan, we exclude Anesthesia from our work. We described an approach that has not been previously implemented in Saudi Arabia. Nine medical programs were included in this process and described as a national standardized needs assessment that can be used for different postgraduate specialties. Therefore, the allocation of the existing training programs under the umbrella of the SCFHS was our first step in this study to facilitate the standardized communication between simulation experts and specialty content experts (specialties scientific councils).
One of our limitations in this study was retrieving the process retrospectively. That was related to the process pattern, which is based on reflective learning until we reach a mature process. Another limitation was listing skills and behaviors in the survey without writing structural objectives, which might not reflect the exact meaning of the gap, e.g., communication with the patient might be related to breaking bad news or dealing with difficult patients... etc. This gap was raised and managed during focused group discussion by participants and filling up the integration simulation sheet by specialty champions. Also, we found in sending the survey in Step 2 that we missed the exact number of targeted groups to calculate the response rate. It happened because we sent the survey to all specialty stakeholders without confirming receiving or counting sent surveys.
The needs assessment detailed gaps in every specialty regarding a list of objectives, themes, modules, suggested educational activities, and others that were not addressed in this paper since it is out of scope. Going deep into these details in every specialty might help content experts review these gaps and support their decisions and future publications. It will be interesting to measure secondary outcomes from applying described methodology on training programs, e.g., level of graduates’ competencies or patient safety ]34[. Many previous studies elsewhere illustrated the importance of assessing the secondary outcomes for any methods used in establishing simulation needs assessment, particularly for required skill and behaviors ]35-39[.
Implementation Challenges:
The approach used and described in our study was a creative idea supported by previous needs assessment work. We described some challenges for better learning in the future and might support the simulation community [11], [40-41[.
1- Time and space
For example, the regular meetings conducted between the simulation experts' panel and champions had enriched the feedback on the process used. There were mainly developed to obtain a concurrent reflection, and they reported immediately after each step with review and feedback. Consequently, significant modifications and reviews of ongoing processes were done, which challenged the speed of the process and prolonged the required time to finish the process. However, we can affirm that our approach would reduce the required time for practicing this needs assessment method for simulation at any program. It would take no more than 10 - 12 weeks to complete each cycle and develop a specialty road map without overstretching involved champions and stakeholders.
2- Champion commitment
One of the challenges was champion commitment. Two of the champions were replaced because of no response. At the same time, another specialty needed a third additional champion to re-do part of their work because it does not follow the same methodology.
3- Variation in required skills by specialties
Other challenge we found was the variation between specialties regarding the pattern of training and method of delivering their education activities. For example, surgical specialties had more skill-based activities. Another example was emergency medicine; they had a long list of required skills and behaviors. Their training was usually based on joined rotations with other specialties.
4- Logistics and Accreditation
For successful implementation, practical steps need to be addressed within the scope of this initiative, such as readiness of simulation centers, availability of equipment, and level of educators in terms of using simulation tools. The panel of three physicians with experience in simulation-based education at SCFHS studied infrastructure preparedness to ensure proper and smooth implementation of the initiative. Consequently, summary road map reports were submitted to simulation centers and program directors, asking for feedback. The feedback focused on the availability of similar training centers, the possibility of creating new activities, and the cost required for establishing such a site.