Funding, ethical considerations and registration: This trial was a self-funded study approved by the local committee of research at the College of Dentistry at Prince Sattam Bin Abdulaziz University (PSAU) (1439-03-001). The trial was registered with ClinicalTrials.gov Protocol Registration and Results System (NCT04336813), and the protocol was not published before.
The authors declare that there is no financial interest or conflict of interest in this trial.
Study design: The trial was designed as a cross-over clustered randomised control trial (each group was a cluster) so that each group acted as their control for their knowledge’s retention and perception.
Setting and consent: The study was commenced at the College of Dentistry, PSAU in Alkharj city in Saudi Arabia. Written consents obtained from all participants before starting the trial.
Participants: The eligibility criteria included undergraduate students in the fourth year of their dental training with no prior orthodontic education. Students who were registered in the course for the second time were excluded to lessen the bias associated with increased knowledge. The cohort of the trial involved the whole fourth-year undergraduate class (34 undergraduate dental students).
Randomisation: Participants were allocated to one of the two even groups using computer-generated randomisation. Participants in the control group (CG) were taught through the conventional model of power-point presentation. Participants in the intervention arm used phone-based audience response system (PB-ARS group) as an adjunct during the presentation. The sequence of random allocation was concealed from the researcher who recruited the participants. Each group consisted of 17 male students.
Lectures: Simultaneously, CG and PB-ARS group attended two lectures, the first lecture (L1) titled "Management of Class III malocclusion" while the second lecture (L2) titled "Management of open bite and cross-bite". L1 and L2 were delivered at the main campus of PSAU College of Dentistry. L1 and L2 delivered identically in all aspects, including
- The presentation platform (PowerPoint, Microsoft Corp, Redmond, WA),
- The lecturer (both L1 and L2 were given by the one registered specialist orthodontist (F.A.), and
- The duration of the lectures which was 60 min.
Learning outcomes of the delivered lectures were based on learning objectives and outcomes as specified by the National Commission for Academic Accreditation and Assessment in Saudi Arabia.
Before L1, students were instructed to register with the PB-ARS and to download its application (Poll Everywhere, San Francisco, California, USA,
https://www.polleverywhere.com). Polleverywhere is a smartphone application that has a feature enabling the administrator to launch open-ended and dichotomous questions using either text or multimedia-based (pictures) materials, and then collects and analyses the answers from the users (students) instantly.
Extra smartphones were accessible to students who did not have smartphones at the time of the lecture. Students were blinded from their allocations until the beginning of L1.
Before the lecture, both groups completed a validated multiple-choice question (MCQs) formative assessment. During L1, the participants of the PB-ARS group had access to an interactive poll of new questions regarding the taught topic, via their smartphones. The participants of the PB-ARS group were allowed to read the questions and answer them. Participants in CG were blinded from those questions. At the end of L1, both groups re-sat the same pre-lecture MCQs assessment. A similar protocol was undertaken during L2 a week later, except that groups were crossed-over. Hence, the group which had had PB-ARS integrated during L1 were blinded from the poll of questions during L2, and vice versa. At the end of L1 and L2, participants of PB-ARS and CG groups filled a set of questions regarding their experience with the lecture.
Formative MCQs formative assessment
MCQs formative tests consisted of 20 questions related to the taught topics during L1 and L2. The maximum achievable score was 20. To reduce the carry-over effect, the PB-ARS questions during the lectures were different to the MCQs formative exam sheet. Two authors piloted the bank of questions to ensure its content validity and reliability. Content validity was tested using test matrix and expert judgment. The test reliability was estimated using inter-rater reliability, a correlation of more than 0.7 was considered acceptable. Appendix 1 and 2 include the poll of questions for L1 and L2.
Both groups attended their final written summative exams ten weeks after L2. The final exam was in MCQ format. The summative exams covered questions from all dental and medical subjects taught during the second semester in the fourth year of undergraduate dental training at the College of Dentistry/ PSAU. The summative exams exam included five questions relevant to the orthodontic subjects taught in L1 and L2. The exam questions were identical for all students and delivered under controlled conditions. The summative exam scores specific to L1 and L2 questions were traced and collected using an excel sheet by an independent tutor to reduce reporting bias. The maximum achievable score for the five questions relevant to the subjects taught in L1 and L2 was 5.
At the end of L1 and L2, participants of the CG group filled a set of questions regarding their experience with the lecture. Similarly, participants of PB-ARS group completed another set of questions (Appendix3). The questionnaire used was a modified version from a previous study  with close similarity of the assessed cohorts (Dhaliwal et al., 2015). The questionnaires of CG consisted of 9 questions that assessed understanding of the topic of the lecture, possibility of participation in the lecture, interaction with the tutor and total level of satisfaction. The questionnaire of the intervention group (PB-ARS) included an additional 4 questions specific to PB-ARS that aimed to assess the perception of using PB-ARs as an adjunctive to conventional teaching. Each question was answered using the 0-10 scale. The response of the students was categorised into five categories (Table 1). The 5 categories of responses were strongly agree, agree, neutral, disagree and strongly disagree.
Analysis of the results
Students' responses and scores were exported into an excel sheet for analysis. Students who failed to attend the summative exam were excluded from this trial to reduce the effect of time as a confounding factor. An intention to treat  analysis was adopted to deal with dropouts and missing data of non-compliant participants. Data was analysed by a blinded statistician using IBM SPSS statistics 22, version 22. Pre- and post-lecture formative assessments' scores were analysed and compared to using cross-over analysis with Mann–Whitney U test while t-test was used to analyse summative exam's score.