Funding, ethical considerations and registration: This trial was self-funded and approved by the local committee of research at the College of Dentistry at Prince Sattam Bin Abdulaziz University (PSAU) (1439-03-001). The trial was registered with ClinicalTrials.gov Protocol Registration and Results System (NCT04336813), however, the protocol was not published. The authors declare that there is no financial interest or conflict of interest in this trial.
Study design: The trial was designed as a cross-over clustered randomised control trial (each group was a cluster).
Setting and consent: The study was commenced at the College of Dentistry, PSAU in Alkharj City in Saudi Arabia. Written consent was obtained from all participants before starting the trial.
Participants: The eligibility criteria included undergraduate students in the fourth year of their dental training with no prior orthodontic education. Students who were registered in the course for the second time were excluded to lessen the bias associated with increased knowledge. The cohort of the trial involved 34 undergraduate dental students.
Randomisation: Participants were allocated to one of the two even groups using computer-generated randomisation. Participants in the control group (CG) were taught through the conventional model using PowerPoint presentation. Participants in the intervention arm used Phone-Based Audience Response System (PB-ARS group) as an adjunct during the PowerPoint presentation. The sequence of random allocation was concealed from the researchers who recruited the participants. Each group consisted of 17 male students.
Lectures: Simultaneously, CG and PB-ARS groups attended two lectures, the first lecture (L1) titled "Management of Class III Malocclusion" while the second lecture (L2) was titled "Management of Open Bite and Cross-bite". L1 and L2 were delivered at the main campus of PSAU College of Dentistry. L1 and L2 were delivered identically in all aspects, including:
- the presentation platform (PowerPoint, Microsoft Corp, Redmond, WA),
- the lecturer (both L1 and L2 were given by the same registered specialist orthodontist (F.A.), and
- the duration of the lectures which was 60 minutes.
Learning outcomes of the delivered lectures were based on learning objectives and outcomes as specified by the National Commission for Academic Accreditation and Assessment in Saudi Arabia.
Before L1, students were instructed to register with the PB-ARS and to download its application (Poll Everywhere, San Francisco, California, USA, https://www.polleverywhere.com). Extra smartphones were accessible to students who did not have smartphones during the lectures. Students were blinded from their allocations until the beginning of L1.
Before the L1 and L2, both groups completed a validated multiple-choice question (MCQs) formative assessment. During L1, the participants in the PB-ARS group had access to an interactive poll of new questions regarding the taught topic, via their smartphones. The participants in the PB-ARS group were allowed to read the questions and answer them. Participants in the CG were blinded from those questions. To assess the improvement in the students’ performance, at the end of L1, both groups again completed the pre-lecture MCQs test. A similar protocol was undertaken during L2 a week later, except that the groups were crossed-over. Hence, the group which had PB-ARS integrated during L1 were blinded from the poll of questions during L2, and vice versa. At the end of L1 and L2, participants in the PB-ARS and CG groups answered a set of questions regarding their experience with the lecture.
Formative MCQs exam
MCQ formative tests consisted of 20 questions related to the topics taught during L1 and L2. The maximum achievable score was 20. To reduce the carry-over effect, the PB-ARS questions during the lectures were different from the MCQ formative written exam sheet. Two authors piloted the bank of questions to ensure its content validity and reliability. Content validity was tested using test matrix and expert judgment. The reliability test was estimated using inter-rater reliability. A correlation of more than 0.7 was considered acceptable.
Both groups attended their final written summative exams ten weeks after L2. The final exam was in MCQ format. The summative written exams covered questions from all dental and medical subjects taught during the second semester in the fourth year of undergraduate dental training at the College of Dentistry, PSAU. The summative written exam included five randomly selected questions relevant to the orthodontic subjects taught in L1 and L2. The written exam questions were identical for all students and delivered under controlled exam conditions. The summative written exam scores specific to L1 and L2 questions were traced and collected using an Excel spreadsheet by an independent tutor to reduce reporting bias. The maximum achievable score for the five questions relevant to the subjects taught in L1 and L2 was 5.
At the end of L1 and L2, participants of the CG completed a set of questionnaires regarding their experience with the lecture. Similarly, participants of PB-ARS group completed another set of questionnaires (Appendix 1). The questionnaire of the CG consisted of 9 questions that assessed understanding of the topic of the lecture, possibility of participation in the lecture, interaction with the tutor, and total level of satisfaction. The questionnaire of the intervention group (PB-ARS) included an additional 4 questions specific to PB-ARS that aimed to assess the perception of using PB-ARS as an adjunct to conventional teaching. To enhance the validity and reliability of the questionnaire, the original English version of the questionnaires, adopted from a previous study with close similarity to our assessed cohorts, was also used in our study . Additionally, the questionnaire was distributed between the authors to reduce ambiguity and modify confusing questions, if deemed necessary. Each question was answered using a 0-10 scale. The response of the students was categorised into five categories. The 5 categories of responses were: strongly disagree (score 0-1), disagree (score 2-4), neutral (score 5), agree (score 6-8) and strongly agree (score 9-10).
Analysis of the results
Students' responses and scores were exported into an Excel spreadsheet. Students who failed to attend the summative exam and decided to take the “resit” exam were excluded from this trial to reduce the effect of time as a confounding factor. An intention to treat  analysis was adopted to deal with dropouts and missing data of non-compliant participants. Data was analysed by a blinded statistician using SPSS 22. Pre- and post-lecture formative assessment scores were analysed and compared using cross-over analysis with the Mann–Whitney U test, while a t-test was used to analyse summative exam scores.