Study Design
Over the span of the first four months of the research (2019 February 3- June 5), a total of 65 students were expected to pass the orthopedic course without using the Orthobox application, and who comprise the control group. Each internship courses lasts two weeks and includes an average of nine students; during the four months, a total of seven internship courses took place which means a total of around 65 students pass the orthopedic course. Accordingly, fourteen different standard exams were designed by the orthopedic standard exam board before study recruitment began, and seven of them were randomly chosen for our seven groups of 9 students. Then, each group is given one of these seven exams randomly at the end of the two-week courses. In a similar manner, after two months of study pause (it will be more clarified in the limitation section), 65 students were considered to participate in the orthopedics course over the span of the second four months of the study (2019 August 5–18 December), and are considered as the case groups. However, this time, the Orthobox application, available in iOs®- or Android®-based operating system, is installed on the students’ smartphones. As such, they use the application during the course, and are tested by the same seven final exam tests given to the control groups. The application panel provides case group activity report to the executors for further correlational analysis. Both the case and the control group participants are asked to fill out an attitude Visual Analogue Scale (VAS) questionnaire at the end of each course before receiving their scores. Data were also collected from students’ scores on the final exams specifically designed for this research. These data were then gathered and compared to each other in order to identify the impact of the Orthobox on students’ knowledge of orthopedics.
Tools
Orthobox software: As mentioned before, the interventional tool used for this study was an application designed specifically for teaching orthopedics named Orthobox. The Orthobox application is free of charge. The app consists of five main parts of educational information (Fig. 1A): 1) frequently used medication in orthopedic wards, 2) order samples for common orthopedic hospitalized patients, 3) prescriptions for common orthopedic ambulatory patients, 4) cast and splint types, and 5) educational movies. Here is the detailed description of each part:
1) The first section is mainly a list of several groups of medication including antibiotics, anticoagulants, analgesics, fluid therapy, vaccines, and casting materials. In each group, we have introduced those medicine which are most frequently ordered or prescribed in orthopedic wards or clinics. The available information about each medicine in this application includes name and classification, forms, indications, contraindications, caution, side effects, and group in pregnancy (Fig. 1B). 2) In the second section, we have selected eight common order notes in orthopedic wards, which include orders for the following orthopedic conditions: active bleeding wound, amputated finger, fat embolism, open fracture, laceration without active bleeding, supracondylar fracture, trochanteric fracture, and deep vein thrombosis (DVT). For each order, a typical history of the patient is provided along with their X-ray or real limb image, and also the appropriate order is written at the end. Certain words in the texts of this section are highlighted, which will provide additional information if you click on them (Fig. 1C). 3) The third section presents several frequently used prescriptions for patients with orthopedic conditions such as electromyography, knee physiotherapy, radiology, and serologic tests (Fig. 1D). 4) The section about casts and splints provides specific descriptions along with their related images about different types of casts and splints, which are used in different parts of the body (Fig. 1E). 5) In an attempt to familiarize students with the casting and splinting skills, useful educational movies about upper and lower limb casting and splinting are provided in the last section of the application. This part of the application requires internet connection to work (Fig. 1F). We are going to make the app publicly available once the investigation is complete. An English-language demonstration video of the application is provided as appendix to the paper (Additional file-1). The development of a new platform enables executors to update and customize the content of the app in order to meet the demands of each specific center.
Application activity report
Each student of the case group is submitted to the application with a defined username and password which is going to be expired after two weeks, at the end of the course. The Orthobox panel enables the executors to get access to the number of times each user visits each part of the application. At the end of the study we had an activity report from students in case group so we could analyze the correlation between student’s exam/ VAS score and his/her activity on application (Additional file-2).
Final exam: Both the teaching plan and final exams for orthopedics internship courses were based on 4 divisions: Order, Prescription, Medication and Skill. The final exams are designed by two designated professors from the orthopedics department. The exam material for both the control and case groups was the Textbook of Orthopedics and Fractures by Dr B. Aalami Harandi, et al., which students are supposed to self-study. For the purposes of this study, the case groups were also given access to the app as a supplementary source of study. The exam consists of four questions requiring the students to write down a complete order for a certain patient history provided on the exam sheet, write a clinic patient prescription for a presented patient, answer questions regarding a certain medicine, and finally explain the techniques of casting or splinting. Before recruitment, the orthopedics exam board prepared fourteen different, but equally difficult (the level of difficulty was carefully balanced by the in-charge professors) questions for each of the four divisions. These questions were then randomly put together to design 14 written question sheets seven of which were finally selected for both groups’ students’ final exam. The exam papers were corrected based on a system of differentiating between major and minor faults. First, the writers carefully defined the standard answer, major fault and minor fault which are reported in Table 1 in detail. This table is based on similar definitions used in driving tests. The term ‘major fault’ is defined here as those answers which miss a necessary point or are completely wrong, whereas the term ‘minor fault’ refers to those answers in which the required point is mentioned but they are not the standard answer. Then, one professor (who was not part of the research team) corrected the exam papers based on these determined guidelines. Therefore, there was no bias in this regard. This method enables us to compare the major and minor faults on different exam questions between case and control students, and analyze their correlation with the number of visits case students had made to each corresponding part of the application.
Table 1
Major and minor fault in exam answers
Definition
|
|
Major
|
Minor
|
Definition
|
Answers which miss a necessary point or are completely wrong, leading to a complication for patients if happens in real.
|
Refers to those answers in which the required point is mentioned but they are not the standard answer.
|
Example
|
|
Major
|
Minor
|
Order
|
Q: Tibia fracture A: Not ordering for check of the compartment syndrome signs.
|
Q: Tibia fracture A: Not ordering the ankle and knee joints radiography
|
Prescription
|
Q: Wound healing discharge A: adult dosing for a child patient.
|
Q: Wound healing discharge A: Writing “Tab” instead of “Cap” for Cephalexin
|
Medication
|
Q: Heparin A: Wrong dosing
|
Q: Heparin A: Not writing complete medicine pharmaceutical forms
|
Procedure
|
Q: Short leg cast A: Wrong foot angle during casting
|
Q: Short leg cast A: Wrong but near to correct number of cast layers
|
Attitude questionnaire
A short VAS scale questionnaire (consisting six question) was designed (Additional file-3). The questions required participants to rate how much they had learned about the different branches of orthopedics knowledge- five questions about medicine, order, prescription, casting and splint skills-so that the amount of learning could be measured quantitatively. Moreover, the last question on this questionnaire asked students about their satisfaction regarding the overall educational programs of the orthopedics course. The ratings given to each question on the questionnaire were then compared between case and control group to determine their relationship with the application usage. Moreover, the correlation of each VAS question score with number of visits made to the corresponding part in the application within case group students were investigated.
Study Population
Using a convenience sampling method, the orthopedic intern students attending to orthopedic departments were recruited for this study. Unfortunately, because of the Covoid-19 pandemic and therefore, changes in educational programs, the last case group was not recruited and we could not reach to the expected 65 students of case group. As a result, a total of 123 students − 66 students in control group and 57 students in case group-. participated in this research. Students who did not own a smartphone, who were a guest student, or those who were passing the course for the second time were excluded from the experiment. Two interns in control and one in case group were then excluded according to the criteria of second time participation. Therefore, final 120 students were studied. Table 2 provides an overview of the study population demographic data. It can be seen from the data that none of the demographic differences were statistically significant between two groups of study. The average score in demographic data refers to the average score of students from the beginning of their studies in medical school. The pre-internship exam is taken twice a year from those students who have passed ten semesters and want to become interns; the score of this exam is also included in the demographic data.
Table 2
Demographic data in case and control groups
variable
|
Whole population
|
Case group (mean ± SD)
|
Control group (mean ± SD)
|
P value
|
Sex*
|
|
|
|
|
|
Women, number (%)
|
68 (56.7)
|
29 ± 51.8
|
39 ± 60.9
|
0.313
|
|
Men, number (%)
|
52 (43.3)
|
27 ± 48.2
|
25 ± 39.1
|
Age** (year)
|
25.33
|
25.30 ± 1.48
|
25.36 ± 1.26
|
0.808
|
Marital status*
|
|
|
|
|
|
Married, number (%)
|
45 (37.5)
|
17 ± 30.4
|
28 ± 43.8
|
0.131
|
|
Single, number (%)
|
75 (62.5)
|
39 ± 69.6
|
36 ± 56.3
|
Internship duration background** (month)
|
8.07
|
6.86 ± 5.96
|
9.28 ± 7.66
|
0.055
|
Average score** (0 to 20)
|
16.05
|
15.95 ± 1.16
|
16.15 ± 0.96
|
0.304
|
Pre-intenship exam score** (0 to 200)
|
121.81
|
119.58 ± 16.63
|
124.04 ± 23.02
|
0.252
|
*mean and SD is calculated using Chi square test for “sex” and “marital status” |
** mean and SD is calculated using T test for other quantitative demographic parameters. |