Study Design and Participants:
This study reports a blinded randomized controlled trial (RCT) designed to investigate an online academic writing and publishing workshop that took place between December 2020 and January 2021. The study was approved by the ethical committee at the faculty of medicine of Damascus University in September 2020.
The announcement of the workshop targeted healthcare professionals as well as post- and undergraduate students in all of Syria using Facebook and WhatsApp groups as well as Telegram channels. Then, a private automatically-generated random number was assigned to every participant using Microsoft Excel 365 version 2011 (year 2020). These numbers were used to concealedly allocate the subjects into four arms: Synchronous Active (Syn_A), Synchronous Passive (Syn_P), Asynchronous Active (Asyn_A), and Asynchronous Passive (Asyn_P). The randomization was stratified based on subjects’ educational level, university of study or graduation, and self-reported English writing skills.
A WhatsApp group was created for every arm to facilitate instructions delivery and encourage group discussions of the training materials. Synchronous (Syn) arms attended the workshop online using Zoom, while asynchronous (Asyn) arms only watched the recordings of the workshop, which were shared privately on YouTube™. Following the lectures of the workshop, participants in the Active (A) groups received six scientific writing tasks and were supposed to solve and submit them to the workshop team. Then they were asked to participate in the peer-revision of other participants’ tasks. In contrast, the tasks were shared with participants of the Passive (P) groups followed by their suggested solutions for self-evaluation. Participants were blinded towards the different arms as well as the planned schedule and assignments for each one of them.
The design of the workshop and the tasks:
The lectures of the workshop covered the structure of scientific papers, as well as tips on scientific writing, plagiarism, and the publication process. The lectures were delivered on three consecutive days in December 2020; Every day had two sessions of 40 minutes each, followed by a 40-minute-session to discuss attendees’ questions. The latter session was not shared with the Asyn group to maintain the blinding.
The workshop team designed six exercises that allowed the participants to practically practice the skills they learnt throughout the theoretical sessions. These tasks were presented to all participants a day after the last session and only the A groups were asked to submit them and participate in the peer-revision assessment. The first task was to find three suitable journals for an unpublished manuscript given its abstract and key words. The second through fifth tasks were based on a preprint -at the time of the workshop- of a systematic review on published health-related articles from Syria during the Syrian armed conflict (Abouzeid et al. 2021). They required the participants to write a short paragraph for the introduction, methods, results, and discussion of this manuscript, respectively. For the introduction task, they were given the aims of the article and three articles as a hint to cite from. The methods task was to draft a paragraph based on a bullet point list of the search strategy and the inclusion and exclusion criteria. Summarizing a given table in a paragraph shaped the fourth task. The fifth task aimed to let the participants exercise discussion writing, the participants were presented with a brief paragraph from the results section of the systematic review and were asked to discuss it using three other papers. The whole preprint without the abstract was sent to the participants as a sixth and final task asking them to summarize it in a short structured abstract. Every exercise was provided separately with enough instructions and a word limit. When the participants had questions, they addressed them to the arm group and were answered by the study team.
When participants of the A groups submitted their works, the study team anonymously shared each submitted solution with two other participants from the A groups as well for peer-revision. Peer-revision was applied in the form of comments and tracked changes, and returned to the study team, who in their role returned the revisions to the primary author also anonymously. Overall, every participant was supposed to provide his solution of the task and to review two solutions of the same task by two peers. Participants in the A groups had two weeks to complete the tasks and the peer review process. On the other hand, the P groups were encouraged to solve the tasks independently during the same two weeks. After this period had passed, the respective paragraphs from the original open-access preprint were shared with all participants (i.e., A and P arms) for self-evaluation, and intra-group questions and discussions were welcomed for all arms.
Workshop assessment:
The registration form was already combined with the pre-course assessment, which assessed participants’ objective knowledge and subjective confidence regarding academic writing. The questionnaire started with the informed consent, demographic information, reasons for joining the course, and previous research experiences. After that, the questionnaire continued with multiple-choice questions that first assessed participant’s knowledge of academic writing and publishing principles. This first main part consisted of 30 questions in total, including 14 that were reused with permission from previous studies (Sabouni, Chaar, et al. 2017; Goyal et al. 2020). The rest were designed as a crossmatch exercise to link each component of research articles to the manuscript section it is usually presented in (Appendix I). This part was assessed as the total number of correct answers, and was referred to as the objective knowledge score, which ranged between zero and 30. The second main part was subjective, where participants rated their confidence in 11 specific aspects of writing a scientific paper (Appendix II). The items covered the writing competence for every section of scientific articles separately as well as additional required skills such as the peer-review process and plagiarism. The rating for each individual item ranged from one representing the least confidence to five, which stood for the highest level of confidence. These ratings were summed and referred to as the subjective confidence score, which ranged between 11 and 55. One additional question probed participants’ estimation of their need for guidance while writing a scientific manuscript with one representing maximal need and five representing no need at all.
After the two weeks allowed for solving and peer reviewing the tasks, the post-course assessment survey was distributed among all the participants. This version contained the same subjective and objective questions of the pre-course questionnaire, in addition to one section that allowed them to evaluate the different personal, organizational, and scientific aspects of the workshop as well as their satisfaction. Participants who attended less than three sessions were excluded from the analysis even if they filled the post-workshop assessment. Participants were considered to achieve the tasks or the peer-revision only if they completed and revised at least three out of the six tasks respectively. In February 2022, the participants were invited again to fill in the same questionnaire in order to check the stability of knowledge and confidence change one year after accomplishing the workshop.
Data collection and analysis:
The pre- and post-workshop assessments were designed using Google forms. The data was exported in Excel format, then analyzed with the Statistical Package for the Social Sciences version 26.0 (SPSS Inc., Chicago, IL, United States). Since the reason behind switching 16 participants from Syn to Asyn was unrelated to the intervention, we used an as-treated protocol for data analysis. The categorical variables were represented as frequencies and percentages, and Chi-square test or Fisher's exact test was used to test their associations. Medians and interquartile ranges were used to represent continuous variables because they were not normally distributed according to Shapiro-wilk test. The associations between unrelated samples were investigated using Mann–Whitney U-test or Kruskal-Wallis test, while the paired Wilcoxon Signed Ranks test was used for related samples. In addition, Mann-Whitney U-test was used as a post-hoc analysis for the pairwise comparisons after Kruskal-Wallis test. An alpha value of 0.05 was used to determine the threshold of statistical significance. With a set power of 80%, our comparisons required a sample size of about 50 participants on each side. However, we aimed to recruit a significantly higher number due to an expected high drop out in similar previous studies (Sabouni, Chaar, et al. 2017). The box plots and stacked bar chart were created using Microsoft Excel.