Is Checklist an Effective Tool for Teaching Research Students? A Survey-based Study

doi:10.21203/rs.3.rs-1447674/v1

Download PDF

Research Article

Is Checklist an Effective Tool for Teaching Research Students? A Survey-based Study

https://doi.org/10.21203/rs.3.rs-1447674/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Students face hardships in determining what are the main points that need more studying in every subject. Checklists are one of the ways that can help students identify the most important pieces of information. Accordingly, in this study, we aimed at examining the impact of using educational checklists on the learning process of postgraduate students in Nagasaki University, Japan.

Methods: Thirty-one Master students, who finished a “how to write a research protocol” course were recruited by sending them an invitation email that had an attached link to a previously developed and tested questionnaire on the SurveyMonkey® platform. After signing the electronic informed consent, twenty-two participants (response rate = 71%) finished the survey. The data was analyzed using Microsoft Excel and expressed in the form of frequencies and percentages.

Results: More than half of the students declared that they know the checklist will be used in the course which we investigated. Only two students used checklists as a means of studying (9%). Twelve students (55%) confirmed that no other courses or lessons in the School of Tropical Medicine and Global Health (TMGH) use checklists. No students found the usage of checklists not easy or not practical to apply. Many students thought the length of the checklist was suitable and not too short (64%), although three students (14%) found it lengthy. Moreover, most students described the checklist as beyond good (86%) and they would recommend using a checklist for teaching other college students (73%).

Conclusion: Using checklists in education can facilitate the learning process, help in memorization, and deepen the concepts being studied. Further studies are required to examine the impact of checklists in teaching undergraduate students and students from other non-healthcare disciplines.

Research education

Checklist

Teaching improvement

Educational checklists are tools to help cover all aspects of the taught material.
Educational checklists help in memorization and deepen the concepts being studied.
Educational checklists should be simple and to the point.

Educational checklists have been used in various settings all over the world for years. However, their positive impacts have rarely been investigated. Only a limited number of studies have investigated the impact of a checklist on a student and the effects on performance. One study has shown that having a checklist to assess individual Doctor of Pharmacy (PharmD) students’ performance in a group has proven to be a reliable and valid tool in the setting of small group sessions in a PharmD curriculum [1]. Another study proved that using a checklist represents a promising approach to improving the quality of care for pregnant women with opioid use disorders [2].

Within the education system, some challenges that are often faced while learning a new subject include difficulty applying learned knowledge to real-life situations and the unavoidable human nature of forgetfulness. By conducting this study on “How to write the protocol”, we can demonstrate that a checklist helps students to overcome the aforementioned challenges. This is because a guiding checklist will alleviate the need to visualize and follow instructions by “imagining” situations in which they will be used in actual practice. A checklist will aid in the student’s ability to put the protocol together in a stepwise fashion, able to see not only where they have been but where they are going. This practicality will also remove obstacles that may otherwise be hidden and help them prepare for this real-world application of writing the protocol with more transparency and much less effort. John Dewey, a leader in education reform, advised instructors that “the most powerful learning would come from combining the intellectual and the practical”, as such, a checklist would benefit the students in “thinking inside and outside of school” [3]. Secondly, forgetting is a necessary component of healthy brain function. The human brain has a default intrinsic pathway for forgetting old information and promoting memory erasure. Without forgetting, our brains would be flooded with unnecessary information, making it more difficult to learn new things. By actively forgetting, our brains are allowing for the formation and consolidation of new memories [4]. In addition, increased stress and emotional distress, which is common in college students, can enhance the forgetfulness pathway [5]. A checklist can alleviate the need for remembering information that can be potentially forgotten. However, further studies are needed to investigate the potential benefits of checklists, especially within an educational setting. This study hopes to investigate the necessity of having a checklist in the classroom and describe the characteristics of a good checklist for students.

Through a survey, this study will evaluate the effects of a checklist on a student's performance, their engagement in the course, and knowledge acquisition, focusing on the development of research protocols. In finding the most effective and successful way to teach and assess students in this area, we can improve the completion and success of future research projects as well as improve the way we relay the basics of research to students.

Study design

This is a cross-sectional, survey-based study conducted at Nagasaki University, Japan. This study was approved on March 31, 2021, by the Ethical Committee (EC) of the School of Tropical Medicine and Global Health Ethical Committee, Nagasaki University, Japan. The approval number is NU_TMGH_2021_171_1 (Appendix 1). The reporting of this study followed the Consensus-based Checklist for Reporting of Survey Studies (CROSS) guidelines [6]. The checklist is provided in Supplementary Table 1.

Participants

The study was based on first-year Master of Tropical Medicine, Master of Public Health, and Master of Science students who had completed a “how to write a research protocol” online course, in March 2021 at Nagasaki University. Participants were included in this study voluntarily. Out of the 31 students who passed the class, 22 students volunteered to participate in the study.

Questionnaire

The questionnaire was developed by our team, but it was self-administered in English using SurveyMonkey®. Participants were sent the survey link through an email after the declaration of their results. The questionnaire was available online for 14 days (from June 1st to June 14th, 2021). The first page of the questionnaire consisted of the description of the study (including aims, benefits, the implication of the study) and the informed consent.

The questionnaire is comprised of three sections. The first one included the socio-demographic details (age, gender, highest qualification, and information related to their use of the checklist). The second section included a detailed assessment to evaluate the effect of the checklist on the performance, engagement, and knowledge acquisition of the student. It included questions regarding the structure of the checklist, benefits, and disadvantages of the checklist. The last section included their final opinion and any recommendations (Appendix 2).

Survey conduction

SurveyMonkey® platform was used to survey the participants, within which the informed consent was embedded. All participants were able to decline or withdraw from participation at any time before clicking the submission at the end of the survey. The survey was sent to School of Tropical Medicine and Global Health (TMGH) Master first-year students via email with two reminders sent each week. Any concerns from participants were addressed by NTH.

Statistical analysis

All the collected data were analyzed using Microsoft Excel. The data were represented as actual frequencies and percentages.

Ethical considerations

Before sending the participants the enrollment emails, ethical approval was obtained from the School of Tropical Medicine and Global Health Ethical Committee, Nagasaki University, Japan. An electronic informed consent was obtained from each participant before the contribution to the study. They were assured that their participation in this study is completely voluntary and that they can disenroll from the study at any time. If any participant had any questions, they would have contacted the principal investigator (NTH) via email. Moreover, there was no incentive provided to the student for completing the survey.

Confidentiality was preserved by a series of steps conducted that included disabling IP address tracking, not collecting personal information that can lead to the identification of the participants (for example, we collected the age using age groups format, not the exact age), and the data collected was encrypted by SurveyMonkey® before the research team handled the analysis. Finally, the SurveyMonkey® account is only accessible by MDNT and NTH.

The survey was sent to 31 students with 22 students completing the entire survey: hence, the survey response rate of 71%. The demographics of all the students are presented in Table 1. Out of the 22 students, seven students (32%) were 18–30 years old, and fourteen students (64%) were older than 30 years, while only one student (5%) refused to provide their age. There were ten male students (45%) and twelve female students (55%). Lastly, there were three students (14%) enrolled in the Master of Tropical Medicine, fourteen students (64%) in the Master of Public Health, and five students (23%) in the Master of Science in Global Health and Medicine.

Table 1

Baseline characteristics of the participants.
	Response rate	Percentage
Age
18–25 years	1	5%
26–30 years	6	27%
> 30 years	14	64%
Unknown	1	5%
Total	22	100%
Gender
Male	10	45%
Female	12	55%
Unknown	0	0%
Total	22	100%
Degree
PhD	0	0%
Master	16	73%
College/University	6	27%
Unknown	0	0%
Total	22	100%
Course
PhD	0	0%
MTM	3	14%
MPH	14	64%
MSc	5	23%
Unknown	0	0%
Total	22	100%
Abbreviations: Ph.D. (Doctor of Philosophy degree); MTM (Master of Tropical Medicine); MPH (Master of Public Health); MSc (Master of Science in Global Health and Medicine).

The results of the survey are included in Table 2. Twelve participants (55%) declared that they knew that the course evaluation would depend on using a checklist that would be made available to everyone throughout the course, while three did not (14%), six students (27%) stated that they could not recall, and one student (5%) did not respond to the question. Among the students, two (9%) usually used a checklist during their studies before attending TMGH, eight (36%) used a checklist intermittently, five (23%) rarely utilized a checklist, and seven (32%) never used a checklist, and no student reported always using a checklist before attending. Fifteen students (68%) found the checklist practical and easy to apply, while seven students (32%) noted they were not sure, and none of the students declared the checklist was impractical or not easy to apply. The length of the checklist was found to be suitable for its contents by fourteen students (64%), three (14%) found that it was too long, five (23%) stated that they were unsure, and no student thought it to be too short. Gaining a better understanding of the lesson by the aid of the checklist was reported by sixteen students (73%), and thirteen (59%) proclaimed there were no disadvantages to the checklist for their learning. The quality of teaching while using a checklist was found to be perfect by 2 students (9%), good by seventeen (77%), not good or bad by three (14%), and none of the students recorded that the quality was bad or awful. Utilizing a checklist for instructing college students was recommended by sixteen of the students (73%), and six (27%) said that maybe they would recommend this teaching style.

Table 2

Participants’ responses to the survey.
Assessing the Checklist	Response rate	Percentage
Were you informed at the beginning of the course that a checklist would be used to evaluate the progress of the course throughout its duration?
Yes	12	55%
No	3	14%
Can’t Recall	6	27%
Unknown	1	5%
Total	22	100%
Have other professors of TMGH used checklists in your courses or lessons?
Never	7	32%
Rarely	5	23%
Sometimes	7	32%
Usually	2	9%
Always	1	5%
Total	22	100%
Have you used or do you use checklists in other studies before attending TMGH?
Never	7	32%
Rarely	5	23%
Sometimes	8	36%
Usually	2	9%
Always	0	0%
Total	22	100%
Was the checklist practical and/or easy to apply?
Yes	15	68%
No	0	0%
Not Sure	7	32%
Total	22	100%
Was the length of the checklist suitable enough for the contents in it?
Yes, it was good	14	64%
No, it was too long	3	14%
No, it was too short	0	0%
Not sure	5	23%
Total	22	100%
Having a checklist in class has helped me… (select all that apply)
Concentrate better in class	7	31.82%
Understand better the lesson and knowledge	16	72.73%
Better for the class	11	50.00%
Study better for the final exam	10	45.45%
Keep focused and prevent unnecessary detours in the course information	10	45.45%
Write the protocol more systematically and organizationally	13	59.09%
Finish writing the protocol in less time	3	13.64%
Communicate better with my colleagues	8	36.36%
Teach another student how to write a protocol easily	3	13.64%
Save my time to remember all items and sections	8	36.36%
The checklist did not provide any significant advantages	1	4.55%
Please select all the disadvantages you found in being taught using a checklist
The checklist has eliminated my constant need to ask for the instructor’s help	3	13.64%
The checklist did not cover all the points required to complete a protocol	3	13.64%
The checklist was so lengthy that whenever I try to use it, I get confused	5	22.73%
Using the checklist lengthened the time needed for writing a protocol	2	9.09%
I found no disadvantages to the checklist	13	59.09%
How would you describe the overall quality of the checklist teaching?
Perfect	2	9%
Good	17	77%
Not good not bad	3	14%
Bad	0	0%
Awful	0	0%
Total	22	100%
Would you recommend using a checklist for teaching college students?
Yes	16	73%
No	0	0%
Maybe	6	27%
Total	22	100%
Abbreviations: TMGH (School of Tropical Medicine and Global Health, Nagasaki University, Japan).

Both teaching and learning medical education content are challenging. A common standard is needed within the medical education system to aid in the improvement of both processes [7, 8]. Checklists are widely used and have shown to be effective in various fields. One study has shown that having a checklist to assess students’ performance in a group proved to be a reliable and valid tool, in the setting of a small group of PharmD students for their curriculum session [1]. Another study has shown that using a checklist might improve the quality of care for pregnant women with opioid use disorders [2]. However, the evidence for the necessity of having a checklist within the classroom, and students’ perception towards it, are undocumented.

Our cross-sectional, quantitative, survey-based study was conducted at Nagasaki University, Japan in 2021. With this study, we aim to fill the gap of knowledge, i.e., investigate students’ perception toward using checklists in class and the necessity of having a checklist to aid learning. We recruited 31 students, twenty-two of them were included in the analysis (three students refused to participate afterward and two were excluded because of missing data). Most of the participants were older than 25 years old (90%, n = 20), sixteen of them were Master's students, and no Ph.D. students were included. More than half of the students declared that they know the checklist will be used in the course which we investigated. Most students do not use checklists in their courses so often, as only two of them (9%) usually used them. Twelve students confirmed that no other courses or lessons in TMGH use checklists. No students found the usage of checklists not easy or not practical to apply. Many students thought the length of the checklist was suitable and not too short, although three students (14%) found it lengthy.

Health education relies largely upon passing knowledge from the instructor to the student. To properly train future healthcare workers and researchers, we must broaden our understanding of how to teach, notably, by including intentional instructional design in the way curriculum is carried out, as a component of clinical education [9]. Undoubtedly, checklists are among one of the best methods for instructional design studies and their protocols. As for demonstrating the necessity of a checklist, our study has shown five major fields students found improvement in as a direct result of its implementation. These areas were: understanding the lesson and curriculum knowledge, writing the protocol systematically and organizationally, studying for the final exam, and keeping focused and preventing unnecessary detours in the course information. Our results were like a study of De La Garza et al., which revealed that checklists also help increase students’ knowledge acquisition with E-learning [10]. More than one-third of our participants responded that the checklist helped them to not only save time remembering details but also facilitate communication between colleagues (n = 8, accounted for 36%). A total of seven students (32%) were able to concentrate better in class. Only one student found the checklist did not provide any significant advantages, which accounted for 5%. No students found the checklist was bad or awful. Most of them described the checklist as beyond good (n = 19; 86%) and they would recommend using a checklist for teaching other college students (n = 16; 73%).

Some limitations of checklists were also investigated. One common disadvantage was that the checklist was so lengthy that it confused students (n = 5; 23%). To a lesser extent, three students (14%) found the checklist was not thorough because of some missing information, and 3 others complained that the checklist has eliminated their need to ask for instructors’ help. In an educational environment, communication between students and teachers/instructors is a crucial part of learning but is often limited. Checklists can further degrade this relationship. We suggest that students use checklists as an organizational tool and a reference source and not for sole dependence of information to resolve this issue. That is because more than the half of students (n = 13, 59%) were satisfied with the checklist and didn’t find any disadvantages. In addition, we have not examined the effect of a checklist on the educational attainment of undergraduate students. Conducting further studies with a bigger sample that includes other medical or research courses is essential to explore the true impact of using educational checklists.

Despite the limitations, our pilot study revealed students have a positive perception of using checklists in class. In the range of our knowledge, our study was the first study in the field of using checklists in education, especially within an educational research setting. The results also suggested that using checklists can ameliorate overall education quality, help to keep students focused and improve their knowledge acquisition, and reduce time-consuming tasks, all the while facilitating communication. It is worth mentioning, a practitioner’s education should be multifaceted and not dependent upon one strategy, such as checklists alone. Furthermore, checklists should be used wisely and alongside other education strategies. For example, a checklist could be used as a supplement for the Rigorous section of the “10R’s of Clinician Education”, to attain maximum effect [9]. This study paves the way for further studies with better designs to investigate the pros and cons of using checklists within a classroom setting. In laying this foundation, we hope to ultimately contribute to the improvement in the quality of the medical education system.

In this study, we aimed at exploring the benefits of using a checklist in research education. This was done by surveying 22 Master's students who attended a “how to write a research protocol” course at Nagasaki University, Japan. All students agreed that the usage of the checklist was not difficult or impractical. And most of them described the checklist as beyond good and recommended its use for teaching other college subjects. Finally, we see that using checklists in education can facilitate the learning process, help in memorization, and deepen the concepts being studied. Further studies are required to examine the impact of checklists in teaching undergraduate students and students from other non-healthcare disciplines.

TMGH: School of Tropical Medicine and Global Health
PharmD: Doctor of Pharmacy
EC: Ethical Committee
CROSS: Consensus-based Checklist for Reporting of Survey Studies

Conflict of interest

The authors declare they have no conflict of interest.

Source of funding

No funding was received for the conduction of this study.

Ethical approval

This research project was performed in accordance with the Declaration of Helsinki. Thereof, an ethical approval was obtained on the 14^th of May 2021, from the School of Tropical Medicine and Global Health Ethical Committee, Nagasaki University, Japan. The approval number is NU_TMGH_2021_171_1.

Consent to participate

An electronic informed consent was obtained from each participant before the contribution to the study. They were assured that their participation in this study is completely voluntary and that they can disenroll from the study at any time by not submitting the survey.

Consent to publish

Not applicable. No images, videos, or other multimedia contents are included in this paper.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available because the sample size is not large enough to guarantee full confidentiality of the participants. Furthermore, we have not obtained a consent from the participants to share the raw material. However, the data can be provided by the corresponding author (Nguyen Tien Huy; [email protected]) on reasonable request.

Authors’ contribution and acknowledgments

NTH accounted for the idea. AMM, GV, JW, DNG, NQK, AI, KM, and NTH contributed to writing the study protocol, obtaining the Ethical Committee approval, and designing the questionnaire. MDNT developed the survey on the SurveyMonkey® platform. NTH, AI, and KM collected the data. GV, JW, and AMM analyzed the data and presented it in the form of tables and figures. GV, JW, DNG, NQK, and AMM wrote the manuscript. All authors revised and approved the final version of the manuscript. The steps of conducting this research were supervised by NTH.

Acknowledgments

The authors would like to thank the participants for undertaking this survey that can improve education of future cohorts.

Salinitri FD, Lobkovich AM, Crabtree BL, Wilhelm SM. Reliability and Validity of a Checklist to Evaluate Student Performance in a Problem-Based Learning Group. American journal of pharmaceutical education. 2019;83(8):6963. doi: 10.5688/ajpe6963.
Goodman D, Zagaria AB, Flanagan V, Deselle FS, Hitchings AR, Maloney R, et al. Feasibility and Acceptability of a Checklist and Learning Collaborative to Promote Quality and Safety in the Perinatal Care of Women with Opioid Use Disorders. Journal of midwifery & women's health. 2019;64(1):104-11. doi: 10.1111/jmwh.12943.
Allen L, Hogan CJ, Steinberg A. Knowing and Doing: Connecting Learning & Work. The Education Alliance at Brown University; 1998.
Davis RL, Zhong Y. The Biology of Forgetting-A Perspective. Neuron. 2017;95(3):490-503. doi: 10.1016/j.neuron.2017.05.039.
Ponds RW, Commissaris KJ, Jolles J. Prevalence and covariates of subjective forgetfulness in a normal population in The Netherlands. Int J Aging Hum Dev. 1997;45(3):207-21. doi: 10.2190/MVQ1-WB58-875H-Y4X0.
Sharma A, Minh Duc NT, Luu Lam Thang T, Nam NH, Ng SJ, Abbas KS, et al. A Consensus-Based Checklist for Reporting of Survey Studies (CROSS). J Gen Intern Med. 2021. doi: 10.1007/s11606-021-06737-1.
Hoffman KM, Trawalter S, Axt JR, Oliver MN. Racial bias in pain assessment and treatment recommendations, and false beliefs about biological differences between blacks and whites. Proceedings of the National Academy of Sciences of the United States of America. 2016;113(16):4296-301. doi: 10.1073/pnas.1516047113.
Meinema JG, Buwalda N, van Etten-Jamaludin FS, Visser MRM, van Dijk N. Intervention Descriptions in Medical Education: What Can Be Improved? A Systematic Review and Checklist. Academic medicine : journal of the Association of American Medical Colleges. 2019;94(2):281-90. doi: 10.1097/ACM.0000000000002428.
Ballantine A, Feudtner C. The 10 R's of clinician education: a checklist. Archives of pediatrics & adolescent medicine. 2010;164(4):389-90. doi: 10.1001/archpediatrics.2010.33.
De La Garza JR, Schmidt MW, Kowalewski KF, Benner L, Muller PC, Kenngott HG, et al. Does rating with a checklist improve the effect of E-learning for cognitive and practical skills in bariatric surgery? A rater-blinded, randomized-controlled trial. Surgical endoscopy. 2019;33(5):1532-43. doi: 10.1007/s00464-018-6441-4.

No competing interests reported.

SupplementaryMaterial.docx

Download PDF

Editorial decision: Major revision
13 May, 2022
Reviews received at journal
10 May, 2022
Reviewers agreed at journal
10 May, 2022
Reviewers invited by journal
10 May, 2022
Editor assigned by journal
10 May, 2022
Editor invited by journal
16 Mar, 2022
Submission checks completed at journal
16 Mar, 2022
First submitted to journal
13 Mar, 2022

You are reading this latest preprint version

Is Checklist an Effective Tool for Teaching Research Students? A Survey-based Study

Status:

Version 1

Abstract

Key messages

Introduction

Methods

Study design

Participants

Questionnaire

Survey conduction

Statistical analysis

Ethical considerations

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1