Assessment of the educational quality of lumbar puncture instruction videos on YouTube.

doi:10.21203/rs.2.16419/v1

Download PDF

Research article

Assessment of the educational quality of lumbar puncture instruction videos on YouTube.

https://doi.org/10.21203/rs.2.16419/v1

This work is licensed under a CC BY 4.0 License

Version 1

posted

You are reading this latest preprint version

Background YouTube has become an important educational platform. For biomedical students the site is a primary information source for informational and instructional videos on a variety of medical topics.This study examined the educational quality of instructional lumbar puncture procedure videos on YouTube.

Methods The search strings “lumbar puncture medical procedure” and “spinal tap medical procedure” were used. The first 100 videos for each search string were assessed for eligibility. Video characteristics were documented. To evaluate the popularity of the videos the Video Power Index was calculated. Four validated checklists were used for video assessment: Lumbar Puncture Assessment Tool, reliability score, Global Quality Score, Attractiveness and Understandability score.

Results Lumbar puncture videos had a median ratio of 8,90 views/day, 0,04 likes/day and a video power index of 14,44, which was lower compared to other categories of instructional videos (CPR and cake baking). The mean LumPAT score was 34/55, with only three videos scoring at least 44/55, the pass ratio of this assessment. The median reliability score was 3/4, the median GQS 3/5 (moderate quality) and the median AU score 6,5/8. Significant correlations were found between the LumPAT score and the video length (p = 0,003), the GQS (p < 0,001) and the AU score (p = 0,01).

Conclusions In this study, only three out of the 23 videos passed the LumPAT score. The majority of the instructional videos are not suitable for the training of health care professionals and students in performing a lumbar puncture procedure.

Medical Informatics

lumbar puncture

spinal tap

instructional video

internet

YouTube

Since its launch in 2005, YouTube has become one of the most popular websites on the internet. With 5 billion videos watched every day and over 30 million active users per day, the site reaches approximately one third of the Internet users (1).

As a result of its popularity, its ease of access and its free content, YouTube has become an important educational platform, especially for the so-called “Millennial” students (2–5). Also in the field of biomedical sciences the site has become a primary information source for students, not only with regard to factual knowledge but also instructional tutorials on a variety of medical topics.

Due to the open nature of the site any individual can upload a YouTube video on nearly any topic. Yet where most scientific or medical publications are subject to a review process, either by a journal editorial board, scientific peers or academic authorities, this is not the case with YouTube videos. The absence of the usual “checks-and-balances” may lead to the uploading of videos with a lower methodological accurateness or evidence-based content. The unregulated nature of the scientific or medical information of these YouTube videos can be potentially detrimental to individuals who intend to use this material in preparation for board exams or clinical rotations.

Some studies have examined the quality of YouTube videos with regard to anatomy, surgical procedures, ECG, ophthalmoscopy, respiratory auscultation and neurological disorders (4,6–12). Yet no systematic analysis has been performed to assess the value of these videos for private study or for integration into classroom teaching.

We hypothesized that YouTube presently contains a large proportion of videos with inadequate educational value. This study aimed to examine the content and viewer perception of instructional videos on the lumbar puncture procedure.

1. Search strategy

YouTube was searched in December (07/12/2018) 2018 via the default settings, and all relevant video metadata were documented for later assessment (13). Searches were conducted using the incognito mode to reduce the influence of location and search history of the computer or user on the displayed videos (14). Two search strings, “lumbar puncture medical procedure” and “spinal tap medical procedure”, were entered in the search bar. Quotation marks were used instead of the logic operator “AND” to ensure the concomitant presence of all search terms. This study considered the first 100 videos for each search string for further analysis.

Videos were included if they were presented in the English language and if they were educational in nature. Duplicate videos, videos about lumbar puncture in children, videos about epidural anesthesia and irrelevant videos (e.g. advertising, presentation of a company) were excluded. All videos were fully viewed to confirm that they included the lumbar puncture procedure. If duplicates of an instructional video were detected, only the video that appeared first on the list was retained for further analysis. All videos that were part of a series were considered as a single file.

For the comparison of “lumbar puncture videos” with YouTube instructional videos from other categories additional search terms were applied, namely “CPR instructions” (as an example of a different medical intervention) and “baking a cake instructions” (as an example of a more general instructional video).

2. Data collection

The following video characteristics were documented: number of views, number of likes and dislikes, number of comments, video length in minutes and days since upload. We documented the number of views, likes, dislikes and comments relative to the number of days on YouTube (15). To evaluate the popularity of the videos a Video Power Index (VPI) was calculated (VPI = like ratio * view ratio /100; with like ratio = like*100/(like + dislike) and view ratio = number of views/days). This index combines number of viewing, user interactions and the audience scale across the entire.

Videos were categorized by source into six groups: patient/individual, government/news agencies, university/school, health care professionals (doctor, hospital), standalone health information websites and other (16,17).

3. Video assessment

Four scoring checklists were used to evaluate the videos. All checklists with detailed item description can be found in the supplementary materials.

The Lumbar Puncture Assessment Tool (LumPAT) was used as the golden standard for the assessment of a lumbar puncture procedure (Supplementary Figure 1). The videos were judged in terms of the planning and preparation, the performance of the procedure, the finalization and the used communication. A total of 11 questions were scored, each with a score from 1 to 5. A pass/fail standard of 44/55 was imposed as this score has been validated to be consistent with trained medical observers’ global judgments of pass/fail (18).

Reliability of information was scored from 1 to 4, based on 4 questions (adapted from the DISCERN tool) (Supplementary Table 1) (19).

The educational quality of each video was rated using the five-point Global Quality Score (GQS), modified from Singh et al (Supplementary Table 2) (19).

Attractiveness and understandability (AU) was scored using an eight-point modified PEMAT (Patient Education Materials Tool) score, adapted from the original PEMAT score for the evaluation of the understandability and actionability of print and audiovisual materials (20). Two organizational aspects, three visual aspects and three auditory aspects were scored for their presence (1) of absence (0) (Supplementary Table 3).

4. Statistical analyses

All data were analyzed using SPSS software (SPSS Inc. Chicago, IL, USA, version 25). The median numeric evaluations of the four investigators were used for statistical analysis. Normality was assessed using the Shapiro-Wilk test (21).

Descriptive statistics analyzed the mean and the standard deviation (if normally distributed), the median and IQR (if not normally distributed), the minimal/maximal value and the proportions (if categorical). Finally, linear regression analyses were performed. P-value < 0,05 represents the threshold for statistical significance.

1. Lumbar puncture video characteristics and comparison with other instructional videos.

A total of 23 lumbar puncture instructional videos were included for analysis out of a total of 200. A detailed overview of the selection process with all relevant exclusion criteria is shown in Figure 1. All videos were independently evaluated by four researchers (R. D. B., M. G., K. H., and E. V. D. B.). Lumbar puncture videos had a median ratio of 8,90 views/day, a median of 0,04 likes/day, a median of 0,001 dislikes/day and a median VPI of 14,44 (Table 1). Compared to videos from the two other instructional categories (CPR and cake baking), lumbar puncture videos scored significantly lower in all measured parameters. The views per day of the CPR and cake baking videos were higher (131 and 4068 views/day respectively). CPR videos also had a much higher popularity as shown by an almost 10-fold higher VPI and the “cake baking” videos had even a 270-fold higher VPI score (Table 1).

The lumbar puncture videos were present in three video categories: health care professionals (34,8%), health information website (26,1%), university/schools (30,4%).

2. Lumbar puncture video content analysis

To assess the lumbar puncture procedure performed in the videos, the validated LumPAT scoring system was used. The mean LumPAT score was 34 (8), on a maximum score of 55. Only three out of 23 videos scored higher than the pass ratio of 44/55 (Figure 2). Most videos received high scores for positioning of the patient, anatomical landmarks, sterility, disinfection and local anesthesia, and insertion of the LP needle. Data that were often missing from the videos were: identification, allergies, contraindications, side effects, replacing the stylet, filling the CSF tubes, post procedure activities (pressure, bandage, cleaning) and informing/comforting the patient. When performing regression analyses a significant relationship was detected between LumPAT score and the length of the video (p = 0,003), the GQS (p < 0,001) and the AU score (p = 0,01). The other video characteristics or scoring systems did not correlate significantly.

An example of the five highest and five lowest scoring LumPAT videos with regard to content is provided in Supplementary Table 4. When comparing the top and bottom five videos, these only differed significantly in the length of the video (p = 0,02). There were no differences in views/days, likes/day, dislikes/day and VPI.

The second quality score with regard to video content, the reliability score, of the 23 videos had a median of 3 out of 4 (Table 2). This was mainly due to the fact that almost no videos provided additional student references.

The third content score, the GQS, showed a median score of 3/5, which corresponds to an overall “moderate” quality (Table 2). Only two videos obtained the highest score of 5 out of 5. These two videos corresponded to the highest LumPAT scores and also the highest reliability scores. These two videos were classified in the source category “health care professionals”.

3. Audiovisual assessment of lumbar puncture videos

The median AU score was 6,5 out of 8 (Table 2). Very few videos contained a summary with key points at the end of the intervention (3/23). Linear regression analysis were found significant results for the reliability and AU score (p = 0,03), and for the AU score and GQS (p = 0,001).

This study analyzed educational videos on the performance of lumbar puncture. We used search terms that would be employed by biomedical students or healthcare professionals in their search for relevant demonstration videos. We assessed the first 100 videos for each search term as it is shown that > 90% of search engine users click on a result within the first 60 results (22).

We used relevant exclusion criteria to obtain only videos that focused on the lumbar puncture procedure and its medical aspects, with the exclusion of therapeutic intervention such as spinal analgesia and epidural anesthesia. This resulted in a total of 23 videos. A significant number of duplicates were detected during the search process. In order to compare these videos with other “how to” videos on YouTube we selected one other medical intervention (CPR) and one general instructional category (“how to bake a cake”). Compared with these videos, it is clear that lumbar puncture videos are less represented and less viewed on YouTube. The VPI was ten times lower in lumbar puncture compared to CPR, and 270 times lower compared to cake baking. Comparing the five highest and lowest videos with regard to views, likes, dislikes and VPI showed no significant differences, suggesting that most viewers are unable to differentiate between procedures of high and low educational quality.

It is striking that only three out of 23 videos had a LumPAT score of 44 or more, which corresponds to a pass grade. As stated before, lumbar puncture videos particularly lacked specific aspects such as identification, allergies, contraindications, side effects, post procedure actions (pressure, bandage, cleaning) and informing/comforting the patient. These deficiencies can probably be explained by the fact that most procedures were performed using simulation models, where ‘identification’ and ‘informing the patient’ are less appropriate. Overall, sterility was well performed, yet in a significant number of videos mistakes were seen. For example, switching from regular to sterile gloves was often ignored and the procedure was performed with regular gloves or palpation of the anatomical landmarks was performed with sterile gloves before disinfection took place… Such sterility missteps are of course less relevant in the case of a manikin, but when they serve as a demonstration for a clinical procedure this imposes a huge risk of complications for the patient, e.g. postpuncture meningitis (23).

The LumPAT score correlated with the GQS and AU score. The correlation between good content and good audiovisual quality suggest that these videos were produced in a professional context with adequate guidance by experts in the field. If points were lost in the AU score this was usually because most videos contained no summary with key points at the end of the intervention (which was also true for the reliability score). The AU score positively correlated with reliability and GQS.

In conclusion, only a few instructional videos on lumbar puncture were available on YouTube, with overall a low to moderate quality. The audience interaction does not differentiate between high and low quality. In this study, only three videos passed the internal validity score and as such are useful for training health care professionals and students in performing this procedure. It is important for students to bear these constraints in mind when searching the Internet for this type of procedural information.

GQS: global quality score

AU score: attractiveness and understandability score

VPI: video power index

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on request.

Competing interests

The authors declare that they have no competing interests.

Funding

No funding was needed for this study.

Authors’ contributions

RB, MG, KH and EB collected analyzed and interpreted the data. JT conceived and designed the study. All authors were major contributors in writing the manuscript. All authors read and approved the final manuscript.

Acknowledgements

Not applicable.

YouTube by the Numbers (2019): Stats, Demographics & Fun Facts [Internet]. 2019. Available from: https://www.omnicoreagency.com/youtube-statistics/
YouTube as a Source of Information on Kidney Stone Disease. Urology [Internet]. 2011 Mar 1;77(3):558–62. Available from: https://www.sciencedirect.com/science/article/pii/S0090429510017085?via%3Dihub
Pitcher GS, Newton DH, Amendola MF. Common Femoral Artery Access on YouTube: What Practices are Being Shown and Who is Delivering the Message? J Surg Educ [Internet]. 2017 May 1;74(3):455–8. Available from: https://www.sciencedirect.com/science/article/pii/S1931720416303245?via%3Dihub#bib7
Jaffar AA. YouTube: An emerging tool in anatomy education. Anat Sci Educ [Internet]. 2012 May 1;5(3):158–64. Available from: http://doi.wiley.com/10.1002/ase.1268
Guraya SY. The Usage of Social Networking Sites by Medical Students for Educational Purposes: A Meta-analysis and Systematic Review. N Am J Med Sci [Internet]. 2016 Jul;8(7):268–78. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27583234
Barry DS, Marzouk F, Chulak-Oglu K, Bennett D, Tierney P, O’Keeffe GW. Anatomy education for the YouTube generation. Anat Sci Educ [Internet]. 2016 Jan;9(1):90–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/26061143
Rapp AK, Healy MG, Charlton ME, Keith JN, Rosenbaum ME, Kapadia MR. YouTube is the Most Frequently Used Educational Video Source for Surgical Preparation. J Surg Educ [Internet]. 2016 Nov;73(6):1072–6. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27316383
Pilieci SN, Salim SY, Heffernan DS, Itani KMF, Khadaroo RG. A Randomized Controlled Trial of Video Education versus Skill Demonstration: Which Is More Effective in Teaching Sterile Surgical Technique? Surg Infect (Larchmt) [Internet]. 2018 Apr;19(3):303–12. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29406814
Akgun T, Karabay CY, Kocabay G, Kalayci A, Oduncu V, Guler A, et al. Learning electrocardiogram on YouTube: How useful is it? J Electrocardiol [Internet]. 2014 Jan;47(1):113–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24119748
Borgersen NJ, Henriksen MJV, Konge L, Sørensen TL, Skou Thomsen AS, Subhi Y. Direct ophthalmoscopy on YouTube: analysis of instructional YouTube videos’ content and approach to visualization. Clin Ophthalmol [Internet]. 2016 Aug;Volume 10:1535–41. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27574393
Sunderland N, Camm CF, Glover K, Watts A, Warwick G. A quality assessment of respiratory auscultation material on YouTube. Clin Med (Northfield Il) [Internet]. 2014 Aug 1;14(4):391–5. Available from: http://www.clinmed.rcpjournal.org/cgi/doi/10.7861/clinmedicine.14–4–391
Muhammed L, Adcock JE, Sen A. YouTube as a potential learning tool to help distinguish tonic-clonic seizures from nonepileptic attacks. Epilepsy Behav [Internet]. 2014 Aug;37:221–6. Available from: https://linkinghub.elsevier.com/retrieve/pii/S1525505014002091
Murugiah K, Vallakati A, Rajput K, Sood A, Challa NR. YouTube as a source of information on cardiopulmonary resuscitation. Resuscitation [Internet]. 2011 Mar 1;82(3):332–4. Available from: https://www.sciencedirect.com/science/article/abs/pii/S030095721001107X
Hillyer GC, MacLean SA, Beauchemin M, Basch CH, Schmitt KM, Segall L, et al. YouTube Videos as a Source of Information About Clinical Trials: Observational Study. JMIR cancer [Internet]. 2018 Jun 26;4(1):e10060. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29945855
Azer SA, Algrain HA, AlKhelaif RA, AlEshaiwi SM. Evaluation of the educational value of YouTube videos about physical examination of the cardiovascular and respiratory systems. J Med Internet Res [Internet]. 2013 Nov 13;15(11):e241. Available from: http://www.ncbi.nlm.nih.gov/pubmed/24225171
Basch CH, Jackson AM, Yin J, Hammond RN, Adhikari A, Fung IC-H. English language YouTube videos as a source of lead poisoning-related information: a cross-sectional study. Int J Occup Environ Health [Internet]. 2017 Jul 3;23(3):222–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29718779
Tolu S, Yurdakul OV, Basaran B, Rezvani A. English-language videos on YouTube as a source of information on self-administer subcutaneous anti-tumour necrosis factor agent injections. Rheumatol Int [Internet]. 2018 Jul 14;38(7):1285–92. Available from: http://www.ncbi.nlm.nih.gov/pubmed/29761222
Henriksen MJV, Wienecke T, Thagesen H, Jacobsen RVB, Subhi Y, Ringsted C, et al. Assessment of Residents Readiness to Perform Lumbar Puncture: A Validation Study. J Gen Intern Med [Internet]. 2017 Jun;32(6):610–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/28168539
Singh AG, Singh S, Singh PP. YouTube for Information on Rheumatoid Arthritis—A Wakeup Call? J Rheumatol [Internet]. 2012 May 1;39(5):899–903. Available from: https://www.ncbi.nlm.nih.gov/pubmed/22467934
Shoemaker SJ, Wolf MS, Brach C. Development of the Patient Education Materials Assessment Tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information. Patient Educ Couns [Internet]. 2014 Sep;96(3):395–403. Available from: https://linkinghub.elsevier.com/retrieve/pii/S073839911400233X
Le Boedec K. Sensitivity and specificity of normality tests and consequences on reference interval accuracy at small sample size: a computer-simulation study. Vet Clin Pathol [Internet]. 2016 Dec 1;45(4):648–56. Available from: http://doi.wiley.com/10.1111/vcp.12390
Jupiter Research. iProspect Search Engine User Behavior Study [Internet]. 2006. Available from: www.iprospect.com
Rössler B, Lahner D, Schebesta K, Chiari A, Plöchl W. Medical information on the Internet: Quality assessment of lumbar puncture and neuroaxial block techniques on YouTube. Clin Neurol Neurosurg [Internet]. 2012 Jul;114(6):655–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22310998

Table 1. Video characteristics of lumbar puncture, CPR and cake baking videos.

Parameter	Lumbar puncture	CPR	Cake baking	P-value
Parameter	Lumbar puncture	CPR	Cake baking	General	LP-CPR	LP-cake	CPR-cake
Viewership/day	8,90 (50,52)	131,27 (178,05)	4068,64 (15430,94)	< 0,01*	0,05	<0,01*	<0,01*
Likes/day	0,04 (0,21)	0,62 (0,89)	19,79 (143,58)	<0,01*	0,04*	<0,01*	<0,01*
Dislikes/day	0,001 (0,01)	0,038 (0,05)	2,30 (5,66)	<0,01*	0,01*	<0,01*	<0,01*
Comment count/day	0,01 (0,03)	0,02 (0,04)	1,55 (5,15)	<0,01*	0,49	<0,01*	<0,01*
VPI	14,44 (77)	119,38 (163,93)	3774,16 (14566,75)	<0,01*	0,08	<0,01*	<0,01*

Presented as median (IQR); VPI = Video Power Index; * indicates a significant p-value

Table 2. Summary of descriptive statistics relating to reliability score, GQS and AU score.

	Reliability score	GQS	AU score
	Reliability score	GQS	AU score
Median (IQR)	3 (1)	3,25 (1,25)	6,5 (2)
Minimum	2	1,5	3,5
Maximum	4	4,75	8
Percentile 25	2	2,25	5
Percentile 75	3	3,5	7

GQS = Global Quality Score; AU score = Attractiveness and Understandability score

Supplementaryfiguresandtables.docx

Download PDF

Version 1

posted

You are reading this latest preprint version

Assessment of the educational quality of lumbar puncture instruction videos on YouTube.

Status:

Version 1

Abstract

Figures

Background

Materials and Methods

1. Search strategy

2. Data collection

3. Video assessment

4. Statistical analyses

Results

1. Lumbar puncture video characteristics and comparison with other instructional videos.

2. Lumbar puncture video content analysis

3. Audiovisual assessment of lumbar puncture videos

discussion

Abbreviations and acronyms

declarations

references

tables

Supplementary Files

Status:

Version 1