YouTube as a Source of Patient Information for Bipolar Disorder: A Content-Quality and Optimization Analysis

BD is a mental disorder characterized by shifts in mood and ranging from depression to mania, with periods of euthymia where the patient does not have mood disturbances. The euthymic phase is the best time to treat and provide psycho-educational advice. Bipolar I (BP1) is defined by prolonged periods of mania and depression, often requiring hospitalization. Bipolar II (BP2) is defined by cycles of hypomanic episodes and depression. It affects 1% of the population worldwide and the median age of onset is 25 years [8]. It is found equally among men and women and it prevails in all social classes, races and ethnicities [8]


Background
Approximately 4.6 billion people are active internet users; this translates to 60% of the world population [1].YouTube is the most common platform for video viewing worldwide [1].The general public uses YouTube primarily as a source of entertainment but it has also expanded as a medium for education.90% of people aged 18 to 24 years trust medical information shared by others online [2].YouTube's large volume of information and accessibility attracts doctors, medical students, patients, and their families to use it as a source of medical information.Healthcare-related videos published on YouTube are often not reliable and may contain misinformation [3,4].While there have been studies regarding the role of YouTube videos disseminating psycho education for dementia, schizophrenia and narcolepsy they have been of poor or limited quality, there are currently no studies on the quality of videos for bipolar disorder (BD) [5][6][7].most do not discuss Internet information concerning BD with their physician [11].
Individuals with BD consistently report social stigma as one of the greatest challenges of living with the condition and thus anonymously search for medical information online (rather than seeking professional help) [12].For this reason, BD is often misdiagnosed or can go undiagnosed for up to 10 years which perpetuates the suffering from the disease [13].However, several of the other factors are involved and the delayed diagnosis of BD including first-degree relatives, high stress, trauma, illicit drug usage and most importantly, depressive disorders/emotional lability like neuroticism.
We aimed to assess the current quality and the reliability of You Tube videos on BD using a comprehensive search strategy.We also sought to determine what optimizes a video for audience engagement.

Search strategy
We searched YouTube on November 11, 2019 using the four key phrases; "bipolar disorder", "bipolar disorder treatment", "bipolar disorder symptoms", and "manic depression".We felt that these search terms offered a sufficient view of BD videos because they are relative synonyms to BD.We intentionally chose these phrases since after being given a diagnosis of "bipolar disorder" we anticipated that a patient would probably use the same search term or a slight variation of it on YouTube.

Data collection
Previous research shows that 90% of search engine users only look at the first 3 pages of searched content.Thus, the first 30 results of each search were viewed and evaluated under the chosen criteria and results were recorded [14].

Inclusion and exclusion criteria
Out of 120 videos, we evaluated a total of 80 videos related to BD.Any duplicates, advertisements, music videos, videos in another language or videos clearly not related to BD (e.g., music videos) were excluded.

Variables extracted
We used the Google Chrome extension "VidIQ Vision for YouTube" to retrieve quantitative information from each video.This included: total number of views, total number of comments, duration, video description word count, video description link count, referrers, likes, dislikes, and video tags and time since upload.Information about the host channel was also recorded: uploader name, average daily views, average daily subscribers, subscribers, and tags.
Qualitative information included: impact of the disorder on everyday life, symptoms of BD, personal story/vignette of person, the difference between BP1 and BP2, results of treatment, discussion of prognosis, animations, diagrams, pathomechanisms explained, a doctor speaker and/or a patient experience.We recorded the video upload source and categorized it into one of the following categories: physician, hospital, patient, educational or other.

Scoring system
Videos were evaluated using the validated DISCERN instrument, by two medical students in the final clinical years of their studies who each had five years of experience using the DISCERN instrument.The DISCERN instrument is a sixteen-point questionnaire designed to allow laypeople to judge the reliability and quality of health information as seen in table 1 [15,16].The first fifteen questions are scored on a scale from 1-5 depending on if they meet the given criteria.A score of 1 means the video did not fulfill the criteria, partial fulfillment of criteria can be scored a 2-4 depending on the independent judgment of the scorer, and a 5 is a definite yes.The total minimum score is 16 and the maximum score is 80 points.
The last question is a summary of the previous 15 DISCERN questions and determines whether the publication can be used as an appropriate source of information.An overall rating of 2 or below indicates the publication is of "poor" quality with many shortcomings and is not useful or appropriate.A score of 3 indicates a "fair" quality of publication that requires additional sources of information due to some limitations.A score of 4 or above indicates "good" quality meaning the publication is useful and appropriate as a source of information about treatment choices [15,16].
The DISCERN score may also be interpreted as a total out of 75 using only the first 15 questions.A score of 63 to 75 indicates an "excellent" score, 51 to 62 indicates a "good" score, 39 to 50 indicates a "fair" score, 27-38 indicates a "poor" score and 16 to 26 indicates a "very poor" score [17,18].

Video optimization
The optimization analysis method in this study is similar to studies previously published [4,7,[19][20][21].The average daily views (total views/ days since upload), the like ratio ((likes/likes+dislikes)*100) and the video power index (VPI) ((like*100/(like+dislike))*(views/day)/100) were used to assess the audience engagement with a video.Of note, standardized cut-off scores do not exist for the VPI since it depends on the number of likes and views and this is highly variable depending on how popular a video is.However, we have provided the mean VPI in the results section so that a reader may use it for reference (to see if a VPI value is higher or lower than the mean).
Videos were grouped based on their qualitative content (e.g., if a video included the treatment of BP disorder).These video groups were analyzed against their average daily views, like ratio, VPI and number of comments.

Statistical methods
The normality was checked and in cases where the data were normally distributed, the Mann-Whitney U test was used to find differences between categorical variables.The intraclass correlation coefficient was used to ascertain inter-rater agreement.P<0.05 was deemed significant.Google Sheets (Google LLC, Mountain View, California, USA) was used for illustrations.Past (Hammer and Harper, Øyvind Hammer, Natural History Museum, University of Oslo) was used for statistical analysis.

Video contents
A total of 80 unique videos were included in our content analysis.Figure 1 shows the qualitative content of videos about BD.Almost all videos covered the symptoms (100%) and the impact (99%) of BD on daily life.Most videos included the treatment (60%), the prognosis (54%), a personal vignette (54%) and a patient experience concerning BD (51%).However, most videos did not have a doctor speaking (48%), animation (28%), BD pathomechanisms (25%), a diagram (23%), and the differences between BD1 & BD2 (31%).

Video upload source
Figure 2 illustrates the source of the videos uploaded.The majority of the videos were uploaded by an educational channel 61.3% (49 videos).The remainder of the videos was uploaded by a hospital 16.3% (13 videos), a physician 11.3% (9 videos), a patient 8.8% (7 videos) and 2.5% (2 videos) were uploaded by other sources.

Video quality evaluation
The mean DISCERN score of the two raters for DISCERN was 63.4 ± 11.3 (38-80) (out of 80 possible points).The first rater and second rater had a mean DISCERN score of 63.5 ± 11.3 (39-80, and 63.3 ± 11.3 , respectively.The intraclass correlation coefficient for the absolute agreement between the two raters was 0.996 (95% Confidence Interval of 0.9938 to 0.9975); this is regarded as "excellent" reliability according to Cicchetti and Koo [22][23][24].
The mean sum of the first 15 items of DISCERN was 59.3 (out of 75 total points).This is considered a "good" quality of information (62-51 points) [17].
The mean total DISCERN scores differed in regards to the uploader: patient videos scored the lowest at 47.64 while physician video uploads scored the highest at 68.11.Hospital videos scored 62.23 and educational channel videos scored 64.93.

Video quality correlations
Table 2 shows video quality in relation to the video content.Videos had a greater DISCERN score when they included: the results of the treatment (P=0.0001), the prognosis (P=0.0001), the pathomechanisms (P=0.0003), a doctor speaker (P=0.0004) and diagrams (P=0.0098).

Optimization analysis
Table 3 shows that audience engagement concerning the video content.Videos that contained an animation (P=0.034) or a doctor speaker (P=0.0006) had significantly higher average daily views and a higher VPI.There was a statistically significant relationship with a doctor speaker (P=0.0005) in the video and a higher number of comments.There were no statistically significant findings in regards to the like ratio.

Top-quality YouTube videos
Table 4 shows the highest-scoring YouTube videos.The three topscoring videos were tied with 80 points and the following 3 were tied with 79 points.The mean DISCERN score of the top six videos is 79.5 (99.375%), which means patients may view them as a trusted source of health content.Half of the videos were uploaded by an educational channel, two were uploaded by hospitals and one was uploaded by a physician.
All these videos included the following five features: symptoms of BD, the impact of the disorder on everyday life, results of treatment, discussion of prognosis and a doctor speaker.In general, these videos were 4 times longer in duration and (on average 4028 seconds in comparison to the average of all videos 1034) covered an entire gamut of information.

Quality analysis
The overall quality and reliability of the bipolar videos were of good quality.Thus, patients may gain basic knowledge about BD on YouTube.Healthcare professionals have a duty to provide their patients with reliable medical information.Since it is impractical to peer-review all videos and other information on the internet, we have listed the top 6 highest quality videos on BD so that physicians may share reliable educational YouTube content to their patients over online communications (e.g., e-mail, social media, hospital websites).While there is research on other neuropsychiatric disorders on the internet and on YouTube, our paper is the first to cover an in-depth analysis of the quality and reliability of BD.
Our study shows that YouTube contains adequate medical information concerning BD.However, several factors were often omitted, these included: the risks of each treatment, the consequences of treatment, the benefits of each treatment, how treatment would affect the overall quality of life, how each treatment works, and the possibility of more than one treatment option.Future video creators may focus on these aspects as they are frequently omitted.Including this information would not only increase the quality of the videos, but also the popularity.We suggest that medical institutions tasked with publishing medical YouTube videos should consider our recommendations and use the DISCERN tool as a checklist for what should be included in their videos to provide a valid information package.We encourage hospitals to publish high-quality and informative videos on BD and other medical topics.

Optimization analysis
Prior studies found that patients do not engage with the highest quality educational videos [25].Our study has comparable findings.Thus, it is critical for content creators (i.e., hospital staff and medical professionals) to understand what makes a video attractive and engaging to the viewer.This way, the most informative videos may be given attention instead of being ignored.
Our results show that videos containing an animation garnered the highest average daily views and VPI.This demonstrates that visual aids may make videos more appealing to viewers.The presence of a doctor speaker resulted in more comments, average daily views and VPI.This indicates that patients are more inclined to watch and interact with

Journal of Psychiatry and Mental Health
Open Access Journal videos uploaded by credible medical professionals.This is a reassuring finding to know that patients appreciate or may be trying to reproduce the classical doctor-patient interaction on a digital interface.The presence of a patient experience was another determinant of high popularity indicating that patients value information coming from a relatable source.
However, we highlight that a direct cause-effect relationship between content (e.g., a doctor speaker) and audience engagement metric (e.g., average daily views) may not be able to be established since the viewer discovers the content of the video only after it is clicked.For example, unless the title or thumbnail indicates the content of the video (e.g., "Doctor X talks about Bipolar Disorder" or a thumbnail of a person in a white coat) the viewer would not know that there is a doctor speaker before clicking it.Of all the audience engagement metrics we have analyzed, we assert that the VPI and like ratio are the best indicators of engagement as a has to like the video while watching it.Therefore, "clickbait videos" (which may have a higher number of views due to a catchy title and thumbnail) are not part of the equation.Clickbait videos would actually have a lower VPI and like ratio score since people would dislike the video more.

Context
After deep analysis, the sources of the videos with the highest popularity were more for entertainment purposes, especially news channels or other educational channels rather than hospitals.Media has a potent influence on the public perception of any kind of illness [26,27].Social media influencers should be mentioned as they can often outshine medical authorities in terms of traffic, views comments and popularity.Not to mention they profit from YouTube with ads and sponsorship deals [28].There is no doubt that social media influencers often engage in mental health awareness as this is a popular topic among their viewers [29].With the majority of YouTube users being between 15-25 years old and 75% of mental health disorders being manifested before the age of 24, it is understandable that mental health is a popular topic among this demographic [30,31].
Taking all this into consideration, it is evident that the risk of misconception is high since media coverage is bound to focus on details that create impact and interest.Therefore, it is of critical importance that health care institutions use media as a type of communication to provide accurate and high-quality information to the public.As noted previously, YouTube and other internet websites have gained a more active role in the life of patients and their management of illness or disease.During epidemics such as the recent novel corona pandemic, the role of telemedicine and online source of health information is emphasized even more.Doctors need to be aware of this and provide reliable resources for their patients.Previous YouTube quality studies found that physicians upload a relatively low number of educational or medical videos on YouTube [3,4].Our study matches this finding as only 11.3% of the videos were published by physicians in our study.We encourage medical professions to use our optimization analysis findings to make their videos more appealing to the viewer.
Patients search the internet for medical information most often to learn about side effects from prescription drugs or to obtain help from coping with their disease [11].This is especially important for patients with BD since the majority of them will inevitably turn to the internet for sources of health information [11].Our study showed that videos often lacked information regarding how to cope with the disorder and what treatment they may undergo.
Patients and physicians may greatly benefit from empowering patients with unbiased and robust medical information.It has been shown to reduce diagnostic times, create online support communities and encourage a more productive patient-doctor dialogue [3,[32][33][34][35].
The ability to provide reliable information at a distance is just one of the ways telemedicine is expanding.Modern methods of psychiatric patient care include E-mental health apps with medication monitoring and individualized messages based on real-time data self-reported by patients [36,37].This is especially critical in the current era of practicing medicine at a distance.However, while these validated digital tools and helpful online resources exist, we wish to highlight the "digital divide" where people with lower technical skills or access to digital technology may not be able to utilize these resources.Indeed, a 2016 study showed that patients with low health literacy were less likely to use health information technology [38].The current coronavirus disease 2019 pandemic has exacerbated this divide as essential services have moved online although some populations still do not have access to the internet [39].
We stress that medical information on YouTube is only supplementary and does not replace seeing a doctor.Bauer M, et al. [40] brings to attention the potential medical harm related to "poor quality online information, self-diagnosis and self-treatment, passive monitoring, and the use of unvalidated smartphone apps [40]." Moreover, there is potential to quickly share fake medical news with the use of social media.Waszak PM, et al. [41] found that popular websites related to common diseases contained misinformation in 40% of cases and were shared 451,272 times from 2012 to 2017 [41].

Limitations
One may argue that a physician with experience with BD, such as a psychiatrist, should be involved in the video evaluation process.However, the DISCERN instrument was created for "patients and information providers" and "was not dependent on specialist knowledge of a health condition or treatment." In addition, the two raters had an excellent intraclass correlation agreement indicating the quality analysis study was reliable and conclusive.Moreover, the two independent raters were medical students both having completed 90 hours of clinical rotations in psychiatry.We based our main keywords on the suggestions and synonyms that the YouTube search provided.Thus, we believe that they are relevant.However, the paper would have even more robust findings if a pilot study was done to see exactly what BD patients search for online.

Future directions
Since this paper is a cross-sectional evaluation, we recommend that it be repeated within a few years to monitor alterations of video content quality.Such studies may focus on how physician and hospital-based health channels on YouTube provide medical contentbased and if it up to date with recent evidence.Additional studies may be done to examine the extent to which patients with BD refer to YouTube specifically to gather information about their disease and whether they regard this information as valuable.Furthermore, the YouTube algorithm that ranks and suggests videos is complex, highly personalized and constantly changing [42,43].Moreover, new videos are being added constantly and new popular trends affect the search results (e.g., the amyotrophic lateral sclerosis ice bucket challenge of 2014).In our experience, we did not notice any popular trends concerning BD.However, we encourage future scientists to publish a follow-up study to monitor the quality of YouTube information on BD over time and obtain a clearer understanding of which video content is most frequently recommended by YouTube.
Since only English videos were analyzed future studies may focus the differences in video quality in various languages and geographical locations where there may be more stigmatization of mental diseases [44].Furthermore, future studies could use the DISCERN instrument to evaluate other sources of medical information about mental health such as phone apps or websites.We encourage physicians to inform patients about DISCERN tool to evaluate medical health information for themselves.

Conclusion
YouTube is often a good source of information on BD as it fulfills the basic information required for patient information.However, the quality of information may still improve particularly in regards to treatment risks, consequences, benefits and overall treatment options.The unregulated nature of YouTube means that videos may fail to meet a higher standard in conveying medical information.Thus, care should be taken by health professionals to provide reliable online information.We suggest that physicians reference our list of the highest quality videos on BD.Since a doctor speaker and animation significantly optimized a video for high audience engagement, we recommend that physicians create medical content on BD and include animation in the video.

Figure 2 :
Figure 2: Source of video uploads on bipolar disorder.

Figure 3
Figure3illustrates the mean score for each of the 16 questions of DISCERN.Questions 1 to 3 scored the highest ratings overall.They pose essential questions in regards to the qualitative video content: are the aims clear, does it achieve its aims, and is it relevant.Questions 4 to 8 all scored above 4, this is considered a "good" score.These five questions rate the publication based on the publication's clarity of sources of information, clarity of when the information was reported or bias, the inclusion of additional sources, and uncertainty.The lowest scores were for questions 9 to 15.These questions are focused explicitly on treatment or lack thereof: how it works, what are the benefits, what are the risks, how they affect the quality of life, and the variety of treatments.

Figure 3 :
Figure 3: The mean score of the two raters for the 16 DISCERN instrument questions.Abbreviation: Q: question

Table 2 :
Statistically significant relationships between the DISCERN score and selected qualitative video content.

Table 3 :
Statistically significant relationships between the average daily views, comments and the video power index against qualitative video content.

Table 4 :
The top six highest quality bipolar disorder videos based on the DISCERN criteria.