Quality analysis of YouTube videos about thyroid orbitopathy

To evaluate the quality and reliability of YouTube videos about Thyroid Orbitopathy. A YouTube search was performed using the keyword 'Graves eye disease’, ‘thyroid eye disease’, ‘thyroid orbitopathy’, ‘thyroid ophtalmopathy' in the search bar of YouTube. The first 50 videos were analyzed for each keywords, and the first 100 videos that came out were included in the study. The numbers of views, likes, dislikes, comments, daily viewing rate (number of views per day), uploaded source, country of origin, video type (patient experience, scholarly information), and described treatment technique were evaluated for all videos. They were also evaluated regarding their DISCERN, Journal of the American Medical Association (JAMA), global quality score (GQS) and usefulness score by two independent ophthalmologists. Of the top 100 videos, 94 videos met the criteria. The mean DISCERN, JAMA, GQS and usefulness score were 55.27 ± 16.57, 3.04 ± 0.64, 3.44 ± 0.93 and 3.23 ± 1.1. If we look at the upload source 53 (56.4%) videos were uploaded by physicians, 30 (31.9%) videos by institutions/private health institutions, 7 (7.4%) videos by health channels, 4 (4.3%) videos by patients. All scoring systems showed a statistically significant and strong positive correlation with each other (p < 0.001). A statistically significant positive correlation between viewing rate, likes and comments was observed. As the duration of the video increased, a significant increase in the scores in other scores was observed. We observed that the tests we used in the scoring were correlated with each other. Most of the thyroid orbitopathy videos on YouTube were of good quality. Ophthalmologists should guide their patients who want to get information on YouTube to watch videos uploaded by health care professionals.


Introduction
Thyroid orbitopathy is an autoimmune inflammatory pathology caused by Graves' disease (GD). Its incidence is 2.67-3.3/100,000 per year in women and 0.54-0.9/100,000 per year in men [1]. Although the disease is more common in women, it is more severe in men [2]. Ophthalmic involvement is absent in the majority of patients when they are first diagnosed, and over the years, two-thirds have mild or no ophthalmic pathology [3]. In the moderate-to-severe form, clinically severe symptoms develop and do not fully respond to medical treatments [4]. The lack of response to current clinical treatment leads patients to seek different treatment methods.
The use of social media, especially for information and research purposes, is increasing over time. You-Tube is a website whose number of users is increasing day by day. It can be used by patients, relatives and clinicians who want to access up-to-date information. YouTube provides free video access and users can easily upload videos. Before the videos are uploaded, no evaluation is made for their reliability and accuracy. Therefore, it becomes important to analyze the content of videos on YouTube [5].
In recent years, many videos containing information about diseases and treatment options have been uploaded to YouTube. There is not yet a study evaluating videos on thyroid orbitopathy. In this study, we aimed to evaluate the quality and reliability of the content of videos related to thyroid orbitopathy.

Materials and methods
This observational study used publicly available data and is exempt from ethics committee approval. This retrospective and cross-sectional study was carried out on February 24, 2022, by typing the phrases 'Graves eye disease', 'thyroid eye disease', 'thyroid orbitopathy', 'thyroid ophtalmopathy' in the search bar of YouTube (http:// www. youtu be. com). The videos were searched after clearing the search history without user login. The search preference was selected as 'by relevance'. The first 50 videos were analyzed for each keywords. The reason for choosing the first 50 videos is that the first 50 videos are in the first 3 pages, and studies have shown that 95% of the researchers do not look beyond the first three pages [6].
All data were recorded double-blindly by two ophthalmologists (A.Ç.Y. and Ç.E.P.). English videos related to the subject were included in the study. Advertisement, non-English videos, videos shorter than 30 s, silent videos, repetitive videos and videos without the like, dislike or comments function were excluded from the study. Videos posted on platforms other than YouTube (such as Vimeo or TikTok) and then uploaded to YouTube are excluded. All videos meeting the study inclusion criteria were recorded for further evaluation.
Time elapsed since the upload date of the videos included in the study (until February 24, 2022), number of views, likes, dislikes and comments, video duration, daily view rate (views per day), origin, uploader source (patient, private or government) university, clinician, health channel), video type (patient experience, scientific content, informational content), and content type (clinical information, treatment, or both). In addition, the audience interaction value (likes-dislikes)/total views × 100) was calculated. All videos were recorded usability scores, global quality scores (GQS), DISCERN, and Journal of the American Medical Association (JAMA) scores. The DISCERN scoring system is divided into three sections, each section contains 5 questions scored from 1 to 5. Score calculation is classified into 5 categories according to the total score received: excellent (63-75 points), good (51-62 points), moderate (39-50 points), poor (27-38 points), and very poor (16-26 points) [7]. The JAMA scoring system is a scoring system that shows the reliability of online health resources in which four different criteria are evaluated, scored as 0-1. Authors, references and sources, financial support and timeliness are evaluated and scored separately. A score of 4 indicates the highest quality. GQS, on the other hand, is a scoring system used to evaluate the instructional, information flow, ease of use and quality of videos. It consists of 5 questions scored 0-1, with 5 indicating excellent quality. Similarly, it is scored as 0-1 in the usability score. Definition, indication, procedure involved, complication, and prognosis-survival are evaluated. Videos with 4-5 points show that they give reliable and accurate information about the disease.

Statistical analysis
Statistical analysis was performed using IBM SPSS, version 24.0 (SPSS Inc., IL-USA) software. To define the sample, descriptive statistics were expressed as mean ± standard deviation and median (minimum-maximum) values, and qualitative data were expressed as frequency and percentage. Kolmogorov-Smirnov test was used for normality analysis; Independent sample t-test was applied to data with normal distribution, and Mann-Whitney U test was applied to data that did not show normal distribution. Spearman correlation coefficient test was used for correlation analysis. The results were evaluated within the 95% confidence interval and all p values less than 0.05 were considered statistically significant. For all numerical data, the ratings of two independent reviewers were averaged and the kappa coefficient was used to evaluate the agreement between them.

Results
Of the 100 videos, 94 videos met the criteria and were included in the study. Six of the videos were excluded because 4 were duplicate videos and 2 videos were less than 1 min. Table 1 shows the descriptive statistics of the 94 videos.
There was a significant level of agreement between the two commentators evaluating the videos. The agreement power among the commentators was found to be 93.4% (kappa score: 0.934). As shown in Table 2, when we compare them according to the sources, views, video age, likes, dislikes, DISCERN, JAMA, GQS, and usefulness score no statistically significant difference was observed (p > 0.05).
In Table 3, the correlation analysis between the video parameters and the scoring systems is  presented. The four scoring systems showed a statistically significant and strong positive correlation with each other (p < 0.001). We observed that there was a statistically significant positive correlation between viewing rate, likes, and comments. As the duration of the video increased, it was determined that there was a significant increase in the scores in other scores except for JAMA.

Discussion
In recent years, the use of social media has increased with the spread of the internet. The use of Youtube is increasing day by day because it is easily accessible, free and information is updated frequently [8]. In the literature, many studies evaluate the quality and content of YouTube videos, however, to the best of our knowledge no study on thyroid orbitopathy has been found in the literature. Thyroid orbitopathy is a disease that affects the quality of life and patients can achieve satisfactory results after treatment and surgery. Before communicating with the doctor and making a treatment decision, patients may need to watch videos that are informed by another doctor and watch videos of patient experiences. There were informational videos about surgery and Tetratumumab treatment on YouTube about thyroid orbitopathy. Compared to previous YouTube related studies, doctors' informational videos about thyroid orbitopathy constituted a greater majority than patient experiences. In most of the similar studies, we observed that there are many videos uploaded by non-physicians [9][10][11][12]. In this study, videos uploaded by patients constituted 4.3% of all videos. This might be due to the need for more information by doctors, since surgery is more complex than other eye diseases and treatment options are more variable from patient to patient. In order for the audience to reach accurate and better quality information, it is important to encourage health professionals to upload videos to video-sharing platforms.
In a previous study, 52 YouTube videos evaluating the quality of the information in YouTube videos about thyroid cancer were examined, longer duration and higher video power index [number of likes/(number of likes = dislikes) × 100] were associated with higher quality scores [13]. Similarly, in our study, we observed that as the duration of the videos increased, the given score increased. Another study evaluated whether DISCERN and JAMA scores were appropriate tools for evaluating Youtube videos about thyroid cancer [14]. It was reported that these two scores are unable to evaluate important areas such as the quality of patients' experience with thyroid cancer; multidisciplinary management; signs and symptoms of  thyroid cancer, risks of developing thyroid cancer, side effects of treatment, and how final diagnosis is reached. As a result, the authors thought that these two scores were not suitable for evaluating videos. In our study, the 4 scores we evaluated were consistent with each other. As the video quality increased, the scores increased. The same scoring systems were used to evaluate Youtube videos in previous publications [15,16]. In those studies, DISCERN, JAMA, GQS, and usefulness coring systems and the mean values of which were 42.92 ± 18.14, 2.7 ± 0.73, 3.07 ± 1.25 and 2.99 ± 1.44 (15), and as 42.11, 1.82, 2.35 and 2.01 [16], respectively. In our study, we used DISCERN, JAMA, GQS, and usefulness coring systems and the mean values of which were 55.27 ± 16.57, 3.04 ± 0.64, 3.44 ± 0.93 and 3.23 ± 1.1. Those scores were much higher than other studies.
These score values show that the thyroid orbitopathy treatment videos are of high quality.
In another study evaluating the quality of You-Tube videos on hypothyroidism, it was stated that the video quality was poor, lacked information sources, and the content often contained incomplete and misleading statements [17]. However, most of the videos we evaluated about thyroid orbitopathy in our study touched on important issues in informing viewers. It could be useful especially in informing viewers who have hesitations about surgery. We consider that the higher quality of the thyroid orbitopathy videos compared to the videos about other thyroid diseases may be due to the more specificity of the subject and the fewer number of patient experience videos. In addition, we think that the videos that last longer than an hour and are uploaded with meeting records attended by more than one doctor are scientifically important and reliable in informing doctors.
Apart from disease information, YouTube publishes video-based information about almost everything, including live surgery. In YouTube studies on thyroid disease, there were studies in which educational videos about thyroid surgery were evaluated. It has been observed that most of the thyroid surgery videos have been published by surgeons affiliated with academic institutions for educational purposes [18]. In that study, it was stated that it would be very valuable to measure the impact of YouTube surgery videos on surgical education by conducting a questionnaire on students, especially to determine whether this content meets an unmet need in surgical education. In addition, developing a rating system according to the perspectives of both the student and the expert educator, it was thought that it could facilitate the critical analysis of the content. In the videos about thyroid orbitopathy, there were patient information videos rather than scientific content about surgery. These videos did not contain sufficient educational information in terms of surgery. Better educational videos can be found by changing the search word. We think that the evaluation of the educational videos about thyroid orbitopathy surgery could be the subject of another study.
In conclusion, we think that such platforms could be important for viewers to be aware of the disease in the early period, to reduce the need for additional treatment with lifestyle changes, to be aware of the disease in the early period if surgery is necessary, to meet with surgeons, or to be aware of the side effects of the new treatment. Watching videos uploaded from trusted sources is important to avoid misleading information. For this reason, ophthalmologists should guide their patients who want to get information on YouTube to watch videos uploaded by healthcare professionals.
Author contributions Conception and design of the study (HDİ), acquisition of data (ÇEP, AÇY), analysis and interpretation of data (HDİ, AÇY, ÇEP); drafting the article or making critical revisions related to relevant intellectual content of the manuscript (HDİ, AÇY, ÇEP); supervision (HDİ) validation and final approval of the version of the article to be published (HDİ, AÇY, ÇEP).
Funding The authors received no funding for the research, authorship, and/or publication of this article from any government or private institution.
Data availability All data and materials are available on request from the authors. The data that support the findings of this study are available from the corresponding author [Ç.E.P.] upon reasonable request.

Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval This study used only observational and publicly available data, therefore exempt from ethics committee approval.
Publisher's Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.