Assessing the Relationship Between Traditional Citation-based Metrics and New Metrics (Altmetrics / Dimensions / Number of Tweets) in COVID-19 & Eye Research

Background: Traditional peer-reviewed publications are the most common forums for disseminating scientic information. With the rising popularity of social media, altmetric studies have gained importance in the assessment of impact values of scientic research. In the framework of COVID-19 and eye research, this study aimed to compare activity on new metrics with traditional bibliometric systems using highly cited publications. Methods: Using the term "COVID-19 & Eye," data from articles in last one year (March 2020 to April 2021) were collected from the Web of Science Core Collection (WoS) database. Author name, publication year, major topic, study type, journal name, journal impact factor, H-index, total citation number (TCN), Altmetric Attention Score (AAS), number of tweets (NT) and Dimensions score were studied in the top 100 cited articles list (T100 list). The bibliometric data was visualized using the VOSviewer software. For analysis, the Spearman correlation test and descriptive statistics were used. Results: In our WoS search, we found 720 articles with the term "COVID-19 & Eye". The mean TCN and AAS in T100 list were 22.51±62.41 and 299.11 ±2406.74, respectively. Total citation number showed moderate positive correlation with AAS and strong positive correlation with NT and Dimensions score (r=0.427, r=0.806, r=0.877, respectively, p=0.001 for all). Journal impact factor and H-index showed substantial signicant positive correlations with AAS and NT. Conclusion: AAS, particularly Twitter and Dimensions score have a substantial positive correlation with traditional bibliometric data of COVID-19 and eye research.


Introduction
Since its discovery in Wuhan, China, in December 2019, the COVID-19 has spread swiftly throughout the world, prompting the World Health Organization (WHO) to declare it a pandemic on March 11, 2020 [1]. COVID-19 is transmitted by close contact, inhalation, and the most common early symptoms of COVID-19 have been documented to be dry cough, dyspnea, fever, and weariness, according to current information [2]. Furthermore, unlike the symptoms of a typical virus infection, COVID-19 patients have been found to have conjunctivitis [3,4]. In addition to that the ocular surface has been suggested as a possible pathway for COVID-19 transfer [5]. Due to these circumstances, COVID-19 and eye research has received more attention. With the emergence of the pandemic, there has been an unusually high number of scienti c studies published on COVID-19 and eye research. As a result, assessing an article's impact is very crucial, especially for readers who do not have the time to read all relevant publications. The number of citations has been commonly used to measure the in uence of works since the development of citation indexing in the 1960s. A greater impact factor equates to more citations per piece. These measurements are regarded as quality indicators, and they can have a signi cant impact on funding decisions for individual researchers and departments [6]. Map knowledge domain is a method for charting and displaying bibliometric data, as well as performing co-occurrence analysis and hotspot identi cation. It informs scholars on research topics, new trends, and the development of new study areas [7].
Bibliometric analyses do not accurately re ect an article's impact outside of academia [8]. An alternative technique is required for rapidly assessing the impact of articles, as well as for quickly selecting and disseminating useful information during the pandemic. Mentions of articles on various social media platforms are included in altmetrics, which were called with the goal of being alternative metrics of publication impact [9]. Alternative metrics, rather than replacing bibliometrics and peer review as indicators of research quality, are meant to be used in conjunction obtained in the "Full record and cited references" forms. All of the nations on the T100 list were subjected to bibliometric coupling analysis. In the co-occurrence keyword analysis, a threshold value of 2 was employed. Maps were used to display the results of country coupling and keyword co-occurrence analyzes.
AAS and Dimensions badge score were provided by Altmetric (https://www.altmetric.com/products/freetools/bookmarklet/) and Dimensions (https://app.dimensions.ai/discover/publication) web sites, respectively (Available till April 8, 2021). AAS were calculated automatically on the website using a system based on a weighted average of all of each article's attention. On the altmetric donut, each color represents a different source of attention ( Figure 1A). Dimension badge of the highly cited article was shown in another gure ( Figure 1B).

2.3.a. Data Extraction from Articles
The authors independently read the abstract and full text of each publication found, extracting the following information: overall AAS, Dimension badge score, NTs, publishing journal, journal impact factor (IF), month and year of publication, the language, origin nation, document type (original articles (clinical observational study, basic study, randomized controlled trial, systematic review/meta-analysis, reviews, case reports, letters, news, editorials, etc.), main topics (virus, epidemiology, immunology, transmission, prevention, clinical manifestations, treatment, public health responses, vaccines, etc). To avoid uctuations in article scores, the Altmetric Explorer search was run on a certain day (8 April, 2021).

Statistical analysis
The majority of the statistical approaches utilized in this investigation were descriptive. In the tables, all of the data was expressed as a percentage, a number, a mean, and a standard deviation (SD). Continuous variables were described using the median and interquartile range (IQRs), whereas categorical variables were de ned using percentages. The Mann-Whitney U and Kruskal Wallis tests were used to compare intragroup and intergroup differences. The relationships between total citations, Altmetric ratings, and dimensional badge scores were also investigated. The linear association between times mentioned and Altmetric scores was determined using Spearman correlation coe cients. Univariate linear regression analysis was used to calculate beta coe cients. Analyses were recorded using Microsoft Excel. Account age was the independent variable in our Twitter analysis, while determined correlation coe cients (r) were the dependent variable. The correlation coe cients were evaluated as follows: less than 0.4 indicates a weak association; 0.4-0.6 indicates a moderate relationship; 0.61-0.8 indicates a strong relationship; and 0.81-1.00 indicates an extremely strong relationship [20]. Univariate linear regression analysis was used to calculate beta coe cients. SPSS for Windows version 23 was used to conduct all of the statistical analyses. Statistical signi cance was de ned as p<0.05.

Results
Using the keyword "COVID-19" in our WoS search, we retrieved 99.323 articles between March 2020 and April 2021. When we re ned the results by using terms " COVID-19 & Eye," the number of articles in the WoS database dropped to 720. The top 100 articles in this result that obtained the most citations according to the WoS database were recorded. For each article, the rst author, publication year, TCN, ACpY, AAS, Dimensions Score, NTs were displayed in the T100 list (Table 1). Although no language was speci ed, it was found that all articles on the T100 list were published in English. In the T100 list, the average rates with each variable's min-max values were recorded as 22,51±62,41 (3-513), 299,11±2406,74 (0-24023), 312,67±2825,01 (0-28256), 36,23±113,65 (0-1012) for TCN, AAS, NT and Dimensions, respectively. Due to the outliers in each variable, the median values were also calculated and stated below in each subsection.

TCN and AAS analysis
In the T100 list, the median values for TCN and AAS were 6 and 4.5, respectively. Chu DK et al's [21] article, "Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis," has the highest TCN of 513 and was published in June 2020. As can be observed, this article omitted COVID-19's ophthalmological characteristics. The second publication, entitled as 'Characteristics of Ocular Findings of Patients with Coronavirus Disease 2019  in Hubei Province, China,' was published by Wu P et al. [22]. The main focus of this article was on the ophthalmological characteristics of COVID-19 with a TCN of 305. At least three citations were received by every article in the T100 list.
The articles by Chu DK et al and Wu P et al get the highest two AAS with the T100 list values of 24023 and 1556, respectively. There were, however, 20 publications in the T100 list that did not yet have AAS.

Twitter analysis
It was determined that 77 of the T100 list's articles had been shared on Twitter. The median value of NT was 4. Chu DK et al. [21]'s paper published in the journal of Lancet had the highest NTs (with a number of 28256), which had also the greatest TCN. The paper with the second highest NTs (a number of 922) was also published in journal of Lancet by MacIntyre CR et al. [23] and was entitled as 'Physical distancing, face masks, and eye protection for prevention of COVID-19.' Both of these pieces were related with eye protection. Wu P et al. [22] published the third article with the highest NTs (a number of 715) in the journal of JAMA Ophthalmology, which also had the second highest TCN. This paper focuses on the peculiarities of ocular ndings in COVID-19 patients in Hubei Province, China.

Dimensions Badge Score
The median Dimensions score was 10.5, and the article with the most was 1012 and it was the one that currently leads the rst place in T100 list according to TCN values. It was published by Chu DK et et al. and entitled as "Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis."

Journal perspective
The T100 articles were published in 44 journals in total, with 19 of the T100 articles were published in the Indian Journal of Ophthalmology. It was noteworthy that the journal with the highest median TCN (number of 267.50), AAS (number of 12731), NTs (number of 14589), and Dimensions score (number of 525) was journal of Lancet. On the other hand, journal of Ocular Surface had the highest median TCN (number of 53.50) and Dimensions score (number of 85.5), while journal of JAMA Ophthalmology had the highest AAS (number of 352) and NTs (number of 197) among ophthalmology-speci c journals (Table 2).

Study types and level of evidence
Thirty-six articles in the T100 list were original articles according to SIGN. When all articles were classi ed according to their level of evidence, there was no statistically signi cant (p=0.076) difference in the median AAS of the four groups. In the same way, there was no signi cant difference in median TCN between the four groups (p=0.275). The median Dimensions score of the four groups did not differ statistically signi cantly (p=0.556). In terms of twitter analysis, there was no statistically signi cant difference in median NTs between the four groups (p=0.14). According to these ndings, the level of evidence had no signi cant impact on the AAS, TCN, Dimensions score, and NTs of the articles (Table 3).

Research topics
When we evaluated the T100 list by main topic, the majority of articles were related to clinical features of eye diseases (n=15), precautions for prevention and transmission (n=14) and teleophthalmology (n=12), (Table 4). However, the main topic with the highest median AAS, TCN, Dimensions score, and NTs were related to precautions for prevention and transmission.

Publication Year and Publication Months
In 2020, 93 publications were published, with July having the highest number of researches (number of 26). The maximum amount of TCN was found in May (number of 30), but the highest amount of AAS was found in June (number of 32). According to TCN, May was also the month with the highest Dimensions score (number of 45). Parallel to AAS, the greatest NTs were cited in June, according to Twitter data (number of 35) (

Correlation analysis
The results of the correlation analysis were displayed in Table 6 and Figure  Using a co-occurrence analysis of high-frequency terms, the hotspots of COVID-19 and eye research were determined. Two was chosen as the minimum number of keyword co-occurrences requirement. 52 of the 205 returned keywords connected to COVID-19 and eye research satis ed the criteria. The network was used to group related keywords, and the colors red, green, blue, yellow, and purple were utilized to symbolize the ve major clusters ( Figure 3B). The keywords 'covid-19' ,'sars-cov-2' and 'conjunctivitis' were the most popular, respectively.

Discussion
The novel coronavirus SARS-CoV-2 has sparked an unprecedented worldwide response in research, medicine, public health, and technology since its discovery in December 2019. Given the extremely contagious nature of the infection, as well as the scarcity of knowledge and present lack of viable treatment options, the necessity for timely exchange and distribution of information has been critical. There have been numerous publications published during the pandemic. Citation counts have traditionally been used to identify high-quality and highly valued articles; but, in a fast-pacing pandemic like the COVID-19 pandemic, this strategy is not always practicable. As a result, interest in the COVID-19 issue will continue to grow both in social media and among academics. Traditional citation-based metrics are statistics of "scholarly impact" on an academic-based audience, but altmetrics may be considered possible measures of a "disseminative impact" on the broader populace. The association between traditional bibliometric analysis (citation count, journal H-index, and IF) and altmetric analysis in the area of COVID-19 and eye research was explored in this work. The e cacy of combining AAS with TCN in evaluating the top 100 "trending" COVID-19 and eye research publications was investigated in this study. To the best of our knowledge, this is the rst study to compare traditional bibliometrics (TCN, journal H-index, and IF) with new metrics like altmetric and dimensions analysis while evaluating publications in the eld of COVID-19 and eye research. Previous research has looked into the relationship between citations and altmetrics in different topics, and some has found that while citations and altmetrics are positively correlated to some extent, their correlations are quite weak [24,25]. According to our ndings, and due to the hot topic, AAS was modestly correlated with TCN (r2=0.427), however NTs was strongly correlated with TCN (r=0.806). This showed that both peers in the scienti c community and members of the general public are on the same page when it comes to knowledge dissemination.
Traditional bibliometric methods are effective for determining the impact value of scienti c articles, but they have a number of drawbacks. Before the published article's citation metrics can be measured, a certain length of time must elapse. As a result, bibliometric analysis cannot be used to assess the in uence of a recently published paper in its early years. It makes it di cult for researchers to identify compelling papers, keep track of new research topics, and come up with new research topics. Furthermore, self-citation or citation of related authors' papers may unjustly in uence citation quantities. As a result, a new technique for quickly analyzing the quality and impact value of the articles is required. Altmetric analysis is a new way of assessing how much attention items get on social media from huge groups of people [26]. Altmetric analysis provides quick insight into an article's worth and in uence [27].
We were granted access to the Altmetric database, which houses nearly 31 million research articles from over 36.000 journals and tracks real-time mentions in public policy documents, blogs, mainstream media, online reference managers like Mendeley, research highlights, post-publication peer review platforms like Open Syllabus, YouTube, and social media networks like Facebook. Altmetric.com calculates AAS to determine the overall level of online effect resulting from a certain research output. The AAS is a weighted number that re ects the expected relative degrees of in uence of sources on potential readers based on total mentions of a story across multiple online media (e.g., default weights of: 8 for news outlets, 5 for science blogs, 3 forWikipedia or policy documents, 1 for Twitter, and 0.25 for Facebook).
New developments, fresh ideas, and active study elds are prioritized on social media platforms, particularly on Twitter. We also looked into how COVID-19-related ocular information was disseminated through the global scope of social media via Twitter. In terms of scienti c knowledge reliability, we found positive connections between the number of tweets and both traditional and altmetric measures. To summarize; AAS had a week positive connection with H-index (r= 0.215) despite having a moderate positive correlation with journal IF (r=0.458) and TCN (r= 0.427).
However, AAS (r=0.998), Dimensions score (r=0.878), TCN (r=0.806), IF (r=0.703), and H-index (r=0.645) were all strongly correlated with NTs. According to these results, items that have been quoted frequently throughout time and are still relevant are more useful on social media and Twitter. While AAS and IF were moderately connected (r=0.458), the NTs and had a strong positive connection (r=0.703). This problem could be explained by the fact that the journal IF is affected by current citations. On Twitter, articles that are still being disputed get a lot more attention. Another explanation is that journals with long-standing Twitter accounts have a higher number of followers. As the number of people who follow you on social media grows, so does your awareness of the journals and your ability to read their articles. It has been discovered that announcing newly published publications on Twitter boosts the number of citations in the years that follow [28,29]. Furthermore, medical journals with Twitter accounts have a higher IF, and there is a positive relationship between journal IF and the number of Twitter followers [30][31][32].
In terms of the NTs vs. TCN debate, our ndings were contradictory with some of the existing literature. The relationship between NTs and TCN was studied by Haustein et al. [33] to be relatively modest. According to Thelwall [34] who has completed over a hundred studies on social media analytics, there was a negative association between NTs and TCN due to the long time required for citation, as opposed to the instantaneous sharing on Twitter. This could be explained by the subject of study. We looked into a topic that was threatening the world, and not only the scienti c community, but also the general population, were interested. being debated and are current. However, it is crucial to note that all of these studies have concluded that AAS is most successful when used in conjunction with established bibliometric measures, and that AAS should not be used simply to determine the quality of an article.
Although AAS can be used to determine how broadly an item has been disseminated, the correlation between TCN and NTs was lower in our study, suggesting that AAS is less bene cial than NTs. This could be due to a variety of factors. There are various obstacles to the usability of AAS, including the heterogeneity of the platforms used by Altmetric to generate the score, the dynamic nature of AAS, and the fact that many of the platforms we looked at (e.g., Twitter, Facebook, etc.) were not geared toward academics. AAS uses Twitter as one of its sources. The substantial relationships between citation count and Mendeley citations, as well as citation count and Dimensions citations, were not surprisingly given the platforms that Altmetric uses to construct its weighted score and those that the scienti c community considers interesting. Mendeley and Dimensions are two well-known websites that scienti c academics use to nd, organize, and cite previously published research. In comparison to other platforms (such as Twitter or Facebook), which appeal to a broader demography of the general public, these sites attract a more professional group of users who often work in the biomedical scienti c sector. Dimensions, a new online scholarly platform for articles, grants, clinical trials, and patents, was introduced in January 2018 by Digital Science, with free partial online access. According to a recent publication, Dimensions' academic database component appears to be a viable alternative to WoS for general citation analysis and citation data in support of certain sorts of research evaluations [38]. We discovered a strong link between TCN and Dimensions score (r=0.877) in our research. In addition, the AAS (r=0.892) revealed a strong association with the Dimensions score, too.
Our research also discovered that during the early stages of the pandemic, the T100 altmetric publications on COVID-19 were published more regularly between March and July 2020. A total of 66 articles were published during the early stages of the pandemic, with the remainder appearing subsequently (between August and December 2020). This could be due to the public's increased interest in and desire for knowledge about the novel infectious disease during the early stages of the pandemic.
COVID-19 has sparked a rapid response from the academic and scienti c communities in terms of gathering evidence to assist and inform. To establish which issues, attract much more attention in the research and the general trend, it is necessary to categorize the T100 list according to the key topic groups. The bulk of articles (n=15) were on "clinical aspects of COVID-Eye disease," while the main issue with the highest median AAS, median TCN, median Dimensions score, and NTs was about "precautions for prevention and transmission." This was simply interpreted to mean that the community was far more interested in eye-based preventative strategies. Despite these drawbacks, social media is a particularly effective method for disseminating intellectual works widely [52]. As a result, our ranking of the top 100 altmetric COVID-19 and eye articles offers insight into the spread of scienti c knowledge via online media over a certain time period following the appearance of a novel infectious disease.

Limitations
There are some limitations in our research. To begin, we only used data from Altmetric.com to evaluate alternative measures. We chose it due to its widespread use and incorporation of a diverse set of web channels. If we had utilized alternative altmetric tools that aggregate and offer article-level metrics, such as PlumX, ImpactStory, and This research only looks at the link between tweets and article citations.

Conclusion
The demand for knowledge about COVID-19 and eye research has resulted in a signi cant increase in the number of papers. This study revealed patterns in the use of social media and publications for rapid knowledge distribution during a worldwide pandemic of infectious disease, using both new metrics and traditional metric. As the globe grapples with this extraordinary public health crisis, our research has examined shifting global tendencies away from traditional methods of spreading knowledge (e.g., through publications) and toward more modern methods, particularly among the Ophthalmology community. Twitter and other social media technologies can be useful for disseminating information and informing audiences in real time and in a participatory way, something that isn't always possible with more traditional techniques (ie, scienti c publications).
Our research provides a complete list and analysis of the top 100 altmetric papers on COVID-19 and eye research; so, providing crucial information about scienti c knowledge transmission during a pandemic caused by this new virus.
Tweets should also be viewed as a metric for social in uence and knowledge translation, as well as a way to gauge public interest in a particular topic, whereas citations should be viewed as a metric for scholarly impact. Because of their potential to re ect evidence of broad public range, tweet counts are one of the most intriguing altmetric variables.
In summary, employing altmetric analysis and paying attention to Twitter analysis in addition to traditional bibliometric analyses in the evaluation of papers provides scholars a plethora of knowledge regarding hot issues in COVID-19 and eye research.

Declarations 6. Financial Disclosure
Each author declared that there is no nancial support or funding in this study.

Declaration of Interest
The authors herein declare that they have no con ict of interest and no proprietary or commercial interest in any materials discussed in this article. The authors alone are responsible for the content and writing of the paper.

Data availability statement
Data of the articles were received from Web of Science (WoS) Core Collection database, and altmetric scores were received from ''Altmetric.com''.
B. Altmetric donut and dimension badge of the highly cited article was shown as a sample