What Are the Variables Associated With Altmetric Scores? – An Overview of Methodological Reviews

Background: Currently, social media has been used to disseminate contents of scientic articles. In order to measure this type of impact a new tool named Altmetric was created. Altmetric aims to quantify the impact of each article through the media online. This overview of methodological reviews aims to describe the associations between the publishing journal and the publishing articles variables with Altmetric scores. Methods: Search strategies on MEDLINE, EMBASE, CINAHL, CENTRAL and Cochrane Library. We extracted data related to the publishing trial and the publishing journal associated with Altmetric scores. Results: A total of 11 studies were considered eligible. These studies summarized a total of 565,352 articles. The variables citation counts, journal impact factor, access counts, papers published as open access and press release generated by the publishing journal were associated with Altmetric scores. The magnitudes of these correlations ranged from weak to moderate. Conclusion: Citation counts and journal impact factor are the most common associators of high Altmetric scores. Other variables such as access counts, papers published in open access journals and the use of press releases are also likely to inuence online media attention.


Background
The most common way to assess the impact of an article is based upon the number of citations [1]. Moreover, the mean number of citations for all the articles published in the journal in the preceding two years is named journal's impact factor [1]. However, the number of citations and the journal's impact factor do not precisely re ect if the message of the paper is reaching a wider audience [2]. Currently, social media has been used to disseminate contents of scienti c articles [3,4]. However until recently, the impact of scienti c articles on social media was not quanti ed. In order to measure this type of impact a new score (named as Altmetric) was created [3,4].
Altmetric measures the impact of each article through the attention attracted online [3]. Moreover, the Altmetric score reveals the instantaneous attention attracted online of the articles in news outlets, commentaries on blogs, number of tweets and mentions on social media. There are two types of Altmetric scores. The Altmetric mentioned score includes data sources involving social media (e.g. Facebook, Twitter); newspapers, encyclopaedias (e.g. Wikipedia); online platforms (e.g. Faculty1000 and Publication Peer-Reviews); videos on YouTube; sites on questions and answers (e.g. Q&A stack over ow) and policy documents on PDF documents available over the internet. The Altmetric reader score includes data sources involving reference managers available online (e.g. Mendeley, CiteULike and Connotea).
Research about Altmetric is increasing and becoming more popular in the recent years [5]. However, most articles about Altmetric published to date are only introductory tutorials or editorials [1,3,4,6,7]. Patthi et al [2] published a systematic review in the eld of dentistry that aimed to analyze the correlations between journal citations and Altmetric scores. This review concluded that journal citations and Altmetric scores are positively correlatated (with pearson's r ranging from 0.30 to 0.61).
Recent articles from several research elds [8][9][10][11][12], showed that the number of citations of articles and Altmetric are positively associated. Finch et al. [10] showed that the number of tweets (i.e. an Altmetric component) could predict citations within the rst 3 days of article publication. Araujo et al. [13] found that number of citations and journal's impact factor were positively associated with Altmetric [13]. These authors also found that the number of years since publication and having a descriptive title (title describes the aim, but does not reveal the main conclusions) were negatively associated with Altmetric [13]. Therefore, it is assumed that the publishing journal and the publishing articles variables, for example citation counts, journal impact factor, access counts (i.e. considered as the sum of HTML views and PDF downloads), papers published as open access, time since publication and press release generated by the publishing journal are likely to be associated with the Altmetric [13]. This overview of methodological reviews aims to summarize all available evidence on the associations between the publishing journal and the publishing articles variables with Altmetric scores.

Research Question
What are the publishing journal and the publishing articles variables associated with Altmetric scores?

Search strategy for identi cation of studies
Systematic searches were conducted on MEDLINE, EMBASE, CINAHL, CENTRAL and Cochrane Library, including publications since the inception of these databases until July 3, 2018 without language restrictions. We used two search terms (Altmetric OR Altmetrics) in all databases.

Inclusion criteria
We included any original research article that measured any type of association between the publishing journal and the publishing article with Altmetric scores, such as: citation counts (i.e. number of citations), journal impact factor, access counts (i.e. considered as the sum of HTML views and PDF downloads), papers published as open access, time since publication and press release generated by the publishing journal.

Exclusion Criteria
Letters to the editor, editorials and conference abstracts were excluded.

Data collection
Two review authors (AA and AV) independently screened all studies for eligibility and data extraction. All discrepancies identi ed during the stages and throughout the review were resolved via discussion, or by arbitration provided by another investigator (DN). The study selection process included: (1) screening the titles and abstracts and (2) screening of full text articles.

Data extraction
Two review authors independently extracted the following data: (1) authors; (2) year of publication; (3) research eld; (4) sample size of articles analised; (5) study designs of the included articles; (6) aims of the article; (7) study results; (8) study conclusions. Variables about the publishing journal of articles included: (9) journal impact factor; (10) access counts (i.e. considered as the sum of HTML views and PDF downloads); (11) papers published as open access and (12) press release generated by the publishing journal. Variables about the publishing articles included: (13) citation counts (i.e. number of citations) and (14) time since publication. We also collected data related to (15) the Altmetric mentioned score and (16) the Altmetric reader score. We contacted authors by email to request the any information that was not reported in the original manuscripts.

Ethics and registration
No ethical approval was required for this type of study. As this review has no clinical outcomes, no registration was needed.

Data analysis
In this study was not possible to perform a meta-analysis of the studies due to the large heterogeneity.
For this reason, our results were reported descriptively.

Search results
The initial electronic database search yielded a total of 334 potentially eligible studies. After screening by title and abstract and removal of duplicates, we considered 22 potentially eligible studies for inclusion and retrieved full-text articles. Eleven published studies [12][13][14][15][16][17][18][19][20][21][22] met the inclusion criteria and were included in this review. The study ow diagram of the eligibility assessment is presented in FIGURE 1.

Characteristics of included studies
A total of 11 studies were considered eligible. These studies summarized a total of 565,352 articles. Eligible studies were published between 2014 and 2018. The study design of the included articles were mixed research designs [12,[14][15][16][17][18][19][20][21][22] and randomized controlled trials in low back pain [13]. The research elds from these articles include Medical Education [14], Physiotherapy [13], Emergency Medicine [21], Biomedical Area [15], Multidisciplinary [16], Dentistry [22], Social Sciences & Humanities, Engineering & Technology, Medical & Natural Sciences [17], Oncology [18], Solid Organ Transplantation [19], Ecology and Conservation [20] and Radiology [12]. The main objective of the included studies was assessing the association with Altmetric scores and the variables: citation counts (i.e. number of citations), journal impact factor, access counts (i.e. considered as the sum of HTML views and PDF downloads), papers published as open access, time since publication and/or press release generated by the publishing journal. The summary of the methods, data analysis, results and conclusions are presented in the TABLE 1.

Statistical analysis and associations of included studies
Different types of analyses were performed in the included studies: correlation analysis [12,14,16,19,21,22], multivariable regression analysis [13,18], boosted regression trees analysis [20], principal component analysis and factor analysis [15,17]. The main results of the included studies demonstrated that the variables citation counts (i.e. number of citations), journal impact factor, access counts (i.e. considered as the sum of HTML views and PDF downloads), papers published as open access, time since publication and press release generated by the publishing journal were associated with Altmetric scores. The magnitude of these associations ranged from weak to moderate (TABLE 2).

Discussion
The objective of this overview of methodological reviews was to summarize all available evidence on the associations between the publishing journal and the publishing articles variables with Altmetric scores. We found that citation counts (i.e. number of citations), journal impact factor, access counts (i.e. considered as the sum of HTML views and PDF downloads), papers published as open access, time since publication and press release generated by the publishing journal were associated with Altmetric scores. The magnitude of these associations ranged from weak to moderate. Besides that, we observed that citation counts or journal impact factor were associated with Altmetric scores in all included studies [12][13][14][15][16][17][18][19][20][21][22].
There is a previous systematic review about the correlation between citations counts and Altmetric in medical research [2]. Besides that, there are articles that have measured associations between the citation counts and Altmetric scores [12,14]. According with the systematic review [2] and these articles [12,14] there is a positive correlation between citation counts and Altmetric scores. Similarly, our overview found positive associations (ranged from weak to moderate) between citation counts and Altmetric scores [12][13][14][15][16][17][18][19][20][21][22]. These results are similar to the ones related to journal's impact factor [13]. This is no unexpected, since the journal's impact factor is based on citations counts of scienti c articles [1].
We also found that most included articles, with the exception of Araujo et al [13] and Knight [19], did not analyze the Altmetric reader score. In this way, our ndings are largely are based on Altmetric mentioned score. We strongly recommend that further investigations on Altmetric reader score should be conducted.
There are no studies that have identi ed speci c characteristics of articles, for example, analysis on studies that published popular/hot topics (e.g. studies on zika virus, miraculous diets, cancer prevention, early life in earth, religious evidence). Moreover, there are no analysis of studies comparing if the direction of the results (i.e. positive versus negative conclusions) in uences Altmetric scores. These characteristics are likely to increase the number of access and sharing these articles on social media [13]. We recommend that future studies should identify if these characteristics are associated with Altmetric scores.
The major strength of this study is about the inclusion of studies from all elds of the research (n = 565,352 articles analised). On the other hand, a possible limitation of this study is about the large heterogeneity of the included studies. Because of this, the data was analised only descriptively. The Altmetric is still a new topic and there are just few avaliable studies.

Conclusion
Citation counts, journal impact factor, access counts (i.e. considered as the sum of HTML views and PDF downloads), papers published as open access, time since publication and press release generated by the publishing journal were associated with Altmetric scores. However, more research is needed to con rm the results obtained in this overview of methodological reviews.

List Of Abbreviations
Not applicable.

Declarations
Ethical Approval and Consent to participate Not applicable.

Consent for publication
Not applicable.

Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.

Funding
Not applicable.
Authors' contributions Altmetric reader score are associated to citation counts.
(FA) and regression models were performed. 5 Costas et al., 2015 [16] (Multidisciplinary) To determine the relationship between citation counts, journal impact factor and Altmetric score.  Study ow diagram of the eligibility assessment.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download.