Using the bootstrapping method to compare research achievements for authors who worked for Departments of Ophthalmology in United States since 2010

Background: The h-index has its popularity in the global scientic community. Despite the h-index being used as an indicator of individual research achievement (IRA), two main disadvantages have not been solved: (1) all coauthors contributing equally to article bylines and (2) the integer nature of the h-index making it dicult to differentiate the IRAs among entities. This article evaluated the most cited authors, institutes, and states in United States in ophthalmology in recent ten years using a proposed hx-index. Methods: Authors who worked for departments of ophthalmology in United States were selected for identifying their IRAs in Pubmed Central(PMC) since 2010. Using the PubMed search engine, we conducted an observational study of citation analyses in aliated research institutes and states of all authors who worked for departments of ophthalmology since 2010. A total of 18,289 published articles from 46,121 authors related to departments of ophthalmology from 50 states were analyzed. The bootstrapping method was applied with an estimated 95% condence interval (CI) to distinguish the differences in IRAs among states and institutes. The x-index and the Kano model were complemental to the h-index for identifying the group IRA characteristics and rankings. A pyramid plot was used to illustrate the importance of the author-weighted scheme(AWS) used for evaluating IRAs in academics. The hx-index combined both advantages of h-/x-index was proposed to assess IRAs for each facility. A signicant difference was identied by observing two bands of estimated 95% CIs that were not overlapped. Furthermore, we drew a choropleth map on Google Maps to visualize the differences of IRA among states. Results: There is a signicant rise over time in the number of publications. The top-ranking states in hx-index based on publications and citations were Massachusetts(42.28),California(39.24), and Massachusetts(42.28). If only the top 100 authors were included for calculating the median hx-index, the top three would be California(6.45), Massachusetts(3.97). and New York(3.07) with no signicant difference found among these three using the bootstrapping method. The institute

The research achievements in the US for authors who worked for department of ophthalmology are rare seen in literature because di culties are to extract data for authors who were only in department of ophthalmology. .
The choropleth maps were plotted in this study. The needs for complementing choropleth maps are those including the Kano diagrams, 95%CI on a dashboard, and pyramid plot being introduced and demonstrated in this study.

Background
A few bibliometric studies have been done in the eld of ophthalmology in recent years [1][2][3]. The h-index has its popularity in the global scienti c community [5]. Despite the h-index being used popularly as an indicator of individual research achievement (IRA) [4], two main disadvantages including (1) all coauthors contributing equally to article bylines [6][7][8] and (2) the integer nature of the h-index making it di cu1t to differentiate the IRAs among entities [9] have been pointed out in the past. The fair authorweighted scheme(AWS) and the modi ed h-index were required accordingly for illustration and application.
In addition, numerous studies have applied the h-index to evaluate IRAs in academic rankings [10][11][12]. However, all those studies [10][11][12] were problematic in assuming h-indices following normal distribution and then using 95% con dence intervals (CI=mean +/-2*SD) to identify the difference between groups. The assumption is not evident and the CI should be calculated by mean +/-2*SE, using standard error instead of standard deviation, unlike the study in ref [12]. The bootstrapping method allows estimation of the sampling distribution for almost any statistic data using random sampling methods with accuracy (as de ned by 95% CI) and is practical in comparing h-indices between groups.
We were thus motivated to use three novel approaches(i.e., AWS, modi ed h-index, and bootstrapping method) to evaluate IRAs for authors, institutes, and the US states. All authors were a liated to departments of ophthalmology in the US.
Three questions are raised in this study: (1) which states and institutes have signi cantly higher IRA, (2) which individual author has substantially higher IRA, and (3)how the AWS plays an important role in assessing IRA.

Data sources
We included only original research articles and eliminated those such as letters, editorials, and comments. Because this study did not involve any examination or treatment on patients or reviews of patient records, it was exempt from review and approval by our research ethics committees.
Three novel approaches used in this study (1) An authorship-weighted scheme (AWS) was proposed in the previous articles [8,14,15] for quantifying author contributions with the de nition of all weights equal to 1.0 for each paper. Accordingly, more importance is given to the rst(primary) and the last (corresponding or supervisory) authors, while it is assumed that the others (the middle authors) have made smaller contributions [1,[16][17][18].
(2) To equip the h-index with higher discrimination power in decimal places [9], the hx-index is de ned in the formula(1), where rh is the ratio (=citations/publications on the x-core of the x-index [ = , where all of the cited papers are denoted by ci] based on the x-core publications at i [19]. hx-index = h + rh/ (1+ rh), (1) For instance, an author whose h-index=2 and x-index with the x-core publications and citations at 3 and 1 has the hx-index at 2.25(=2 + 1/3(1 + 1/3)).
(3) The bootstrapping method [20] was performed to verify the difference in hx-indices among groups. A total of 1,000 median metrics were retrieved from random samples of 100 repetitions of median values for each hx-index in each group. Next, the mean and 95% con dence intervals (CI) were obtained to compare differences in hx-index among departments by inspecting whether two 95% CI bands were not overlapped. In this part of the study, the criterion of sampling authors to be included is only those with the top 100 hx-index in descending order for each group (i.e. regions or institutes).

1) Choropleth maps
The term "choropleth map" was coined by John Kirtland Wright in 1938 [21]. The most famous example of CM used was the results of the 2000 US presidential election [22]. Other examples include showing the disparities in health outcomes across areas related to dengue outbreaks[23=24], disease hotspots [25], and the Global Health Observatory (GHO) maps on major health topics [26].

2) Kano diagrams
The Kano Model is based on a theory of product development and customer satisfaction developed in 1984 by Professor Noriaki Kano [27], who classi ed products or items into three main categories of quality: basic requirement, one-dimensional quality, and excitement(or, say wow) feature. These are diagrammed according to the satisfaction perceived by customers on Axis Y and the effort achieved by providers on Axis X. In this study, we attempt to demonstrate the use of the Kano diagram to characterize the feature of authors using the hx-index in bubble and x-core publications and citations on axes X and Y, respectively.
3) Pyramid plot for comparing IRAs with AWS or non-AWS in comparison A productive author will be illustrated using the pyramid plot to present the feature of IRAs with AWS or non-AWS.

Creating Dashboards on Google Maps
All gures but the pyramid are shown by author-made modules in Excel (Microsoft Corp). We created pages of HTML used for Google Maps. All relevant h-index information on the entities can be linked to dashboards on Google Maps. The absolute advantage coe cient(ACC) was applied to assess the strength for the rst entity(i.e., A1) in comparison with the next two(i.e., A2 and A3. The ACC is determined by the formula(=W/(1+W), where W=V1/V2, and V1=A1/A2, V2=A2/A3), Cutting points is set at 0.70 [28].

Results
The overall impact factor is 10.38(=137345/13236) as shown in Table 1. There was a signi cant rise over time in the number of publications, referring to Table 1 Figure 5, where we can see the author's impact factor increased from 20.5 to 28.3. The hx-indexes identi ed by either AWS or non-AWS are substantially different because the weighted hx-index will be smaller due to both weighted publications and citations less than the non-AWS ones.

Discussion
In this study, there was a signi cant rise in publication number over time in original research published by authors who worked for departments of ophthalmology. Citations were not included in the case.
Although choropleth maps provide a better interpretation of the disparities in health outcomes across countries/areas [31], the bootstrapping method we used in Figures 2 and 3 helps us understand the difference between entities. For instance, by examining whether the 95% CI bands overlaps can be complemental to the choropleth maps in a study.
There are other features in this study. First, the hx-index with decimal places can complement the original h-index to increase the discrimination power [9] for identifying the group IRA characteristics and rankings.
The second feature is the application of the Kano model [27] to interpret the attributes for one's IRA using a visual representation.
Particularly, the dashboard-type display allows us to see details of the bibliometric indices while scanning the QR-code and clicking the entity of interest on Google Maps. Interested readers are invited to read the number of citations for each author in Figure 4 when the author bubble has been clicked.
The reasons for using x-index on two axes in Figures 4 are (1) to closely correlate to h-index [19]; (2) it was newly developed in 2018., Despite being a simple and easy-to-use metric in comparison to others, the disadvantage of using h-index has been addressed [19]. For example, an h-index of 5 means that a scientist has published ve papers that each has at least ve citations [10]. The h-index is little affected by those with a high volume of low-impact papers or only a few high-impact articles [10]. Excessive citations or publications were totally neglected [31,32]. The x-index was thus used to complement h-index to clearly interpret one's feature toward the citation-oriented or the productively-oriented. The hx-index shown on the Kano diagram (Figure 4) is novel and innovative in showing the author's IRAs on a dashboard.
The third feature is the dashboard of the Kano diagram combined with the hx-index on Google Maps, which is harder to create using the traditional method [33].
The fourth feature is the rankings based on the 95% CIs comparison, which provides us a deeper insight into the knowledge and information we need. The bootstrapping procedure and the dashboard creation are detailed in an MP4 video in Additional File 2.
Finally, the ACC was proposed to present the dominant role in members, which is unique to inspect whether a role is overwhelmingly played or concentrated in a eld or discipline. The way using Gini coe cient for evaluating the research domain of an author [34,35] is challengeable and comparable in future studies.

Limitations and suggestions
Although our analysis provides quanti ed ndings, several potential limitations were noticed and may encourage further research. First, this study only focused on authors in China, Hong Kong, or Taiwan. The results cannot be generalized to other areas and countries.
Second, certain biases might have occurred during citation extraction because of the increasing number of citations with the elapsed date. Namely, the IRA might differ if the time periods and the citation sources of the data are disparate.
Third, many authors work for multiple departments, such as in university, medical school, and a liated hospital. Only the unit listed rst was included in this study. The results of hx-index in comparison in Figures 3 might be biased if we were to apply the author-weighted scheme [8,14,15].
Fourth, although our proposed bootstrapping method was to display the 95% CIs for comparison of IRAs, many statistical software does not provide such an approach to users. The Microsoft Excel version of the bootstrapping method is provided on request to the corresponding author of this study.
Finally, despite the hx-index is considered not normally distributed in nature, the bootstrapping method we used in this study is merely selecting the mean from random samples of 100 repetitions of median values for each hx-index in each group. The 95% CI would be different if the median was retrieved from 100 repetitions of median values for each group. Future studies are recommended to compare the difference between the methods of using the mean and the median in this bootstrapping method.

Conclusions
The hx-index and the Kano models can complement the h-index for identifying the various IRA characteristics among groups.
The bootstrapping method allows estimation of the sampling distribution for almost any statistic data using random sampling methods with accuracy (as de ned by 95% CI). We highly recommend these three features used in this study, AWS, hx-index, and bootstrapping method, to be applied in future studies.

Availability of data and materials
All data used in this study is available in Additional File les.

Competing interests
The authors declare that they have no competing interests.

Funding
There are no sources of funding to be declared.

Authors' Contributions
YT developed the study concept and design. TWC, WC and SC analyzed and interpreted the data. SBS monitored the process of this study and helped in responding to the reviewers' advice and comments. TWC drafted the manuscript, and all authors provided critical revisions for important intellectual content. The study was supervised by SC. All authors read and approved the nal manuscript.  By selecting only the top 100 authors for calculating the median hx-index, the top three were California(6.45), Massachusetts(3.97). and New York(3.07), no signi cant difference was found among these three using the bootstrapping method Page 12/14  It is worth noting that the article [29] cited by 353 times since 2014 was authored by Weinreb and Medeiros from California) highlighted