Journal analysis
When it comes to journal publications, it is might not give a clear picture to evaluate only the impact factor of journals or the number of citations of papers. Therefore, we simultaneously used the total publications, total citation count, and impact factor to make the results more comprehensive [8]. The total publications, total citation count, average citations, 5-year impact factor of the top 10 journals in health policy and services research are summarized in Table 1. Health Affairs published the largest number of health policy and services related publications (4319, or 25.46% of the total number of publications), followed by Medical Care (3305, or 19.48% of the total number of publications) and Health Services Research (2198, or 12.96% of the total number of publications). Health Affairs was the most cited journal, and Milbank Quarterly had the most average citations in a single publication and had the highest 5-year impact factor in the top 10 journals.
Table 1
The top 10 journals in health policy and services research
Journal Title
|
TP(%)
|
TC
|
TC/TP
|
IF(5 year)
|
Health Affairs
|
4319 (25.46)
|
136214
|
31.54
|
5.53
|
Medical Care
|
3305 (19.48)
|
135036
|
40.86
|
3.67
|
Health Services Research
|
2198 (12.96)
|
62326
|
28.36
|
3.25
|
Value in Health
|
1811 (10.67)
|
42432
|
23.43
|
5.64
|
Health Policy and Planning
|
1414 (8.33)
|
28901
|
20.44
|
3.28
|
Implementation Science
|
1335 (7.87)
|
31846
|
23.85
|
5.36
|
BMJ Quality & Safety
|
796 (4.69)
|
14060
|
17.66
|
6.46
|
Administration and Policy in Mental Health and Mental Health Services Research
|
771 (4.54)
|
13655
|
17.71
|
2.98
|
Pharmacoeconomics
|
678 (4.00)
|
12776
|
18.84
|
3.50
|
Milbank Quarterly
|
338 (1.99)
|
16241
|
48.05
|
7.19
|
TP, total publications; TC, total citation count; TC/TP, average citations; IF, 5-year impact factor |
Characteristics of publication outputs
Table 2 summarizes the temporal evolution of major scientific productivity descriptors between 1999 and 2018. The annual total publications, number of authors, cited references, page count, total citation count in the area of health policy and services research slightly fluctuated but considerably increased. The average number of authors, references, pages, and citations per article slightly fluctuated during this period with an overall average of 4.69, 34.31 and 11.70, respectively. Only 5820 (34.31%) health policy and services related articles were published between 1999 and 2008, and the number of these publications nearly doubled to 11,145 (65.69%) between 2009 and 2018. The average number of references per article varied over the years, from 26.65 in 1999 to 38.43 in 2018. The significant increase in the number of publications and references per article between 1999 and 2018 indicated a stable growth and communication in the health policy and services research field during the past two decades. To some extent, the number of citations can mostly reflect the influence of an article on its research field. The number of citations per article decreased from 50.18 in 1999 to 1.45 in 2018. Despite this, the influence of these research outputs remained unknown considering the time of publications.
Table 2
Characteristics of publication outputs by year
Years
|
TP
|
Au
|
Au/P
|
NR
|
NR/P
|
PG
|
PG/P
|
TC
|
TC/P
|
1999
|
450
|
1595
|
3.54
|
11993
|
26.65
|
5559
|
12.35
|
22581
|
50.18
|
2000
|
433
|
1501
|
3.47
|
11401
|
26.33
|
5294
|
12.23
|
20674
|
47.75
|
2001
|
430
|
1607
|
3.74
|
12669
|
29.46
|
5458
|
12.69
|
25807
|
60.02
|
2002
|
514
|
1812
|
3.53
|
15311
|
29.79
|
6451
|
12.55
|
29037
|
56.49
|
2003
|
546
|
1953
|
3.58
|
16087
|
29.46
|
6705
|
12.28
|
26655
|
48.82
|
2004
|
596
|
2169
|
3.64
|
18131
|
30.42
|
7023
|
11.78
|
27299
|
45.80
|
2005
|
627
|
2465
|
3.93
|
18476
|
29.47
|
7001
|
11.17
|
35624
|
56.82
|
2006
|
677
|
2621
|
3.87
|
20558
|
30.37
|
7536
|
11.13
|
25947
|
38.33
|
2007
|
751
|
3253
|
4.33
|
22090
|
29.41
|
8295
|
11.05
|
33677
|
44.84
|
2008
|
796
|
3454
|
4.34
|
25439
|
31.96
|
8638
|
10.85
|
31614
|
39.72
|
2009
|
937
|
4026
|
4.30
|
29338
|
31.31
|
9780
|
10.44
|
37013
|
39.50
|
2010
|
981
|
4350
|
4.43
|
31564
|
32.18
|
9554
|
9.74
|
33162
|
33.80
|
2011
|
1126
|
5492
|
4.88
|
38141
|
33.87
|
11122
|
9.88
|
33929
|
30.13
|
2012
|
1130
|
5689
|
5.03
|
39862
|
35.28
|
11955
|
10.58
|
30349
|
26.86
|
2013
|
1061
|
5374
|
5.07
|
39715
|
37.43
|
11186
|
10.54
|
26512
|
24.99
|
2014
|
1190
|
6184
|
5.20
|
42769
|
35.94
|
12085
|
10.16
|
20202
|
16.98
|
2015
|
1085
|
5764
|
5.31
|
41774
|
38.50
|
11561
|
10.66
|
14354
|
13.23
|
2016
|
1168
|
6444
|
5.52
|
44729
|
38.30
|
12478
|
10.68
|
11292
|
9.67
|
2017
|
1201
|
6770
|
5.64
|
45418
|
37.82
|
12561
|
10.46
|
5921
|
4.93
|
2018
|
1266
|
7294
|
5.76
|
48656
|
38.43
|
14922
|
11.79
|
1838
|
1.45
|
TP, total publications; AU, number of authors; NR, cited references; PG, page count; TC, total citation count; AU/TP,NR/TP,PG/TP, and TC/TP, average number of authors, references, pages, and citations per article. |
Geography analysis
A total of 16,485 articles were used for the geography analysis, as 480 articles lacked the information of authors’ address. We extracted the geographic coordinates of 52,452 authors using python geocoding, then we used ArcGIS to present the latitude and longitude coordinates in the form of geocoded dots to reflect the spatial distribution of the number of articles published. The higher the density of dots in an area, the more articles published in that area. As can be seen from Fig. 1, the authors who studied health policy and services were distributed worldwide. The major spatial clusters of these authors were mainly located in North America and Europe, the other minor clusters were distributed in Eastern Asia, Southeastern Australia, Southern South America, Eastern and Western Africa, and only a few cluster dots were distributed in North Africa, Central Asia, and North Asia. These phenomena illustrated that the research interest of health policy and services varies among regions around the world, but authors in Europe and North America do better.
Countries/territories analysis
The contribution of different countries/territories was estimated based on the location of the authors’ affiliated countries/territories. Articles signed by authors from the same countries/territories were defined as single-country publications (SP); articles signed by authors from different countries/territories were defined as internationally collaborative publications (CP). The top 20 most productive countries/territories between 1999 and 2018 were summarized in Table 3, and they were ranked in terms of their total number of publications. The US outranked 149 other countries/territories with 12,421 publications, where 10,490 were solely produced by authors from the US and 1931 were international collaborations. The UK published the second largest number of total publications, followed by Canada, Netherlands and Australia. Among the top 20 most productive countries/territories, the number of internationally collaborative publications in 19 countries/territories outnumbered their single-country publications except for the US. Uganda had the highest proportion of internationally collaborative publications in its total publications (92.93%), whereas the US had the highest proportion of single-country publications in its total publications (84.45%). Out of these 20 countries/territories, 12 were from Europe, three were from Asia, two were from North America, two were from Africa and only one was from Oceania. We also found that the number of academic outputs was affected by the economic development of a country. Most of these top 20 countries/territories were developed countries/territories.
Table 3
The top 20 most productive countries/territories between 1999 and 2018
Country/territory
|
TP
|
%
|
SP
|
SP/TP
|
CP
|
CP/TP
|
USA
|
12421
|
73.22
|
10490
|
84.45
|
1931
|
15.55
|
UK
|
2190
|
12.91
|
766
|
34.98
|
1424
|
65.02
|
Canada
|
1360
|
8.02
|
597
|
43.90
|
763
|
56.10
|
Netherlands
|
719
|
4.24
|
305
|
42.42
|
414
|
57.58
|
Australia
|
618
|
3.64
|
224
|
36.25
|
394
|
63.75
|
Switzerland
|
329
|
1.94
|
34
|
10.33
|
295
|
89.67
|
Germany
|
313
|
1.84
|
90
|
28.75
|
223
|
71.25
|
Sweden
|
233
|
1.37
|
63
|
27.04
|
170
|
72.96
|
France
|
211
|
1.24
|
45
|
21.33
|
166
|
78.67
|
South Africa
|
196
|
1.16
|
25
|
12.76
|
171
|
87.24
|
Belgium
|
186
|
1.10
|
37
|
19.89
|
149
|
80.11
|
Norway
|
157
|
0.93
|
39
|
24.84
|
118
|
75.16
|
Spain
|
155
|
0.91
|
50
|
32.26
|
105
|
67.74
|
China
|
144
|
0.85
|
28
|
19.44
|
116
|
80.56
|
Italy
|
140
|
0.83
|
41
|
29.29
|
99
|
70.71
|
India
|
138
|
0.81
|
32
|
23.19
|
106
|
76.81
|
Denmark
|
110
|
0.65
|
38
|
34.55
|
72
|
65.45
|
Taiwan
|
106
|
0.62
|
50
|
47.17
|
56
|
52.83
|
Uganda
|
99
|
0.58
|
7
|
7.07
|
92
|
92.93
|
Ireland
|
96
|
0.57
|
23
|
23.96
|
73
|
76.04
|
TP total publications, SP single-country publications, CP internationally collaborative publications, SP/TP percentage of single-country publications in the total number of publications, CP/TP percentage of collaborative publications in the total number of publications, % percentage of total articles. |
The collaborative network of the top 20 most productive countries/territories was further visualized using Gephi (Fig. 2). Each node symbolizes a country/territory. The larger the nodes in the figure, the larger number of collaborative publications published in that country/territory. The thickness of the connecting line represents the strength of collaboration between any two countries/territories [9]. As can be seen from Fig. 2, the top 20 most productive countries/territories frequently collaborated. The US was identified as the major collaborator of some productive countries, including the UK, Canada, Switzerland, and the Netherlands. In addition, the UK had also a lot of collaboration with Canada and the Netherlands.
Institutions analysis
The contribution of different institutions was estimated according to the affiliated institution of authors. Similar to countries/territories analysis, articles signed by authors from the same institution were defined as single-institution publications (SP); articles signed by authors from different institutions were defined as cross-institution collaborative publications (CP). The most productive institution was Harvard University with 1292 articles, followed by the University of California Los Angeles with 624 articles, the University of Michigan with 580 articles, the University of Washington with 578 articles and RAND Corporation with 577 articles (see Table 4). Harvard University published the largest number of single-institution publications with 140 articles, followed by the University of Pennsylvania, almost half of the former with 75 articles. Harvard University also published the largest number of cross-institution collaborative publications, followed by the University of California Los Angeles, and the University of Washington. Most of these institutions were universities, only four of them were companies or hospitals; a total of 18 institutions were from the US, one was from Canada, and one from the UK.
Table 4
The top 20 most productive institutions between 1999 and 2018
Institution
|
TP
|
%
|
SP
|
SP/TP
|
CP
|
CP/TP
|
Harvard Univ,USA
|
1292
|
7.62
|
140
|
10.84
|
1152
|
89.16
|
Univ Calif Los Angeles,USA
|
624
|
3.68
|
34
|
5.45
|
590
|
94.55
|
Univ Michigan,USA
|
580
|
3.42
|
71
|
12.24
|
509
|
87.76
|
Univ Washington,USA
|
578
|
3.41
|
52
|
9.00
|
526
|
91.00
|
RAND Corp,USA
|
577
|
3.40
|
74
|
12.82
|
503
|
87.18
|
Univ Penn,USA
|
537
|
3.17
|
75
|
13.97
|
462
|
86.03
|
Univ Calif San Francisco,USA
|
500
|
2.95
|
72
|
14.40
|
428
|
85.60
|
Brigham & Womens Hosp,USA
|
395
|
2.33
|
6
|
1.52
|
389
|
98.48
|
Univ Toronto,Canada
|
372
|
2.19
|
9
|
2.42
|
363
|
97.58
|
Univ N Carolina,USA
|
368
|
2.17
|
49
|
13.32
|
319
|
86.68
|
Stanford Univ,USA
|
360
|
2.12
|
34
|
9.44
|
326
|
90.56
|
Boston Univ,USA
|
349
|
2.06
|
15
|
4.30
|
334
|
95.70
|
Yale Univ,USA
|
347
|
2.05
|
33
|
9.51
|
314
|
90.49
|
Johns Hopkins Bloomberg Sch Publ Hlth,USA
|
340
|
2.00
|
18
|
5.29
|
322
|
94.71
|
Univ Minnesota,USA
|
331
|
1.95
|
48
|
14.50
|
283
|
85.50
|
Johns Hopkins Univ,USA
|
324
|
1.91
|
32
|
9.88
|
292
|
90.12
|
Duke Univ,USA
|
324
|
1.91
|
43
|
13.27
|
281
|
86.73
|
Columbia Univ,USA
|
308
|
1.82
|
40
|
12.99
|
268
|
87.01
|
London Sch Hyg & Trop Med,UK
|
296
|
1.74
|
19
|
6.42
|
277
|
93.58
|
Univ Chicago,USA
|
278
|
1.64
|
36
|
12.95
|
242
|
87.05
|
TP total publications, SP single-institution publications, CP cross-institution collaborated publications, SP/TP percentage of single-institution publications in the total number of publications, CP/TP percentage of collaboration publications in the total number of publications, % percentage of total articles. |
The collaboration network of the top 20 most productive institutions is shown in Fig. 3. As a central point, Harvard University had strong collaborative relationships with Brigham Women's Hospital, RAND Corporation, the University of California Los Angeles, the University of Michigan and Yale University. In addition, the University of California Los Angeles had strong collaborative relationships with RAND Corporation, the Universities of Washington and California San Francisco. The Universities of Washington, Pennsylvania and Duke also had close collaborations, while the collaboration among other institutions was relatively fewer.
Keywords analysis
Keywords usually reflect the topic of an article. Keyword analysis can help us to identify research hotspots and trends. The keywords in the collected articles were divided and ranked in five time periods (1999–2003, 2004–2008, 2009–2013, and 2014–2018) (see Table 5). The arrows in Table 5 indicate changes in the rankings of these keywords. The five most frequently used keywords for all periods were “Medicare”, “Quality of care”, “Implementation,” “Cost-effectiveness”, and “Primary care”. In the last two decades, the rankings of keywords in this area have been constantly changing, but the rankings of “Medical”, “Quality of care”, and “Primary care” have been consistently ranked in the top ten, indicating that these topics were invariable hotspots in this field. The three keywords with the biggest changes in the ranking were “Implementation”, “Patient safety”, and “Disparities”, rising from the 130th, 740th, and 266th in 1999–2003 to the 1st, 14th and 16th in 2014–2018, respectively. These words attracted growing attention during the past 20 years and might become a new focus in the future.
Table 5
Temporal evolution of the 20 most frequently used keywords
Key words
|
Total
|
1999–2003
|
2004–2008
|
2009–2013
|
2014–2018
|
P
|
R
|
P
|
R
|
P
|
R
|
P
|
R
|
P
|
R
|
Medicare
|
416
|
1
|
72
|
3
|
81
|
3
|
110
|
3
|
153
|
3
|
quality of care ↓
|
398
|
2
|
76
|
2
|
96
|
2
|
113
|
2
|
113
|
6
|
Implementation ↑
|
349
|
3
|
6
|
130
|
13
|
82
|
92
|
4
|
238
|
1
|
Cost-effectiveness↑
|
326
|
4
|
31
|
12
|
71
|
6
|
127
|
1
|
97
|
9
|
Primary care↑
|
324
|
5
|
37
|
6
|
59
|
7
|
71
|
7
|
157
|
2
|
Medicaid↑
|
324
|
5
|
41
|
5
|
100
|
1
|
57
|
11
|
126
|
4
|
Quality improvement↑
|
277
|
7
|
19
|
24
|
50
|
8
|
84
|
5
|
124
|
5
|
quality of life↓
|
239
|
8
|
55
|
4
|
77
|
5
|
73
|
6
|
34
|
42
|
Mental health↑
|
229
|
9
|
24
|
19
|
49
|
9
|
50
|
14
|
106
|
7
|
managed care↓
|
217
|
10
|
97
|
1
|
79
|
4
|
28
|
44
|
13
|
202
|
health policy↑
|
207
|
11
|
14
|
40
|
25
|
39
|
63
|
9
|
105
|
8
|
health insurance↓
|
178
|
12
|
31
|
12
|
48
|
10
|
38
|
26
|
61
|
17
|
Veterans↓
|
174
|
13
|
34
|
9
|
30
|
28
|
31
|
38
|
79
|
11
|
Depression↓
|
170
|
14
|
34
|
9
|
47
|
11
|
42
|
20
|
47
|
24
|
patient safety↑
|
166
|
15
|
1
|
740
|
39
|
17
|
57
|
11
|
69
|
14
|
health services research↑
|
161
|
16
|
17
|
31
|
31
|
26
|
43
|
19
|
70
|
13
|
quality↑
|
160
|
17
|
19
|
24
|
42
|
13
|
49
|
16
|
50
|
22
|
mortality↓
|
160
|
17
|
37
|
6
|
39
|
17
|
46
|
18
|
38
|
35
|
Diabetes↑
|
160
|
17
|
7
|
104
|
34
|
24
|
62
|
10
|
57
|
19
|
Disparities↑
|
158
|
20
|
3
|
266
|
45
|
12
|
47
|
17
|
63
|
16
|
P number of publications, R rank |