1. Data
Data for network analysis are shown in Table 4. Designated evaluation items and comprehensive evaluation items were assigned codes from A to I, and the results of the inter-item correlation investigation were entered in the matrix of Fig. 3. The survey was conducted with 50 supercomputer experts participating in the joint utilization system, and items with correlation between evaluation items were marked with 1, and those with no connectivity were marked with 0.Network analysis programs used UCINET6 and Netdraw.
Table 4
Code
|
Evaluation item
|
Code
|
Evaluation item
|
A
|
Related Performance
|
E
|
Operation Achievement (Mid)
|
B
|
Validity of operation purpose and plan
|
F
|
Operational performance (Mid)
|
G
|
Internal and external collaboration performance(Mid)
|
C
|
Suitability of center manpower
|
H
|
Operational performance (Comprehensive)
|
D
|
Facility and Equipment Securing Plan
|
I
|
Publicity and performance in the year of the end of the operating period
|
Unlike the existing network analysis, this paper applies new weights for each evaluation item. The reason for applying the weight is, first, to reduce the error caused by the small number of survey respondents, and second, to minimize the statistical difference in the distribution of points for each evaluation item because designated evaluation and comprehensive evaluation are essentially different evaluations. The weight \({\text{W}}_{\text{N}}\) considers the distribution of points for each evaluation item and is calculated as shown in Equations (5) to (6). N is the score of the evaluation items, \({\text{S}}_{1}\) is the total score of the designated evaluation item, and \({\text{S}}_{2}\) is the total score of the comprehensive evaluation item.
![](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAdIAAAAmCAYAAABwFBqNAAALYElEQVR4nO3dz2vb5h8H8Le+fK9rKrunDkqwctuGxypnoziHHhKZDsYgAXs/DoEGOnkw2FicVMlu+VF5bIUw7GakEMY2O2vHYMzemkAv1mFb3WKz7dTIlB52khKy/gHP95CvNMu/ZTtx7HxeIGgeydIj6/HzefRIz1OOMcZACCGEkLb8p9cZIIQQQvoZBVJCCCGkAxRICSGEkA5QICWEEEI6QIGUEEII6QAFUkIIIaQDfR9I4/E4OI5DKBSy00KhEDiOQzwe72HOyElSq0xwHAeO46BpWg9zRgjpd9wgjCMNBALY39/H7Owsrl27Bk3T8OGHH+LBgwe9zho5IUzTxKuvvgpd11EoFOD3+xGPx3FwcIDl5eVeZ48Q0sf6/o4UALxeLzY3N3H9+nWUSiU7jRCL1+vFyMgIVFXFzMyMnT40NNTDXBFCBsFABFIACAaDmJ+fRzgcRjAY7HV2yAkVi8Xg9XqxsLCAS5cu9To7hJAB8N9eZ6CbYrEY7ty5Q89GSUOJRAKiKNLdKCGkKwbijtQ0TZimCQDY2NjAjRs3Wv5cPB7HwsLCUWavbywsLAxsI6S8jPh8PqyurrZcTkqlEqLRKNLp9FFm8cRJp9OIRqP290ZIP4pGo01fKAyFQvZjwbawNhUKBcbzPOtgF12hqioDwCRJcqSV/12LYRhMFEWWSqXsNOt8VFW108rPs1AodP8EWpDL5RgAlsvl2vp8+TlYC8/zTJZlpuu6Y9tUKsVEUexGtu39VV6fXpAkqeraSpLk+LuWQqHABEGwr32hULC/w/LrYZ0nz/NHcwJ1hMNhx3UtX9yyyoggCHaadf6V5YSQfiBJkqOOZ4wxQRDs30h5vdRJHdtRFLSCWD9SFKWqEs1kMoznecbzvKPiUFWVKYpy3Fm0ybLMeJ7vKA+ZTMZxrXK5HBNFkfE8X9VAkGW5aYBxQ5KkngfSdomiWPXjUhSF8Txf1eCQJIllMpnjzB4zDKOqArAaTm5ZZaTyfFOpVN9eP3J6qapaVY/lcrmqwFqu3Xg2EF27bpmmiZWVFUxNTTnSz5w5g9HRUQiCAEVRHOu6+TxN07SWuwlN08T29jZWV1extbXV9jHPnDnj+DsYDOKXX34BAKyurjrWffzxx7hx48ap79LLZrPY39+venltaGgIkUgEuq5jfX3dsa7ye+7E+vo6isViw21qvZ3u9/uxt7fn+nj18h6JRLC7u0vjbUnfKJVKmJubQywWc6R/++23iEQidT+nqiqi0ajr47kKpKZpIhKJgOM4eDwe3Llzx/UBT4KdnR2Iogifz1dz/c2bN7G1tXVkFUcwGMTm5mZLwXRnZwezs7MYHx+HrutVFWuxWITH40EgEKhal81mG/b7e71eyLJcFaB9Ph88Hg9+++03F2flzFMgEADHcRgZGcHu7m5b++m1n376CeFwuOa64eFhzM/P4/r160fW4JiamsLk5GTTYFpO0zTH5CTA4e82EAjA4/Egm8061pVKpaq0WiYmJvDzzz+3nA9Ceunu3buQZdmRpmkakslkw8l6Ll26hGQy6fp4rgLpO++8A13XYRgGHj9+DEEQXB+wnKZp9uwy9ZbKSqEb/vjjj4Z5DwaDCIfDmJ6e7vqxLYlEAouLi00ryc8++wzj4+Pw+XwQBAG//vqrY/3q6ioeP36MmZkZTE5OOir1r776qm5jwVLvTntkZAR//vlni2fzL9M0cfnyZQQCATDGsL297XoflXpVTh48eIALFy7UXX/16lV4PB588sknXT82cNjQWVtbw8zMTNNgPTY2Bo7jMDY2VrXu888/x8bGBn788Ue8++67jjJ39+5dPP/8803zMjw8jIcPH7o/CUJ64P79+xgeHnaknT9/HuzwcSbm5ubAcVzV56zeJ7cvHrUcSEulEu7du4ebN2/C6/XC6/XilVdecXWwSsFg0D6xeovV/VipWcVauZR7+PBh07yvrKxgb2+v6VusrVTytRZBEKDrOubm5uruu1gsQhAEOxiGw2FsbGzY67PZLN588014vV5cu3YN4XDYfvssnU53fH3asbOzg/39fSQSCQCHd7cjIyMd7bNX5SSfz+OFF16omy8r0CWTyaa9F9ZUlm6X119/Hfl8Hrdv3264/1wuB8YYcrmcI900Tezv78Pv9yMYDOLrr7/G5cuXoWkaisUiHj16BL/f33DfhAyC8psKxhgkSarbjfv333+72nfLgbTZjq2A4vF47NZzrXlwu6VZxVq5uOXz+TA/P990iEQrlXytRdd1CIIAVVXr7vu7777D1taWXamurKwgn887Wkvj4+P2v5eXlzExMQFBELC5uVn1fKCWg4ODjnsWyj19+rTh+kErJ1euXIEkSVhaWsLZs2frbheLxdoqJ5lMBqIo4urVqy3lp/zZN3B4fd9++21HfhOJBN544w1MTk5iZWXF9TkTMggWFxc7G/JSpmsvGwWDQSiKAo/Hg/fffx8A7Ir8m2++qfmZXnXZXbx4EY8ePWq6XSwWg8fjwZdfftn1PESjUSwtLTW8G0gmkzAMw1Gx8jyPnZ0dAIeVYuXLJsvLyw3v0MqZpolkMln3OeBR6KdyIooi/vrrr6bbJRIJ3Lt3D/l8vqvHN00TH3zwATY2NlxPeRkIBAAcNggrX5aKRCLY29vD7u5u065/QvpRq+W6W+W/5UAaDAbB8zy++OILAIddvU+ePHFsMzQ0hLW1NWxvbzteYKhXCXTSZdeJl156CbquV6X/888/VWmbm5s1t+1ENpvF9PR0w7fH0uk0RkdHq7670dFR/PDDD66PWXluxWIRoVAIgiDgo48+qtp+d3cXL774ouvjSJIEAPbbrLW6PPulnAQCgZp32AcHB46/fT4fVFXtejm5ffs2vv/++4aNrVrPTjVNayuo1yr/lidPnuDixYuu90lILwwPD+P+/fsNt5menrYfQVms+sr1NLMtDpNhjB2OwbEGsyqKYo87k2WZMXY4bieXy7FcLmePxTyJ48+ssXeVg8ytAem3bt1ypIfD4a6Oq2zGGtxvfc8WRVHsdOs7b0WtCRkEQWCyLDPDMKq213Wd8Txvr7PGC7f6Hdy6dcsej5tKpex8W+O3+qWcZDIZx+QEjDknZCgff2sYBhMEoe0B3e066gkZLL04N0Lapet61W9AlmXH76PWJCOqqrqqWy1dnU3BqiAZO6z0j3ogvlVh4/8zyriZeUhRlJ5OsnCS1ZuQoZ0CVstxl5PyH1CtINGIKIrHPsnCSUMTMpB+VGtChmbaaYAy1uVAat2lMvZvC/2opkyzWhxWq0JRFFct5lpTBJLaUwRad8jdmibuOMuJdT6GYTDDMFwHhMopAk8bmiKQ9LNWpgG1dNLr0rVAas0HWx7RrbSjYHU/dhIIDcPo+fR/J0mtaRO77bjLidU920m3pK7rTJblU9foSqVSdbv/CekXsiw3/f1LktRRY5FjrI13/k8ITdOwtLSE33//HfPz8y0N9yCnTzqdxuLiIgBgbW0NV65c6XGOCCGDpK8DqaVUKmFiYgJLS0sN34Qlp5umaRgbG4Ou6zTsgxDSNX07ab2maXbQHBoaAs/z9rCEdDrd8Ww6ZDDE43F7KM758+cBAM+ePbPLyFGNQSWEnB59G0ife+456LoOjuNw7tw5CIKAqakpFItFjI6OUiAlAIALFy7g008/BcdxEEURiqLA7/fj6dOnmJ2dhWEYfTupPiHkZBiIrt1aQqHQkQzSJ4NlfX3d/m/RCCGkHX17R9qMaZqn/v/TJI2tr6/jtddeA1B7BiZCCGnFwAVSa17WfD6Pc+fONf3fW8jplE6n8d577+Hll1/GW2+91evsEEL62MB27RJCCCHHYeDuSAkhhJDjRIGUEEII6QAFUkIIIaQDFEgJIYSQDvwPYy6wwoEGxDkAAAAASUVORK5CYII=)
$${\text{W}}_{\text{N}}=\frac{{\text{d}}_{\text{N}}}{\sum {\text{d}}_{\text{N}}}$$
6
The calculated weight is reflected in the matrix value as in Eq. (7). \(\text{V}\) is an existing matrix value and \({\text{V}}_{\text{M}}\) is a matrix value considering weights.
$${\text{V}}_{\text{M}}={\text{W}}_{\text{N}}\bullet \text{V}$$
7
2. Network Centrality Analysis Result
Figure 4 shows a schematic diagram of the evaluation item network. The line means connection, and the thickness of the line means the strength of the connection. Item A is usually not strongly linked to other items. Items that are relatively strongly connected are C and D, which are all evaluation items of designated evaluation. B is strongly connected with D, and the strength of connection with comprehensive evaluation items such as F, E, and H was also strong. On the other hand, the connection strength with the I evaluation item is relatively low. C showed a high connection strength with comprehensive evaluation items E and F, and a low connection strength with I.D has a low connection strength with C and I, and shows a similar level of connection strength to the other items. E, the evaluation item of the comprehensive evaluation, is most strongly connected to H, and also shows relatively high connectivity to C, F, and G. F is most strongly linked to H and weakest to A. G is very strongly connected to E and H to F and E. I has the lowest connectivity with all evaluation items.
The characteristic parts derived from the analysis results are as follows. Item A did not show any connection with the evaluation items of the comprehensive evaluation, but according to the “operating guidelines”, it means the performance before designation, so it is appropriate that there is no connection. In the case of B, it was confirmed that there was overall correlation with the evaluation items of the comprehensive evaluation. Therefore, in the designated evaluation system, it was confirmed that the weight of the evaluation item B should be higher than that of other evaluation items, and it is reflected at an appropriate level in the operation guidelines. Similar to B, C also showed a connection with the comprehensive evaluation items except for I. In the case of I, there was no correlation with any of the evaluation items. Therefore, I can be regarded as relatively inappropriate as an evaluation item for re-designation.
Quantitative analysis was conducted through eigenvectors to derive improvement measures considering the characteristic parts.
The eigenvector values and descriptive statistics for each evaluation item for the network are shown in Table 5, 6. Evaluation items were divided into three groups using the eigenvector value. Groups with a value of 2 or less with relatively low connection strength include A and I, and groups with a value of 2 or more and 4 or less with a medium level of connection strength include C, D, E, F, G, and H. B is evaluation items with a high connection strength exceeding 4. Next, implications were derived through the eigenvector value for each evaluation item. Similar to the results analyzed by visualizing the network, the eigenvector values of A and I were also very low, resulting in relatively low influence in the evaluation model or independent items with no linkage between evaluation items. In the case of the A evaluation item, it is reasonable to exclude it from the designated evaluation item considering the linkage with the comprehensive evaluation. However, since the necessity exists as an important criterion for evaluating the capacity of the applicant organization at the time of designation, it is judged appropriate to lower the score as an independent item even if there is no linkage. On the other hand, in the case of I, it is judged that the necessity of evaluating the re-designation of the center is relatively low for the promotion performance item of the end year. Furthermore, it is reasonable to exclude I because the promotion performance of the final year overlaps with the operational performance (whole period) of evaluation item H. In the case of G, the value of 0.225 showed relatively low centrality. In order to analyze the cause of the low centrality of evaluation item G, a second content review was conducted targeting the respondents. Most of them interpreted the definition of internal/external cooperation performance more broadly compared, and interpreted the external cooperation part by reducing it to overseas cooperation. Therefore, the need to change the name of evaluation item G more clearly was raised. The eigenvector variance value was calculated as 0.032 for the designated evaluation item and 0.013 for the comprehensive evaluation item, showing a difference of about 47% in the variance value of the two groups. Through this, it is necessary to adjust the evaluation items to improve it into a balanced evaluation model. In addition, the maximum value of the eigenvector was derived as 0.528 as the designated evaluation item B. Through this, it was confirmed that the center of gravity of the specialized center designation evaluation was in the designated evaluation, and the necessity of changing the comprehensive evaluation items for re-designation to be the center was derived through the improvement of the evaluation model.
Table 5
Eigenvector centrality value
Item
|
Value
|
Var
|
A
|
0.099
|
0.032
|
B
|
0.528
|
C
|
0.317
|
D
|
0.384
|
E
|
0.386
|
0.013
|
F
|
0.354
|
G
|
0.225
|
H
|
0.354
|
I
|
0.117
|
Table 6
N
|
Min
|
Max
|
Sum
|
Avr
|
SD
|
Dist
|
9
|
.10
|
.53
|
2.76
|
.3071
|
.13767
|
.019
|
Table 7
|
DesignatedEvaluation
|
|
ComprehensiveEvaluation
|
Evaluation item
|
points
|
Evaluation item
|
points
|
A
|
Related Performance
|
10
|
E
|
Operation Achievement (Mid)
|
30
|
B
|
Validity of operation purpose and plan
|
40
|
F
|
Operational performance (Mid)
|
20
|
C
|
Suitability of center manpower
|
20
|
G
|
Internal and external collaboration performance(Mid)
|
10
|
D
|
Facility and Equipment Securing Plan
|
30
|
H
|
Operational performance (Comprehensive)
|
40
|
Table 8
Eigenvector centrality value
Item
|
Value
|
A
|
0.135
|
B
|
0.362
|
C
|
0.314
|
D
|
0.346
|
E
|
0.437
|
F
|
0.357
|
G
|
0.170
|
H
|
0.534
|
4. Improvement Model Evaluation
The improved evaluation items are shown in Table 3. Comprehensive evaluation item I with a very low eigenvector value was excluded, and A was not excluded because it had a low eigenvector value but had an independent influence on the evaluation results, and only some points were adjusted downward. In addition, the points assigned to the evaluation items D, E, F, and G were partially increased considering the points assigned to the existing evaluation items and the eigenvector values.
Table 9
|
DesignatedEvaluation
|
|
ComprehensiveEvaluation
|
Evaluation item
|
points
|
Evaluation item
|
points
|
A
|
Related Performance
|
10
|
E
|
Operation Achievement (Mid)
|
30
|
B
|
Validity of operation purpose and plan
|
40
|
F
|
Operational performance (Mid)
|
20
|
C
|
Suitability of center manpower
|
20
|
G
|
Internal and external collaboration performance(Mid)
|
10
|
D
|
Facility and Equipment Securing Plan
|
30
|
H
|
Operational performance (Comprehensive)
|
40
|
As a result of the network visualization in Fig. 5, the connection strengths of most evaluation items except for B-D, F-H, and E-H were similar. It can be seen that the connection strength of F-G and F-C centered on F has slightly lowered compared to the previous part.
The eigenvector values for the evaluation items are shown in Table 10. The eigenvector values of A, B, C, F, and G decreased compared to the previous ones, while D, E, and H increased. H showed the largest increase, and G showed the largest decrease. A qualitative analysis on this confirmed that in the case of G, the evaluation range included in E, F, and H as “Internal and external collaboration performance (Mid)” was relatively low, and it was improved to an appropriate level. H is the “Operational performance” of the entire operating period and is the key evaluation item with the highest score in the comprehensive evaluation. As a result of the analysis, the allotment of the peripheral evaluation items, which are related to most of the evaluation items such as E, F, and G, and have a high linkage, was raised, indicating high centrality. In addition, it was confirmed that the evaluation model was improved as the evaluation model, which had been focused on the existing designated evaluation, moved to the comprehensive evaluation.
Table 10
Eigenvector centrality value
Item
|
Value
|
A
|
0.135
|
B
|
0.362
|
C
|
0.314
|
D
|
0.346
|
E
|
0.437
|
F
|
0.357
|
G
|
0.170
|
H
|
0.534
|
The descriptive statistics are shown in Table 11. It was confirmed that the network balance of the evaluation items was improved to the level of 5–10% of the evaluation items compared to the existing evaluation model, as both the standard deviation and variance values were reduced compared to the previous evaluation model.
Table 11
|
N
|
Min
|
Max
|
Sum
|
Avr
|
SD
|
Dist
|
Deg
|
9
|
.10
|
.53
|
2.76
|
.3071
|
.13767
|
.019
|
Com
|
8
|
.14
|
.53
|
2.66
|
.3319
|
.13034
|
.017
|