The unguaranteed quality of scientific articles in the field of medical popularization of science is an important problem facing the popularization of science in China at present. Determining the scientific evaluation criteria of popular science articles is an important prerequisite for seeking change. This study confirms the consensus of experts on the scientific evaluation criteria of medical popular science article. This consensus shows that it is feasible to establish a unified scientific evaluation standard for medical popular science articles in the medical popular science industry, which covers many specialties. The establishment of this evaluation standard plays a positive role in standardizing the medical popular science industry.
From the consensus results, the expert opinion is concentrated in three areas directly related to the selected evidence, namely, A. Choice of evidence, B. Evidence evaluation, C. The application of evidence, among which the metric A and B present a 100% agreed result. It is shown that, similar to the evaluation standard of scientific papers, the credibility of evidence is the most important thing in evaluating the scientific nature of popular science articles.
However, there are still some disputes on the specific evaluation criteria of evidence. In particular, experts argue on whether newer research evidence should be selected for popular science (77.8%). The opposition view is that the new evidence needs to be repeatedly verified, which is not enough to spread as definite knowledge to the public with weak judgment ability, and there are still many mature knowledge in the field of medical popular science, which is still not understood by the public and needs to be popularized, so it is not cost-effective to carry out such popularization in the situation complicated enough. In addition, the degree of recognition of “Total number of evidences selected” metric is not high (77.8%). The choice of evidence in the evidence rating standard of evidence-based medicine cannot be used as the evaluation standard, but the reason for adding this item to the questionnaire is due to the fact that a large number of pseudo science popularization articles contain only "how to do" but not "why". The disagreement on this metric can be expected before the research.
Of the other three metrics, the most significant differences in expert opinions are concentrated in F. “conflict of interest”. The uniformity of expert opinion for the primary metric is only 55.6% ((SA+A) / n*100%). The opinion uniformity of the three secondary metrics is 74.1%, 85.2% and 77.8%, respectively. Only from the scientific point of view, the introduction of any kind of conflict of interest may affect the objectivity of the author, resulting in the bias of author's choice of evidence because of his own point of view, which is not conducive to the scientific nature of the article. However, different from the evidence rating standard of scientific papers, popular science articles are facing two important problems: first of all, different from general scholars, the authors of popular science articles usually need to rely on popular science creation as the main source of income, and it may be too idealistic to require that they do not involve interests at all; in addition, as a means of media education, the popularization of science must include the author's personal position, that is, what the author thinks is correct and should be understood by the public, so it may not be necessary to be completely neutral and objective.
The experts invited in this research have high authority and comprehensiveness. The experts invited for research were all from popular medical writing Committee, CSWA (China Science Writer Association) and have rich experience in the field of medical science popularization in China. The average number of years of experts engaged in science popularization related work is 9 years, and the experts involved in the investigation cover a number of fields involved in the general science popularization in nutrition, medicine and pharmacy.
Strengths and limitations of this study
The Delphi method is used in the study, and its advantages mainly lie in anonymity and iteration. Anonymity is achieved by contacting participants alone, avoiding the influence of expert prestige and authority or opinion leaders in a small range of discussions. It also avoids the problem of group integration and preserves the difference of expert views. In addition, different from the single round of questionnaire survey, because of the three consecutive rounds of Delphi consultation, the members of the group can adjust their views according to the feedback of others over time to promote the achievement of consensus.
However, the potential limitation of the study is the cultural specificity. The design of the questionnaire concerns mostly Chinese society, and the experts who fill out the questionnaire are all from China. In view of the typical problems existing in the popularization of medical science in China, the scores of experts in specific metrics may be higher than the objective standard.