Systematic Analysis and Quality Evaluation for Pharmacoeconomic Studies on Traditional Chinese Medicines

Background: This study aimed to describe the current situation of the published Pharmacoeconomic researches on Traditional Chinese Medicine and evaluate the quality of the researches to analyze existing problem and make suggestions for improvement. Methods: We developed a search strategy, set inclusion and exclusion criteria and searched the articles of Pharmacoecnomic studies on Traditional Chinese Medicine from CNKI, VIP, Wanfang, Sinomed database and Pubmed from their inception to February 2020. Basic information of the literatures was extracted. Then quality of each article was evaluated by using Guidelines for authors and peer reviewers of economic submissions to the BMJ. Results: 430 articles with 433 pharmacoeconomics studies were included. And the information of time of publication, journal, author, funding, diseases, interventions, research design, evaluation methods, cost identification, health outcomes, uncertainty analysis and so on. The overall quality score of the literatures was 0.62 according to the evaluation standard of BMJ guidelines. The highest mark was 0.85 and the lowest mark was 0.38. Conclusions: Quality of pharmacoeconomic studies on traditional Chinese medicines was generally low.There were some problems in the published pharmacoeconomic articles, such as lack of fund support, short research horizon, unclear perspective and research design, misidentification of cost, poor comparability of health outcome and so on. We suggested to develop guidelines for pharmacoeconomic evaluation on Traditional Chinese Medicine to standardize the research procedure and establish reasonable health outcome indicators of Traditional Chinese Medicine to enhance comparability and generalization of studies.

Traditional Chinese Medicine (TCM) has been playing an important role in the field of medical treatment in China with Chinese patent medicine making up 38% share of the drug market. In recent years, with the increasing attention to economic evaluation of medicines, there were more and more pharmacoeconomic researches that could comprehensively evaluate the effect and economy of TCM.
However, due to data gathering, standardization of clinical research and so on, pharmacoeconomic researches on TCM were of variable quality. This study aimed to describe current situation, analysis the existing problems of pharmacoeconomic researches on TCM and make suggestions for further improvement of the pharmacoeconomic research on TCM.
The search strategy was ("Chinese medicine" OR "Traditional Chinese Medicine" OR "Chinese patent medicine" OR "herbal medicine" OR "natural medicine" OR "Chinese Medicinal Materials" OR "Botanical medicine" OR "Chinese herbal pieces" OR"Chinese medicinal granula") and ("pharmacoeconomics" OR "economics" OR "Cost Effectiveness Analysis" OR "Cost Utility Analysis" OR "Cost Benefit Analysis" OR "Cost Minimization Analysis" OR "Markov" OR "Decision Tree").
Chinese and English terms for search keywords were used when searching the databases. In addition, the references in all retrieved articles as well as our department files were searched. were marked with funding support, which included 9 state projects, 22 provincial and ministerial funded projects, 22 municipal funded projects, 4 University funding projects, 4 funding projects of the authors' units, 3 project supported by the foundation, and 2 Enterprise funded projects.

Disease
According to the International Classification of Diseases (ICD-10), the 430 published literatures ranged over 148 kinds of diseases' treatments, among which the top five kinds of disease included nervous system, cardiovascular system, respiratory system, gynecology and digestive system as noted at There were 262 kinds of Chinese patent medicines and 70 kinds of traditional Chinese medicine prescriptions (such as prescription, pieces decoction, powder, extraction, tincture, pills) in the studies.

Sample size and perspective
Expect 21 studies extracting parameters from other literatures, the sample size of other studies ranged from 38 to 2820 subjects, which was shown in Figure 4. The age of subjects was from 0 to 90 years old. 376 (86.84%) studies did not clearly define the perspective of their pharmacoeconomic evaluations. 24 of the remaining studies were from the perspective of medical institutions, 13 from the patients, and 11 from the whole society, 11 from the payers, 7 from the patients, and 2 from health system.

Evaluation techniques
More than 82% of the studies chose cost-effectiveness analysis (CEA), and the others study used at least one method of cost analysis, such as Cost Minimization Analysis (CMA), Cost Benefit Analysis (CBA), and Cost Effectiveness Analysis (CUA). In the 28 studies, the above methods were used in combination with the Decision Tree (DT) model for short-term predictions or Markov model for longterm predictions ( Table 2).  Uncertainty analysis and generalizability 66.51% the studies (288) carried out uncertainty analysis, and the main uncertainty analysis variables were shown in Table 4. Uncertainty analysis of 219 (76.04%) studies were assessed by using one-way sensitivity analysis with reducing drug prices by 10% -20%. 69 (23.96%) studies used multi-way sensitivity analysis for uncertainty. If the evaluation result of each item was marked as "yes", 1 point would be scored. If the evaluation result is marked as "no" or "unclear", 0 point would be scored.
Each study was scored based on the proportion of items it should have met according to the guideline of BMJ, the maximum score of which was 1. The overall quality score of a pharmacoeconomics evaluation researches on TCM was the average score of each study. If the score is greater than 0.90, it indicated that the literature quality was high [6] . And we also calculated the score of every item in guideline of BMJ. The specific quality evaluation scoring results were shown in Table 6.

Adjustments for Timing of Costs and Benefits Time horizon of costs and benefits was stated
The discount rate(s) was stated The results showed that the overall quality score of 433 pharmacoeconomics studies on TCM was 0.62 according to the evaluation standard of BMJ guidelines. The highest mark was 0.85 and the lowest mark was 0.38. Scores of majority studies were concentrated in 0.5-0.7. The distribution was showed in Table 7 ranking according to marks. Among them, items of "stating methods to value health states and other benefits", "reporting quantities and their unit costs of resources separately", "adjustment details of currency of prices for inflation or currency conversion", "the choice of rate(s)", "the discount rate(s)", "an explanation without being discounted", and "details of statistical tests and confidence intervals for stochastic data" scored below 0.1. Items of "perspective/viewpoints", "productivity change reporting", "Time horizon", and "conclusions accompanied by the appropriate caveats" also scored low. And items on the rationale stating for choosing the alternative programmes or interventions compared, source(s) stating of effectiveness estimates did not score high.
When calculating separately, the average quality score of the 6 literatures in English was 0.73. The results showed better quality of pharmacoeconomics research literatures on TCM from SCI journals, especially in the items of research perspective and discounting. However, uncertainty analysis and productivity measurement of the study remained to be improved.

Discussions
At present, the published pharmacoeconomic studies on TCM were in small number of sample size, conducted by numerous research institutions and lack of fund support. In the past decade, the number of the published pharmacoeconomic research articles on TCM had being increasing annually, but the total was less than one tenth of the number of published pharmacoeconomic research articles on chemical drugs. Moreover, researchers of those studies were mostly from various medical institutions, which were decentralized and only 15.35% studies had funding support.
Chinese patent medicines had rose to 49.98% of all drugs listed in National Drug Catalogue for Basic Medical Insurance of China, which was released in 2019 [7] . Although the government had considered the factor from pharmacoeconomics evaluation on TCM in the policy design of medical insurance reimbursement, governmental drug procurement and pricing, it was not clear how much current published studies were worth.
Overall quality of published pharmacoeconomic research articles on TCM was low. Although nearly half of the studies were published in Chinese core journals criterion and even journals of Science Citation Index, compared with the study of Sun [1] in 2015, the quality of pharmacoeconomics research articles on TCM has not improved in recent five years; compared with the studies of Chen [8] , Li [9] , Lei [10] in 2004, 2008 and 2010 respectively, the quality of pharmacoeconomics research articles on TCM was lower than that on chemical drugs. On the one hand, none single consistent guideline for TCM pharmacoeconomic evaluation has been accepted by the industry, and there may be some dislocation and bias when using pharmacoeconomic evaluation guideline on chemical drugs for TCM. On the other hand, the researchers of pharmacoeconomics evaluation research on TCM mostly came from medical institutions, were lack of thorough understanding of the basic theory of pharmacoeconomic, which led to poor quality of overall researches.
Most researches did not clearly described perspectives and study design of the researches lacked scientific basis. The perspective of pharmacoeconomics evaluation would affect cos identification and measurement, the selection and calculation of health outcomes and be consistent in a study [11] , however, more than 85% of the studies did not mention their research perspectives, which had negative impact on the integrity and reliability of the study. The sample size of 433 studies ranged from dozens to thousands, with great differences. Although the China guideline for pharmacoeconomic evaluation suggested the sample size should meet the needs of different pharmacoeconomics evaluation [11] , there was only four studies describing the process of sample size estimation. More than half of the studies reported randomized controlled trials, but the vast majority of the studies did not provide specific methods for randomization. Moreover, some studies obviously confused retrospective cohort study, mixed study and randomized clinical intervention study. In terms of control group, nearly half of the studies did not explain the reasons of setting. If researchers only compare several interventions at will, it would be difficult to ensure the generalizability and application of the results. Time horizon of the studies was commonly short. Nearly 70% studies were within one month and only 1.04% studies lasted more than one year. This might be why only 2.3% costs in the literatures were discounted. Except a few studies on acute diseases, most studies on chronic diseases with relatively long treatment cycle. However, these studies did not fully reflect the advantages of TCM in improving long-term clinical efficacy, reducing recurrence rate and treatment costs.

Conclusions
Quality of pharmacoeconomic studies on traditional Chinese medicines was generally low, which was consistent with the results of previous quality assessment of economic evaluation on complementary and integrative medicine [12] .
We suggested researchers to formulate pharmacoeconomic evaluation guidelines being suitable for the characteristics of TCM and standardize the research procedure, such as complex prescription and excessive confounding factors. In 2013, Xie et al, published 'technical specification for pharmacoeconomic evaluation on post marketed Chinese patent medicine (Draft)' in China Journal of traditional Chinese Medicine [13] ; and in 2019, Gao et al, published 'Guidelines for Quality Assessment of Pharmacoeconomic Evaluation Report' as group standardization [14] . Those guidelines considered the basic ideology and characteristics of traditional Chinese medicine to certain extent, however, they did not solute the specific problems faced by the pharmacoeconomic evaluation of TCM, such as how to map the effectiveness and health outcome of TCM and chemical medicines. Therefore, a scientific and unified pharmacoeconomic evaluation guideline for TCM should be performed combined with expert opinions, focusing on the characteristics and current research situation of TCM. Reasonable and operable requirements and suggestions should be put forward from the aspects of sample size, study design, perspective stating, intervention selection, evaluation method selection, cost identification, health outcomes determination, uncertainty analysis, report form, etc., so as to standardize research procedure and improve the quality and reliability of results.
Reasonable health outcome index for TCM should be establish to enhance clinical comparability.
Traditional Chinese medicine takes holistic treatment as main concept, so it's better to evaluate overall outcome of TCM by using health utility as its clinical output index [15] . It is difficult to evaluate the utility of TCM due to lack of health-related quality of life scale of traditional Chinese medicine in China. Therefore, it is necessary to establish an objective, scientific and quantifiable evaluation system of TCM effect indicators. On the one hand, there is need to establish the judgment standard of Chinese Medicine syndrome efficacy generally accepted to reduce the bias of subjective factors [15] .
On the other hand, researchers should explore the corresponding relationship between symptoms, syndrome types of Chinese Medicine and disease diagnosis and classification of Western Medicine, so as to enhance the comparability between different treatments, expand the clinical applicability and generalizability for government's decision-making of the results of pharmacoeconomics research on TCM.
Strengthening cooperation of clinical and academic insititution might be helping to improve research quality. The number of pharmacoeconomic studies on TCM has been increasing in the past 40 years, but most clinicians and health policy makers remain skeptical of the results [16] .
Many pharmacoeconomics researchers also found that the research quality was generally low when Not applicable.

Consent for publication
Not applicable.

Availability of data and materials
All data are fully available without restriction.

Competing interests
The authors declare that they have no competing interests.

Funding
This research has been supported by National Key R&D Program of China (Grant No. Tables Table 5 was omitted by the authors in this version of the paper. Figure 1 Flow Chart of Study Search and Selection Distribution of Sample Size