Development of a Core Outcome Set for Clinical Trials of Traditional Chinese Medicine in Lung Cancer: a Study Protocol

Background: Lung cancer (LC) is currently the leading cause of cancer death globally. LC accounts for a high mortality and incidence rate of cancer in both men and women. Radiotherapy and chemotherapy, though effective for some patients, have strong side effects and most of them only have palliative effect. Though many studies reported the effectiveness of traditional Chinese medicine (TCM) treatment on LC, the lack of methodological quality in clinical trials resulted in heterogeneous reporting of outcomes, making it dicult to compare and combine in different trials, limiting the validity of meta-analysis and reducing the value of clinical studies. A core outcome set (COS) could reduce outcome reporting bias and heterogeneity across studies of LC using TCM-related interventions. The aim of this study is to develop a standardized COS for LC in TCM clinical trials (COS-TCM-LC) to improve the methodological quality of TCM clinical studies to serve as a guidance in healthcare decision making for LC. Methods: The study has been developed according to the Core Outcome Set-STAandards for Development standards for the design of a COS study. The study process consisted 4 stages as follows: (1) Identifying a list of potential outcomes through systematic reviews of TCM RCTs and 2 clinical registry databases, qualitative surveys on patients and healthcare professionals to form an outcome pool and nally establishing a preliminary checklist of outcomes. (2) Selection of stakeholder groups. (3) Representatives of stakeholder groups will be invited to participate in a two-round Delphi survey. (4) A face-to-face consensus meeting will be held to determine the nal COS-TCM-LC. Discussion: In this study protocol, we have followed the guidelines of COS-STAndardized Protocol (STAP) statement and checked the items in COS-STAndards for Developement (STAD). Developing a COS-TCM-LC will improve the quality of future RCTs on LC with the TCM interventions and promote better evidence-based clinical decision-making. Trial registration: This study is registered with the Core Outcome Measures in Effectiveness Trials database as study 1483 (http://www.comet-initiative.org/studies/details/1483).

from 56-90% depending on different pathologies. For advanced or late stage LC not suitable for surgical resection, radiotherapy and chemotherapy will be chosen as the standard treatment procedures [8], but the results are not satisfactory with high rate of recurrence. Radiotherapy and chemotherapy, though effective for some patients, have strong side effects and most of them only have palliative effect [3][4].
China's medical situation is to attach equal importance to traditional Chinese medicine (TCM) and Western medicine via integration of both treatments, especially for certain refractory tumor diseases.
TCM can be used in improving therapeutic e cacy, reducing toxicity, improving immunity of the body, as well as improving quality of life and prolonging survival time for LC [9][10]. It has been reported that TCM can enhance cytotoxic activity against LC via stimulating the host's immune response through inhibiting tumor cells' proliferation and promoting apoptosis, which will reduce side effects of chemoradiotherapy or gene therapy used in LC [11][12]. In a retrospective cohort study which included 111564 newly diagnosed LC patients in 2000 to 2009 from, Taiwan National Health Insurance Program database, patients using TCM as an adjuvant treatment showed a 32% decreased risk of death as compared to patients not using TCM [13].
Though there are many studies reporting the effectiveness of TCM treatment on LC, these results should be dealt with caution due to the lack of methodological quality in clinical trials [14]. One important aspect of poor methodological quality in clinical trials is the heterogeneous reporting of outcomes. Several problems concerning the choice and reporting of outcome measures for TCM clinical trials: (1) outcome measured and reported in clinical trials of the same condition varied greatly, with an existence of apparent selective reporting bias. (2) surrogate endpoints, such as biochemical markers were widely used, with few reporting on primary or long-term outcomes. (3) there is no agreed upon or standardized outcomes. (4) lack of agreed and standardized outcomes to measure TCM-related syndrome changes [ 15]. Selective reporting of outcomes resulted in di culty to compare and combine different trials, limiting the validity of meta-analysis and reducing the value of each clinical research. Hence, the selection of appropriate outcomes is crucial in designing clinical trials, thereby reducing waste in research [16].
A set of standardized outcomes, termed as a core outcome set (COS), which should be measured and reported, as a minimum in trials of the same healthcare area [17], can be utilized to help address this existing problem of heterogeneity in outcomes. Currently, there are 4 COS studies related to LC registered in the Core Outcome Set for Effectiveness Trials (COMET) initiative database. There were 2 studies relating to consensus of standard treatments for advanced stage NSCLC [18,19] and 2 studies on the development of COS. One study was to develop a COS in clinical trials using Chinese patent medicine as an adjuvant treatment for advanced NSCLC, but no published article was being recorded and could not be found. Another study done by International Consortium for Health Outcomes Measurement (ICHOM) focused on de ning a LC standard outcome set based on patient-reported outcomes [20]. However, the patient population was mainly from North America, Europe, Brazil and Australia, lacking a clear perspective from the low-and middle-income countries as well as the Asian and Chinese population. The purpose of this study is to develop a COS for clinical trials of TCM in LC (COS-TCM-LC) which can represent the perspectives of the stakeholders particularly in China where most TCM related trials are being carried out. The COS will reduce outcome reporting heterogeneity in future TCM clinical researches to support data synthesis in addressing the effectiveness of TCM treatment, which plays an important supplementary role in the treatment of LC.

Objective
The objective of this study protocol is to develop a core outcome set to be used in future TCM clinical trials for LC, with reference to a series of standards made by Core Outcome Measures in Effectiveness Trials Initiative (COMET) group [21][22][23].

Scope
The scope of the COS-TCM will include four main aspects based on the Core Outcome Set-STAndards for Development (COS-STAD) [23] as follows: (1) Population: All patients diagnosed with LC.

Steering Committee
The Steering committee will include 5 members, including 2 medical oncologists in the eld of LC (1 TCM expert and 1 western medicine expert), 1 methodologist, 1 policy maker and 1 COS developer. The committee will review and provide guidance at each stage of the study protocol.

Working group constitution
The working group will be made up of 10 individuals, including 4 professionals, 2 methodologists, 2 PhD students from the team of COS-TCM-LC development from Chinese clinical trial core outcome sets research center(ChiCOS) and 1 patient representative. The role of the working group will include organizing regular meetings, facilitating communication and hold discussion meetings to seek advice from the steering committee if there are any differences to be resolved.

Patient and public involvement (PPI)
Patients or their representatives will be involved in the semi-structured interviews, two rounds of Delphi survey and the nal consensus meeting.

Design
The development of this COS-TCM-LC will be conducted in 4 stages ( Fig.1): 1. Identifying a preliminary checklist of outcomes, after constructing a pool of outcomes and classifying into outcome domains (Stage 1).
Stage 1: Identifying a preliminary checklist of outcomes The rst step in the development of COS-TCM-LC is to collect all existing outcomes to form an outcome pool, which is an initial list of potential outcomes retrieved from TCM related clinical trials in LC. An outcome pool for TCM related clinical trials refers to a collection of outcomes that have been used in clinical trials concerning a certain disease or speci c disease type with application of TCM related interventions. The outcome pool can generate a preliminary list of outcomes which is used in Delphi process in the development of a COS. In this study, we use four steps which include a systematic review of published literature, a systematic review of clinical trial registration protocols, clinician questionnaire surveys and patient semi-structured interviews to generate a pool of outcomes in identifying the preliminary checklist of outcomes.
Step 1: Systematic review To identify outcomes used previously in clinical trials, we will perform a systematic review of randomized controlled trials (RCTs) for TCM.

Literature search and screening
A comprehensive, electronic search strategy of 3 English databases which include PubMed, Cochrane Library, and Embase as well as 3 Chinese databases which include the China National Knowledge Infrastructure CNKI , WanFang Database and SinoMed. The search time range will be from 1 January 2010 to 31 December 2020. Key terms used to guide the search will include 'lung cancer', 'small cell lung cancer', 'Large cell lung cancer', 'fei ji', 'clinical trial' and 'randomized controlled trial'. Two reviewers will independently screen the abstracts and titles of studies identi ed in the literature search.
Relevant studies will be selected according to the prespeci ed inclusion and exclusion criteria as listed in Table 1. Any disagreements will be resolved through discussion after a thorough reading of the paper or by consulting a third researcher.

Data extraction
Two reviewers will independently extract relevant data by reading the full texts from included studies. Data extraction will include the rst author's name, number of recruited patients, patients' age, TCM syndrome types, intervention details (medication name, course of treatment, treatment frequency and dosage), name or de nition of outcomes, outcome measurement instruments or methods, time-point at which the outcome is being measured. In addition, we will assess the quality of outcome reporting. As there is currently no consensus method for assessing the quality of outcome reporting, we refer to relevant COS development studies that have utilized the quality assessment of outcome measures reported [24][25][26]. The list of criteria to assess the reporting quality of the outcome measures is shown in Table 2. If the eligible study completely meets an item, it will receive 1 point. If it does not meet any item, it will receive 0 points. Step 2: Identifying additional potential outcomes from 2 clinical trial registry databases Two clinical trial registry databases, Chinese Clinical Trial Registry (ChiCTR) and ClinicalTrials.gov will be searched to include all RCTs on LC using TCM as an intervention to retrieve potential outcomes. The retrieval time is from inception to December 31, 2020. Two reviewers will independently screen the abstract and title of the studies identi ed in the clinical trial registry databases. If a study has been registered, completed or published, we will prioritize inclusion of the published full text to avoid duplication with published literature retrieved in Step 1. Relevant articles will be selected according to the inclusion and exclusion criteria shown in Table 1. The reviewers will independently extract relevant data by reading the full texts from included studies to determine its eligibility. Any disagreements will be resolved through discussion after a thorough reading of the paper or by consulting a third researcher.
An extraction table will be predesigned by the working group to collect data from the included studies. Data to be extracted will include country of registered organization/researcher, status of registered trial (ongoing or completed), ethics, funding resources, mode of intervention/s, name or de nition of outcomes, outcome measurement instruments or methods and the time-point at which the outcome is being measured.
Step 3: Patient semi-structured interviews and clinician questionnaire surveys Patients and clinicians will be invited to participate in semi-structured interviews and questionnaire surveys respectively. Recruitment will be carried at the respiratory and pulmonary department of inpatient wards and outpatient clinics in Tianjin University of TCM (TJUTCM) a liated First Hospital and TJUTCM a liated Second Hospital. The inclusion and exclusion criteria for the patient participants and healthcare professionals are shown in Table 3.
The recruitment sample size of patients and healthcare professionals will be 25 respectively. Healthcare professionals panel will include 13 TCM doctors and 12 western medicine doctors. We will solicit and respect the legitimate rights and interests of these subjects. If they agree to participate in the semistructured interviews, we will ask them to provide a signed written consent form. In addition, they can withdraw unconditionally from the study without affecting any other interests.
At the beginning of the interview, a quali ed and trained researcher of the working group will explain the study in plain language to prepare patients or their caregivers for the interview. The demographic and LC disease information of patients will be collected. At the same time, the trained investigator will ask the patient two questions to obtain the most important outcomes reported by patients (PRO) on LC.
Examples of the questions during the interview are as listed below: 1. "what is the most important outcomes for you in the treatment of LC " 2. "In the treatment of your LC, what aspects do you most hope to get better improvement " If the participants are not able to answer the questions, the investigator will provide the list of outcomes collected from the systematic review of published literature and registered studies for them to choose. The content of this semi-structured interview will be audio recorded.
For healthcare professionals, we allow them to have the freedom to list outcomes which are of importance to them for TCM treatment in LC using a questionnaire in an open-ended format. Participants will not be constrained by a de nite list of outcomes to choose from. Participants are required to list up to a maximum of 5 outcomes. The limit set on the selection number can help to achieve the selection of outcomes which are of high importance to healthcare professionals. Step 4: Constructing an outcome pool Outcomes collected from systematic reviews, clinical trials registry databases and qualitative surveys will be combined to form a pool of outcomes. An initial translation of audio-recorded qualitative surveys into texts will be performed and outcomes retrieved will be imported into an Excel table. This will be followed by different names of a similar outcome will be standardized and uni ed as one, which is done in accordance to the de nition of the outcomes in original studies or videos of the qualitative surveys. Any duplicates will be removed from the outcome list. Similar outcomes with overlapping de nitions will be merged. Finally, an outcome pool of LC using TCM interventions will be developed. Outcomes will then be classi ed 7 outcome domains, namely TCM syndrome or symptom, clinical symptoms and signs, physiological and biochemical indexes, quality of life, long term prognosis, economic evaluation and adverse events. In this step, two researchers will independently standardize, merge and group the outcomes, and then cross-check the results after completion. If there is any disagreement it will be resolved by discussion with a third researcher.
Step 5: Establishing a preliminary checklist of outcomes After constructing a pool of outcomes, an initial list of outcomes will be formed. The initial list of outcomes can be a long list. Implementing a long list of outcomes will deter participants in the Delphi survey (in the later stage) from completing and thus affect the response rate in the Delphi survey [21]. Thus, we need to reduce the length of the list when considering the practicality of the study. The preliminary checklist of outcomes will be determined based on the following (Fig.2): 1. If there are more than 80 outcome items in the pool of outcomes, speci c criteria will be applied to remove some of the outcomes from the pool of outcomes. The outcomes will be assessed according to the 6 principles, which include the clinical importance of the outcomes, the combination of disease and symptom, the recognizability of the outcome, the speci city, consistency and practicality of the outcome in the study. If at least 90% of the members voted for an outcome as "not important", the outcome item will be dropped from the list, with guidance by the Steering committee in an internal anonymous voting process. Finally, the retained outcomes will be approved by the Steering committee as the preliminary checklist of outcomes for COS-TCM-LC.
2. If there are 80 or less outcomes in the pool, all of the outcomes can be retained as items in the preliminary checklist.

Stage 2: Selection of stakeholder groups
The selection of stakeholder groups is crucial in the process of COS development [27][28][29]. Representatives of the stakeholder groups will determine the representation of a COS. In the study, the stakeholder groups include COS users, healthcare professionals (medical doctors and TCM doctors), clinical trialists, representatives of pharmaceutical enterprises, health policy makers, researchers, methodologists, patient representatives, COS developers and journal editors. A list of all potential stakeholders is shown in Box 1.

Box1 Stakeholder involvement
Healthcare professionals The Delphi survey will comprise of two rounds of electronic questionnaires. Delphi round 1 will include the generation of scores from each outcome in the preliminary checklist. Additional outcomes which are deemed to be important, and previously not included in the preliminary checklist, can be added into the list of items by participants. The added outcome items will be cross-checked to determine if these items are new and unique. The newly added items will be con rmed and added to the list of outcome items in Delphi round 2. Feedback scores from the rst round will be presented in Delphi round 2 and the scoring statistics from all participants in various stakeholder groups will be collected and analyzed.
Participants who have completed the rst round will be invited to score all items in the rst round as well as newly added items in Delphi round 2.

Delphi survey participants
There is currently no recommendations for the sample size of a Delphi survey. We aimed to include a range of stakeholders' perspective and ensuring an equal representation of healthcare professionals and patients. Healthcare professionals will include a fair mix of western doctors and TCM doctors. We will also ensure that patients in the early and late stage of LC currently or previously on TCM medication will be represented in the Delphi survey. The stakeholders will be invited to participate in the Delphi survey on a voluntary basis. We will invite 200 representatives identi ed through certain academic organizations such as China Association of TCM.

Delphi survey process
An electronic mail will be sent out to all participants (excluding patient representatives), which includes a link to the questionnaire survey. Each round of Delphi survey should be completed within four weeks. The patient representatives will complete the survey in a face-to-face interview with the trained researcher from the working group during outpatient consultation under the supervision of their medical doctor or TCM doctor.
Delphi scoring Participants will be asked to score each outcome according to the importance using a 9-point Likert scoring scale, where 1 to 3 is labeled as 'not important', 4 to 6 as 'important but not critical' and 7 to 9 as 'critical' [30]. Participants will have an option of 'unsure' if they are unable to assess the importance of the outcome items.
Delphi round 1 Participants are required to score every outcome.
Simultaneously, an open-ended question will be placed at the end of the questionnaire where participants can suggest additional outcomes which they think is necessary and important to be in COS-TCM-LC. After the rst round of Delphi the working group will proceed to calculate the number of respondents and the distribution of scores for each outcome item.
Outcomes which are de ned as 'important' (if at least 70% or more participants scored 4 to 9) will be retained. For the remaining outcomes, the working group will discuss and retain outcomes with controversial opinions. In addition, outcomes added will be cross-checked to con rm if they are new items. Newly added items will be retained. All of the retained outcomes will be included in Delphi round 2.
Delphi round 2 On completion of Delphi round 1, the following participant will automatically be included in round 2 of Delphi survey. In Delphi round 2, aggregated results of the number of respondents and distribution of scores, together with their own score for every outcome in round 1 will be presented to all participants. They are required to rescore all outcomes, and if the score of some outcome changed signi cantly, for example, a score of less than 4 (not important) to a score of 7 and above (critical), or vice versa, they will be required to provide a reason for the scoring changes.
After the second round, the working group will then proceed to generate statistics of the number of participants, distribution of scores and assessment of attrition bias. The changes in scoring between rounds will be examined and veri ed, as well as summarizing the reasons for the change in scores between rounds. The outcomes will be ranked according to the level of importance rating. Outcomes which have achieved consensus in at least one of the stakeholder groups will be included in the candidate list of core outcomes for the consensus meeting.

Stage 4: Consensus meeting
The objective of a consensus meeting is to determine the nal core outcome items in a COS development study. A one-day face-to-face meeting of approximately 20 individuals will be held to discuss the results from the Delphi survey to agree upon a nal COS. Apart from the Steering committee and the working group members, participants are mainly representatives of the stakeholder groups and senior experts in related elds who have completed 2 rounds of Delphi survey.
The meeting process will involve ve sections: (1) a short review of the COS study.
(2) summary of the scoring results of all outcomes from each stakeholder groups will be shown during the meeting, introducing all outcomes in the candidate list and obtaining the number of stakeholder groups which have achieved consensus.
(3) Participants will rate the candidate list of core outcome through anonymous voting.
(4) An outcome will be considered to have reached a consensus if 70% of participants vote in favour of the outcome to be included in the COS [31] . The determination of COS-TCM-LC will be via discussion of the voting results to achieve consensus. All participants have the rights to discuss any items, if there are any disputes, it will be settled via the guidance of the Steering committee. .
(5) Suggestions for later revision and promotion.

Consensus de nition
Consensus de nition is classi ed into 3 categories: 'consensus in', 'consensus out' and 'no consensus'. These consensus criteria will be used for outcomes to be retained or dropped at the end of Delphi round 2. 'Consensus in' is de ned if at least 70% of the participants in at least one stakeholder group score outcomes as 7 to 9, and will be given priority as a recommended item in the candidate list of core outcomes. 'Consensus out' is de ned when more than 70% of all participants score outcomes as 1 to 3. 'No consensus' refers to outcomes which does not meet 'consensus in' and 'consensus out' requirements (Table 4). The selection of 70% for 'consensus in' relatively broadens the criteria for outcomes to be included in the candidate list of outcomes for the consensus meeting in determining the nal COS-TCM-LC.

Discussion
As a complementary and alternative medicine, TCM plays an important role in the healthcare system [32].
With the increasing number of TCM clinical trials carried out, to develop COS-TCM will standardize the use of outcomes in TCM clinical trials and improve the quality of TCM evidence.
This study protocol has reached the standard and have a clear methodology in compliance to the 13 items of COS-STAP (Core Outcome Set-STAndardised Protocol) and 11 minimum standards of COS-STAD (Core Outcome Set-STAndards for Development) [22,23]. In addition to PPI, we have also included clinicians to participate in the qualitative semi-structured interviews. This will increase the feasibility and usefulness of the outcomes to be included in the nal COS for implementation into future clinical studies.
The protocol of COS-TCM-LC development portrays a standardized methodology which will serve as a reference for COS development guidance in China, especially in the eld of TCM. There is a necessity to strengthen the application and promotion of COS-TCM in China. Hence, the development of COS-TCM-LC will ensure more consistency in outcome reporting, reducing reporting bias, with a set of standardized outcomes, as a minimum, to be used in the speci c healthcare area, with additional TCM related characteristic outcomes, in the hope of increasing methodological quality in TCM related clinical trials to achieve international standards. manuscript; MZ and JZ provided supervision for all aspects of the project. All authors agreed on the nal draft manuscript for submission and are accountable for all aspects of the work.  Flow diagram on determining preliminary checklist of outcomes