Evaluation of the clinical guidelines for the diagnosis and treatment of non-small cell lung cancer (NSCLC) in Iran via the guideline appraisal with AGREE II

doi:10.21203/rs.3.rs-3558208/v1

Introduction:

Guidelines are systematically developed. They are compiled by analyzing the available knowledge, and the country's situation and conditions that will enable health service providers to make the most appropriate decision for patients in different situations. Consequently, the purpose of this study was to evaluate clinical guidelines for diagnosing and treating non-small cell lung cancer (NSCLC) in Iran.

Methods

In this descriptive-analytical study, four experienced evaluators independently reviewed the AGREE II tool for Iran's NSCLC treatment and diagnosis guidelines, and at the end, for each of the items and in general, using the scaled domain score, we reviewed the quality and value of this guideline. "Highly recommended" refers to a standardized score of 50% in all areas. "Recommended with changes" refers to a standardized score of 50% in the overall assessment. Guideline that did not achieve a standardized score of 50% or more were designated as "not recommended."

Results

In the review of NSCLC diagnosis and treatment guidelines in Iran using the AGREE II tool, which revealed the following items related to the evaluated domains: In the scope and purpose, the standardized score is 70.8%, the stakeholder involvement is 75%, the rigor of development is 35.7%, the clarity of presentation is 83.3%, the applicability is 45.8%, and the editorial independence is 62.5%.

Conclusions

It takes considerable time and resources to develop, compile, and update quality guidelines. The development of guidelines in Iran has received much attention in recent years. According to this study, some changes should be made to the guideline for treating non-small cell lung cancer.

Pulmonology

Internal Medicine

Oncology

Diagnosis

Non-small cell lung cancer

Guideline

Lung neoplasms

Carcinoma

According to the World Health Organization, lung cancer is the most common type of cancer among men in terms of mortality, and it is the second most common type of cancer in terms of mortality. It is the third most common cancer in terms of incidence and second most common cancer in terms of mortality among women [1, 2]. Lung cancer has the highest death rate as compared to breast, prostate, and other cancers [3]. Among Iranian men, lung cancer incidence is 7.09%, and among Iranian women, it is 3.38% [4]. In 2012, about 1.82 million new lung cancer cases were reported, and about 1.56 million deaths from lung cancer were recorded [5]. Lung cancers commonly metastasize to the brain, bones, liver, contralateral lung, and adrenal glands [6]. Lung cancer can be caused by a number of factors such as radon gas, asbestos, people's genetics, air pollution, chronic and inflammatory lung diseases, some metals, ionizing radiation, and smoking [7–12].

For treatment purposes, lung cancer cells are classified based on the size and shape of the malignant cells. Their broad categories include non-small cell lung carcinoma (NSCLC) and small cell lung carcinoma (SCLC) [13]. Accurate staging of lung cancer at the beginning of the disease diagnosis is essential because the choice of treatment and the determination of the prognosis of the disease are wholly determined based on it [2]. Lung cancer is staged using the TNM staging system [4]. Chemotherapy, chemoradiation, and neoadjuvant chemotherapy are non-surgical treatments for non-small cell lung cancer [14].

In a specific clinical situation, guidelines help the treatment team and the patient make informed decisions about the most appropriate health care based on the best available evidence [15, 16]. It should be noted that as much as using standard and evidence-based guidelines can increase the health level of society, using low-quality guidelines will also cause irreparable damage in the different physical, financial, and psychological dimensions.

Four of the NSCLC treatment guidelines include: the national scientific societies (American Society of Endocrinology (ASCO), the Italian Society of Endocrinology (AIOM), the international (European Society of Endocrinology (ESMO), and the American Cancer Society (NCCN). ASCO guidelines are prepared through a well-defined method, making them a suitable tool for decision-making. However, it lacks regular updates, limiting its use in such rapid development fields [1]. The ESMO manual has the same limitations as the ASCO; however, more frequent online updates are provided. Although ESMO members have recently increased (even outside Europe), ESMO guidelines have been prepared with the aim of using them in European countries and, therefore, only include methods approved by the EMA [17]. The advantage of the NCCN guidelines is that they are constantly updated after FDA approval in the USA when a new method is introduced. These guidelines are essential in providing all available options for a particular clinical situation, especially when new drugs are being considered. However, they are weaker in providing comparative tools to assist physicians in decision-making and lack a strong methodological track record. In contrast, the AIOM guidelines are presented with a solid methodology. They are updated annually and include suggestions for all available options for a specific clinical problem in the Italian context. For this reason, their transferability outside the borders of Italy is a significant limitation [18].

The most standard tool available for evaluating guidelines is the appraisal of guidelines for research & evaluation (AGREE) tool [16]. This tool was updated in 2010 [19]. The tool "AGREE II" evaluates both the quality of the presentation of the content and the quality of some aspects of the recommendations. This tool examines the probability of the guide's success in achieving the desired behavioral outcomes (predictive validity of the guide).

Non-small cell lung cancer treatment plans are currently based on Iran's NSCLC diagnosis and treatment guidelines (The PDF is available [20]). This guideline has not been evaluated. Also, the most accurate tool for evaluating guidelines is AGREE II. Considering how the guidelines can improve the quality of care provided to patients as well as their outcomes, we decided to evaluate the "non-small cell lung cancer treatment guideline in Iran" based on AGREE II instrument for the first time.

Information Sources:

This descriptive-analytical study investigated Iran's NSCLC diagnosis and treatment guidelines that was compiled in 2018 [20]. Iran's national committee has drafted this guideline based on the world's most reliable guidelines like NCCN, ASCO, and ESMO. Doctors and specialists are generally the target audience for this guideline. This guideline is widely used in medical diagnostic centers and hospitals in Iran and has not been reviewed for quality.

Clinical practice guidelines are assessed using the AGREE II instrument. Assessors must respond to 23 items based on examples and instructions provided in the AGREE II manual on a scale of 1 to 7 for ''strongly disagree'' to ''strongly agree'. The assessors rate each item from 1 to 7 [19]. A quality assessment of guidelines is based on six domains: 1- Scope and purpose (criteria 1–3): Examines the general purpose of the guideline, the clinical questions covered, and the guideline's target population. 2- Stakeholder involvement (criteria 4–6): Examines how well competent stakeholders have developed the guideline and how much it reflects the views and wishes of users. 3- The rigor of development (criteria 7–14): Gathering and summarizing the evidence and examining the methods of compiling and updating the recommendations. 4- Clarity of presentation (criteria 15–17): Examine the text's verbosity and the guideline's structure. 5- Applicability (criteria 18–21): Examines the barriers and facilities of the application and ways to improve the application of the guideline. 6- Editorial independence (criteria 22–23): Examines the independence in drafting recommendations and the possible conflict of interests of the guideline drafting group. To increase the reliability of this tool in this study, two experienced pulmonologists and two experienced hematologists as evaluators familiar with the AGREE II tool, as well as the standards and methods of guideline development, independently evaluated the Iran's NSCLC guideline [20]. The AGREE II Instrument along with the Iran's NSCLC guideline PDF [20] was given to 4 evaluators (two experienced pulmonologists and two experienced hematologists) as evaluators familiar with the AGREE2 tool, as well as the standards and methods of guideline development, in September 2020. We gave them a week to answer and evaluate this guideline written by the national committee. These evaluators were selected from different hospitals in Tehran and did not work together in the same place.

Data Analysis:

Our database was built in Excel 2019 using the data we collected. Our qualitative data were defined as frequency and percentage. In order to calculate the grades for each of the areas of AGREE II, the following formula was used:

\(\text{T}\text{h}\text{e} \text{s}\text{c}\text{a}\text{l}\text{d} \text{d}\text{o}\text{m}\text{a}\text{i}\text{n} \text{s}\text{c}\text{o}\text{r}\text{e}=\frac{\text{o}\text{b}\text{t}\text{a}\text{i}\text{n}\text{e}\text{d} \text{s}\text{c}\text{o}\text{r}\text{e}-\text{m}\text{i}\text{n}\text{i}\text{m}\text{u}\text{m} \text{p}\text{o}\text{s}\text{s}\text{i}\text{b}\text{l}\text{e} \text{s}\text{c}\text{o}\text{r}\text{e}}{\text{m}\text{a}\text{x}\text{i}\text{m}\text{u}\text{m} \text{p}\text{o}\text{s}\text{s}\text{i}\text{b}\text{l}\text{e} \text{s}\text{c}\text{o}\text{r}\text{e}-\text{m}\text{i}\text{n}\text{i}\text{m}\text{u}\text{m} \text{p}\text{o}\text{s}\text{s}\text{i}\text{b}\text{l}\text{e} \text{s}\text{c}\text{o}\text{r}\text{e}} \times 100\text{%}\) . If the guideline achieved a standardized score of 50% or more in all areas, it was assigned to "highly recommended." If the guideline obtained a standardized score of 50% in the overall assessment, it was assigned to "recommended with changes." If the guideline did not get a standardized score of 50% or more in all areas or the overall evaluation, the option "not recommended" was assigned to it.

In the Iran's NSCLC guideline, four domains of “clarity of presentation,” “stakeholder involvement,” “scope and purpose,” and “editorial independence” were obtained high score, respectively. The "Applicability" and "rigor of development" domain scores were below 50% among six AGREE II domains. The answers of the evaluators to the review of Iran's NSCLC guideline are given in Table 1. Additionally, we reported the quality of each domain of this guideline by means of AGREE II using the example shown in Fig. 1.

Table 1

Summary of items and domains related to the AGREE II instrument for the review of Iran's NSCLC guideline along with appraisal answers.

Figure 1

An example of how AGREE II domain scores are calculated

Finally, four evaluators recommended the desired guideline with modifications. the overall evaluation of guideline quality and how to use the guideline in the clinical situation are shown in table 2.

Table 2

Overall Iran's NSCLC guideline Assessment.

We investigated the quality of Iran's NSCLC diagnosis and treatment guideline using the most standard tool (AGREE II) for evaluating guidelines. We found that except for the “Applicability” and “rigor of development” domain, the rest of the domains were highly recommended. Therefore, considering the importance of timely diagnosis of this type of cancer, it is recommended to use this guideline with some changes. Since it has been mentioned that if the guideline gets 50% or more in the overall evaluation of the standardized score, the option with recommended changes is assigned to it.

In general, the compilation of guidelines is done by writing original solutions and localization. It is believed that in the developing countries like ours, the localization of guidelines is more critical and a priority. Because in compiling an original solution and complete scientific coverage of the topic in question and having an opinion in that field, the editors must be equipped with vast and diverse methodological knowledge and sciences. In addition, developing an original solution suitable for a specific region requires rich support from epidemiological studies, clinical trials, review studies, and meta-analysis, which is lacking in our country; Therefore, developing an original solution in our country is impossible or very difficult for most areas [21]. In order for the following treatment activities to be effective, it is important to diagnose the stage of the disease. Therefore, there is no unique treatment model for lung cancer, as each patient's diagnosis stage, treatment options, age, and health status differ [22]. Since lung cancer does not have any clinical symptoms in the early stages and patients often go to the doctor when the disease is in advanced stages; as a result, the death rate related to cancer is high [23]; therefore, related guidelines need to be in a compilation as a section in the field of broader diagnostic procedures in old age. To collect information on the clinical guidelines for the diagnosis and treatment of non-small cell lung cancer in Iran, data were collected from various databases, which seems necessary to mention these databases at the beginning of the guideline. One of the most critical changes in the field of methodology is the time range of the search and the keywords used—the selection criteria of the evidence that needs explanations in this field.

A search of various sources revealed that this study is the first to evaluate the quality of Iran's NSCLC diagnosis and treatment guideline, which is one of its strengths. However, in the field of analysis of the obtained results, it is impossible to compare this study due to the lack of similar studies. Below is the overview of the shortcomings and problems of this guideline based on the AGREE II standard tool:

1. Scope and Purpose: In the present study, the general objective of the guideline and clinical questions were covered, and the target population of the guideline was clearly stated. The level of development, views, and wishes of the users were also considered. To some extent, the views of these people were given enough attention. Therefore, experts participated in the development of the relevant service. 2. Stakeholder Involvement: It has been the responsibility of oncologists, radiotherapy-oncology specialists, internal specialists, radiology specialists, pathology specialists, policymakers from the Ministry of Health and Medicine, health managers, statisticians, analysis specialists, and insurance managers to develop guidelines that include the diagnosis and treatment of non-small cell lung cancer. 3. Rigor of Development: In the accuracy and quality of the compilation methodology, the criticism of the evaluators on the methods of compilation and updating of the recommendations has been entered, and it has been less in the field of gathering and summarizing the evidence. As there is a belief that guidelines are formed based on summarizing and compiling evidence, the relevant guideline requires more information in gathering evidence to determine the patient's condition at the beginning of the disease for diagnosis. Furthermore, in this guideline, complete explanations about the compilation methodology are not provided. Iran's guideline on NSCLC diagnosis and treatment could boost its score if the authors specified the databases used. Additionally, they can provide evidence to support their recommendations. The guidelines should also include information on when and how they should be updated. It is also important to note how the editing team reached its final decision. The AGREE II score can be increased by incorporating these contents during the writing of this guideline. 4. Clarity of Presentation: This area examines the text's clarity and the guide's structure. The guideline has obtained an acceptable score in this area. Hence, all the specialists who performed the services were available, the priority was to provide the service by the radiotherapy-oncology specialist and the oncologist. 5. Applicability: this domain had low score among the other domain. According to the evaluators, the reason for this was due to the guidelines compiled systematically by analyzing the existing knowledge and the state and conditions of the country. These guidelines can be used in the simplest possible way to make the most appropriate decision for different patients in the conditions used differently. However, in the case of the guideline in question, it has not been possible to use this guideline easily. Maybe using the algorithm to show the diagnosis and treatment process is more effective in the quality of using the guidelines. In the area of applicability, it is suggested that the key recommendations be clearly defined. In addition, developing, compiling, and updating quality guidelines requires considerable time and resources. 6. Editorial Independence: Observance of the principle of impartiality in the formulation of guidelines is one of the main principles in the formulation of guidelines. The members of the drafting group should clearly state any compromise of their interests. On the other hand, the method of compiling the relevant guideline is one of the other ambiguous areas of this guideline.

Finally, Expanding information about each stage of the cancer diagnosis and treatment process can allow decision makers to compare diagnostic and treatment measures open mindedly. It should be noted that as much as using standard and evidence-based guidelines can increase the health level of society, using low-quality guidelines will also cause irreparable damage in various dimensions of physical, financial, and psychological. Although effective treatment is essential for the doctor and the patient in the first place, the guideline low quality may delay the treatment and progress of the disease.

Developing, compiling, and updating quality guidelines require considerable time and resources. In recent years, much attention has been paid in developing guidelines in Iran. This study showed that the guideline for treating non-small cell lung cancer is "recommended with some changes". In the "applicability" domain, maybe showing the process of diagnosis and treatment with the algorithm would be more effective. Adding information about quality levels of recommendations, supporting evidence for each recommendation, and updating the guideline in the "rigor of development" domain can improve Iran's NSCLC diagnosis and treatment guidelines.

Ethics approval and consent to participate:

This is a descriptive-analytical study for investigating Iran's NSCLC diagnosis and treatment guideline. The Baqiyatallah University of Medical Sciences Ethics Committee has waived the need for ethical approval in this study. Each participant in the study provided informed consent. All experiments were performed in accordance with relevant guidelines and regulations (such as the Declaration of Helsinki).

Consent for publication:

Not Applicable.

Availability of data and materials:

On reasonable request, the corresponding author's datasets are available.

Competing Interests:

Neither financial nor non-financial interests are disclosed by the authors.

Funding:

No funding has been allocated to this project study.

Authors contribution:

Conceptualization and design of the study were contributed by all authors. M.G, R.M.A, and A.G prepared the materials, collected the data, and analyzed the results. P.E and K.P wrote the first draft, and all authors commented on previous versions. The final manuscript was read and approved by all authors.

Acknowledgments

Not applicable.

Samet JM: Smoking kills: experimental proof from the Lung Health Study. Ann Intern Med 2005, 142(4):299–301.
Silvestri GA, Gould MK, Margolis ML, Tanoue LT, McCrory D, Toloza E, Detterbeck F: Noninvasive staging of non-small cell lung cancer: ACCP evidenced-based clinical practice guidelines (2nd edition). Chest 2007, 132(3 Suppl):178s-201s.
Choong NW, Vokes EE: Adjuvant and neoadjuvant therapy for early-stage non-small-cell lung cancer. Clin Lung Cancer 2005, 7 Suppl 3:S98-104.
Edge SB, Compton CC: The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol 2010, 17(6):1471–1474.
Chemotherapy in non-small cell lung cancer: a meta-analysis using updated data on individual patients from 52 randomised clinical trials. Non-small Cell Lung Cancer Collaborative Group. Bmj 1995, 311(7010):899–909.
Waller D, Peake MD, Stephens RJ, Gower NH, Milroy R, Parmar MK, Rudd RM, Spiro SG: Chemotherapy for patients with non-small cell lung cancer: the surgical setting of the Big Lung Trial. Eur J Cardiothorac Surg 2004, 26(1):173–182.
Arriagada R, Bergman B, Dunant A, Le Chevalier T, Pignon JP, Vansteenkiste J: Cisplatin-based adjuvant chemotherapy in patients with completely resected non-small-cell lung cancer. N Engl J Med 2004, 350(4):351–360.
Visbal AL, Leighl NB, Feld R, Shepherd FA: Adjuvant Chemotherapy for Early-Stage Non-small Cell Lung Cancer. Chest 2005, 128(4):2933–2943.
Wozniak AJ, Gadgeel SM: Adjuvant therapy for resected non-small cell lung cancer. Ther Adv Med Oncol 2009, 1(2):109–118.
Pirker R: Adjuvant chemotherapy of non-small cell lung cancer. Tanaffos 2012, 11(1):12–17.
Winton T, Livingston R, Johnson D, Rigas J, Johnston M, Butts C, Cormier Y, Goss G, Inculet R, Vallieres E et al: Vinorelbine plus cisplatin vs. observation in resected non-small-cell lung cancer. N Engl J Med 2005, 352(25):2589–2597.
Douillard J-Y, Rosell R, De Lena M, Carpagnano F, Ramlau R, Gonzáles-Larriba JL, Grodzki T, Pereira JR, Le Groumellec A, Lorusso V: Adjuvant vinorelbine plus cisplatin versus observation in patients with completely resected stage IB–IIIA non-small-cell lung cancer (Adjuvant Navelbine International Trialist Association [ANITA]): a randomised controlled trial. The lancet oncology 2006, 7(9):719–727.
Feld R, Rubinstein LV, Weisenberger TH: Sites of recurrence in resected stage I non-small-cell lung cancer: a guide for future studies. J Clin Oncol 1984, 2(12):1352–1358.
Berghmans T, Paesmans M, Meert AP, Mascaux C, Lothaire P, Lafitte JJ, Sculier JP: Survival improvement in resectable non-small cell lung cancer with (neo)adjuvant chemotherapy: results of a meta-analysis of the literature. Lung Cancer 2005, 49(1):13–23.
Graham ID, Harrison MB: Evaluation and adaptation of clinical practice guidelines. Evid Based Nurs 2005, 8(3):68–72.
Choi TY, Choi J, Lee JA, Jun JH, Park B, Lee MS: The quality of clinical practice guidelines in traditional medicine in Korea: appraisal using the AGREE II instrument. Implement Sci 2015, 10:104.
Cherny NI, Dafni U, Bogaerts J, Latino NJ, Pentheroudakis G, Douillard JY, Tabernero J, Zielinski C, Piccart MJ, de Vries EGE: ESMO-Magnitude of Clinical Benefit Scale version 1.1. Ann Oncol 2017, 28(10):2340–2366.
Bunn PA, Jr.: Early-stage non-small-cell lung cancer: current perspectives in combined-modality therapy. Clin Lung Cancer 2004, 6(2):85–98.
Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE et al: AGREE II: advancing guideline development, reporting and evaluation in health care. Cmaj 2010, 182(18):E839-842.
Clinical guidelines for diagnosis and treatment of non-small cell lung cancer (NSCLC) in Iran [https://oldbaharestan.iums.ac.ir/files/baharestan/files/riyee.pdf]
Zargham-Boroujeni A, Zoafa A, Marofi M, Badiee Z: Compilation of the neonatal palliative care clinical guideline in neonatal intensive care unit. Iran J Nurs Midwifery Res 2015, 20(3):309–314.
Akbarzadehbaghban A, Esmaeili M, Kimiafar K: Medical information management and assessment of direct costs of treatment of lung cancer. Health Information Management 2008, 5(2).
Kalemkerian GP, Akerley W, Bogner P, Borghaei H, Chow LQ, Downey RJ, Gandhi L, Ganti AK, Govindan R, Grecula JC et al: Small cell lung cancer. J Natl Compr Canc Netw 2013, 11(1):78–98.

Table 1: Summary of items and domains related to the AGREE II instrument for the review of Iran's NSCLC guideline along with appraisal answers.

Domain	Item	Content	Appraiser 1	Appraiser 2	Appraiser 3	Appraiser 4	Total	The scald domain score (%)
Scope and Purpose	1	The overall objective(s) of the guideline is (are) specifically described.	6	6	5	6	23	70.8
	2	The health question(s) covered by the guideline is (are) specifically described.	5	4	4	5	18
	3	The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.	6	5	6	5	22
Stakeholder Involvement	4	The guideline development group includes individuals from all relevant professional groups.	7	6	6	6	25	75
	5	The views and preferences of the target population (patients, public, etc.) have been sought.	6	5	6	5	22
	6	The target users of the guideline are clearly defined.	5	5	5	4	19
Rigour of Development	7	Systematic methods were used to search for evidence.	2	2	3	3	10	35.7
	8	The criteria for selecting the evidence are clearly described.	4	4	4	4	16
	9	The strengths and limitations of the body of evidence are clearly described.	2	2	3	1	8
	10	The methods for formulating the recommendations are clearly described.	2	2	2	2	8
	11	The health benefits, side effects, and risks have been considered in formulating the recommendations.	3	3	3	2	11
	12	There is an explicit link between the recommendations and the supporting evidence.	3	3	3	2	11
	13	The guideline has been externally reviewed by experts prior to its publication.	5	4	5	4	18
	14	A procedure for updating the guideline is provided.	2	2	1	1	6
Clarity of Presentation	15	The recommendations are specific and unambiguous.	6	6	6	6	24	83.3
	16	The different options for management of the condition or health issue are clearly presented.	6	6	6	5	23
	17	Key recommendations are easily identifiable.	7	7	6	5	25
Applicability	18	The guideline describes facilitators and barriers to its application.	3	3	3	2	11	45.8
	19	The guideline provides advice and/or tools on how the recommendations can be put into practice.	3	2	3	3	11
	20	The potential resource implications of applying the recommendations have been considered.	2	3	3	2	10
	21	The guideline presents monitoring and/or auditing criteria.	4	3	3	3	13
Editorial Independence	22	The views of the funding body have not influenced the content of the guideline.	6	6	6	6	24	62.5
Editorial Independence	23	Competing interests of guideline development group members have been recorded and addressed.	4	3	4	3	14	62.5

NSCLC: non-small cell lung cancer; AGREE II: the appraisal of guidelines for research & evaluation II

Table 2: Overall Iran's NSCLC guideline Assessment.

AGREE II Item		Appraiser 1	Appraiser 2	Appraiser 3	Appraiser 4
Overall Guideline Assessment	Quality	6	5	5	5
Overall Guideline Assessment	Recommendation	Yes, with modifications	Yes, with modifications	Yes, with modifications	Yes, with modifications

NSCLC: non-small cell lung cancer; AGREE II: the appraisal of guidelines for research & evaluation II

Evaluation of the clinical guidelines for the diagnosis and treatment of non-small cell lung cancer (NSCLC) in Iran via the guideline appraisal with AGREE II

Status:

Version 1

Abstract

Introduction:

Methods

Results

Conclusions

Figures

1. Introduction

2. Materials and Methods

3. Result

4. Discussion

5. Conclusion

Declarations

References

Tables

Status:

Version 1