Representativeness of CPGs
The systematic search strategy described above (Fig. 1) alongside the CPGs’ varied authors, collaborative organizations, and countries of origin (Table 2) highlight that the included studies represent most, if not all, MCC CPGs [3, 9, 12–13, 22, 26–28]. Moreover, the CPGs likely incorporate the most modern clinical evidence regarding MCC. All CPGs utilize the most recent (eighth) edition of the AJCC staging manual [1, 3, 9, 12–13, 22, 26–28]. Some CPGs explicitly list themselves as updates of prior versions, highlighting which evidence-based changes were made [3, 9, 13, 26]. Furthermore, the SITC guideline exclusively deals with one of the newest treatment modalities of MCC–immunotherapy [27]. The role of immunotherapy in MCC began receiving serious attention over the past decade when clinical trials began reporting preliminary but promising data [10, 17, 23, 30]. Altogether, the collection of CPGs included in this article are up-to-date representatives of the body of literature and will likely serve as frameworks for future CPGs.
Domain Strengths
Table 3 demonstrates that domains 4 (clarity of presentation) and 6 (editorial independence) are strong across almost every CPG. High clarity of presentation implies that main recommendations from each guideline are well demarcated and that multiple treatment options are explicitly presented. This is an important strength in the context of MCC, as dermatologists, otolaryngologists, plastic surgeons, and medical and radiation oncologists often work alongside each other when caring for these patients. For instance, during tumor boards, multiple specialists can easily reference these CPGs and quickly understand specialty-specific decision-making when deciding on the next step of MCC treatment. High editorial independence implies that neither funding nor conflicts of interest significantly altered each CPG’s recommendations. This ensures that management algorithms are dictated by evidence-based outcomes rather than financial incentives. Focus on best clinical practice is critical in MCC given its aforementioned aggressiveness and high rate of recurrence; accurate diagnosis with targeted treatment can also help limit the high healthcare costs imposed on this patient population [5–6, 18, 29, 33–34].
Domain Weaknesses
The selected CPGs are noticeably weaker across domains 2 (stakeholder involvement), 3 (rigor of development), and 5 (applicability). Upon closer inspection of individual appraiser values, the low scaled domain score for stakeholder involvement mainly arose from low item 5 scores (Table 1). This suggests that most CPG authors struggled to include perspectives from the target population. The exact reasons behind this are unclear but likely stem from the small patient population and costliness of gathering public opinion [2, 24]. However, despite domain 2’s low overall scaled score, most CPGs did well in including relevant professionals and defining target users during guideline creation.
The low rigor of development (domain 3) value followed a similar trend in that very low item 13 and low 14 scores (Table 1) brought down the overall scaled domain score. Regarding item 13, most, if not at all, CPGs had a distinct absence of any external expert review prior to publication. This is an important flaw in validity, as lack of outside opinion could cause groupthink amongst CPG authors and subsequently limit generalizability [11]. More variability was present among item 14 as half of the CPGs provided a clear protocol for incorporating new updates into their guidelines while the remaining half failed to mention it altogether. Authors who are interested in contributing to new CPGs would benefit from including detailed update instructions. The CPGs did well in other aspects of domain 3 such as: efforts to include systematic searches, inclusion/exclusion criteria and critical evaluation for evidence, and methods and expert appraisal behind recommendation formulation. This suggests that those domain features are ubiquitous among higher quality CPGs.
In contrast to the two previously discussed domains, the low applicability (domain 5) value was due to uniformly low scores across all domain items (items 18–21, Table 3). These items are all closely linked to the topic of implementing each CPG’s recommendations (i.e. facilitators, barriers, resource limitations, and periodic auditing). Issues in this domain may stem from the physician authors’ limited exposure to the medical supply chain, non-patient-related clinical tasks, and quality assurance metrics. Therefore, future CPG development boards may benefit by including a health economist who can give input about those realms [15].
Global Quality of Guidelines for MCC
In terms of overall appraisals, most MCC CPGs were determined to be of moderate quality or higher (Table 3). The EDF/EADO/EORTC guideline had the highest overall score with no significant deficiencies across any domain. Although the AHS guideline was also high quality, it was relatively lacking in domain 5 (applicability). The authors of this study therefore recommend updaters of current CPGs and authors of future ones reference the EDF/EADO/EORTC as a framework. It should be noted that the high quality guidelines (EDF/EADO/EORTC and AHS) are less widely known than some of their moderate quality counterparts (i.e. NCCN), showcasing how a guideline’s popularity does not necessarily dictate its developmental excellence [14]. The DEG guideline was the only low quality CPG, lacking in most of the six overarching domains. Interestingly, it did perform relatively well in domains 4 (clarity of presentation) and 6 (editorial independence). This illuminates how low quality guidelines may not only serve as a weaker example to learn from but also provide positive value in certain aspects.
Validity of Findings
The ICC values shown in Table 4 validate the reliability of this systematic evaluation’s findings. Having excellent interrater reliability across four of the six domains as well as good and moderate values for the remaining two, showcase that each appraiser (DL, CF, JH, TA) evaluated the CPGs with a near identical interpretation of the AGREE II instrument. This implies both that the training provided online by the AGREE Next Steps Consortium is comprehensive and that having the recommended number of appraisers is effective in eliminating biases between raters [4, 7]. The excellent ICCs present among domains 1 (scope and purpose), 2 (stakeholder involvement), 3 (rigor of development), and 5 (applicability) suggest that these topics require no additional discussion between appraisers. The good ICC value for domain 6 (editorial independence) may have stemmed from some mild ambiguity in how appraisers define funding or competing interests as influential. Thus, future appraisers may marginally benefit from additional discussion on this topic before grading CPGs. The moderate ICC for domain 4 (clarity of presentation) might have arisen from minor subjective bias in how each evaluator prefers data presentation within CPGs. Evaluators should consequently consider establishing objective checklists between themselves on what is considered “clear” beforehand.
Study Limitations
As with any study, it is important to note the associated limitations. Beginning with the literature search, there is possible publication bias from non-published negative data and lack of public access to institution-specific MCC treatment protocols. Excluding five non-English articles also may have introduced minor amounts of selection bias. Another important limitation lies with inherent subjectiveness of the AGREE II instrument. Despite the appraisers being specifically trained to use the tool and having the electronic manual on-hand while evaluating CPGs, the use of a Likert scale relies on subjective judgment to differentiate between point values [20]. The impact of this subjectivity was likely mitigated by the strong interrater reliabilities described above but cannot be eliminated altogether. The scaled domain scores referenced when creating overall quality impressions are also unevenly subject to interrater fluctuations. For example, domain 6 (editorial independence) contains only two items while domain 3 (rigor of development) has eight. So, despite having four reviewer scores for each item, the scaled domain score for domain 6 has six fewer data points in its calculation than that for domain 3. Finally, it should be noted that the AGREE II instrument is not designed to evaluate the validity of each CPG’s recommendations. The overall quality assessments are consequently limited to the developmental methodology, presentation, and clarity of each CPG. Even low quality CPGs should therefore not be completely discounted as they may provide valuable pearls of clinical management despite lacking various components in the final version.
In conclusion, the majority of CPGs for MCC are of acceptable quality with the potential to standardize management of the disease. We recommend use of the EDF/EADO/EORTC guideline as a developmental framework with the AHS guideline as a valid alternative. These suggestions are based on an objective, validated measurement tool with a high degree of interrater consistency. Future updates or new CPGs may benefit from garnering patient and public perspectives, inviting external expert review, and considering available resources and implementation barriers during their creation.