The Value of Different Types of Economic and Budget Analysis for Informing Real World Decision Making: the Case of Cervical Cancer Screening in South Africa

Objectives South Africa recently updated its cervical cancer screening policy, which we used as a case study to compare the outcomes of different methods of economic evaluation and their potential contributions to decision making. Methods We applied three evaluation methods – cost-effectiveness analysis (CEA), budget impact analysis (BIA) and multi-criteria decision analysis (MCDA) – to six strategies for screening and treatment of cervical pre-cancer lesions in HIV-positive women. Strategies included screening with visual inspection with acetic acid (VIA); cytology, followed by colposcopic biopsy; and HPV DNA testing. Strategies also differed by number of visits for screening and management, and treatment (cryotherapy or LEEP). Results Ranking of strategies differed by evaluation method. HPV DNA testing with LEEP ranked highest in the CEA but had the largest budget impact. VIA 1- visit ranked highly across the methods. Cytology was dominated in the CEA but was more affordable than HPV DNA testing, and scored lowest on the MCDA despite being the method chosen in the policy for the scale up of services. Conclusions Standard economic evaluations often rely on CEA for priority setting. In our study, the difference in strategy rankings between the evaluation methods demonstrates the tension between CEA, budget constraints, and other decision-maker priorities. While MCDA may be best placed to offer multi-faceted input, it struggles to resolve this tension when policy decisions are made over a number of sequential steps.


Introduction
The most popular approach to guiding resource allocation decisions by formal economic evaluation is costeffectiveness analysis (CEA) (1). This method compares the costs and health effects of different interventions to current practise or other interventions, typically expressed as an incremental cost effectiveness ratio (ICER). An intervention may be considered cost-effective if its ICER falls beneath a speci ed willingness-to-pay threshold representing a society's willingness-to-pay for an improvement in health (2)(3)(4). The World Health Organization's WHO-CHOICE project in 2002 suggested a threshold of 1-3 times gross domestic product (GDP) per capita per disability-adjusted life year (DALY) averted. While this approach has been widely used, it has been criticized for not taking affordability into account; not re ecting a true willingness-to-pay; not aiding in decision making, as a substantial number of interventions are deemed cost-effective; and not re ecting true opportunity cost (1,3,5). A possible way to avoid these limitations is to de ne thresholds by considering the supply side by de ning spending thresholds based on the health system's current productivity under a xed budget, with productivity de ned as the cost per unit of health gained. Any spending diverted to a new intervention is then viewed as an opportunity cost (2).
Another approach for explicitly incorporating affordability into decision making is budget impact analysis (BIA) (6,7). CEA and BIA require some of the same data and methodology, but BIA estimates an intervention's nancial costs in the short to medium term from the payer's perspective, and does not consider health outcomes (7,8). Thus, while it may not be considered a full economic evaluation, it provides information on whether an intervention is affordable but none on value for money.
Another evaluative framework for evidence-based priority setting, known as multi-criteria decision analysis (MCDA), includes decision criteria such as completeness and consistency of reporting evidence, improve in safety and tolerability in addition to cost, effectiveness and affordability (9,10).
In 2017 South Africa updated its cervical cancer screening policy (11,12). Cervical cancer is the second most commonly diagnosed cancer and the leading cause of cancer-related deaths among South African women (13).
Most cervical cancer is caused by persistent human papillomavirus (HPV) infection (14,15). HIV co-infection increases susceptibility to HPV and the risk of HPV persistence and progression to cervical cancer (16,17). South Africa has the largest HIV epidemic in the world and a high rate of HPV infection (12). A major goal of the updated policy was to expand cervical cancer screening, diagnosis, and treatment of pre-cancerous lesions. The policy recommended that HIV-positive women be screened at the time of HIV diagnosis and every 3 years thereafter.
Policy makers considered visual inspection with acetic acid (VIA), cytology (Pap smear, including using liquid based cytology) and HPV DNA testing as screening options; cryotherapy and loop electrosurgical excision procedure (LEEP) were considered as treatment options. The policy called for a continuation of cytology as the screening method of choice for both HIV-positive and HIV-negative women, with liquid-based cytology and HPV DNA testing phased in as funding allowed, as well as LEEP as the preferred treatment option and cryotherapy where LEEP services are unavailable, despite the limited availability of cryotherapy in South Africa.
Our objective was to evaluate and compare the ranking of cervical cancer screening and treatment strategies in HIV-positive women in South Africa using three different decision-making frameworks: CEA, BIA, and MCDA.

Methods
Under each analytical framework, we considered six different strategies for the screening and treatment of cervical pre-cancerous lesions in HIV positive women, guided by the policy options under consideration by decision makers in 2017.

Comparison strategies
Strategy 1 was VIA with cryotherapy treatment for screen-positive women, or LEEP if a woman was not eligible for cryotherapy, in 1 visit; Strategy 2 was 2-visit VIA with cryotherapy (or LEEP if ineligible); Strategy 3 was 3-visit cytology, followed by diagnostic testing with colposcopy and biopsy for screen-positive women, and treatment with cryotherapy (or LEEP if ineligible); Strategy 4 was 3-visit cytology, followed by colposcopy and biopsy for screenpositive women, and treatment restricted to LEEP; Strategy 5 was 2-visit HPV DNA testing followed by treatment with cryotherapy-+y (or LEEP if ineligible); Strategy 6 was HPV DNA testing followed by treatment with LEEP only (Table 1). For strategies involving cryotherapy, we assumed that 83% of women were eligible for cryotherapy (i.e., had lesions that did not extend to the endocervix and covered less than 75% of the cervix) (18); women ineligible for cryotherapy received LEEP.

Data sources
Data on the success rates of screening and treatment interventions were collected in two clinical trials conducted in Johannesburg, South Africa (19,20). Both trials enrolled HIV-positive women only. The screening trial compared the effectiveness (i.e., proportion of cervical intraepithelial neoplasia grade 2 or higher (CIN2+) detected) of VIA, cytology, and HPV DNA testing. Effectiveness was measured as the proportion of cases of cervical intraepithelial neoplasia grade 2 or higher (CIN2+) detected. The treatment trial compared cryotherapy versus LEEP for treatment of cervical dysplasia. Treatment success, or cure, was de ned as the absence of lesions at 12 months (19,20).
We obtained provider cost and health care utilization data for these strategies from published cost-effectiveness studies conducted at the same sites and in parallel with the clinical trials mentioned above (21,22). Personnel, consumables, equipment and laboratory costs were included, but building costs were excluded. We obtained costs incurred by patients for screening and pre-cancer treatment as well as cervical cancer treatment from a separate cost-effectiveness study comparing cervical cancer screening and treatment for HIV-positive women in South Africa (18). Patient costs include both direct costs, such as travel costs, as well as the opportunity cost of lost time, valued as the median income of the sample.

Economic evaluation
For the CEA, we used a Monte Carlo simulation model previously developed to t the natural history of HPV infection and cervical cancer among HIV-infected women in South Africa (18). Girls entered the model at 9 years of age, were assumed to be infected with HIV at 20 years and diagnosed at age 25 years. HPV incidence was based on age, and probabilities of HPV genotype-speci c acquisition were calibrated to age-speci c HPV prevalence data among HIV-infected women in South Africa. Transition probabilities between HPV-related health states, including HPV clearance, progression to precancer (i.e., CIN2, CIN3), and progression to invasive cervical cancer were strati ed by HPV genotype and duration of HPV infection. Costs were evaluated from the provider as well as patient perspective and calculated in 2017 USD. ICERs were calculated by dividing the incremental cost by the incremental effectiveness of each strategy relative to the next most costly strategy after eliminating strategies that were dominated (i.e. more costly and less effective, or have a higher ICER than a more effective strategy). Based on these ICERs, we determined the value of the interventions by considering a) a commonly used threshold of 1-3 times of per capita GDP, and b) a threshold based on supply side opportunity cost. Under the rst threshold, strategies are considered very cost-effective if they fall below per capita GDP, which in 2017 was $6,180 in South Africa. The opportunity cost threshold for South Africa has been estimated at between $1,175 and $4,714 (23).
Next, we calculated the budget impact of each strategy over ve years starting in 2017 at full implementation scale. We used the same cost and effectiveness data as for the CEA, but applied these data to the estimated population of HIV-positive women in South Africa. Age-strati ed transition probabilities between states in the BIA were based on literature (19,(24)(25)(26). The BIA compared strategies to the baseline equivalent of the current budget, whereas the CEA uses a baseline of no screening and treatment of pre-cancer. The "current budget" re ects expected costs if screening and treatment remain consistent with the old policy and documented coverage.
Finally, we conducted a multi-criteria decision analysis (MCDA), using a questionnaire with select criteria from the EVIDEM framework (10). Criteria for inclusion were determined through analysis of literature and past policy decisions as well as current decision-making processes and included domains such as disease impact, intervention effectiveness and budget impact (Table 4). We used criteria weights established for an EVIDEM MCDA focused on health care interventions to be offered in the private sector in South Africa (27); this study assembled a panel of experts including doctors, pharmacists and nurses with at least one year's experience in health policy decisionmaking for a health plan.
To implement the survey and obtain scores relevant for our strategies, we selected potential participants purposively based on their expertise or their recent experience in drafting the updated cervical cancer policy, including experts from the South African National Department of Health and National Treasury, clinicians, and health economists with experience in cervical cancer care in South Africa. Participants were emailed the questionnaire and relevant information to assist with scoring (based on data in Tables 2, 3, 5-9). All ve selfadministered the survey between November 2018 and April 2019. Scores were entered per criterion and multiplied by the criterion-speci c weights in order to arrive at a total, weighted score. The survey is included as Appendix A.

Results
We present the main results from the three economic evaluation methods as well as a summary of the rankings of the strategies across the methods. Additional outputs from the models used as inputs to the MCDA are contained in Appendix B. Table 5 shows the results of the cost effectiveness analysis, including the total lifetime cost per woman, as well as the average population life expectancy based on screening starting at the age of 25. One-visit VIA and HPV DNA testing with LEEP were the only e cient strategies; other strategies were more costly and less effective. While VIA 1-visit has the lowest ICER, HPV DNA testing with LEEP would be considered the optimal strategy, as it is the most effective strategy (in terms of life years saved) with an ICER below both the standard threshold of $6,180 and the opportunity cost threshold of between of $1,175 and $4,714.

Budget impact analysis
The budget impact, in terms of the average annual costs for national cervical screening and treatment of HIVpositive women using the six strategies, is shown in Table 6. With the current budget, the low screening rate leads to a high burden of cancer, at a cost of $39m. In contrast, using VIA as the screening method would result in cost savings of approximately $11m over baseline as screening shifts away from cytology and more cancer cases are prevented due to higher screening coverage. While the HPV DNA testing strategy is the most effective, it also has the largest impact on the budget, increasing the total budget needed by 82% over the baseline.

Multi-criteria decision analysis
The average and total weighted MCDA scores in each domain for each strategy are presented in Table 7. Cytology scored relatively poorly compared to VIA and HPV DNA testing, as it was less effective than HPV DNA testing and costlier than VIA, both of which were heavily weighted criteria. Cytology scored highest when considering the quality of evidence. HPV DNA testing scored highest in those criteria that considered characteristics of the strategy itself, such as effectiveness and improvement in patient tolerability. This is likely due to its superior effectiveness in terms of the reduction in the lifetime risk of cervical cancer. Cytology with cryotherapy scored higher than VIA in the "Intervention" criteria (as listed in table 4), but scored poorly in the economic criteria, resulting in its lower total score. HPV DNA testing scored particularly poorly in the economic criteria due to its large impact on the health budget. Table 8 summarizes the ranking of the strategies based on each of the evaluation methods. Each method ranks a different method the highest. HPV DNA testing with only LEEP was the optimal strategy in the cost-effectiveness analysis, while VIA 1-visit had the lowest budget impact and HPV DNA testing with cryotherapy/LEEP scoring highest in the MCDA. Both cytology strategies were dominated in the CEA and scored lowest on the MCDA but only had a moderate effect on the budget.

Discussion
Economic evaluations of health care interventions are used routinely today to prepare policy decisions, but different methods of evaluation can lead to different recommendations. This study highlights the types of and possible variations in outcomes that one might encounter when using three different kinds of economic evaluations.
HPV DNA testing with LEEP was the optimal strategy regardless of which willingness-to-pay threshold was used. However, CEA does not provide guidance on whether a new, albeit cost-effective, intervention is affordable. In this analysis, VIA had the smallest impact on the ve-year budget. In fact, it was cost saving when compared to current practice. In contrast, HPV DNA testing would have the largest impact on the budget, exceeding the costs of cytology by $20 million per year.
While economic considerations are important, they are not the only issues considered by policymakers when deliberating on new health interventions. MCDA takes this into consideration, by including criteria such as the quality of evidence demonstrating effectiveness and the severity of the disease addressed by the intervention. Our MCDA showed that respondents felt that the clinical evidence for cytology was superior, but VIA performed better when they considered economic factors. HPV DNA testing with cryotherapy or LEEP as treatment scored the highest in the MCDA, scoring well in intervention such as effectiveness, improvement in safety and tolerability. The quality of evidence for HPV DNA testing was also scored higher than for VIA.
While weighting each criterion in the MCDA the responders to the survey seemed to reveal their own implicit weighting based on their background experience. Those with a stronger economics background differentiated their scores between strategies within the economic criteria, while the respondents with a clinical background tended to do the same within the intervention criteria such as effectiveness and tolerability. This implicit weighting could potentially have had more of an effect in the economic criteria due to the differences in the scoring scales across all questions in the survey. Considering this, the scales used to score questions in the MCDA most certainly had an impact on the nal result. This difference in value systems between experts and stakeholders has previously been noted as an issue which needs to be addressed (28).
Consideration also needs to be given as to how these different methods would practically t into the policy determination process, which in our experience happens sequentially, with different criteria being of importance at different time points in the decision-making process. For example, in the development of the updated cervical cancer screening policy in South Africa, VIA was excluded by decision makers early in the process for due to a number of factors, including perceived ineffectiveness. There were evidence-based concerns as the study used to inform VIA performance for the CEA and BIA found relatively favourable sensitivity, in part because the comparator was colposcopic biopsy (which is also a visual method and thus highly correlated with VIA). After VIA was excluded, decision makers debated between shifting from cytology to HPV DNA testing (as well as shifting to new cytological methods). At this point in the decision-making process a BIA was conducted to ensure affordability.
Ultimately, the policy included shifting to new cytology methods in the near term, with the plan of introducing HPV DNA as additional resources become available.
This decision was supported given our BIA results. Although HPV DNA testing was more cost-effective than cytology, it was extremely costly when modelled at the national level. In contrast, the decision to choose cytology was not supported in the MCDA analysis where it scored lowest. This raises the question of whether the exercise of scoring by criterion in an MCDA can add value to a decision-making process as, while each of the criteria included are important to decision makers, in practice they are applied sequentially and in a binary fashion, almost as an algorithm. In other words, an early assessment of an intervention not being effective or affordable can end the decision-making process, without additional consideration being given as to whether it is cost effective or has better population impact. A more stream-lined and transparent process using a range of publicly communicated decision criteria and evaluation processes, such as those used as part of the recommendation process of the UK National Institute for Health and Care Excellence (NICE) or Thailand's Health Intervention and Technology Assessment Program (HITAP), would be an improvement on current decision making. Such a more centralised process would be a worthwhile consideration for South Africa as it seeks to implement a new health technology assessment structure as part of its plans to implement universal healthcare in the shape of national health insurance [29].
This study has a number of limitations. The rst is that the CEA and BIA analyses used different methods to estimate the number of cervical cancer cases. The CEA model strati ed transition probabilities by HPV genotype and duration of infection, while the BIA strati ed transition probabilities by age. The CEA and BIA did not include capital costs and will thus not provide exact estimates of the budget consequences of each testing and treatment strategy. For the MCDA we used existing weights from a survey of private sector representatives as eliciting weights from public sector players was beyond the scope of this study. We assumed that public sector weights might have resulted in a higher importance being placed on economic criteria; however, as the private sector sample from which we obtained the weights included clinicians alongside decision makers in a private health insurance organisation, the effect is not likely to be large. Importantly, in the CEA results were very sensitive to the assumed number of visits, loss to follow-up between visits, test performance, and costs. Finally, our ndings are not intended to guide actual policy, but rather as a stylized example to demonstrate differences in policy rankings when different decision-making tools are used.
Despite these limitations, our study has shown how the use of different methods of evaluation can lead to different policy recommendations. While an intervention may be deemed cost-effective it may not be seen as affordable, or even desirable by policy makers or experts, and those interventions that are dominated in a CEA may still be desirable policies. This illustrates the tension between CEA, budget constraints and other decision criteria, highlighting the need for further research on the values that policy makers and the general public attach to health decision criteria and the trade-offs between them.

Declarations
Ethics approval and consent to participate MCDA survey respondents provided consent to participate. Ethics approval was obtained from the Human Research Ethics Committee at the University of the Witwatersrand and the study was determined to be exempt by the O ce of human Research Ethics at the University of North Carolina.

Consent for publication
Not applicable Availability of data and materials The datasets during and/or analysed during the current study available from the corresponding author on reasonable request.

Competing interests
The authors declare that they have no competing interests.

Funding
This study has been made possible by support from the President's Emergency Plan for AIDS Relief (PEPFAR) through USAID under the terms of Cooperative Agreements AID-674-A-12-00029 and 72067419CA00004 to HE 2 RO. The contents are the responsibility of the authors and do not necessarily re ect the views of PEPFAR, USAID or the United States Government. CvR also received funding through the NIH Fogarty International Center Grant #R25TW009340.

Authors' contributions
CvR, NLD and GMR conceptualised the work. CvR, NLD and GMR assisted in the analysis of results for each evaluation method. NGC performed analysis for the CEA. JM provided expert opinion and assisted analysis of MCDA. CvR, GMR and NLD drafted the manuscript and NGC and JM substantially revised it. All authors read and approved the nal manuscript