The DES methodology is an individual agent-based strategy that allowed us to follow individual patients from 19 and through their lifetimes while allowing their individual sampled characteristics (age, cancer genesis, breast density) to determine their future pathway progression. These characteristics were sampled using published data and estimations developed in1. Once sampled, women follow the pathway described in the main paper (Fig. 1) and depending on a set of probabilities the patient may develop a tumour that may progress over time. The tumour may be detected in one of the patients 3-yearly visits depending on the sensitivity and specificity of the readers (adjusted to account for the woman’s breast density). Therefore, tumours may or may not be detected. If not detected they will continue to grow (in size and stage) based on a cancer growth model from Weedon-Fekjær et al., (2008), allowing for the model to simulate the impact of missed diagnosis or false negative results. The latter affect the patient’s quality of life and costs incurred. A detailed description of the process that women follow though the model pathway and details on the parameters including the costs and utilities, are described below and in Extended Data Tables 1, 2, 3 and 4.
Movements from the false positive and true negative stages
If false positive, the recall stage will reveal that the woman’s’ preliminary diagnosis was mistaken. Therefore, as with the true negative cases, the woman will be issued a three years’ invitation.
Movements from the cancer detected stage
As a simplification, the model did not follow the cancer disease progression and assumed that women who have screen detected cancer will either move to the death from cancer or to the death from natural causes end stages. This assumption implies that the women either die from the disease progression despite treatment or die from natural causes if recovered from cancer or die from natural causes before their cancer progression. Costs and outcomes derived from cancer are considered by stage and severity (more details below).
Movements from the false negative stage
As movements from the screen detected cancer stage, woman whose cancer was missed could move to death from cancer (if the undetected cancer grows symptomatic or progresses to cause death with or without treatment), death from natural causes (if the patient survives cancer or dies before disease progression), and to the invitation to next scan, if the undetected cancer growth does not manifest in a symptomatic disease or if its progression is slow at the time of being the missed diagnoses. Similarly, costs and outcomes derived from cancer are considered by stage and severity (more details below).
Invitation to next scan
Women are invited to a three-year scan if their mammography assessment was either, false positive, true negative or false negative. As the next scan invitation is to occur three years after the current scan, the woman may: die from natural causes before the next scan, develop cancer cells (carcinogenesis), or if they already have an undetected cancer growth, this would continue to grow. If alive, after three years have elapsed, the woman will receive an invitation to attend the next cycle of mammography according to the NHS scan service. She may accept or reject to attend. If, she decides to attend, the process described above will be repeated. If however the woman misses the three years invitation, she will be re-invited after three years. As before, during these three years, she could either, die from natural causes, develop cancer cells (carcinogenesis) or if already she has an undetected cancer this will continue to grow. For women whose cancer has continued to grow, if it becomes symptomatic, they will have the option to seek medical attention or wait for their next scan appointment.
The described process will continue until the woman dies either for natural causes, or due to causes related to a cancer progression.
Starting age and carcinogenesis
Although the model focuses on the NHS breast cancer screening, in which woman between the age of 50 to 70 receive an invitation for a mammography scan every three years, the model starting age is 19. The latter as the model uses the annual probability of a woman to develop cancer cells from inception (carcinogenesis). These cells may growth silently for many years before being detected or become symptomatic. In some cases, by the time the woman has her first scan, these cells may have increased their size or mutated and fall into an advanced cancer stage. This also allows for the possibility of women developing breast cancer and therefore requiring treatment prior to their involvement in the national screening programme.
The annual probability of developing cancer over the lifetime of the woman was estimated using data from the office of national statistics data 2012 21 and based on Gray et al., (2017) lifetime risk of developing cancer1.
Despite the model staring age of 19, the model would not account for any costs or QALYs until the women are eligible for the cancer screening programme (age 50). Similarly, only women who have carcinogenesis before the age of 50 but have not sought medical attention before their eligible age for screening are being considered.
Breast cancer growth
Breast cancer growth was estimated based on Gray et al., (2017) using the formulations used in the continuous growth model described in Weedon-Fekjær et al., (2008)1,12. As DES is a discrete time event model (rather than continuous), we evaluate the cancer growth every three years. This information is later used to determine the type and stage the cancer is at (type and stage subsection). Table 1 contains the initial parameters used to populate it and estimate it.
Detected and undetected tumours
Tumours can either be detected via the breast screening programme, which is triennial, the symptomatic breast service due to breast symptoms or as an incidental finding by another clinical team. A tumour can also go undetected, by being missed in screening service and being asymptomatic. If the tumour is missed, it either continues to grow until the next screening invitation or in some cases, it will progress further to an advanced condition which could lead to death. These instances will be determined by the stage and severity of the tumour when missed by the screening service. However, if the tumour presence becomes symptomatic the woman may seek medical attention between screening invitations. This was assumed mid-way through the three-year cycle.
Cancer type and stage
After a screen detected tumour, the model evaluates if the detected tumour is a non-invasive or invasive tumour, defined as ductal carcinoma in situ (DCIS). This is based on the probability of a detected tumour being a DCIS reported in Tan et al., (2013).22 If non-invasive, the tumour was assumed to remain constant, and although the patient was assumed to receive treatment, it was assumed not to be life-threatening. We assume that these tumours do not progress to an invasive disease. Although if undetected the tumour will not be treated and will not affect the overall life expectancy of the women, if detected, it was assumed that the patient will undergo treatment and incurred costs and quality of life decrements until healed.
If, however, the tumour is invasive, the model uses its size to determine the stage at diagnosis. We use the Nottingham prognostic index (NPI). This index takes in to account the size of the tumour, the number of lymph nodes involved and the tumour grade. We then use the probability of NPI group membership conditional on tumour size as reported by Kollias et al., (1999)13. All invasive cancers irrespectively of the NPI category have a probability of being detected at an advanced stage (or stage IV per the Metastasis classification system 23. As in Gray et al., (2017) these probabilities were also associated with the tumour size based on data from the NHS audit of screen-detected breast cancers (2013)1.
Cancer survival and background mortality
The model does not follow cancer progression, it assumes that cancer survivability is linked to the stage the cancer is when detected (cancer detected stage in Fig. 1). Given that the model is evaluating the impact on scan detection rate and accuracy, this assumption implies that cancers detected at earlier stages will have better prognosis, whilst progression and treatment once detected was assumed to be the same irrespectively of how or where the cancer was detected. Years remaining of life given stage at diagnosis were estimated based on Gray et al., (2017) using Fong et al., (2015) estimates.1,19 The only exception was DCSI, as these types of cancers were assumed to have the same survival as that of the general population.
General population survivability was based on the Office for National Statistics, (2019)24. Life expectancy for each woman in the cohort is estimated and assigned at the start of the simulation. This is adjusted in the event of the women developing a carcinoma at any stage (NPI, II or III or advanced). Therefore, risk of death from natural causes was present throughout the simulation.
Sensitivity and specificity
The scan accuracy was measured by the sensitivity and specificity of the independent readers, including those from the Mia AI technology and arbitration (assumed as an independent Radiologist reader). Data to populate the model was obtained from a retrospective multi-centre, multi-national clinical investigation that evaluated the effectiveness of the Mia AI technology (Extended Data Table 2)10.
Sensitivity was adjusted for breast density. We used the Volpara Density Group (VDG) to determine woman breast density. Scans from breasts with a VDG grade 1 will be easier to read than VDG 4, hence the sensitivity will be reduced. Since the data we used to populate the model is likely to randomly include woman with different breast density, we assumed no reduction in sensitivity for breast density VG1 with slight incremental penalties for VG2 to VG4. The reductions in sensitivity for breast with VG2 to VG4 were based on based on estimates by Gray et al., (2017)1 (Extended Data Table 1). To assign women with a breast density we used estimates from 14. This was done randomly at the start of the simulation.
Costs and Quality of life
Costs and QALYs were estimated in the model depending on each woman’s health status. Women who did not have a tumour during their lifetime (either invasive or non-invasive), were assumed to have a quality of life according to their age from the UK tariff 25. For women with a non-invasive and invasive tumour, we assumed a utility value based on the stage at detection and time from diagnosis: from diagnosis to 6 months, from 6 months to year one, from year one and up to year 9. If the women survive cancer, their utility will be returned to the national UK average according to their age. This information was based from estimates by Hall et al., (2015)16. If, however, the women die from cancer, during her last year of life we assumed a utility based on advance cancer, while the last 6 months will assume utility values for palliative care using estimates from Rautalin et al., (2018)18
Costs of treatment were calculated based on estimates based on grade from Hall et al., (2015) and Laudicella et al., (2016), for costs on the first year of treatment and year 1 to year 9, if the woman survives cancer or advanced and palliative care costs for their last year of life if not.16,26 Although Hall et al., (2015) and Laudicella et al., (2016) estimates were for grade (I, II, III) we assumed equivalence for NPI I to grade 1, NPI II to grade 2 and NPI III and advanced to grade 3.
Costs of the Mia technology were obtained from discussion with Kheiron Medical Technologies based on their potential price and costing estimates at the time of this study. The implementation of the AI technology requires a set-up cost, annual maintenance and a cost per scan read. All these items were included in the model accordingly. Mia was assumed to act as second reader. Cost of the first reader and those for the first and second reader for the standard practices were costed based on the time a consultant radiologist takes to evaluate a scan. These were assumed the same irrespectively of the treatment arm (Mia or standard practice). The hourly costs of these were obtained from the PSSRU 2019 20. The model base case scenario assumes that all scans are read by a consultant radiologist. This proportion is modified in the sensitivity analysis.
Other, common costs included were costs of the mammography, cost of arbitration and recall. Extended Tables 3 and 4 include the Utility and Costs values used in the model. All costs have been updated to 2020 GBP prices using the ONS consumer price index (CPI) health index17.
Reporting and Sensitivity analysis
We estimate cost-effectiveness using incremental cost-effectiveness ratio. The model accounts for uncertainty via a probabilistic sensitivity analysis (PSA). The variables and distributions used for this analysis can be found in Extended Data Tables 1, 3 and 4. Sensitivity and specificity were assumed fixed, however, to account for the potential influence of these variables, we preform several scenario sensitivity analyses. The main objectives of this analysis, however, was to estimate the potential combination of factors that will indicate if the Mia AI or SP may be cost-effective, depending on the results from the base case analysis. The variables that were evaluated in these scenarios are described in Extended Data Table 4.
First order uncertainty
The DES approach also requires additional considerations to avoid potential bias due to its stochastic nature. Therefore, one model run assumes 100,000 patients. This number was defined along with the mammography service as they indicated to be an average number of patients being seen by the service each year. This large number of patients also has the advantage to limit the bias of the stochasticity of the DES approach. However, this large number of patients, resulted in long running times, resulting in limiting the number of PSA iterations to 1,000. To evaluate the impact of this low number of runs, we estimated JackKnife confidence intervals on the incremental cost-effectiveness ratio to determine if the number of runs was sufficient to determine cost-effectiveness 27. Additionally, to limit the bias on the deterministic sensitivity analysis, this analysis was performed as probabilistic running 100,000 patients over 1,000 iterations. This will avoid potential bias due to a deterministic run.
Locum assumptions and detailed results
Cost of Locum was assumed based on data published in the Clinical Radiology UK workforce census 202028 assuming a £88 million total locum costs paid by the NHS. Around 20% of that sum was expected to be related to the breast cancer service (£17.6 million) to complete an average of 2 million studies (£8.80 per study or £4.40 per reader). This extra cost per reader was added to the current cost per scan incurred by the NHS (£5.90) to give the £10.30 per scan used.
The results of the use of 10% of Locum at the three different prices analysed (£8.80; £10.30 and £12.36) indicate that the use of Mia is cost-effective when compared against standard practice. These results indicate a probability of cost effectiveness of 56, 58 and 59%, respectively.
Although the cost savings by using Mia are relatively small (£6.75; £8.38 and £10.02 per woman), the cost difference would allow for an increase on the MRP that Mia could reach (maximum price at which Mia would be consider equally cost-effective as standard practice). If 10% of the service relies on Locum at a cost of £10.30 per scan the MRP of Mia would be up £6.28; £5.87 for a cost of £8.80 per scan and £6.69 when the cost is set at £12.36 per scan (against the MRP of £5.50 estimated in the base case scenario). The later can be roughly be indicative that an increase in 10% on the cost of the Locum increases the MRP in 3%.
The additional scenario where we assume that the Locum requirements were of 20%, indicate Mia is the cost-effective strategy with a probability of cost-effectiveness of 61%, while the MRP that Mia could reach £7.16 per scan. The latter indicates that for every 10 percentual points increase in the use of locum, the MRP will increase in 14%, while the probability of cost-effectiveness of Mia will increase by three percentage points on average.
Overall analysis of the results
The different scenario analysis, testing different specificity, sensitivity levels, impact of the price per scan, set-up and maintenance cost, offer an overview on the requirements for Mia (or a similar technology) to be a suitable alternative for the NHS service in the UK. A key element is the effectiveness of the device. If the technology proves to be as effective (or very similar) to standard practice, then a price up to £5.50 per scan plus a set up cost of £35,000 would make this technology suitable for the NHS. Further improvements in its specificity over standard practice would allow the MRP to reach over £8.80 per scan (an increase of 87%). The inclusion of maintenance costs has a relatively small impact on cost-effectiveness; however, the company must observe the maximum amount an individual Trust or the NHS will be willing to invest in the technology. A suitable comparison could be the current salary of a radiographer per year (including overheads and training). Different combinations of set-up costs, price per scan and a maintenance fee can be explored while maintaining cost-effectiveness (or indifference between the SP and the AI technology). If maintenance fees are considered, given almost identical effectiveness, a MRP of £4.72, a set-up fee of £35,000 and a maintenance below £17,000 per year may allow for indifference or cost-effectiveness for the technology.
Given the current pressure to the service, it is not unusual that they require the use of a Locum to meet demand, if that is the case, the use of an AI technology such as Mia instead would be the preferred option as the services using this technology as second reader will be the cost-effective strategy when all other variables are set to the base case scenario (equivalent sensitivity and specificity levels, set-up costs at £35,000). The MRP in such scenario will increase depending on the percentage of Locum required and cost per scan charged by the Locum.
Other limitations and assumptions to consider
Although the analysis performed was aimed at minimising the potential impact of the DES stochasticity, the small difference in favour of the Mia, found in the base case results may be relate to the later, as shown with the Jackknife CI of the ICER. While the base case results (Mia as a cost-effective strategy) are uncertain, our results suggest that assuming the current sensitivity and specificity performance (based on Sharma N et al.10 is at least equivalent to SP and therefore is a viable strategy for use in the NHS.
While the stage of the detected cancer is based on the relative size of the tumour, once detected the model only assumes treatment based on the stage upon detection. The model does not follow individual patient’s cancer treatment pathway. Similarly, the proportion of non-invasive tumours was based on data published by Tan et al., (2013)22. We assume that these tumours do not progress to an invasive disease. Although if undetected the tumour will not be treated and will not affect the overall life expectancy of the women, if detected, it was assumed that the patient will undergo treatment and incurred costs and quality of life decrements until healed.
The results are based on all scans being read by a consultant radiologist. Whilst the model can estimate different ratios between radiologist and radiographer, this was not explored due to running time restrictions, but it is expected that a lower ratio between Radiographers/Radiologist will increase standard of practice cost-effectiveness. Conversely, if the service is understaffed and scans are read by more expensive subcontractors this will increase the cost of SP and result in Mia being the cost-effective strategy.
The model does not account for QALY losses due to a false positive result. Given the small, estimated difference in QALYs between the intervention and control, adding a utility decrement due to a false positive result may lead to an increase in QALYs for Mia if the technology offers a higher specificity compared to standard practice which may then result in Mia being the cost-effective strategy even if sensitivity is lower.
Reference for extended material
21. Royal College of Radiologists. Clinical Radiology UK Workforce Census 2020 Report. R. Coll. Radiol. 72 (2021).
22. Tan, K. H. X. et al. Quantifying the natural history of breast cancer. Br. J. Cancer 109, 2035–2043 (2013).
23. Office of National Statistics. Office of National Statistics Cancer Incidence and Mortality in the United Kingdom—2008–10. London: (2012).
24. TNM Classification of Malignant Tumours. (Wiley-Blackwell, 2009).
25. Office for National Statistics. National life tables: UK. https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/lifeexpectancies/datasets/nationallifetablesunitedkingdomreferencetables (2019).
26. Self-Reported Population Health: An International Perspective based on EQ-5D. (Springer Netherlands, 2014). doi:10.1007/978-94-007-7596-1.
27. Laudicella, M., Walsh, B., Burns, E. & Smith, P. C. Cost of care for cancer patients in England: Evidence from population-based patient-level data. Br. J. Cancer 114, 1286–1292 (2016).
28. Iglehart, D. Simulating stable stochastic systems, V: Comparison of ratio estimators. Nav. Res. Logist. Q. 22, 553–565 (1975).
29. National Cancer Registration & Analysis Service. Routes to Diagnosis: Tumours Diagnosed 2006–2016. https://www.cancerdata.nhs.uk/routestodiagnosis/routes (2016).