Our findings demonstrate that in trials using follow-up through national cancer registration alone in England, Wales and Northern Ireland, 15% of tubo-ovarian cancer cases may be missed. Similarly, use of national death registration alone would result in 11% of tubo-ovarian cancer deaths being missed. The proportions of missing cancer registrations showed no decline over time unlike the deficit in disease specific death registrations. These rates could be halved if additional national electronic datasets such as hospital episode statistics are used. When extrapolating these results, two facts must be taken into consideration. We included cancer and death registrations of primary peritoneal cancer in ovarian cancer incidence and mortality statistics. This is not the norm adopted by UK Office of National Statistics or US SEER Registries. We obtained final follow-up data from the registries three months after the censorship date in order to fulfil reporting guidelines for the trial. Longer intervals from censorship to obtaining registry data are likely to reduce the rate in the last year.
Adjudication by an independent review process can improve the accuracy of cancer and death registrations. In those where a cancer registration was available, 9% of registry reported tubo-ovarian cancers would have inaccurate cancer site assignment. Additionally, 4% of the final outcomes confirmed cancers would have been missed as they were registered as non-ovarian cancers. Reliance on death registration alone, would have resulted in a similar proportion (7.8%) of deaths being wrongly assigned to tubo-ovarian cancers and 6.2% of ovarian cancer deaths being missed.
Strengths and Limitations
A key strength of our analysis is that we compared cancer diagnoses and causes of death reported by the national registries of England, Wales and Northern Ireland with those assigned through independent central adjudication in a population cohort of over 3000 women. Of the 202638 women, 202632 (99.9%) women were electronically flagged using their NHS number to the relevant national cancer and death registries. The multiple additional data sources (Table 1) we used to identify potential ovarian cancer cases further ensured that we had a complete population dataset of cases.
We did not restrict the study to women with the tubo-ovarian cancer specific ICD-10 codes (C56, C57.0, C48.1 and C48.2) but used 19 possible ICD-10 codes, in particular malignant neoplasm primary site unknown (ICD-10 C80) that might have potentially included tubo-ovarian cases.
The peritoneal cancers reported by cancer registries were included as tubo-ovarian in this analysis. While this is the norm among clinicians and researchers, these cancers are not included in national tubo-ovarian statistics. We did this to ensure that registry completeness rates were not underestimated due to the 2014 WHO revision of the primary site definition.8 As the latter is adopted by pathology departments the world over, it is likely that cancers previously denoted as primary peritoneal will be registered as tubo-ovarian.
A limitation is that the cases were diagnosed over a prolonged period between 2001 and 2014. To address this we explored time trends. While there were no time trends in missing cancer registration data, the completeness of death registration data improved over the years.
Use of UK cancer registry data alone would have allowed us to detect 85% of the tubo-ovarian cancer cases. This is an improvement from the 78% sensitivity that we reported in our previous trial where women were diagnosed with these cancers between 1986 and 1993.12 It is likely that the rates have improved further in the more recent years. Use of multiple national electronic data sets especially hospital administration records can augment these rates. A recent report on tubo-ovarian cancers diagnosed between 2004 and 2012 in Switzerland also found that the hospital registry data provided complementary information to that from the cancer registries.13 Sourcing data directly from live healthcare systems allows trials to overcome the latency in the national cancer registration systems. A median latency of 18 (range: 4–60) months to completion of incidence ascertainment was reported in a recent survey of European Network of Cancer Registries.14
The overall agreement regarding ovarian cancer site between OR and cancer registries was moderate. A key area of discrepancy was related to tumours defined by the ICD-10 code D39.1; neoplasms of the ovary of unknown or uncertain behaviour. On outcome review, the majority were reclassified as malignant neoplasm of the ovary (ICD-10 codes C56), either borderline epithelial ovarian or granulosa cell tumours. Most of these discrepancies likely reflect historical coding practices and classification systems used by regional registries prior to centralization of services in 2013. Furthermore, coding for ovarian cancers since 2011 uses the ICD-O-3 system,15 where the topographical code (site of cancer) is clearly separated from the morphological code which includes the behaviour (malignant/ benign). In the current system, as borderline ovarian tumours are classified as C56 (topographical code) with a morphology code of 1, there is less likelihood of the cases not being registered as ovarian cancer. However our data for 2012 to 2014 are not sufficiently large to confirm this.
The 9% of tubo-ovarian cancers reported by the registries that were classed as ‘not ovarian’ cancers on review were equally distributed between three main groups: (1) miscoding of a benign mass as a cancer, (2) other primary cancer, and (3) malignant neoplasm, primary site unknown (C80). The other primary group included just under a third of cancers that metastasized to the ovary, which might have led to the confusion. The ovaries are a frequent site for metastases, with 5-30% of ‘ovarian masses’ being secondary metastasis reported from non-ovarian cancers.16
The cause of death on death certificates is not always accurate17 due to a variety of reasons (inexperience of certifying physician, lack of sufficient time/information, coding errors). However, in our trial there was substantial agreement regarding primary site between death registry reported ovarian cancer deaths and those assigned by the review committee. This has also been reported in screening trials of other cancers (e.g. prostate) which used adjudication panels.18,19 Nonetheless 6.2% additional tubo-ovarian cancer deaths were identified through the adjudication process. A high proportion were registered as malignancies with an unknown primary site (C80) and all were advanced stage. In advanced cancer, where multiple sites are involved, it may be difficult to assign the primary site.20 The lack of consistency in the approach adopted by pathologists to assignment of primary site during our trial led to a proposal for unified criteria for tubo-ovarian site assignment21 which has since been adopted in international pathology reporting22 and clinical guidelines.23
The adjudication process used in the UKCTOCS trial is not dissimilar to that used in the Prostate Lung Colorectal and Ovarian Cancer screening trial.24 In contrast to the Cluster randomised trial of PSA testing for prostate cancer25 where data was abstracted into vignettes for expert review, we provided copies of original clinical notes to the ORC members. To minimise ascertainment bias any reference to the trial/randomisation arms was redacted in these documents. Review only occurred if the minimal required number of documents was available. ORC members who were highly experienced gynaecological oncology surgeons or pathologists used an algorithm developed specifically during the trial to assign cancer site. To ensure accuracy and reproducibility over the years and across assessors, we audited the reviews and introduced a classification algorithm (Supplementary Figure 3, see Additional File 5). The resource implications of central adjudication can be significant, as it requires many hours of highly trained senior staff time. Often as in our trial, it is challenging to estimate costs as reviewers usually volunteer their time and work outside normal office hours. To save the effort and cost of adjudication, we used individual as opposed to group adjudication, a work flow that involved more than one assessment only when there was discrepancy between the primary reviewer and registered ICD-10 code with group review limited to those where there was discrepancy between two reviewers.
Adjudication also provides information on other key variables such as staging and morphology which in the past have been poorly documented by registries26 but now improving.27 In our trial we observed a stage shift in the multimodal screening arm.6 This would not have been quantifiable using data from the registries alone without the detailed review of all available clinical documents as undertaken in UKCTOCS.
Kahan et al28 using a simulation model concluded that outcome misclassification can lead to biased treatment effect estimates and reduced power, potentially resulting in an erroneous conclusion regarding efficacy. The implementation of strategies to reduce misclassification is therefore of critical importance in clinical trials. In UKCTOCS, the main impact of adjudication was in improving accuracy of tubo-ovarian cancer. On death outcomes, there was no significant difference on the mortality impact of screening between the analyses which used ORC adjudicated disease specific deaths and the sensitivity analysis limited to deaths with the disease specific ICD-10 codes (ICD-10 C56, C57 and C48) for tubo-ovarian cancer.10