Completeness and accuracy of national cancer and death registration for outcome ascertainment in trials - an ovarian cancer exemplar

doi:10.21203/rs.3.rs-31549/v2

Download PDF

Research

Completeness and accuracy of national cancer and death registration for outcome ascertainment in trials - an ovarian cancer exemplar

https://doi.org/10.21203/rs.3.rs-31549/v2

This work is licensed under a CC BY 4.0 License

Journal Publication

published 25 Jan, 2021

Read the published version in Trials →

You are reading this latest preprint version

Background: There is a trend to increasing use of routinely collected health data to ascertain outcome measures in trials. We report on completeness and accuracy of national ovarian cancer and death registration in the United Kingdom Collaborative Trial of Ovarian Cancer Screening (UKCTOCS).

Methods: Of the 202638 participants, 202632 were successfully linked and followed through national cancer and death registries of Northern Ireland, Wales and England. Women with registrations of any of 19 pre-defined ICD-10 codes suggestive of tubo-ovarian cancer or notification of ovarian/tubal/peritoneal cancer from hospital episode statistics or trial sites were identified. Copies of hospital and primary care notes were retrieved and reviewed by an independent outcomes review committee. National registration of site and cause of death as ovarian/tubal/peritoneal cancer (C56/C57/C48) obtained up to three months after trial censorship was compared to that assigned by outcomes review (reference standard).

Results: Outcome review was undertaken in 3110 women on whom notification was received between 2001 and 2014. Ovarian cancer was confirmed in 1324 of whom 1125 had a relevant cancer registration. Sensitivity and specificity of ovarian/tubal/peritoneal cancer registration was 85.0%(1125/1324; 95%CI 83.7%-86.2%) and 94.0%(1679/1786; 95%CI93.2%-94.8%), respectively. Of 2041 death registrations reviewed, 681 were confirmed to have a tubo-ovarian cancer of whom 605 had relevant death registration. Sensitivity and specificity was 88.8%(605/681; 95%CI86.4%-91.2%) and 96.7%(1482/1533; 95%CI95.8%-97.6%%) respectively. When multiple electronic health record sources were considered, sensitivity for cancer site increased to 91.1%(1206/1324; 95%CI89.4%-92.5%) and for cause of death 94.0%(640/681; 95%CI91.9%-95.5%).

Of 1232 with cancer registration, 8.7%(107/1232) were wrongly designated as ovarian/tubal/peritoneal cancers by the registry and 4.0%(47/1172) of confirmed tubo-ovarian cancers were mis-registered. In 656 with death registrations, 7.8%(51/656) were wrongly assigned as ovarian/tubal/peritoneal cancers whilst 6.2%(40/645) of confirmed tubo-ovarian cancer deaths were mis-registered.

Conclusion: Follow-up of trial participants for tubo-ovarian cancer using national registry data will result in incomplete ascertainment particularly of site due in part to the latency of registration. This can be reduced by using other routinely collected data such as hospital episode statistics. Central adjudication by experts though resource intensive adds value by improving accuracy of diagnoses.

Trial registration: ISRCTN22488978, date of trial registration: 6/4/2000

Translational Medicine

Internal Medicine

Integrative & Complementary Medicine

outcomes review

adjudication

randomised controlled trial

ovarian cancer

screening

UKCTOCS

registry

Currently most trials use multiple sources for outcome ascertainment such as case record forms, patient questionnaires and electronic national health datasets. The latter includes cancer and death registrations, hospital administrative records (e.g. hospital episode statistics in the UK), national audit data and specialised health datasets. This is often followed by central adjudication to ensure consistency and minimize bias.¹ Central review requires medical records to be retrieved, checked for completeness, redacted with regards to any reference to randomisation group and patient identifiers and the collated information reviewed by a specialist committee. Given the expensive and time-consuming nature of this process, there is a clear need to evaluate alternative strategies.

There has been an explosion in the use of electronic national datasets for clinical research. As the data quality of such resources improve and access processes for research become more streamlined, this is likely to be a less resource intensive and less costly option for outcome ascertainment in trials. Use of cancer and death registration alone is especially relevant to large randomised controlled trials of cancer screening where cancer site and disease specific cause of death (CoD) are primary outcome measures. Data from the U.S. National Lung Screening Trial (NLST) suggests that using death certificate CoD alone (18%; 95% CI: 4.2-25.0) would not have impacted on the published (20%; 95% CI: 6.7-26.7) lung cancer mortality reduction where the process included central adjudication.² This was also noted in the Finnish prostate cancer screening trial.³ However, other trials have raised concerns about completeness and accuracy of such data.⁴ In the Health Insurance Plan of New York breast screening trial⁴, there was no screening benefit based on death certificates alone whereas analysis performed using adjudicated data showed a significant effect. This raises the need for further investigation before universal adoption.

In the United Kingdom Collaborative Trial of Ovarian Cancer Screening (UKCTOCS)^5.6follow-up relies on multiple data sources including national electronic health related datasets with expert adjudication (outcomes review committee) being used for confirmation of ovarian cancer diagnosis and CoD. This was felt necessary as ovarian cancer often presents late with widely disseminated disease, heightening the possibility of disease misclassification. In addition, decreased investigation in older women, accident and emergency presentation and mortality within the first month of presentation could contribute to ovarian cancer being reported as a malignant neoplasm of unknown origin (ICD-10 code C80)⁷. The trial data provides an opportunity to determine the completeness and accuracy of national ovarian cancer and death registration in England, Wales and Northern Ireland using outcomes review as the reference standard.

Between April 2001 and September 2005, 202638 postmenopausal women aged 50-74 from the general population were recruited. They were randomised to annual screening by transvaginal ultrasound (50639) or a multimodal strategy using serum CA125 interpreted by the Risk of Ovarian Cancer Algorithm (ROCA), followed by ultrasound as a second-line test (50640) or control (no intervention; 101359).^5,6The full trial protocol is accessible at https://www.ctu.mrc.ac.uk/studies/all-studies/u/ukctocs/. Trial registration is ISRCTN22488978. This analysis is based on data collected prospectively up to the end of the initial follow-up phase of UKCTOCS (2001-2014).

Multiple sources were used for follow-up of trial participants (Table 1).⁶ Women who provided written consent for follow-up through national registries were flagged using their NHS number, date of birth and address for cancer and/or death registrations with NHS Digital (England and Wales) and the Northern Ireland Cancer Registry and Business Services Organisation Health and Social Care Northern Ireland (BSO Northern Ireland), respectively. Censorship data for the initial analysis of trial data was 31^st December 2014.⁶ The last notification prior to data freeze for trial outcome analysis was received for both cancer and deaths for England and Wales on March 25^th, 2015 (NHS Digital); deaths for Northern Ireland on April 9^th, 2015 (BSO, Northern Ireland), cancers for Northern Ireland on April 15^th, 2015 (Northern Ireland Cancer Registry). For women resident in England, data was also available for 2001-2012 from Hospital Episodes Statistics (HES) and up to 2014 from the National Cancer Intelligence Network (NCIN). Additionally, all women were sent two follow-up postal questionnaires, 3-5 years after randomisation and in 2014. The UKCTOCS coordinating centre also received direct notification from the 13 trial centres, and trial participants or their relatives.

Outcomes Review Committee

An outcomes review committee (ORC) was set up to confirm ovarian cancer diagnosis and death. The committee consisted of two pathologists with specialist interest in gynaecological cancer and two gynaecological oncologists. The ORC members were independent of trial conduct and blinded to the randomisation group.

Identification of cases and collation of evidence

Cases were identified for OR based on a cancer/death registration with any one of 19 preselected International Classifications of Disease (10^th Revision, ICD-10) codes (WHO 2003)⁷ (Supplementary Table 1, see Additional file 1) or a notification of possible ovarian cancer through the other sources listed above. The preselected ICD-10 codes were defined at trial initiation as those potentially associated with an underlying ovarian cancer.⁶Cases with an ICD-10 C80 (n=936) death registration and a cancer registration of a non-ovarian malignancy were reviewed by a designated gynaecological oncologist and excluded from the below process.

Dedicated Coordinating Centre personnel initiated collection of pertinent documents from National Health Service (NHS) Trusts, general practices, local health authorities, private hospitals, and hospices. These included copies of medical records, histology, cytology reports, operative notes, diagnostic imaging reports, hospital letters and discharge summaries, multidisciplinary meeting summaries, chemotherapy/radiotherapy notes and autopsy reports. Procurement of the histology report was obligatory in cases where pathological examination had been undertaken. The documents were organized in chronological order and any direct or indirect mention of randomization group redacted, prior to submission to ORC. The minimum dataset required for review comprised three separate documents which could include in addition to those listed above, death certificates and cancer registrations.

Outcome review

Figure 1 shows a schematic for outcomes review. For all cases, a cancer review form (Supplementary Figure 1, see Additional file 3) and where applicable, a death review form (Supplementary Figure 2, see Additional file 4) was completed. If the reviewer deemed the documentation adequate, then a cancer diagnosis or CoD was assigned. An algorithm with detailed rules for site allocation (Supplementary Figure 3, see Additional file 5) ensured robust, reproducible and transparent assignment in various clinical scenarios. In all cases where there was a discrepancy regarding ovarian cancer classification between the reviewer and cancer or death registry, the case was forwarded to a second ORC member for an independent review. If both reviewers were in agreement, the assigned diagnosis/CoD was accepted. If there was disagreement, a third ORC member independently reviewed the case and a consensus decision was made based on the majority view. If a reviewer was unable to make a definitive diagnosis, the reviewer could request either further information or review of pathology slides, or a case discussion at an ORC meeting.

Outcomes review initially used the WHO 2003⁷ rules for classifying cases into tubo-ovarian or primary peritoneal. More recently all the peritoneal cancers were reviewed and reclassified as tubo-ovarian cancers based on WHO revised 2014 criteria⁸. This states that peritoneal cancers can only be diagnosed when there is no demonstrable gross or microscopic ovarian or tubal involvement.⁹

Analysis

All trial participants were included in this analysis (Figure 2). Sensitivities and specificities of cancer/death registration of ovarian cancer obtained up to three months post censorship date were calculated, using central adjudication as the reference standard. We also calculated the sensitivity if all available national electronic health related datasets sources (Table 1) were used. The definitions for sensitivity and specificity were as defined by the STARD 2015 guidelines (http://www.equator-network.org/reporting-guidelines/stard)

In addition, in all cases with a cancer registration and/or death certification, overall agreement of disease site and CoD between cancer/death registries and OR was assessed using the Cohen’s kappa (κ) statistic¹⁰, with confidence intervals calculated using 1000 bootstrap samples. Zero weighting was applied to all off-diagonal cell entries.

We also considered the false negative rate (FNR=1-sensitivity) of cancer registration and death certification over time, with ORC confirmed diagnosis and CoD as the gold standard and summarising with a test for proportional trend (Stata package ptrend). This test takes a standard χ² test of cancer registration (or death certification) versus year, on k=(row-1)×(column-1) degrees of freedom (df) and partitions the χ²value into a linear component (trend test) on 1 df and a ‘departure’ from linearity component on k-1 df. We excluded the year 2014 from the cancer registration analysis, as during exploratory analysis there was a large number of missing registrations.

Funding

The current analysis was funded by a peer reviewed NIHR HTA grant (16/46/01) and The Eve Appeal. The funders were not involved in the analyses nor in the writing of this report.

Baseline characteristics of the trial population have been previously described.¹¹ Of the 202638 postmenopausal women, 202632 (99.9%) women were electronically flagged for cancer and death registrations in the relevant national registries of England, Wales and Northern Ireland (Figure 2).

Performance characteristics of cancer and death registration for tubo-ovarian cancer

Our algorithm identified 3110 women requiring outcomes review for tubo-ovarian cancer diagnosis. The ORC confirmed 1324 as having tubo-ovarian cancer. Of them, 1125 women had an ovarian, tubal or primary peritoneal cancer registration. Overall sensitivity and specificity of ovarian, tubal or primary peritoneal cancer registration within 3 months of censorship date was 85.0% (1125/1324; 95% CI 83.7%-86.2%) and 94.0% (1679/1786; 95% CI 93.2%-94.8%), respectively (Figure 2). In an additional 81 of the 1324 women, we received an ovarian, tubal or primary peritoneal cancer notification from other national datasets (death registration 51, HES 27, NCIN 3). If all available national electronic health related datasets (Table 1) were considered, sensitivity would be 91.1% (1206/1324; 95% CI 89.4%-92.5%). Figure 3a and Supplementary Table 2 (see Additional file 2) show proportion of missing cancer registrations over time. Excluding the last year (2014), there was no statistical evidence of a trend in the proportion of missing cancer registrations (p=0.261) or any other form of temporal variability (p=0.549).

Our algorithm identified 2214 women who died and required outcomes review for CoD. Only one of these women did not have a death registration as she had died outside the UK. The ORC confirmed 681 as having died due to tubo-ovarian cancer. Of them, 605 women had an ovarian, tubal or primary peritoneal cancer CoD registration (Table 2). Overall sensitivity and specificity of ovarian, tubal or primary peritoneal cancer death registration within 3 months of the censorship date was 88.8% (605/681; 95% CI 86.4%-91.2%) and 96.7% (1482/1533; 95% CI 95.8%-97.6%%) respectively (Figure 2). In the remaining 76 women, 29 women had an ovarian, tubal or primary peritoneal cancer registration, six had an ovarian cancer HES record. If all available national electronic health related datasets (Table 1) were considered, sensitivity would be 94.0% (640/681; 95% CI 91.9%-95.5%). The false negative rates by year are presented in Figure 3b (Supplementary Table 2, see Additional file 2). We found strong evidence of a negative linear trend in proportion of missing death certificates (p=0.00002) but no further departure from that linearity (p=0.474).

Agreement of registry and outcome review assigned tubo-ovarian cancer diagnosis

There was a relevant ICD-10 cancer registration in 1474 of the 3110 women who underwent outcomes review for potential tubo-ovarian cancer (Table 2). 107 (8.7%; 107/1232) of 1232 cases registered as ovarian, tubal or primary peritoneal cancer were reclassified on outcome review as malignant neoplasm of unknown origin (2.7%; 33/1232), no cancer (2.1%; 26/1232) and other primary cancer (3.9%; 48/1232). Over one third were endometrium (41.6%; 20/48) About one fourth (27.1%; 13/48) of the other primary cancers were metastatic to the ovary.

In addition, 47 (4.0%; 47/1172) of the 1172 women confirmed as having tubo-ovarian cancer by outcomes review would have been missed (Table 2). Almost three-fourths of these women were registered as neoplasms of the ovary of uncertain behaviour (ICD-10 D39.1). The majority of these were reclassified on outcomes review as borderline epithelial or non-epithelial ovarian cancer (ICD-10 C56.0). Majority (80.3%; 10/12) of the missed invasive epithelial cancers were at advanced stage (Table 3). The overall agreement (85.4%; 1259/1474) between disease site ascribed by the cancer registries and those designated by the ORC was moderate (Cohen’s kappa (κ) 0.55; 95% CI 0.5-0.6).

Agreement of registry and outcome review assigned ovarian cancer deaths

At censorship, 2041 cases had undergone outcomes review and had a relevant ICD-10 death registration. Of the 656 ovarian cancer death registrations, 92.2% (605/656) were confirmed on outcomes review. The CoD in 51 (7.8%; 51/656) of 656 women registered as having died of ovarian, tubal or primary peritoneal cancer were reclassified on outcome review as non-ovarian. This include malignant neoplasm of unknown origin (51.0%; 26/51), other malignancy (37.3%; 19/51) or non-cancer causes (11.8%; 6/51). Over half of the other cancer deaths were due to endometrial cancer (52.6%; 10/19) (Table 2). In addition, 40 (6.2%; 40/645) of 645 women confirmed by outcomes review as having died of tubo-ovarian cancer were registered as having died of non-tubo-ovarian cancer causes (Table 3). The agreement (86.3%; 1763/2041) between cancer registry and OR was substantial (Kappa 0.78; 95% CI 0.76-0.80).

Main Findings

Our findings demonstrate that in trials using follow-up through national cancer registration alone in England, Wales and Northern Ireland, 15% of tubo-ovarian cancer cases may be missed. Similarly, use of national death registration alone would result in 11% of tubo-ovarian cancer deaths being missed. The proportions of missing cancer registrations showed no decline over time unlike the deficit in disease specific death registrations. These rates could be halved if additional national electronic datasets such as hospital episode statistics are used. When extrapolating these results, two facts must be taken into consideration. We included cancer and death registrations of primary peritoneal cancer in ovarian cancer incidence and mortality statistics. This is not the norm adopted by UK Office of National Statistics or US SEER Registries. We obtained final follow-up data from the registries three months after the censorship date in order to fulfil reporting guidelines for the trial. Longer intervals from censorship to obtaining registry data are likely to reduce the rate in the last year.

Adjudication by an independent review process can improve the accuracy of cancer and death registrations. In those where a cancer registration was available, 9% of registry reported tubo-ovarian cancers would have inaccurate cancer site assignment. Additionally, 4% of the final outcomes confirmed cancers would have been missed as they were registered as non-ovarian cancers. Reliance on death registration alone, would have resulted in a similar proportion (7.8%) of deaths being wrongly assigned to tubo-ovarian cancers and 6.2% of ovarian cancer deaths being missed.

Strengths and Limitations

A key strength of our analysis is that we compared cancer diagnoses and causes of death reported by the national registries of England, Wales and Northern Ireland with those assigned through independent central adjudication in a population cohort of over 3000 women. Of the 202638 women, 202632 (99.9%) women were electronically flagged using their NHS number to the relevant national cancer and death registries. The multiple additional data sources (Table 1) we used to identify potential ovarian cancer cases further ensured that we had a complete population dataset of cases.

We did not restrict the study to women with the tubo-ovarian cancer specific ICD-10 codes (C56, C57.0, C48.1 and C48.2) but used 19 possible ICD-10 codes, in particular malignant neoplasm primary site unknown (ICD-10 C80) that might have potentially included tubo-ovarian cases.

The peritoneal cancers reported by cancer registries were included as tubo-ovarian in this analysis. While this is the norm among clinicians and researchers, these cancers are not included in national tubo-ovarian statistics. We did this to ensure that registry completeness rates were not underestimated due to the 2014 WHO revision of the primary site definition.⁸ As the latter is adopted by pathology departments the world over, it is likely that cancers previously denoted as primary peritoneal will be registered as tubo-ovarian.

A limitation is that the cases were diagnosed over a prolonged period between 2001 and 2014. To address this we explored time trends. While there were no time trends in missing cancer registration data, the completeness of death registration data improved over the years.

Interpretation

Use of UK cancer registry data alone would have allowed us to detect 85% of the tubo-ovarian cancer cases. This is an improvement from the 78% sensitivity that we reported in our previous trial where women were diagnosed with these cancers between 1986 and 1993.¹²It is likely that the rates have improved further in the more recent years. Use of multiple national electronic data sets especially hospital administration records can augment these rates. A recent report on tubo-ovarian cancers diagnosed between 2004 and 2012 in Switzerland also found that the hospital registry data provided complementary information to that from the cancer registries.¹³ Sourcing data directly from live healthcare systems allows trials to overcome the latency in the national cancer registration systems. A median latency of 18 (range: 4–60) months to completion of incidence ascertainment was reported in a recent survey of European Network of Cancer Registries.¹⁴

The overall agreement regarding ovarian cancer site between OR and cancer registries was moderate. A key area of discrepancy was related to tumours defined by the ICD-10 code D39.1; neoplasms of the ovary of unknown or uncertain behaviour. On outcome review, the majority were reclassified as malignant neoplasm of the ovary (ICD-10 codes C56), either borderline epithelial ovarian or granulosa cell tumours. Most of these discrepancies likely reflect historical coding practices and classification systems used by regional registries prior to centralization of services in 2013. Furthermore, coding for ovarian cancers since 2011 uses the ICD-O-3 system,¹⁵ where the topographical code (site of cancer) is clearly separated from the morphological code which includes the behaviour (malignant/ benign). In the current system, as borderline ovarian tumours are classified as C56 (topographical code) with a morphology code of 1, there is less likelihood of the cases not being registered as ovarian cancer. However our data for 2012 to 2014 are not sufficiently large to confirm this.

The 9% of tubo-ovarian cancers reported by the registries that were classed as ‘not ovarian’ cancers on review were equally distributed between three main groups: (1) miscoding of a benign mass as a cancer, (2) other primary cancer, and (3) malignant neoplasm, primary site unknown (C80). The other primary group included just under a third of cancers that metastasized to the ovary, which might have led to the confusion. The ovaries are a frequent site for metastases, with 5-30% of ‘ovarian masses’ being secondary metastasis reported from non-ovarian cancers.¹⁶

The cause of death on death certificates is not always accurate¹⁷ due to a variety of reasons (inexperience of certifying physician, lack of sufficient time/information, coding errors). However, in our trial there was substantial agreement regarding primary site between death registry reported ovarian cancer deaths and those assigned by the review committee. This has also been reported in screening trials of other cancers (e.g. prostate) which used adjudication panels.^18,19 Nonetheless 6.2% additional tubo-ovarian cancer deaths were identified through the adjudication process. A high proportion were registered as malignancies with an unknown primary site (C80) and all were advanced stage. In advanced cancer, where multiple sites are involved, it may be difficult to assign the primary site.²⁰ The lack of consistency in the approach adopted by pathologists to assignment of primary site during our trial led to a proposal for unified criteria for tubo-ovarian site assignment²¹which has since been adopted in international pathology reporting²²and clinical guidelines.²³

The adjudication process used in the UKCTOCS trial is not dissimilar to that used in the Prostate Lung Colorectal and Ovarian Cancer screening trial.²⁴ In contrast to the Cluster randomised trial of PSA testing for prostate cancer²⁵ where data was abstracted into vignettes for expert review, we provided copies of original clinical notes to the ORC members. To minimise ascertainment bias any reference to the trial/randomisation arms was redacted in these documents. Review only occurred if the minimal required number of documents was available. ORC members who were highly experienced gynaecological oncology surgeons or pathologists used an algorithm developed specifically during the trial to assign cancer site. To ensure accuracy and reproducibility over the years and across assessors, we audited the reviews and introduced a classification algorithm (Supplementary Figure 3, see Additional File 5). The resource implications of central adjudication can be significant, as it requires many hours of highly trained senior staff time. Often as in our trial, it is challenging to estimate costs as reviewers usually volunteer their time and work outside normal office hours. To save the effort and cost of adjudication, we used individual as opposed to group adjudication, a work flow that involved more than one assessment only when there was discrepancy between the primary reviewer and registered ICD-10 code with group review limited to those where there was discrepancy between two reviewers.

Adjudication also provides information on other key variables such as staging and morphology which in the past have been poorly documented by registries²⁶ but now improving.²⁷ In our trial we observed a stage shift in the multimodal screening arm.⁶This would not have been quantifiable using data from the registries alone without the detailed review of all available clinical documents as undertaken in UKCTOCS.

Kahan et al²⁸ using a simulation model concluded that outcome misclassification can lead to biased treatment effect estimates and reduced power, potentially resulting in an erroneous conclusion regarding efficacy. The implementation of strategies to reduce misclassification is therefore of critical importance in clinical trials. In UKCTOCS, the main impact of adjudication was in improving accuracy of tubo-ovarian cancer. On death outcomes, there was no significant difference on the mortality impact of screening between the analyses which used ORC adjudicated disease specific deaths and the sensitivity analysis limited to deaths with the disease specific ICD-10 codes (ICD-10 C56, C57 and C48) for tubo-ovarian cancer.¹⁰

Our data suggests that follow-up of trial participants for tubo-ovarian cancer using national registry data will result in incomplete ascertainment particularly of site due in part to the latency of registration. This can be reduced by using hospital administrative data, which has shorter latency. Central adjudication by experts though resource intensive adds value by improving accuracy of diagnoses. Revised classification systems for ovarian cancer⁸ will improve accurate identification and aid reproducibility of site assignment but this will require pathologists and clinicians to apply this new system uniformly across the globe.

United Kingdom Collaborative Trial of Ovarian Cancer Screening (UKCTOCS)

Outcomes review committee (ORC)

Cause of death (CoD)

Risk of Ovarian Cancer Alogorithm (ROCA)

Office of National Statistics (ONS)

Business Services Organisation, Health and Social Care Northern Ireland (BSO Northern Ireland)

International Classifications of Disease, (Revision 10)(ICD10)

National Health Service (NHS)

General practitioners (GP)

Invasive epithelial ovarian cancer (iEOC)

Ethics approval and consent to participate

The study was approved by the UK North West Multicentre Research Ethics Committees (North West MREC 00/8/34) on 21^st June 2000 with site-specific approval from the local regional ethics committees and the Caldicott guardians (data controllers) of the primary care trusts. All trial participants provided written informed consent. The study was conducted in accordance to the Declaration of Helsinki and Good Clinical Practice guidelines.

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing Interests

UM has stock ownership in Abcodia awarded to her by UCL. IJ is a co-inventor of the Risk of Ovarian Cancer Algorithm (ROCA). The ROCA has been licensed to Abcodia Ltd by the owners of the ROCA, Massachusetts General Hospital (MGH) and Queen Mary University of London (QMUL). IJ has a financial interest in Abcodia Ltd as a shareholder and director and is entitled to a royalty payments via MGH and QMUL from any commercial use of the ROCA. He is a trustee (2012–14) and Emeritus Trustee (2015 to present) for The Eve Appeal. SJS co-developed the risk of ovarian cancer algorithm with all rights assigned to MGH and QMUL. MGH has co-licensed software to Abcodia implementing the algorithm. SJS receives personal fees from the LUNGevity Foundation and SISCAPA Assay Technologies as a member of their Scientific Advisory Boards and from Abcodia as a consultant. SA is funded by a research grant from Abcodia. All other authors declare no competing interests.

Funding

The current analysis is supported by National Institute for Health Research (NIHR) HTA grant (16/46/01) and The Eve Appeal. UKCTOCS was funded by Medical Research Council (G9901012 and G0801228), Cancer Research UK (C1479/A2884), and the Department of Health, with additional support from The Eve Appeal. Researchers at UCL are supported by the NIHR University College London Hospitals (UCLH) Biomedical Research Centre and MRC CTU at UCL core funding (MR_UU_12023).

Disclaimer: Views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Author Contributions

This analysis was conceived by UM, JKK and AR. SM, SA, MH, DMC, CK collated clinical notes for review by Outcomes Review Committee. NS, MW, EB and RW performed the Outcome Review. JKK, UM and AGM performed the literature search. AR prepared dataset and MB performed the statistical analysis. JKK, AR, MB, AGM and UM interpreted the data. JKK, AR, AGM and UM prepared the figures and tables. AR, JKK, AGM, MB and UM contributed to writing of manuscript. AGM, SA, MH, DMC, JKK, SM, CK and UM were involved with trial implementation and data collection. All authors (JKK, AR, DMC, NS, MB, EB, SA, MH, SM, CK, AGM, RW, MW, LF, SC, SS, AM, MP, IJ, UM) critically reviewed and approved the report before submission. Authors LF, SC, SS, AM and MP contributed to the trial design initiated by IJ and UM.

Acknowledgments

We thank the volunteers without whom the trial would not have been possible and everyone involved in conduct and oversight. We are also very grateful to the current members of the UKCTOCS Trial Steering Committee - Prof Henry Kitchener (Chair), Prof Julietta Patnick, Prof Jack Cuzick and Ms Annwen Jones.

Dechartres A, Boutron I, Roy C, Ravaud P. Inadequate planning and reporting of adjudication committees in clinical trials: recommendation proposal. Clin. Epidemiol. 62, 695-702 (2009).
Marcus PM, Doria-Rose VP, Gareen IF, Brewer B, Clingan K, Keating K, et al Did death certificates and a death review process agree on lung cancer cause of death in the National Lung Screening Trial? Clin Trials. 2016 13:434-8.
Mäkinen T, Karhunen P, Aro J, Lahtela J, Määttänen L, Auvinen A. Assessment of causes of death in a prostate cancer screening trial. Int J Cancer. 122, 413-417 (2008).
Doria-Rose VP, Marcus PM, Miller AB, Bergstralh EJ, Mandel JS, Tockman MS, et al. Does the source of death information affect cancer screening efficacy results? A study of the use of mortality review versus death certificates in four randomized trials. Clin Trials. 7, 69-77 (2010).
Menon U, Gentry-Maharaj A, Ryan A, Sharma A, Burnell M, Hallett R, et al. Recruitment to multicentre trials--lessons from UKCTOCS: descriptive study. 337, a2079 (2008).
Jacobs I, Menon U, Ryan A, Gentry-Maharaj A, Burnell M, Kalsi JK et al Ovarian cancer screening and mortality in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trial. Lancet 387, 945–956 (2016).
World Health Organisation: International Statistical Classification of Disease and Related Health Problems Codes, (tenth revision) 2^nd (2003)
World Health Organisation: International Statistical Classification of Disease and Related Health Problems Codes, (tenth revision) (2014)
Prat J; FIGO Committee on Gynecologic Oncology. Staging classification for cancer of the ovary, fallopian tube, and peritoneum. Int J Gynaecol Obstet. 124(1):1-5. (2014)
Cohen, J. "A coefficient of agreement for nominal scales". Educational and Psychological Measurement. 20, 37–46 (1960).
Menon U, Gentry-Maharaj A, Hallett R, Ryan A, Burnell M, Sharma A, et al. Sensitivity and specificity of multimodal and ultrasound screening for ovarian cancer, and stage distribution of detected cancers: results of the prevalence screen of the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS). Lancet Oncol. 10:327-40 (2009)
MacDonald N, Sibley K, Rosenthal A, Menon U, Jayarajah A, Oram D et al A comparison of national cancer registry and direct follow-up in the ascertainment of ovarian cancer. Br J Cancer. ;80:1826-7. (1999)
Wieser S, Schmidt M, Kind AB, Heinzelmann-Schwarz VA. Ovarian cancer in Switzerland: incidence and treatment according to hospital registry data. Swiss Med Wkly. 2018 Jul 29;148:w14647. doi: 10.4414/smw.2018.14647. eCollection 2018 Jul 16.
Zanetti R, Schmidtmann I, Sacchetto L, Binder-Foucard F, Bordoni A, Coza D, et al. Completeness and timeliness: Cancer registries could/should improve their performance. Eur J Cancer. 51, 1091-1098 (2015).
Internationational Classification of Diseases for Oncology http://codes.iarc.fr/topography Accessed 1st Aug 2018
Sung-Jong Lee, Jeong-Hoon Bae, A-Won Lee, Seo-Yun Tong, Yong-Gyu Park, and Jong-Sup Park. Clinical Characteristics of Metastatic Tumors to the Ovaries. J Korean Med Sci 24, 114–119 (2009).
Mieno MN, Tanaka N, Arai T, Kawahara T, Kuchiba A, Ishikawa S, et al Accuracy of Death Certificates and Assessment of Factors for Misclassification of Underlying Cause of Death. J Epidemiol. 26, 191-198 (2016).
Godtman R, Holmberg E, Stranne J, Hugosson J. High accuracy of Swedish death certificates in men participating in screening for prostate cancer: a comparative study of official death certificates with a cause of death committee using a standardized algorithm. Scand J Urol Nephrol. 45, 226-232 (2011).
Turner EL, Metcalfe C, Donovan JL, Noble S, Sterne JA, Lane JA, et al. Contemporary accuracy of death certificates for coding prostate cancer as a cause of death: Is reliance on death certification good enough? A comparison with blinded review by an independent cause of death evaluation committee. Br J Cancer. 28;115, 90-94 (2016).
Singh N, Gilks CB, Wilkinson N, McCluggage WG. Assessment of a new system for primary site assignment in high-grade serous carcinoma of the fallopian tube, ovary, and peritoneum. Histopathology. 67, 331-337 (2015).
Singh N, Gilks CB, Hirschowitz L, Kehoe S, McNeish IA, Miller D, et al. Primary site assignment in tubo-ovarian high-grade serous carcinoma: Consensus statement on unifying practice worldwide. Gynecol Oncol 141, 195-198. (2016)
McCluggage WG, Judge MJ, Clarke BA, et al. Data set for reporting of ovary, fallopian tube and primary peritoneal carcinoma: recommendations from the International Collaboration on Cancer Reporting (ICCR). Mod Pathol 28,1101-1122. (2015)
Colombo N,Sessa C,du Bois A,Ledermann J,McCluggage WG,McNeish I, et al ESMO-ESGO Ovarian Cancer Consensus Conference Working Group. ESMO-ESGO consensus conference recommendations on ovarian cancer: pathology and molecular biology, early and advanced stages, borderline tumours and recurrent disease Ann Oncol.30:672-705. (2019)
Miller AB, Yurgalevitch S, Weissfeld JL; Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial Project Team.Death review process in the Prostate, Lung,Colorectal and Ovarian (PLCO) Cancer Screening Trial. Control Clin Trials 21(6 Suppl):400S-406S (2000).
Williams NJ, Hill EM, Ng SY, Martin RM, Metcalfe C, Donovan JL, et al. CAP Cause of Death Committee (2015). Standardisation of information submitted to an end-point committee for cause of death assignment in a cancer screening trial. BMC Med Res Methodol 15, 6.
Walters S, Maringe C, Butler J, Brierley JD, Rachet B, Coleman MP. Comparability of stage data in cancer registries in six countries: lessons from the International Cancer Benchmarking Partnership. Int J Cancer. 132, 676-685. (2013).
McPhail S, Johnson S, Greenberg D, Peake M, Rous B. Stage at diagnosis and early mortality from cancer in England. Br J Cancer. 112, Suppl 1:S108-15
Kahan BC, Feagan B, Jairath V. (2017). A comparison of approaches for adjudicating outcomes in clinical trials. Trials.18, 266. DOI 10.1186/s13063-017-1995-3

Due to technical limitations, Tables 1-3 are only available as downloads in the supplemental files section.

R1KalsietalTable1.pdf
R1KalsietalTable1.pdf
R1KalsietalTable2.pdf
R1KalsietalTable2.pdf
R1KalsietalTable3.pdf
R1KalsietalTable3.pdf
KalsietalUKCTOCSTrialsSupplFigure1.pdf
Supplementary Figure 1: Outcomes Cancer Review Form
KalsietalUKCTOCSTrialsSupplFigure1.pdf
Supplementary Figure 1: Outcomes Cancer Review Form
KalsietalUKCTOCSTrialsSupplFigure2.pdf
Supplementary Figure 2 Outcomes Death Review Form
KalsietalUKCTOCSTrialsSupplFigure2.pdf
Supplementary Figure 2 Outcomes Death Review Form
KalsietalUKCTOCSTrialsSupplFigure3.pdf
Supplementary Figure 3 Algorithm for assignment of tubo-ovarian cancer site based on available data
KalsietalUKCTOCSTrialsSupplFigure3.pdf
Supplementary Figure 3 Algorithm for assignment of tubo-ovarian cancer site based on available data
KalsietalUKCTOCSTrialsSupplTable1.pdf
Supplementary Table 1: International Classifications of Disease, (Revision 10) used for data linkage with national cancer registries in England, Wales and Northern Ireland
KalsietalUKCTOCSTrialsSupplTable1.pdf
Supplementary Table 1: International Classifications of Disease, (Revision 10) used for data linkage with national cancer registries in England, Wales and Northern Ireland
KalsietalUKCTOCSTrialsSupplTable2.pdf
Supplementary Table 2: Year of diagnosis / death of tubo-ovarian cancers with missing registration
KalsietalUKCTOCSTrialsSupplTable2.pdf
Supplementary Table 2: Year of diagnosis / death of tubo-ovarian cancers with missing registration
R1KalsietalSTARD.docx
R1KalsietalSTARD.docx

Download PDF

Journal Publication

published 25 Jan, 2021

Read the published version in Trials →

Editorial decision: Accept
10 Dec, 2020
Review #1 received at journal
06 Dec, 2020
Reviewers invited by journal
03 Dec, 2020
Reviewer #1 agreed at journal
03 Dec, 2020
Editor assigned by journal
25 Nov, 2020
Submission checks completed at journal
25 Nov, 2020

You are reading this latest preprint version

Completeness and accuracy of national cancer and death registration for outcome ascertainment in trials - an ovarian cancer exemplar

Status:

Journal Publication

Version 2

Abstract

Figures

Introduction

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Tables

Supplementary Files

Status:

Journal Publication

Version 2