This study describes the execution of an updated computable phenotype for the rare disease IPF using PCORnet data infrastructure. We demonstrate how a single computable phenotype can be evaluated for different use cases, each with its predefined gold standard. This provides a flexible approach to meet diverse clinical research needs: the Restrictive Use Case (consensus diagnosis of IPF but not FPF or CPFE) is commonly used in pharmaceutical research studies while the Inclusive Use Case (consensus diagnosis of IPF, FPF, or CPFE) allows more comprehensive characterization of the spectrum of IPF.
Utilizing duplicate chart review and adjudication of 100% of the cases, the computable phenotype showed a positive predictive value of 57% for the Inclusive Use Case and 47% for the Restrictive Use Case. This accomplishes a marked improvement in recruitment efficiency from 0.3% (source population) to 50% (computable phenotype sub-population) when the task is to identify candidate individuals within the EHR for specific purposes such as clinical trials. However, individual level chart adjudication is still necessary.
Our IPF population prevalence estimates (12.6–26.7/100,000) are concordant with previously published US estimates of 14–43/100,000 in a large health plan (11) and 2–29/100,000 in the general population (13); they are lower than the estimates of 276–725/100,000 in a population of US veterans, possibly attributable to high levels of exposure to risk factors in that population (14). A comprehensive dataset from Quebec, Canada estimates a prevalence of 78.4/100,000 (27). International estimates compiled by Maher et al (28) demonstrate geographic variation in prevalence: 5.7–45.1/100,000 in Asia Pacific; 3.3–25.1/100,000 in Europe; and 24-29.8/100,000 in North America; with an adjusted prevalence of 3.3–45.1/100,000 globally.
The comparative demographics of the source population and IPF Computable Phenotype cohort align with established demographic characteristics of this disorder (13). Further, the slight rise in mean age seen with the Restrictive Use Case compared with the Inclusive Use Case (Table 1) fits with the acknowledged presentation of FPF at a younger age than sporadic IPF. Thus we addressed the fitness/utility of this computable phenotype and found it to be favorable through a multidimensional qualitative assessment (5).
The ideal computable phenotype will identify people accurately and automatically. For IPF, this ideal does not yet exist. There is no single biomarker for IPF (29–31) and the current most accurate diagnostic approach is multidisciplinary (32). Absent this ideal, current computable phenotype methodologies offer flexible, agile approaches to harnessing the power of the EHR for clinical research. As demonstrated in this paper, it was possible to develop a single computable phenotype and ultimately have that work for different purposes based on using different gold standards for different use cases. Whether the goal is to look at disease burden broadly or to find people to contact and recruit, this EHR-based approach is feasible.
Computable phenotypes are of growing interest to the rare-disease research community, defined in the U.S. as those diseases affecting fewer than 200,000 people (33). Although individually rare, these diseases collectively affect 25–30 million Americans of whom an estimated 1–2 million have a primary lung disorder (34). Rare diseases in Europe are defined as those affecting less than 1 in 2000 individuals and collectively affect 30 million European Union citizens (35). There is an “unprecedented interest” from industry to study IPF (36) and other ILDs resulting from an accelerated interest in pharmacotherapy for IPF in the last 25 years (37).
Our work builds on and extends the work of previous investigators. Esposito et al (8) developed claims-based algorithms to identify IPF using the 2011 criteria, thereby estimating IPF incidence and prevalence in the US. This approach compared three different algorithms to identify patients from a claims database comprised of 14 million persons enrolled between 2006–2012 for at least 6 months. The dataset excluded persons under 50 or over 100 years and required at least 1 physician diagnosis of IPF and the absence of an alternative diagnosis. Three increasingly narrow algorithms identified n = 4598 (broad case identifying algorithm), n = 2052 (narrow case identifying algorithm), and n = 1354 (IPF score algorithm) persons. The PPV rose from 44–62% to 76% using the treating clinician’s diagnosis and from 54–58% to 83% using the expert clinician’s diagnosis. Acknowledged limitations include that the dataset included only commercially insured patients, limiting generalizability. Medical record review was used as the gold standard without independent review of HRCT and pathology specimens. Clinical adjudicators determined both the treating clinician’s diagnosis and their own diagnosis. The adjudication of the IPF score was performed on only 3.7% of cases (n = 50). This was judged to be a poorly-performing algorithm (38) for estimating disease incidence and prevalence.
Ley et al evaluated a computable phenotype for IPF using a single payer health maintenance organization, reviewing/validating through chart review of a 10% sample (15). They evaluated two algorithms: the IPF algorithm required age over 18 years and a diagnosis of IPF in the absence of an alternative diagnosis. Their broader IIP algorithm required age greater than 50 and a diagnosis of IPF or the less specific IIP while excluding those with an alternative diagnosis. Through adjudication of two random samples of n = 75 cases for the IPF algorithm and one random sample of n = 75 for the IIP algorithm, they found a PPV of 42.2% and 12% for the IPF and IIP algorithms, respectively. Our computable phenotype for IPF does not exclude prior or subsequent diagnosis of IIP, but rather estimates their importance through the chart review process for specific use cases. Among the false positive cases identified by our algorithm, there were n = 44 with other ILDs [other pulmonary fibrosis, not otherwise classifiable (n = 17); non-specific interstitial pneumonia (n = 8); connective tissue-associated ILDs (n = 6); occupational, radiation or drug-induced ILD (n = 6); granulomatous diseases (n = 3); respiratory bronchiolitis-ILD (n = 2); cryptogenic organizing pneumonia (n = 1), and hypersensitivity pneumonitis (n = 1)]. Our algorithm is also revised from that of Ley et al recognizing the increasing importance of the antisynthetase syndromes and undifferentiated connective tissue disease.
The present study represents a different framework for computable phenotypes that accounts for varying use cases than has been used to date for IPF. Previous investigators compared distinct computable phenotypes (e.g., broad and narrow), aiming for estimates of disease prevalence or burden (8, 11, 12, 15). For the use case of incidence/prevalence estimates, a PPV of 50% is considered poor (38). However, for the purposes of recruitment efficiency and developing a candidate pool for clinical trials, this approach has merit. Prior studies and ours, taken together, indicate that IPF computable phenotypes can be applied to diverse geographic areas, payer mixes and EHR systems to identify people with IPF with a PPV of 40–50%. This represents a marked recruitment efficiency, compared to beginning with an unfiltered EHR pool. It also reduces, although does not eliminate, selection bias that often exists in clinical trial recruitment.
Strengths of our study include the source population from a health system comprising multiple insurers as well as the uninsured. By using a health system as the source population, instead of a claims database, we included all individuals regardless of insurance enrollment status. We utilized PCORnet’s Common Data Model (21) architecture, which makes the computable phenotype portable across PCORnet. Applied widely, this EHR-based computable phenotype will provide information complementary to claims-based studies (8, 11, 12). Another strength was building from a previously-published algorithm (11), which allows the results to be compared to other epidemiologic estimates based on a similar case definition. Strengths of our validation procedure included a chart review process with the use of consensus diagnostic criteria (13), two independent reviewers and a reconciliation process for disagreements. We were also able to perform a chart review validation, an opportunity not present when working with claims databases. We evaluated the computable phenotype with two predefined use cases, and validated each with chart review performed on 100% of the identified patients.
An inherent limitation of this and other studies (8, 15) is the inability to ascertain the true prevalence of IPF in our population, due to the practical barriers to identifying the false negatives from the source population. Based on published estimates of IPF prevalence (8, 11, 12, 28), our population of 588,000 patients might contain between 74 and 249 people with IPF. We identified 89 cases, confirmed by chart review. A well-designed population study could address this limitation but would be impractical due to the cost (39); we calculate that we would need to review 3675 charts at random to find one of the possible 160 missed diagnoses. This is a limitation shared by EHR-based rare disease research. An advantage of the detailed chart-level validation is the ability to estimate the magnitude of misclassification, which can be of value in interpreting claims-based studies.
Data for this single-center study was from a tertiary care academic medical center, which limits the generalizability of the results. However, the PCORnet CDM data architecture was chosen as it is standardized for use across more than 60 sites and 66 million people in the US, including diverse EHR systems, payer mixes and care delivery settings(40). Despite the common data architecture, analyses in other EHRs and/or in other health systems will be needed. Variability in EHR and coding practices may influence the performance of this computable phenotype in other health systems. As with any computable phenotype, the results are also limited by the quality of EHR data. Our computable phenotype was also limited by the data available. Addition of variables including HRCT findings, as identified by natural language processing, would likely improve the PPV substantially but possibly at the cost of sensitivity. As personalized medicine identifies biomarkers predicting responsiveness to specific therapies, these can also be incorporated into future computable phenotypes (41, 42).
The field of IPF research and treatment is changing to reflect a multi-targeted approach potentially with combination therapies (42) and precision medicine (41). EHR and claims-based studies have complementary but distinct advantages. For a rare disease, it becomes imperative to have efficient methods to identify potential study participants. The use of a computable phenotype within the EHR allows for identification of a source population for clinical trial recruitment and may help to address the acknowledged need to find people who can participate in evaluating new therapies. Chart review is still needed as a gold standard due to the integration of tests and history required for diagnosis of IPF and other disorders without a single biomarker. The positive predictive value calculated in this and other studies is poor for estimates of incidence and prevalence but excellent for recruitment efficiency. Thus, chart review is still necessary but many fewer charts need to be reviewed to identify eligible participants. In this way, computable phenotypes represent part of the approach to connecting people with ILD to clinical trials (43–48) and, as they become available, personalized therapies.
Future applications of the computable phenotype in EHR-based populations include ICD-10 coding, measures of disease severity, and changes in disease management, i.e., inclusion of newly-available therapies. Work is also needed to assess the broader landscape of fibrotic lung disease, such as identifying cases of progressive fibrotic interstitial lung disease. Additional uses for computable phenotype populations in IPF and fibrotic disease more broadly include biomarker-based studies and evaluation of practice variation and clinical outcomes (38). These applications provide an opportunity to evaluate computable phenotypes for additional use cases based on an understanding of disease patterns and the researchers’ goals for the computable phenotype.