Study design and data
For this retrospective cross-sectional study, we used data from patients registered in Primary Care. Patients were registered during 2014 with one of the five participating GP practices in Nijkerk, the Netherlands. Data at individual patient level was extracted from the practices’ electronic health records. Obtained data included age, gender, and coded healthcare procedures, diagnoses and pharmaceutical data for 30,596 patients over the year 2014. Diagnoses were registered as ICPC-1 diagnoses codes, as used in the Netherlands (11). For proper recognition by the ACG system, ICPC-1 codes were converted to ICPC-2 codes. Prescribed medication was registered as Anatomical Therapeutic Chemical (ATC) codes (12), the classification system for pharmacy products. The number of GP visits was extracted from all healthcare procedures in 2014. GP visits were defined as all GP encounters, including physical and telephone consults and home visits by either GPs or nurse practitioners working at the GP practices.
From the original datasets 4,289 cases were removed, due to corrupted patient identification numbers. Another 2,689 cases belonging to three specific ACG categories, were left out of the analyses: No Diagnosis or Only Unclassified Diagnosis (n=281), Non-Users (n=2,407) and Invalid Age or Date of Birth (n=1). The final analyses were performed with data for 23,618 persons (77% of 30,596 registered people).
Data preparation and analyses were performed with IBM SPSS Statistics 24.
ACG System software
The study was conducted, using the Johns Hopkins University’s ACG® System software 11. The ACG® System software 11 is a risk stratification tool, assigning each patient to one of the 98 mutually exclusive ACG categories. Assignment to ACG categories is based on combinations of diagnoses types. With the ACG system the diagnoses for each patient are grouped into 32 Aggregated Diagnosis Groups (ADGs), based on type of diagnoses rather than on specific diagnoses, i.e. specific ICPC codes. Individuals’ patterns of ADGs determine the assignment of patients to one of the 98 mutually exclusive ACG categories (8).
Information on diagnoses and medication, in addition to age and gender, were used as input data for the ACG® System software 11.
Assessment of the ACG system
To assess the applicability of the ACG system in Dutch primary care, we looked at two aspects: face validity and model performance.
Face Validity
According to Mosier (13) an important aspect of the testing of an instrument lies in the ‘consumer acceptance’. The first step in effective use of a test, is the actual selection for use and acceptance of the results. Mosier describes one of the translations of face validity as the appearance of validity: the test must appear valid in addition to the statistical validity. In this study we defined face validity as this appearance of validity described by Mosier (13).
We assessed the ACG system’s face validity by exploring the actual ACG categorization with regard to age. Age distributions for each ACG category were created and ACG categories were assessed on recognition of multimorbidity in relation to age.
Model Performance
To investigate the impact of the ACG system in Dutch primary care, four different logistic regression models were estimated.
Dependent variable
The outcome variable, number of GP visits, was transformed into binary variables according to four definitions. According to the first definition, no GP visits was defined as no utilization of care, whereas one or more GP visits were defined as utilization of care. With the second definition, a distinction between zero or one GP visit and two or more GP visits was made. With the third definition, a distinction between zero to two GP visits and three or more GP visits was made. Accordingly, for the final definition the outcome was defined as a distinction between zero to three and four or more GP visits. The performance of each of these models was investigated.
Independent variables
In the null or base model only age as a continuous variable and gender were included as explanatory variables.
Model 1 included age, gender and ICPC chapters as independent variables. ICPC diagnosis codes are divided into 17 different chapters including ‘General and unspecified’, ‘Blood, blood forming organs, lymphatics, spleen’, ‘Digestive’, ‘Eye’, ‘Ear’, ‘Circulatory’, ‘Musculoskeletal’, ‘Neurological’, ‘Psychological’, ‘Respiratory’, ‘Skin’, ‘Endocrine metabolic and nutritional’, ‘Urology’, ‘Pregnancy, childbirth, family planning’, ‘Female genital system and breast’, ‘Male genital system’ and ‘Social problems’. Different ICPC chapters can be registered to a single person. Therefore, the ICPC chapters were added to the model as 17 different dummy variables.
Model 2 included age, gender and ADG diagnoses as independent variables. As an individual can have more than one ADG, the 32 ADGs were added to the model as 32 dummy variables.
Model 3 included age, gender and mutually exclusive ACGs. Before estimating the logistic regression, the numbers of individuals in each ACG category were checked. Aggregation of some ACG categories was necessary due to categories with small numbers of individuals. In the supplementary file 1 the aggregation of the original ACG categories is presented.
Table 1 gives an overview of the four different models estimated. To select the best model, the performance of each logistic regression with outcome variable as defined above, was investigated. The Area Under the Curve (AUC) values were calculated for each model.
Ethics approval and patients’ consent
The need for ethical approval was waived by the medical ethical committee of Leiden University Medical Center (CME - LUMC), the Netherlands.
Participants were not asked for their consent because we used routinely collected de-identified data.
Table 1
Independent variables of logistic regression models to investigate association with utilization of GP care
Independent variables included in different logistic regression models |
Null model | - Age - Gender |
Model 1 | - Age - Gender - ICPC chapters |
Model 2 | - Age - Gender - ADGs |
Model 3 | - Age - Gender - Mutually exclusive ACG categories |