Study data and population
Data used for this study were obtained from the Partners For Kids (PFK) pediatric accountable care organization database which includes the date and type of medical encounter(s), diagnosis, procedure(s), medication(s), treating physician(s), and facilities [21, 22]. PFK contracts with Medicaid-managed care plans in 34 counties in central and southeast Ohio, providing healthcare to approximately 330,000 children aged 0 to 21 years. For the purpose of this study, healthcare claims for concussion-related medical encounters by actively enrolled PFK members aged 0 to 18 years between January 1, 2008 and December 31, 2016 were analyzed. Visits to multiple medical providers in the same day were treated as one concussion-related medical encounter.
Concussions were identified using the International Classification of Diseases, Ninth and Tenth Revisions, Clinical Modification (ICD-9-CM and ICD-10-CM) codes for concussion: 850.0, 850.1, 850.11, 850.12, 850.2, 850.3, 850.4, 850.5, 850.9, and those beginning with S06.0 [6, 23]. Only patients with one or more of the above concussion codes were included in the analyses. Concussions with a co-occurring severe TBI diagnosis code(s) (2.8%) were excluded.
To ensure accuracy of the initial concussion-related medical encounter, the study inclusion criteria were defined as follows: (1) an injury was sustained between April 1, 2008 and December 31, 2016, and (2) the patient was continuously enrolled in PFK for at least 30 days prior to the initial concussion-related medical encounter [22]. For patients with multiple concussion-related encounters, at least 90 days without a concussion-related encounter was required to denote unique injuries. This study was approved by the Institutional Review Board at the authors’ primary institution.
Outcomes of interest
The main outcome was monthly rate of concussion-related medical encounters per 10,000 member months, calculated as the number of initial concussion-related medical encounters in a month divided by the total number of PFK members in that same month, and multiplied by 10,000. A similar calculation was used to determine yearly rates.
Time series analysis
The Box-Jenkins ARIMA intervention time series analysis was used to quantify the impact of Ohio’s concussion law on trends in the monthly rates of concussion-related medical encounters. Seasonal ARIMA models were specified to account for the inherent dynamics in the series, and were expressed as (p, d, q) and (P, D, Q). The p, d, and q specified the order of the autoregressive, differencing, and moving average processes of the regular noise model, while P, D, and Q corresponded to the parameters for the seasonality component [24]. A log transformation was applied to the monthly rates series to ensure the normality and homogeneity of variance of the residuals. The concussion law was treated as the intervention and coded as a binary variable (0 = pre-law, before April 26, 2013; 1 = post-law, on and after April 26, 2013).
To analyze the time series of monthly rates of concussion-related medical encounters, we defined S = 12, corresponding to 12 observations per year. We assessed stationarity and used plots of autocorrelation function (ACF) and partial autocorrelation function to identify the six parameters in the model. Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were used to compare models; the optimal model was based on the lowest AIC and BIC values. The mean absolute percentage error (MAPE) was calculated to assess forecast accuracy and to select an optimum model; the lower the MAPE value the better data fit.
The Ljung-Box Portmanteau (or Q) was used to examine the randomness of residuals of the estimated model. We assessed the effect of the intervention by interpreting the coefficient β for the indicator variable. The percent change in the post-law period was estimated as exp(β)-1.
Additionally, a traditional Poisson regression model was used to estimate the monthly rate ratio of concussion-related medical encounters. Results were then compared to the results of the ARIMA intervention time series analysis to validate our findings (see eAppendix in the Supplement for model specification; results are presented in eTable 1 and eFigure 1). The two independent variables (pre- or post-law group and dummy variable of month identification) were included in the Poisson models, with a pre-defined reference month (either April 2008 or April in 2013). The traditional curve fitting method was employed to assess the trends of yearly rates of concussion-related medical encounters, using the coefficient of determination (R2) to determine goodness of fit. All of the data analyses were conducted using SAS 9.4 and the TSA package in R statistical software [24, 25], as appropriate. A P < 0.05 was considered statistically significant.