Study data and population
Data used for this study were obtained from the Partners For Kids (PFK) pediatric accountable care organization database which includes the date and type of medical encounter(s), diagnosis, procedure(s), medication(s), treating physician(s), and facilities [21, 22]. PFK contracts with Medicaid-managed care plans in 34 counties in central and southeast Ohio, providing healthcare to approximately 330,000 children aged 0 to 21 years. For this study, healthcare claims for concussion-related medical encounters by actively enrolled PFK members aged 0 to 18 years between January 1, 2008 and December 31, 2016 were analyzed. Visits to multiple medical providers in the same day were treated as one concussion-related medical encounter.
Concussions were identified using the International Classification of Diseases, Ninth and Tenth Revisions, Clinical Modification (ICD-9-CM and ICD-10-CM) codes for concussion: 850.0, 850.1, 850.11, 850.12, 850.2, 850.3, 850.4, 850.5, 850.9, and those beginning with S06.0 [6, 23]. Only patients with one or more of the above concussion codes were included in the analyses. Concussions with a co-occurring severe TBI diagnosis code(s) (2.8%) were excluded.
To ensure accuracy of the initial concussion-related medical encounter, the study inclusion criteria were defined as follows: (1) an injury was sustained between April 1, 2008 and December 31, 2016, and (2) the patient was continuously enrolled in PFK for at least 30 days prior to the initial concussion-related medical encounter [22]. For patients with multiple concussion-related encounters, at least 90 days without a concussion-related encounter was required to denote unique injuries. This study was approved by the Institutional Review Board at the authors’ primary institution.
Outcomes of interest
The main outcome was the trend in monthly rate of concussion-related medical encounters per 10,000 member months, calculated as the number of initial concussion-related medical encounters in a month divided by the total number of PFK members in that same month, and multiplied by 10,000.
Time series analysis
The Box-Jenkins ARIMA intervention time series analysis was used to quantify the impact of Ohio’s concussion law on trends in the monthly rates of concussion-related medical encounters. Seasonal ARIMA models were specified to account for the inherent dynamics in the series, and were expressed as (p, d, q) and (P, D, Q). The p, d, and q specified the order of the autoregressive, differencing, and moving average processes of the regular noise model, while P, D, and Q corresponded to the parameters for the seasonality component [24]. A log transformation was applied to the monthly rates series to ensure the normality and homogeneity of variance of the residuals. The concussion law was treated as the intervention and coded as a binary variable (0= pre-law, before April 26, 2013; 1= post-law, on and after April 26, 2013).
To analyze the time series of monthly rates of concussion-related medical encounters, we defined S=12, corresponding to 12 observations per year. We assessed stationarity and used plots of autocorrelation function (ACF) and partial autocorrelation function (PACF) to identify the six parameters in the model. Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were used to compare models; the optimal model was based on the lowest AIC and BIC values. The mean absolute percentage error (MAPE) was calculated to assess forecast accuracy and to select an optimum model; the lower the MAPE value the better data fit.
The Ljung-Box Portmanteau (or Q) was used to examine the randomness of residuals of the estimated model. We assessed the effect of the intervention by interpreting the coefficient β for the indicator variable. The percent change in the post-law period was estimated as exp(β)-1.
Additionally, we validated the results of the ARIMA model using both traditional Poisson regression and curve fitting models. We first estimated the monthly rate per 10,000 member months and rate ratio of concussion-related medical encounters by including two independent variables (pre- or post-law group and the dummy variable of month identification) in the Poisson models, with a pre-defined reference month (either April 2008 or April 2013). We then employed the traditional curve fitting method to assess the trends of yearly rates of concussion-related medical encounters, using the coefficient of determination (R2) to determine goodness of fit. Finally, we compared the results of the ARIMA intervention time series analysis to the findings from the two traditional methods. All data analyses were conducted using SAS 9.4 and the TSA and forecast packages in R statistical software [24, 25], as appropriate. A P<0.05 was considered statistically significant.