Symptoms are the Most Effective Child SCAT5 Component for Recognizing Concussion on the Day of Injury

Background: The Child Sport Concussion Assessment Tool 5th Edition (Child SCAT5) was developed to evaluate children between 5-12 years of age for a suspected concussion. However, limited empirical evidence exists demonstrating the value of the Child SCAT5 for acute concussion assessment. Therefore, the purpose of our study was to examine differences and assess the diagnostic properties of Child SCAT5 scores among concussed and non-concussed middle school children on the same day as a suspected concussion. Methods: Our participants included 34 concussed (21 boys, 13 girls; age=12.8±0.86 years) and 44 non-concussed (31 boys, 13 girls; age=12.4±0.76 years) middle school children who were administered the Child SCAT5 upon suspicion of a concussion. Child SCAT5 scores were calculated from the symptom evaluation (total symptoms, total severity), child version of the Standardized Assessment of Concussion (SAC-C), and modi�ed Balance Error Scoring System (mBESS). The Child SCAT5 scores were compared between the concussed and non-concussed groups. Non-parametric effect sizes (r=z/ √ n) were calculated to assess the magnitude of difference for each comparison. The diagnostic properties (sensitivity, speci�city, diagnostic accuracy, predictive values, likelihood ratios, and diagnostic odds ratio) of each Child SCAT5 score were also calculated. Results: Concussed children endorsed more symptoms (p<0.001, r=0.45), higher symptom severity (p<0.001, r=0.44), and had higher double leg (p=0.046, r=0.23), single leg (p=0.035, r=0.24), and total scores (p=0.022, r=0.26) for the mBESS than non-concussed children. No signi�cant differences were observed for the SAC-C scores (p’s ≥ 0.542). The quantity and severity of endorsed symptoms had the best diagnostic accuracy (AUC=0.76–0.77), negative predictive values (NPV=0.84–0.88), and negative likelihood ratios (-LR=0.22–0.31


Background
There are approximately 12.3 million middle school (grade level 6-8) age children in the United States and an estimated 36% (≈4.4 million) will participate in organized intramural or interscholastic sport annually.[1,2] Participation in organized sport for children under 14 years of age is the leading cause of concussion (43%) [3] and has been associated with a six times greater risk of concussion as compared to other leisure physical activities.[4] Limited on-site medical coverage of school-sanctioned sports [5] contributes to a majority (86%) of children seeking medical care from healthcare professionals in direct access settings (e.g., emergency department, outpatient clinics) [6,7] where follow up visits are uncommon (1-3%).[8][9][10] Several governing bodies [11][12][13][14] recommend the implementation of a multimodal assessment for the evaluation of children following a suspected concussion.The Sport Concussion Assessment Tool 5th edition (SCAT5) [15] and the Child Sport Concussion Assessment Tool 5th Edition (Child SCAT5) [16] are two of the most commonly used multimodal assessments.[17][18][19][20] The Child SCAT5 is a modi ed version of the SCAT5 which was developed for administration to children between 5-12 years of age.[15] Previous literature has demonstrated poor psychometric (e.g., test-retest reliability) and diagnostic properties (e.g., sensitivity, speci city, predictive value, likelihood ratios) for the individual components of prior iterations of the SCAT[16, [21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39][40] and the Child SCAT 3rd Edition (Child SCAT3) [41][42][43][44].Despite consensus recommendation and wide adoption, scant evidence exists regarding the psychometric and diagnostic properties of the Child SCAT5.[45] The lack of empirical evidence regarding the Child SCAT5 necessitates that healthcare professionals rely upon their subjective interpretation of Child SCAT5 scores to inform acute clinical management.An approach which may contribute to misdiagnosis and inconsistent treatment for those children acutely evaluated for concussion using the Child SCAT5.[46] Therefore, the purpose of our study was to evaluate for differences and assess the diagnostic properties of Child-SCAT5 scores on the same day as a suspected concussion among middle school children who were ("concussed") or were not ("non-concussed") subsequently diagnosed with a concussion.We hypothesized that (i) the concussed children would endorse signi cantly more symptoms and report a signi cantly higher symptom severity as compared to the nonconcussed children; (ii) the concussed children would score lower on the SAC-C and commit signi cantly more errors on the mBESS as compared to the non-concussed children; and (iii) the symptom evaluation (total symptoms endorsed, symptom severity) would demonstrate better diagnostic properties than the SAC-C or mBESS on the same day as the suspected concussion.

Design and Settings
The Advancing Healthcare Initiatives for Underserved Students (ACHIEVES) Project provided on-site medical care to sixteen middle schools within a large socio-demographically diverse school district in Virginia, USA.[47]The George Mason University Institutional Review Board approved the construction of the deidenti ed database for research purposes as part of the ACHIEVES Project and waived assent and consent.All participants in our study competed in school-sanctioned sports between the 2017-2018 and 2019-2020 academic years.

Participants
Our sample originally consisted of 186 (103 concussed, 83 non-concussed) middle school children that were evaluated for a suspected concussion (Figure 1).Exclusion criteria were set to ensure that injury and assessment data was available from the concussion assessment that was administered at the time of the suspected concussion.Participants were excluded if their suspected concussion occurred outside of sport, the initial assessment was not completed on the same day as the suspected concussion, or there were any missing data elements from the Child SCAT5 (Figure 1).After applying all of our exclusion criteria, our nal sample consisted of 78 (34 concussed, 44 non-concussed) middle school children that were evaluated for a suspected concussion while participating in school-sanctioned sports.

Testing Procedures
Consistent with the school system's concussion management protocol, the Child SCAT5 was administered to middle school children after removing them from participation in school-sanctioned sports due to the suspicion of a concussion.The children in our study were allocated into groups dependent on whether they were ("concussed") or were not ("nonconcussed") subsequently diagnosed with a concussion.The de nition of a concussion was consistent with the most recent international consensus statement on concussion in sport.[11] Regardless of their Child SCAT5 scores, the children were not permitted to return to sport participation on the same day as the suspected concussion which is in alignment with international consensus statements, several governing bodies, and the state law of Virginia.[11-14, 48, 49]

Outcomes
The administration and scoring of each component of the Child SCAT5 (symptom evaluation, child version of the Standardized Assessment of Concussion [SAC-C], modi ed version of the balance error scoring system [mBESS]) are described in detail elsewhere.[16] For the symptom evaluation, the total number of symptoms endorsed (range=0-21) and the total symptom severity (range=0-63) endorsed by the children were calculated.For the SAC-C, points earned for the immediate memory (range=0-15 points), concentration (range=0-6 points), and delayed recall (range=0-5 points) domains were recorded and summed to calculate a composite score (range=0-26 points).For the mBESS, errors committed (range=0-10) during each stance (double leg, single leg, tandem) were recorded and summed to calculate the total score (range=0-30).Lower scores on the SAC-C and higher scores on the mBESS were indicative of worse performance on the respective components of the Child SCAT5.

Statistical Analyses
Nonparametric analyses were performed due to the non-normal distribution (Shapiro-Wilk=0.78-0.94,p's<0.05) of the Child SCAT5 scores.Mann-Whitney U-tests were used to assess for differences between the concussed and non-concussed children for each Child SCAT5 score.Nonparametric effect sizes [50] ( ) were calculated and interpreted as small (r=0.10-0.30),moderate (r=0.30-0.50),or large (r≥0.50).[51] Receiver operator curve analyses were performed to calculate the diagnostic accuracy (area under the curve [AUC]) of each Child SCAT5 score.[21,24, 25, 52-55] Youden's Index (J)[56] was calculated to determine the cutoff score that optimized the combination of sensitivity (Sn) and speci city (Sp) for each Child SCAT5 score.[57][58][59][60] Values closer to 1.0 for the Youden Index are indicative of a greater combination of sensitivity and speci city. [56] The sensitivity and speci city values of each Child SCAT5 score was used to calculate their positive (PPV) and negative (NPV) predictive values, positive (+LR) and negative (-LR) likelihood ratios, and diagnostic odds ratios (DOR).Positive and negative predictive values that are closer to 1.0 are indicative of a more valid test result.
[61] Higher positive likelihood ratios indicate a greater likelihood that the patient has the test condition (e.g., concussed) while lower negative likelihood ratios indicate a greater likelihood that the patient does not have the test condition (e.g., not concussed).
[62] Higher diagnostic odds ratios indicate that the test result has a greater ability to differentiate between patients with (e.g., concussed) and without (e.g., non-concussed) the test condition.All analyses were performed using SPSS (Version 27, IBM Corp., NY, USA).Alpha was set a priori at p<0.05.

Discussion
Our study evaluated for differences in Child SCAT5 scores, and their subsequent diagnostic properties, among concussed and non-concussed middle school children on the day that they were evaluated for a suspected concussion.Our ndings demonstrate that concussed children endorse a greater number and severity of symptoms and suggest that the symptom evaluation is the most effective component of the Child SCAT5 for differentiating between concussed and non-concussed children on the same day as a suspected concussion.The concussed children also committed signi cantly more errors on the mBESS than the non-concussed children, however, the magnitude of these differences were relatively small as supported by the effect sizes.Our data suggests that the SAC-C is the least meaningful component of the Child SCAT5 as no signi cant differences were observed between the concussed and non-concussed children which resulted in poor diagnostic accuracy values.Overall, our study reinforces the importance of the symptom evaluation as an integral part of the clinical assessment of children on the same day as a suspected concussion.
Self-reported symptomology has been a cornerstone of concussion evaluation and remains the key component for informing clinical judgement.[11] Our data demonstrates that concussed middle school children endorse a signi cantly greater number and severity of symptoms than children who are not diagnosed with a concussion.Speci cally, the concussed children in our sample endorsed nearly double the number of symptoms (8.0±5.50 vs. 3.7±3.97)and symptom r r r severity (12.1±10.5 vs. 12.1±10.9)as compared to the non-concussed children.Previous literature has demonstrated that concussed children evaluated in the emergency department using the Child SCAT3 endorsed signi cantly more symptoms and greater severity than non-concussed children, which is alignment with our results.[44] Our ndings also align well with previous literature that have reported elevated endorsement and severity of symptoms in older athletic populations (e.g., high school, collegiate) following a diagnosed concussion as compared to preinjury (baseline) data or matched comparisons.[23,63,64] We did not incorporate baseline assessments or matched comparisons, thus, we encourage future research to investigate these differences in symptom reporting of middle school children following a suspected concussion.
Our study also observed that concussed children committed signi cantly more errors in two stances (double leg, single leg) of the mBESS and had signi cantly higher mBESS total scores as compared to the non-concussed children.
Furthermore, our mBESS total scores for the concussed children (7.8±4.53 errors) were slightly elevated compared to normative reference values for this population (5.0±3.7 errors).
[65] However, the small effect sizes (r=0.23-0.26)calculated for the concussed versus non-concussed comparisons suggest that the differences may not be clinically meaningful.Findings from our sample of middle school children align well with extensive literature that have reported that concussed high school and collegiate athletes perform worse on the BESS than non-concussed athletes.[22,23,63,64,66,67] Collectively, our ndings and those of previous literature that assessed older athletic populations suggest that balance assessment following a suspected concussion may elicit subtle de cits to inform clinical diagnosis and management.
The concussed children in our study did not perform signi cantly different on any component of the SAC-C as compared to the non-concussed children on the same day as the suspected concussion.The SAC-C composite scores were also not signi cantly different between the concussed and non-concussed groups in our sample and the values for each group align within the "broadly normal" interpretation of normative reference values for this population.
[65] The lack of signi cant ndings between the concussed and non-concussed groups is different than previous literature that evaluated older athletic populations.[22,23,63,64,66,67] Possible rationales for the lack of signi cant differences in SAC-C scores include discrepancies in the age of the participants in our study (middle school children) as compared to those of previous literature study populations (high school or collegiate athletes) and different ranges of composite scores for the SAC (range=0-25) and SAC-C (range=0-26).[15] To our knowledge, our study is the rst to provide evidence of the diagnostic properties of the Child SCAT5 in the population for which it was designed.The total number and severity of endorsed symptoms were found to have the highest levels of diagnostic accuracy (AUC=0.76-0.77)and sensitivity (Sn=0.79-0.88) of the Child SCAT5 scores.Based on the calculated cutoff scores, children who endorse less than four symptoms or report a severity less than six points on the same day as the suspected concussion were less likely (-LR=0.22-0.31) to be diagnosed with a concussion.As mentioned previously, it is important to note that none of the children assessed for a suspected concussion were permitted to return to sport participation on the same day as the assessment regardless of their Child SCAT5 scores.This is in alignment with the recommendations from the leading international consensus group on concussion in sport, [11] position statements from several governing bodies, [12][13][14] and the legal requirements of state laws [49].
The highest positive predictive values and likelihood ratios were observed for interpretation of the double leg stance of the mBESS followed by the delayed recall domain of the SAC-C.However, inordinately low thresholds for a positive test result (e.g., diagnosed concussion) for the double leg stance of the mBESS (≤1 error) and the delayed recall domain of the SAC-C (≤1 point) yielded high speci city values (Sp=0.98-1.00)which arti cially elevated the calculated positive predictive values and likelihood ratios.Therefore, we caution healthcare professionals from independent interpretation of the double leg stance of the mBESS or the delayed recall domain of the SAC-C in their clinical decision-making at this time.Future research is warranted to validate our ndings related to the diagnostic properties of the individual components (symptom evaluation, SAC-C, mBESS) of the Child SCAT5 in an independent sample of children on the same day as a suspected concussion.
The calculated diagnostic properties of the Child SCAT5 scores in our study align with those reported for previous iterations of the SCAT and the Child SCAT3.[23,44,63, 67] More speci cally, the values of sensitivity, speci city, and diagnostic accuracy observed in our study are similar to those calculated for the individual components of the SCAT for the assessment of older athletes.[23,63,67] A similar methodology as our study has been implemented to evaluate for differences and assess the diagnostic properties of Child SCAT3 scores among children who were evaluated for a suspected concussion in the emergency department.[44] The authors of this prior study reported similar diagnostic accuracy values for Child SCAT3 scores to those observed in our study which utilized the Child SCAT5.[44] Overall, our ndings support those of previous literature [23,44,63,67]  We recognize that our study is not without limitations.All of the children in our study were participating in schoolsanctioned sports at middle schools within a single county in the northern Virginia which limits our generalizability.However, the middle school student population in our study has a unique socio-demographic pro le including students of diverse racial backgrounds (e.g., 36.7%Hispanic, 27.8% White/Caucasian, 20.7% Black/African-American) and high academic achievement (e.g., less than a 2% course failure rate overall).[47]Another limitation is the variability in the time between the removal from sport and the concussion evaluation, however, all participants were evaluated on the same day as the suspected concussive event.Lastly, the diagnosis of a concussion was made by the on-site healthcare professional using their own clinical decision-making which improves the external validity of our study.The healthcare professionals participating in the ACHIVES Project also completed annual training on concussion assessment using the Child SCAT5 and followed an established concussion management protocol which limited variability in their clinical evaluation.

Conclusion
The concussed middle school children endorsed more symptoms, reported greater symptom severity, and committed more errors during administration of the mBESS on the same day as the suspected concussion than children who were not diagnosed with a concussion.Clinical interpretation of the total quantity and severity of the endorsed symptoms resulted in the greatest combination of diagnostic properties as supported by the best sensitivity, diagnostic accuracy, negative predictive values, and negative likelihood ratios.Our data does not support the clinical interpretation of the SAC-C on the same day as a suspected concussion due to the lack of signi cant differences observed between the concussed and nonconcussed groups and highly variable diagnostic properties.Overall, our ndings demonstrate that the symptom evaluation may be the most effective component of the Child SCAT5 for differentiating between concussed and nonconcussed middle school children on the day of a suspected concussive event.

Figures
Figures

Figure 1 Flow
Figure 1

Table 2
Diagnostic properties for the Child Sport Concussion Assessment Tool 5 th Edition (Child SCAT5) scores.
which suggest that the symptom evaluation has the best combination of diagnostic properties and is the most effective component of the Child SCAT for differentiating between concussed and non-concussed children.Healthcare professionals in direct access settings (e.g., emergency department, outpatient clinics) may be the rst to evaluate a child following a suspected concussion[6] and likely will not have access to preinjury (baseline) scores for comparison.It is vital that healthcare professionals in direct access settings are equipped with age-appropriate assessment tools that can effectively differentiate between those who are and are not concussed in order to appropriately inform patient care.Our ndings reinforce the importance of the symptom evaluation of the Child SCAT5 and suggest that healthcare professionals can be con dent in the clinical interpretation of acute symptom reporting of children following a suspected concussion.The poorer diagnostic accuracy of the SAC-C and mBESS highlights the inability of these assessments to adequately differentiate between concussed and non-concussed children which limits their clinical utility on the same day as a suspected concussion.Future research should investigate alternative assessment tools (e.g., tandem gait test[68]) or strategies (e.g., the dual task paradigm[69]) for the acute evaluation of children following a suspected concussion.Findings from this future research may provide additional objective data to assist in the evaluation of children with a suspected concussion.