Variable reliability of components of the QTTAC-PREM was observed when applied to hospitalized injured adolescent patients and their parents. These observations can be attributed to situational and survey administration factors.
Many patients only stayed 1 or 2 days in hospital making the need for education supports inconsequential. In addition, 13 of the 17 months over which this study was run was during the SARS-CoV-2 (COVID-19) pandemic, when on-line schooling prevailed for many schools in the trauma centers’ catchment areas. These two situations likely resulted in questions related to school support needs less relevant to all respondents; superficial reflection on these could thereby make them prone to poor test-retest results. Alternatively, needs for outside supports for post-discharge issues such a homecare, and dealing with frustration and school performance issues, may have been compounded by limited access to supports because of the pandemic, confounding responses between the test and retest phases. Similarly, poor reliability on questions related to social network supports may have affected questions related to having friends visit. During the pandemic, visitors were extremely limited in acute care settings, likely rendering these questions inconsequential and subject to inference rather than true experience.
Asking questions pertaining to in-hospital care, eight weeks post-discharge, may have decreased their reliability due to respondent recall. Amongst parent responses, a stronger directionality, namely in-hospital experiences being reported as more negative and post-discharge experiences as more positive suggest that overall perceptions of these two phases may have changed somewhat overtime. If recall bias is influencing the retest results, the lack of clear overall directional change of responses over time with the adolescent group may suggest more randomness of their retest responses. Similarly, poor reliability of questions pertaining to post-discharge supports and information/communication may be related to timing of survey administration if some patient completed the first survey prior to follow-up with their health care practitioner and the re-test after follow-up.
Recommendations And Future Directions
Given the limitations outlined above and, notably, the situational factors that may have influenced the test-retest process (COVID-19 pandemic), we would still recommend use of the QTTAC-PREM for examining adolescent and parent experiences during in-hospital and post-discharge care of adolescents hospitalized for traumatic injuries. Retesting of reliability of sections pertaining to maintenance of social networks and schooling reliability and supports needed post-discharge, when there is no concurrent pandemic, may be of value. We also recommend it be delivered in two phases temporal to the in-hospital and post-discharge phases of care, to avoid recall bias.
The COVID-19 pandemic likely influenced the interpretation of some of the questions related to supports for schooling and maintenance of social networks, suggesting need for repeat reliability testing outside the pandemic.
The potential for recall bias must also be acknowledged as the survey inquired about in-hospital care but was administered at eight weeks post-discharge. This 8-week period was chosen with an assumption that most patients would have had their follow-up care within this period. However, the effects of the pandemic on scheduling follow-up appointments may have made this assumption invalid and we would suggest future querying of post-discharge experiences include a screening item to exclude respondents who have not yet had follow-up care. Finally, administering survey questions relating to in-hospital versus post-discharge care at time frames more temporal to these phases of care should be considered.
Our reliability study may also have been limited by our sample size. We note the original study examining the reliability of the QTAC-PREM SF included 117 retest cases, with a calculated sample size requirement of 75 [6, 7]. We were only able to recruit 40 patients and 40 parents and although there were similarities in reliability results between the two respondent groups, variations in directionality suggest that they may not be comparable.