We conducted qualitative interviews with PACTS clinicians working in public mental health settings across Philadelphia in 2018. PACTS clinicians were asked about their decision-making processes in implementing the TN. We adapted a behavioral insights approach to systematically stage the analysis—Narrow, Understand, Discover, Generate, and Evaluate (NUDGE)—and coded the interview data using a guide from the behavioral insights literature—the Behavioral Economics Guide—to arrive at behaviorally informed hypotheses about the determinants of clinicians’ TN use (36,37). We leveraged these hypotheses to generate implementation strategies using the behavioral insights informed “Easy Attractive Social and Timely” (EAST) framework, which organizes strategies for researchers and policy-makers (38).
Participants and Study Procedure
Participants were clinicians who had completed training in TF-CBT through PACTS. Clinicians were contacted either by e-mail or at “booster” training session in spring of 2018 and asked to complete a 10-15-minute survey about their perceptions of and past use of TNs. See (39) for more information about the initial survey clinicians completed. Of the 65 clinicians that completed the survey, a subset (n=26) were selected for in-depth qualitative interviews using purposive sampling. Participants who completed qualitative interviews were sampled in order to capture variability in clinician use of the TN. As part of the survey, participants were asked to indicate the percentage of their TF-CBT clients in the past six months with whom they used the TN; whether participants intended to use the TN with their TF-CBT clients in the next six months; and how likely it was that they would use the TN with their TF-CBT clients in the next six months. Clinicians fell into three groups and were purposely sampled from each, including: 1) clinicians with high intentions and high likelihood of using TNs, but who had used TNs with none or few clients in the past (n=8); 2) clinicians with high intentions and high likelihood of using TNs who reported using TNs with all or most of the their clients in the past (n=5); and 3) clinicians who reported low intentions but medium to high likelihood of using TNs who had variable levels of past TN use (n=4). Of the 26 participants who completed the survey and were invited to partake in the qualitative interviews, 17 participants completed interviews by phone or in person (65%). Those who declined either did not respond to attempts to contact them or reported insufficient time to complete an interview. All procedures were approved by the City of Philadelphia and the University of Pennsylvania Institutional Review Boards.
Semi-structured interviews focused on clinician perceptions of the TN, as well as factors that interfere with or assist their use. Several questions prompted clinicians to consider their most recent sessions with a client and the determinants to TN implementation in a single session (40). These questions were intended to elicit in-depth descriptions of clinicians’ judgment and decision-making in order to analyze the interviews using behavioral insights (see Appendix A for the interview guide).
Each participant completed one interview lasting between 30-60 minutes. The interviews were audio-recorded and conducted individually in person or over the phone. Interviews were conducted by BSL and HEF, both doctoral students with familiarity with TF-CBT and the PACTS study. Interviews were recorded and later transcribed by undergraduate research assistants. Participants received a $50 gift card for completing the hour-long interview.
We used an integrative approach to code the data informed by thematic analysis and a flexible adaptation of existing approaches from the behavioral insights literature to enrich our interpretation of the qualitative data. As no single approach was sufficient to guide the hypothesis generation process, our study team integrated several guides, coding processes, and frameworks from the behavioral science literature into our analyses. Broadly, our analytic approach had three major phases, elaborated extensively below.
First, in order to distill qualitative interview transcripts, thematic analysis was applied to organize the qualitative data into a manageable and interpretable amount of text (41). Second, in order to systematize the hypothesis generation process, we selectively borrowed elements from the NUDGE framework, which has been used to design behavioral insights-derived implementation strategies based on hypothesized determinants (36). To structure this part of the analytic process, we relied heavily, though not exclusively, on the Behavioral Economics Guide to code hypothesized behavioral insights determinants of TN implementation (37). Third, in order to generate behavioral insights informed implementation strategies, we used EAST as a design framework (38).
NUDGE is a behavioral insights approach to rigorously identify what drives EBI implementation (36). NUDGE lays out a multi-step process from “Narrowing” the focus to a specific behavioral target through “Understanding” the context of the behavior, “Discovering” the underlying behavioral insights, “Generating” implementation strategies, and “Evaluating” them through trials. In previous work, the NUDGE approach was used to analyze qualitative data to discover what drives EBI implementation in publicly funded mental health agencies (36). We adapted the “Discover” step of NUDGE into a coding process in which we applied codes for various behavioral insights largely drawn from “The Behavioral Economics Guide” (37). Note that this guide is not exhaustive, and that given their training, coders were also familiar with other behavioral insights guides that they drew upon in this step (42). To structure the “Generate” step of NUDGE, we used the EAST framework to propose behavioral insights-derived implementation strategies (38). EAST was developed by the UK Behavioral Insights Team, a group of scientists and policymakers who apply findings from cognitive science, social psychology, and behavioral economics to a host of policy domains. EAST was developed as a practical and comprehensive tool for researchers and practitioners to arrange evidence in a digestible format. EAST primarily organizes behavioral insights strategies according to the principles that underlie their effectiveness. These strategies work because they make the optimal choice easier, more attractive, more social, and/or timelier than other choices. EAST offers a structured way to comprehensively consider all the different mechanisms by which to address hypothesized implementation determinants.
It is important to note that in the current study we did not generate an exhaustive list of all potential implementation strategies. Rather, we designed several possible strategies to illustrate the promise of this structured brainstorming process informed by behavioral insights and lived experience.
Behavioral Insights Coding Process. Figure 1 displays the multi-step process we used to analyze the qualitative interviews in detail. In Step 1, three investigators (BSL, CET, and SHS) separately reviewed all of the qualitative interview transcripts. Investigators met and coded the determinants of TN use. Codes reflected what clinicians stated got in the way and what helped them use the TN with clients and came directly from clinicians’ responses to questions about the barriers (i.e., what prevents) and facilitators (i.e., what helps) TN implementation (see Appendix A for interview questions), whereas other TN determinants were inferred. For example, some codes were “complex trauma,” “community violence,” “back-to-back sessions.” In Step 2, using a thematic analytic approachIn Step 3, as a validation check, investigators coded four transcripts together, and further synthesized and deduplicated the codes. Through discussion and consensus, coders distilled and reduced the codes and broad themes.
In Step 4, the three investigators (BSL, CET, and SHS) then mapped the TN determinants, coded in the clinicians’ own language, onto a predetermined set of behavioral insights that are described in the “Behavioral Economics Guide of 2018” (37). Table 1 provides definitions of the behavioral insights that we mapped onto the TN determinants (for a full list of the possible insights see the guide). There were a few cases in which well-established behavioral insights were not listed in the guide. For example, “reinforcement” is a psychological principle (43) that is not explicitly in the guide (though related insights such as “incentives” are). When these occasions arose, coders discussed whether it was appropriate to include these behavioral insights in our coding process, which resulted in several additions. We consider the few additions to be behavioral insights insofar as they derive from the behavioral science literature, have been shown to determine behavior, and can be shaped through behavioral insights strategies. In this step, it was possible for several behavioral insights to map onto one TN determinant, which explains why several behavioral insights are associated with one TN code (see Table 3). For example, “social norms” and “defaults” map onto the TN determinant code relating to common practices at a particular agency.
In Step 5, to ensure the validity of the TN determinants and behavioral insights generated from Steps 1-4, we conducted an “expert validation check.” The first author (BSL; a graduate student in clinical psychology with clinical experience conducting TF-CBT and research expertise in clinical decision-making) worked with AMB (an expert in behavioral insights) and RSB (a licensed clinical psychologist and expert in implementation science) to validate the TN determinants and behavioral insights based on the literature and their research and clinical expertise. After this final list of hypotheses was validated, for Step 6, BSL and RSB integrated the behavioral insights and implementation science literature to generate implementation strategies, using the EAST framework to structure this process. EAST helps link the hypothesized behavioral insights to the implementation strategies mechanistically.