The following findings from the interviews and surveys at the two school districts are organized by the CFIR domains and summarized across the two sites and the various staff interviewed. CAI included specific quotes that demonstrate salient findings, labeled by the interviewees and their role; interviewees are also labeled by a randomly-generated identification number. Table 3 provides an overall summary of themes and quotes organized by CFIR domains and constructs. Tables 4 and 5 present key findings from the pre- and post-surveys and are embedded within the specific CFIR domains (Domains 1 & 5) that they sought to measure.
Table 3. Summary of emergent themes from qualitative interviews that influence RS implementation
CFIR Domain 1: Intervention Characteristics
The CFIR domain, intervention characteristics, contains constructs that influence the implementation of the RS (intervention), including evidence strength and quality, and complexity.
Evidence Strength and Quality
Overall, interviewees felt that schools are an ideal location to address the epidemic of STIs, HIV, and unintended pregnancy because students may not be able to access SHS resources at home or in their greater community. As one clinical services staff at a CBHP mentioned, “some young people don’t have supportive parents, they live in a strict environment…for them it provides an opportunity to be able to get access to sexual health which is so important.” (Participant 02-CBHP)
Most interviewees also felt that having a strong RS can ease students’ stress and anxiety in determining who and where to ask for help, thereby improving students’ ability to focus at school, school attendance, and educational outcomes.
The surveys sought to measure the construct of complexity overall, and interviewees reported a moderate level of difficulty implementing an RS (average score (AS)=5.50) (Table 2). Some components of the CCRS were rated as more difficult to implement than others, such as ensuring broad knowledge of the RS among staff and students (AS=5.79); developing district policy about making SHS referrals (AS=5.63); and developing district-specific referral procedures (AS=5.57).
During the interviews, participants overwhelmingly reported that the many layers of bureaucracy and the “chaotic” nature of the school system make implementation difficult. As one district staff mentioned, “the challenge we face is our bureaucracy…there's so many people that have to be involved that sometimes it hinders the process.” (Interviewee 04-District Staff)
Table 4. Results from pre-survey assessing perceived difficulty in implementation, and measuring the complexity construct
CFIR Domain 2: Outer Setting
The second domain, Outer Setting, explores external factors that may influence the RS, such as student needs, state and district policies, and partnerships.
Needs and Resources
Most of the interviewees believed that connecting students to SHS is a high need. As one interviewee states, “On a scale of one to ten based on how much students need SHS, I'd probably give it an 11". (Interviewee 02 -CBHP)
One reason is the high rates of STIs among the adolescents, specifically where the school districts are located: “We have a hot spot for chlamydia that we’ve seen, more outbreaks of syphilis than we have seen in the past.” (Interviewee 02-CBHP). Interviewees also expressed that many students are sexually-active but are misinformed about the risk, as they often share incorrect information with one another. Many interviewees expressed specific concerns for disproportionately affected populations, such as students of color and those identifying as LGBTQ.
In this context, cosmopolitanism can be defined as the degree to which each site is networked with other external organizations.17 While each site has myriad local and state partnerships, two main types of organizations emerged as crucial partners: local health departments (LHD) and CBHP. Interviewees noted that through the planning and implementation of the RS, partnerships among these organizations were strengthened. "It was a tenuous relationship in the past but…this grant has really increased our ability to access the school board and have a seat at the table" (Interviewee 03-CBHP)
Specifically for LHD, the interviewees expressed that they played a prominent role in supporting professional development (PD) sessions for teachers and staff, offering SHS to students through SBHC and assisting LEAs with monitoring and evaluation activities.
CBHP, described as “youth-friendly, LGBT-inclusive, more up-to-date than typical clinicians on the latest birth control methods and STI testing, HIV testing” (Interviewee 11-District Staff), also play active and crucial roles in the implementation of the RS. Interviewees reported their involvement in writing project proposals and work plans, providing SHS to students, delivering training and technical assistance to other CBHP, and in creating resources for LEAs (e.g., referral guides, policies, and staff PD curricula). Interviewees also felt that these partnerships increased CBHP’s awareness of state and local policies regarding minors’ access to health services.
Additionally, interviewees from the Southeastern site mentioned that the relationship between the organization and the school district has strengthened significantly in the last several years. In the past, the CBHP was not allowed on school campuses due to the larger political climate and a restrictive district policy. Several interviewees mentioned that through collaborating to create an RS, the partnership has been further solidified. One interviewee summarized this evolving relationship by saying, “people in the school system are much more open now and understand that having a relationship with a youth advocacy organization is a strengthening component”. (Interviewee 03-CBHP)
State and District Policies
While interviewees identified how some state laws facilitate RS—the Western site’s state law allows students to access STI services (including HIV testing and treatment) without parental consent, other policies serve as barriers—the Southeastern state’s guidance for abstinence-only education is an obstacle for schools to deliver comprehensive sexual health education. The Southeastern site’s state policy prohibiting school district staff from providing contraceptives, contraceptive counseling, and referrals for contraceptive services without parental consent, was also discussed as a barrier to student access to SHS: “not being able to talk about condoms or any contraceptives in the school is a humongous barrier” (Interviewee 12-CBHP).
CFIR Domain 3: Inner Setting
The third domain, Inner Setting, refers to the location where the intervention takes place. Constructs that were explored included networks and communications, culture and climate, leadership, and available resources.
Networks and Communications
RS in both sites had distinct networks and communication systems, drawing from pre-existing staffing structures and innovative marketing and communication approaches. In the Western site, the RS was supported by the existing network of the referral staff, who were nurses across different schools organized into teams called “clusters.” Clusters met monthly and often discussed issues related to referring students to SHS.
In the Southeastern site, one primary communication tool used to refer students to services is a palm-sized ‘chat card’. Staff who were knowledgeable about the RS refer students to the counselor or provide students via ‘chat cards.’ These include information about locations of teen health centers, types of services offered, and contact information. However, many interviewees from this site noted that the ‘chat cards’ might not be directive enough to link students to SHS.
Organizational Culture and Access to Knowledge and Information
In the Western site, most staff mentioned that the organizational culture is supportive of sexual health prevention services. However, they also mentioned that the county they work in is more conservative, relative to the rest of the state, and that negative attitudes and stigma towards sexual health education and services exist. Interviewees felt that other school staff did not feel capable of promoting the RS (e.g., linking students to designated referral staff or giving referral guides to students) due to fears related to parents, legal issues, and general lack of knowledge about the RS. One interviewee expressed that, “there is this nervousness that principals and other people have when kids go to confidential appointments when they're entrusted by their parents to be at school”. (Interviewee 16-District-Level Staff) Similarly, many staff in the Southeastern site highlighted the culturally conservative climate as negatively impacting the RS.
Since school principals have a lot of authority on the day-to-day operation of schools, they can influence the degree of RS implementation. Most interviewees at both sites reported that principals’ acceptance and commitment to implementing a sexual health RS varies. A few of the interviewees stated that school-building level champions were at times hard to identify due to lack of buy-in from principals. However, district-level leadership (e.g., superintendents) and CBHP were described overall as supportive, involved, and motivated. As one interviewee stated, “the leaders here are on board with it, and they really encourage us to get the program out there for the students”. (Interviewee 07-CBHP)
Many interviewees discussed the importance of having a full-time referral staff as crucial in the success of the RS. The Southeastern site mentioned that the district is spread out geographically and that public transportation is limited; therefore, their five SBHC are critical resources for students. As one interviewee stated, “it’s hard for a lot of the students to get to health clinics unfortunately, which is why it’s important that we are in the schools.” (Interviewee 05-District Level Staff). Challenges were also mentioned around SBHC, which included space, staffing, inadequate hours, and difficulty for students who attend other schools to access.
CFIR Domain 4: Characteristics of Individuals
The fourth domain describes the individuals involved with the RS, such as district staff, school staff, CBHP and leadership. Specifically, these individuals involved with the RS’ knowledge and beliefs regarding the importance of SHS emerged as a salient construct.
Knowledge and Belief
Across the two sites, interviewees reported that staff who were the most actively involved in RS (e.g., referral staff, program coordinator, champions) were also the most knowledgeable, especially regarding the logistics of making a referral, policies, and promoting student access to local health service providers. Interviewees stated that knowledge about the RS is generally higher among all staff and students if the school has a SBHC onsite or if the RS is heavily marketed through posters, flyers, and/or announcements.
However, interviewees reported that beliefs and attitudes towards the RS among school-building staff were mixed. Particularly in the Southeastern site, interviewees reported that their colleagues are hesitant to make referrals as they do not want to appear that they are endorsing adolescents having sex. One school district staff said “…We’re in the Bible Belt and I think some of the mindsets…about the way that children should be behaving…hinder the referral process.” (Interviewee 04—District Staff)
Additionally, many interviewees stated that despite having supporting policies in place at the state- or district-level, lack of knowledge or misunderstanding of the policies are barriers to SHS service provision. As one interviewee explained, laws surrounding consent are often difficult for school staff to comprehend because it is outside of typical practice for schools to allow students to do anything without parental consent. “The school system does not do anything with young people without parental consent and engagement, except for in rare situations” (Interviewee 03—CBHP). Furthermore, many interviewees mentioned that sometimes, local- and state-policies conflict, which is confusing for school and district staff.
CFIR Domain 5: Process
The last domain looks at the entire process of the RS through different stages of implementation. Salient constructs in this domain include engaging staff and students in the referral process and reflecting and evaluating.
Many interviewees spoke of the necessity of the Program Coordinator position to employ different strategies to engage the appropriate individuals to ensure successful implementation and use of the RS. As one CBHP manager stated, “I've been really impressed with her ability to sort of navigate all of this…a lot of it just falls on her. She's really coordinating with everyone and working with the schools and coordinating with the partner agencies.”
Also, PD proved to be important to engage designated referral staff (e.g., counselors, school nurses), to ensure that they know about CCRS and the health resources available for students, and to increase their knowledge in local STI, HIV, and pregnancy statistics.
Engagement of students mostly occurred through posters, flyers, palm cards, school announcements, banners, and calendars. Social media approaches (e.g., Instagram, Snapchat) are currently being developed as well in these sites. Health educators interviewed also identified incentives to help students fill out paperwork to take home to their parents to get permission for SHS, such as providing lunch during outreach and education encounters.
Reflecting and Evaluating
Methods to conduct ongoing tracking and evaluating of the RS varied depending on the school district. In the Southeastern site, the LHD develops tracking tools, collates data, and ensures data quality. In the Western site, the program coordinator developed a paper log for nurses to track data and distribute a monthly survey to collect data. Across the two sites, tracking referrals was identified as very difficult as the current data collection systems does not capture passive referrals (e.g., self-referral when student seeks out SHS on his/her own after seeing a poster) or efforts that do not result in a referral. Further, only the staff who were most involved in RS implementation regularly receive data reports and take the time to use data to evaluate their work.
The sustainability construct was assessed through post-interview surveys to measure perceived difficulty implementing an RS in the future absence of funding. Interviewees reported a high level of overall difficulty (average score=7.96) (Table 5). In particular, continuing to provide management and oversight regarding RS implementation (average score=8.54) was rated as the most difficult component.
Table 5: Results from post-survey assessing complexity and perceived difficulty in sustaining implementation