Quantitative Analysis
Demographic characteristics of participants (n = 26) are displayed in Table 1. Most participants were white women physicians. Participants were representative of the overall health system primary care provider population with respect to provider sex and clinical degree. Almost all had heard about PrEP prior to the study and over one-third self-reported prescribing PrEP. Providers rated the potential use of EHR-based CDS tools to improve HIV testing and PrEP delivery as acceptable (median score 5, IQR [4–5]) and appropriate (5, IQR [4–5]). However, feasibility ratings were slightly lower (4, IQR [3.75–4.75]), with 26% of participants disagreeing that implementation would be “easy to do.” Of the seven proposed CDS tools (Table 2), the highest-ranking was the patient electronic self-screening tool, followed by alerts to providers for HIV screening for patients with STIs. The lowest overall ranking tool was an alert for PrEP-eligible patients based on STI test results.
Table 1
Demographics and provider experience in adolescent primary and sexual health care
Provider Characteristics (n = 26) | n (%) [Median (IQR)] |
Age (years) | 46 (38–56) |
Race | |
White | 24 (92) |
Black/African American | 1 (4) |
Declined to Answer | 1 (4) |
Ethnicity | |
Hispanic or Latino | 1 (4) |
Non-Hispanic or Latino | 25 (96) |
Sex | |
Male | 3 (12) |
Female | 23 (89) |
Gender | |
Male | 3 (12) |
Female | 23 (89) |
Primary practice location | |
Government subsidized (“Title X”) Clinic | 15 (58) |
Non-Title X clinic | 11 (42) |
Years in practice | [18.5 (8–25)] |
Clinical degree | |
Doctor of Medicine/Doctor of Osteopathic Medicine | 19 (73) |
Certified Registered Nurse Practitioner | 7 (27) |
Adolescent medicine training | |
Yes | 7 (27) |
No | 19 (73) |
Number of adolescent patients seen per month | [40 (15–60)] |
Frequency of documentation of sexual history in EHR | |
Infrequently or never | 1 (4) |
About half of the time | 2 (8) |
Most or all of the time | 23 (89) |
Prior knowledge of HIV pre-exposure prophylaxis (PrEP) | |
Yes | 23 (89) |
No | 3 (12) |
Prior experience prescribing PrEP | |
Yes | 10 (39) |
No | 16 (62) |
Prior experience referring patients to another clinical site for HIV PrEP | |
Yes | 13 (50) |
No | 13 (50) |
Table 2
Ranking of electronic health record (EHR)-based CDS options for HIV/STI prevention care
CDS Tool | Top 3 (Ranks 1–3) n (%) | Middle 2 (Ranks 4 & 5) n (%) | Lower 2 (Ranks 6 & 7) n (%) |
Electronic questionnaire for adolescents that populates sexual activity data into EHR2 | 14 (56) | 8 (32) | 3 (12) |
Automated referrals to HIV prevention counselors for patients with STIs | 13 (50) | 4 (5) | 9 (35) |
Electronic alert to test for HIV for patients with STI diagnoses | 13 (50) | 8 (31) | 5 (19) |
Electronic alert to test for HIV in all adolescents | 12 (46) | 6 (23) | 8 (31) |
A bundled tool with both alerts and templated HIV test orders | 10 (39) | 9 (35) | 7 (27) |
An alert to counsel regarding PrEP for youth with STIs | 8 (31) | 9 (35) | 9 (35) |
Default HIV testing orders within the adolescent visit template (providers must uncheck to cancel order) | 8 (31) | 7 (27) | 11 (42) |
1 Ranking from 1–7, where 1 is highest rank and 7 is lowest rank. |
2 Missing n = 1. |
Qualitative Analysis
The 10 unique decision-making steps of the pediatric well visit identified in the work domain analysis are displayed in Fig. 1. Key themes describing contextual barriers and facilitators of implementing CDS within the pediatric visit workflow are described below with exemplar quotes displayed in Table 3.
Table 3
Theme | Exemplar Quotes |
Theme 1: Optimizing CDS integration within the primary care workflow (CFIR domain: Intervention) | “I think having something come up for HIV testing if their chlamydia is positive isn’t as helpful because by then the patient’s out of the office, so I think if we’re going to be ordering it, it makes more sense to order it at the time of the visit regardless of what the results of the [STI test] are” |
"…I think, you’d find all the people onboard. I think, as I said, just please pilot things. Because we always start getting told that things are getting piloted, but sometimes they just get introduced. And then, having like…someone who doesn’t do this all the time onboard to kind of give feedback…” |
“It comes from feeling like all we do is click now. It comes from feeling like we can’t – we’re not talking to our patients, we’re clicking boxes….” |
Theme 2: Designing for standardization, with room for adaptable CDS design (Intervention) | “I would strongly suggest in areas where chlamydia prevalence is greater than 20 percent that we engage in universal screening, for at least GC and chlamydia, and then maybe starting HIV at 15, doing it with rapid HIV, removing stigma, having everyone screened before they even see their provider, to remove bias.” |
“If it’s a negative screen it’ll say negative screen. If it’s a positive, then it’ll give you suggestions and links on things that you can do. And if it’s emergent it’ll also save that for you. So, you want the tool to kind of not just ask the questions but help guide you through what to do with the answers.” |
Theme 3; Inner Setting (III) Recognizing the need for confidential care (Outer Setting) | “…I feel like it’s possible that a female patient with private insurance might be more inclined to not want their parents to know – which would decrease testing.” |
Theme 4: Improving accurate HIV risk perception (Outer Setting, Individuals) | “I think females just don’t perceive that they’re as much at risk [of HIV infection]…adolescents just have a very skewed view of risk in general.” |
“I think education ..about numbers of people that are HIV positive in proportion of males versus females would be helpful…maybe looking at the data and seeing who's being tested and seeing where we miss might help us realize that we should be testing more than we are – so doing some data analysis and looking at the percent of males versus females that I test personally may really hit at home for me that I'm falling short even though right now I feel like I test as many, but I might not be.” |
Theme 5: Recognizing limitations of time and staff as available resources (Inner Setting) | “I know one of our nurses at [clinic] has been thinking about sort of like a much more nurse-led visit. And I think that could be something that allows you to better use all of your team members and take advantage of the time that the families and the – like less wasted time with them sitting around and more time when people are really working with them to address issues and prioritize.” |
Theme 6: Prioritizing HIV relative to other primary care needs (Inner Setting) | “I think, again, importance in priorities, and a lot of things are happening at that [visit], and you’re dealing with depression and school issues, and so [providers] have to pick and choose.” |
“I don't think people here deliberately ignore PrEP. I think in the grand scheme of everything that you have to deal with, with a teenager and you’re given 15, maybe 30 minutes at best… If you want to talk about every single thing that you could possibly talk about that this kid needs, you're well over an hour visit.” |
Theme 7: Increasing provider knowledge about HIV testing and PrEP (Individuals) | “So, I have never prescribed PrEP, so I wouldn't feel 100 percent comfortable prescribing it. But I would feel comfortable saying that she should consider that and recommend it.” |
Theme 1: Optimizing CDS integration within the primary care workflow (CFIR domain: Intervention)
Providers stressed the importance of placing CDS within the “right step” of the visit workflow, with particular consideration to efficiency and confidentiality. For example, one provider noted that an EHR alert for PrEP counseling should not be triggered when a patient’s parents are in the room due to risk of breaching confidentiality. Another consideration was a desire for CDS to be deployed while a patient is still physically present in the clinic, to ensure CDS could be acted upon immediately, as opposed to during EHR chart documentation which often occurs at the end of a provider’s day. Related to timing, participants expressed a desire for any new CDS to be appropriately piloted prior to implementation, recalling negative experiences with previous primary care EHR-based CDS tools that had not been piloted and were unsuitable for their practice's workflow. Lastly, providers identified the need to consider the presence of other CDS in the workflow; too many unrelated CDS systems could contribute to a sense of burnout. While some providers highlighted successful changes in their own practice stemming from existing immunization and asthma-related CDS, providers also noted feeling overwhelmed by the sheer volume of EHR alerts and nudges. They raised concerns about additional CDS increasing the potential for alert fatigue, the desensitization to EHR alerts due to an overwhelming number of alerts.
Theme 2: Designing for standardization, with room for adaptable CDS design (Intervention)
Providers raised the need for CDS to be standardized for all patients, irrespective of HIV risk factors. Universal application of CDS could eliminate the need for the provider to do risk-based calculations using sexual history and lab data, thus reducing cognitive load. Universal CDS was also noted to have the potential to minimize provider implicit bias, particularly sex-based bias, in decision making around HIV testing and PrEP. Finally, providers felt that standardization would increase fidelity to clinical practice guidelines and avoid patient stigma surrounding HIV testing. As a complementary approach, other providers desired CDS that could tailor the intensity of recommended resources and clinical guidance based on individual HIV risk. These providers desired algorithmic approaches using EHR data, such as STI results, to further customize decision support. In this approach, the format/type of CDS (e.g., alerts to test for HIV, links to a templated PrEP order set with CDC guidelines) could parallel the level of the patient’s HIV risk.
Theme 3: Recognizing the need for confidential care (Outer Setting)
Similar to the ubiquitous barrier of time pressure, the need to protect confidential sexual health information was also recognized as a consideration at each step of the visit. Participants noted that CDS alone would be unlikely to yield gains in HIV prevention in the absence of changes to funding and insurance billing practices to protect minor confidentiality. Specifically, providers discussed the utility of increasing access to federal Title X funding to allow expansion of confidential sexual health services. Providers had concerns about threats to confidentiality for privately insured patients due to explanations of benefits for HIV testing or PrEP services being sent to home addresses, thus risking disclosure of unknown sexual activity and/or orientation, to patients’ parents. Providers noted that confidentiality concerns may be particularly salient for female patients as providers perceived that adolescent girls typically experience more stigma around sexual activity than adolescent boys.
Theme 4: Improving accurate HIV risk perception (Outer Setting, Individuals)
Many providers noted that their patients, particularly those with STIs, did not perceive themselves as at-risk for HIV or in need of PrEP. This mismatched risk perception was seen as a barrier to HIV testing and PrEP uptake, even with effective CDS implementation, suggesting that additional interventions augmenting CDS would be needed to increase HIV testing and PrEP uptake. Distinct from adolescent patients’ self-assessment of their HIV risk, providers’ perceptions of patient risk also influenced their clinical decision-making about HIV testing and PrEP. Some providers expressed that CDS focused on universal HIV prevention could be a waste of resources, perceiving that cisgender female adolescents are at very low risk of acquiring HIV.
Theme 5: Recognizing limitations of time and staff as available resources (Inner Setting)
Providers emphasized that CDS development should consider clinic time as a critical resource, given the common challenge of addressing HIV prevention with the extensive array of clinical responsibilities at well visits. Providers also expressed the need for additional resources to maximize CDS effectiveness, such as rapid HIV tests, implicit bias training for providers, and patient education materials. Providers noted that time challenges were often greatest for female patients, given the need to also discuss menstruation and contraception during sexual health conversations, diminishing time to address HIV testing and PrEP counseling. With respect to insufficient staffing, participants suggested that CDS alone would not improve HIV prevention without changes to staff roles and workflow. Providers stressed the need for involvement of clinic staff other than primary care providers to optimize HIV testing and prevention counseling. Two examples included introducing medical assistant-led HIV rapid testing and nurse-led HIV prevention counseling, in which the responsibilities of pre-test counseling and result delivery were shifted to allied members of the healthcare team.
Theme 6: Prioritizing HIV relative to other primary care needs (Inner Setting)
Many providers felt that while HIV and PrEP were important topics, they fell below some of the more prevalent and immediate health concerns, such as depression, school performance, and obesity. In addition, providers reported concerns about having enough time to address HIV and PrEP given the need to prioritize other responsibilities, such as depression screening, that are mandated performance measures within the health system. Perceptions of the relative priority of HIV also influenced providers’ views surrounding the implementation of HIV prevention CDS. Some providers had concerns about resistance from fellow providers if these providers felt “forced” to discuss PrEP without sufficient knowledge or time to do so. However, others noted that their clinic would and should be prepared to talk about HIV prevention and PrEP as part of their purview as primary care providers.
Theme 7: Increasing provider knowledge about HIV testing and PrEP (Individuals)
Providers noted that their knowledge (or lack thereof) regarding HIV testing and PrEP would influence how effectively they could utilize HIV prevention CDS. For example, for providers who reported no prior knowledge of PrEP before the study, CDS would also need to efficiently provide education or links to clinical practice guidelines about HIV prevention care or be accompanied by separate educational efforts to increase competency across clinical staff.
Integration of findings: In Fig. 2, we synthesize our key findings and provide an Implementation Research Logic Model for CDS implementation for HIV prevention-oriented pediatric primary care. Within the model, we propose seven essential strategies for CDS implementation grounded in the ERIC taxonomy: 1) Conducting cyclical usability testing to optimize CDS within the workflow, 2) adapting to context by using pre-visit confidential patient-collected sexual health information to trigger CDS early in the visit, 3) providing a strong “nudge” toward equity by introducing defaulted HIV testing orders to improve universal testing, 4) using EHR alerts to remind providers of need for HIV prevention within busy clinical care encounters, 5) embedding clinical practice guidelines within CDS to increase provider knowledge, 6) expanding clinical teams to shift HIV prevention responsibilities away from pediatricians and APPs and 7) educating patients to increase awareness of their need for HIV testing.