Community and Partner Engagement in Dissemination and Implementation Research at the National Institutes of Health: An Analysis of Recently Funded Studies and Opportunities to Advance the Field

doi:10.21203/rs.3.rs-2476446/v1

Download PDF

Research

Community and Partner Engagement in Dissemination and Implementation Research at the National Institutes of Health: An Analysis of Recently Funded Studies and Opportunities to Advance the Field

https://doi.org/10.21203/rs.3.rs-2476446/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 11 Jul, 2023

Read the published version in Implementation Science Communications →

You are reading this latest preprint version

Background: As the focus has grown in recent years on both engaged research and dissemination and implementation (D&I) research, so too has federal funding to support these areas. The purpose of this analysis is to provide an overall perspective about the range of practices and approaches being used to engage partners in D&I research, with special attention to disparities-relevant research, and to identify gaps and opportunities in research funded by the U.S. National Institutes of Health (NIH) in this space.

Methods: This analysis examined a portfolio of active D&I research grants funded in fiscal years 2020 and 2021 across the NIH. Grant applications were deductively coded and summary statistics were calculated. Cross-tabulations were used to identify trends by engagement and disparities foci.

Results: There were 103 grants included in the portfolio, of which 87% contained some form of community or partner engagement, and 50% of engaged grants were relevant to health disparities. Engagement was planned across the research continuum with each study engaging on average 2.5 different partner types. Consultation was the most common level of engagement (56%) while partnership was the least common (3%). On average, each study used 2.2 engagement strategies. Only 16% of grants indicated formally measuring engagement. Compared to non-disparities studies, disparities-relevant studies were about twice as likely to engage partners at the higher levels of partnership or collaboration and were also more likely to be conducted in community settings (26% vs. 5%).

Conclusions: Based on this portfolio analysis, D&I research appears to regularly integrate engagement approaches and strategies, though opportunities to deepen engagement and diversify who is engaged remain. This manuscript outlines several gaps in the portfolio and describes opportunities for increasing engagement to improve the quality of D&I research and application to advancing health equity. In addition, opportunities for leveraging the consistent and systematic application of engagement approaches and strategies to advance the science of engagement are discussed.

community engagement

partner engagement

health disparities

health equity

dissemination and implementation

This analysis of D&I research grants funded by the National Institutes of Health (NIH) provides evidence that community and partner engagement is common.
These findings contribute to a gap in the literature by describing the extent and qualities of engagement activities in NIH-funded D&I research and the intersection of engagement and disparities-relevant research in the D&I portfolio.
These findings also contribute to a gap in the literature by outlining opportunities to increase and deepen engagement efforts in support of NIH’s commitment to health equity and NIH’s mission to generate knowledge and the application of that knowledge to enhance health.

In recent years, it has become increasingly clear that for efficacious health and public health interventions to be effectively implemented, broadly scaled up, and sustained over time, there needs to be increased appreciation for and investment in areas of scientific inquiry that can provide insight and perspectives as to how to best reach these goals. Specifically, leveraging the behavioral and social sciences, community engagement, health equity, and implementation science models and methods is critical to advance such efforts.

Dissemination and Implementation (D&I) research at the U.S. National Institutes of Health (NIH), essentially synonymous with implementation science, has evolved over the past few decades to now include dedicated staff, a standing program announcement (1–3), a study section (4), an annual conference (5), and ongoing training programs for extramural investigators (e.g., Training in D&I Research in Cancer [TIDIRC]) (6). In addition, most Institutes, Centers, and Offices (ICOs) feature some aspect of D&I research in strategic plans or initiatives as there is an increasing recognition of the value of implementation science approaches to facilitating effective uptake, sustainability, and scale up of efficacious medical and behavioral interventions to impact public health.

Several recent efforts have recognized the importance of integrating the methods and approaches of community engagement with D&I research to enhance rigor and relevance and to address health disparities. A special issue of Translational Behavioral Medicine highlighted research gaps and opportunities and included a reflection on NIH efforts in this space (7, 8). There appears to be broad interest in bringing not just community, but also other partner engagement into D&I research. This is evidenced by an increasing number of perspective pieces and commentaries highlighting the potential benefits of this integration to both research and practice, outlining examples of federal efforts to support this research, and offering of potential frameworks to accomplish that integration (9–13). There have also been a few published case studies documenting interventions that worked to integrate engagement in D&I research for a diverse range of health and related outcomes including violence, heart disease, and cancer (14–16). It is suggested that this integration of engagement methods and approaches into D&I research can lead to improved health outcomes, but this has yet to be documented in most cases (17).

One of the major lessons learned from the experiences during the COVID-19 pandemic is that even with the best scientific advancements (such as vaccines), these life-saving interventions will not be widely taken up and scaled without a concerted effort to understand and deploy implementation strategies that match each unique setting, context, and population. A primary goal of D&I research is to achieve broad, sustainable use of an evidence-based program, policy, or practice, which requires inclusivity, partnerships, and shared ownership and buy-in from a range of partners and community members. As a result of this understanding, the NIH launched two major COVID efforts designed to focus on community engagement, partnerships, and D&I approaches in order to engage populations most at risk and who simultaneously face the most barriers to access and participation in COVID testing (Rapid Acceleration of Diagnostics-Underserved Populations) (18) and vaccine uptake (Community Engagement Alliance against COVID-19) (19). Further, in 2022, a new NIH Common Fund initiative was launched, Community Partnerships to Advance Science for Society (ComPASS), which focuses on advancing health disparities and health equity research through community organizational partner-led, multisectoral research.

The few examples above notwithstanding, there remain clear and notable gaps in community and partner engagement in D&I research. There do not appear to be many “standard” D&I studies that routinely and rigorously integrate community or partner engagement approaches into their work, and engagement is not well-integrated into most D&I theories, models, or frameworks (20). Further, implementation scientists may not have the competencies necessary to lead engaged or participatory research (21). Given recent public attention to social inequities and health disparities, it seems imperative that D&I research be done in partnership with the communities that are most in need and the organizations that serve them. Yet, D&I research often fails in this regard. Indeed, a recent reflection on “promises and pitfalls in implementation science” by ten leading mid-career implementation scientists identifies this lack of connection with community partners and lack of focus on health equity as fundamental to the field’s challenges (22). These identified pitfalls include re-creation of the research to practice gap, misalignment of timelines incentives and priorities with community partners, and complexity of strategies that may not match community needs. These authors offer several ways forward, many of which involve increased attention to better integration of engagement approaches to inform D&I efforts since community engaged research is rooted in social justice and advancing health equity (23–25).

Given this imperative and increased attention to health disparities and health equity, several NIH initiatives across multiple ICOs over the last few years have attempted to stimulate the field. These initiatives invest in programs of research that seek to bring community and partner engagement into D&I research to tackle entrenched health disparities and challenges related to diseases such as opioid addiction, HIV, cancer, and environmental health (Fig. 1). [INSERT FIGURE 1 NEAR HERE] These efforts are promising, yet we currently lack a comprehensive understanding of how NIH investments are working to support this integration, whether they are in fact successfully stimulating the field to conduct studies that increasingly incorporate and integrate engaged research approaches in D&I research. Such an understanding would allow us to identify gaps and opportunities to further stimulate and support the field and ultimately to produce more rigorous, relevant, and impactful public health science. Thus, we undertook the current analysis to provide such insight.

The purpose of this analysis of NIH-funded D&I research from the last two years is to provide a snapshot of the current range of practices and approaches being used to engage communities and partners in D&I research, with special attention to disparities-relevant research, and to identify gaps and opportunities. To that end, the research questions we sought to answer were as follows:

To what extent and how are community members and other partners engaged in recent NIH-funded D&I research?
Are there common themes that emerge with respect to engagement across implementation phases, strategies, or other key domains?
1. To what extent are NIH-funded D&I-engaged research studies relevant to health disparities?
2. How do NIH-funded D&I research studies relevant to health disparities involve community or partner engagement?

This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for reporting observational studies (26). This analysis examined active research grants funded in fiscal years 2020 and 2021 across all NIH ICOs that were funded through D&I program announcements (1–3) or reviewed in the Science of Implementation in Health and Healthcare (SIHH) (4) (Dissemination and Implementation Research in Health (DIRH) through 10/2020) study section. An internal NIH portfolio analysis platform, iSearch, was used, which provides comprehensive, easy-to-use access to carefully curated, extensively linked datasets of funded and unfunded grant applications, historic and current.

The study team developed a codebook directly stemming from the research questions through an iterative process that included review of other NIH-specific portfolio analysis codebooks and publications. There was also a review of the literature on D&I and community and partner engagement, and discussions among the study team and collaborators. Grant characteristics were extracted from the metadata in iSearch. Variables coded captured information about study population demographics; details regarding interventions under study and where and by whom they are delivered; and details about the study design and methods (27, 28), implementation phase, implementation strategies used (29), and use of theories, models, and frameworks (30, 31). Grants were coded for where on the research continuum (e.g., intervention design, evaluation, etc.) and through what approaches (e.g., practice-based research (32), community-based participatory research (CBPR) (33), etc.) engagement was undertaken. Grants were also coded for the planned audience for engagement according to the 7Ps framework proposed by Concannon et al. (34) with definitions adapted from the Patient Centered Outcomes Research Institute (PCORI) (35), the highest level of engagement described according to the continuum proposed by Sanders Thompson et al. (36), and any indicators of equity in engagement as discussed by Key et al. (37). [INSERT FIGURE 2 NEAR HERE] Grants were also coded as disparities-relevant if they related to minority health or health disparities, as defined by the National Institute on Minority Health and Health Disparities (38). The codebook was programmed into iSearch’s curation tool and piloted by two members of the study team.

Full text grant proposals were accessible through iSearch. Each grant proposal (research summary sections, human subjects, and budget justifications) was independently coded directly in iSearch by two assigned coders. Coding discrepancies were resolved through discussion to consensus between the coding pairs and amended in iSearch. If intercoder consensus could not be reached, a third coder from the study team reviewed the grants and assisted in resolving disagreements. Coders (N = 21) were generally assigned to grants from their own ICOs, with additional support provided by the NIH Office of the Director. The study team held office hours three times over the course of the coding period to answer questions and offer clarifications for coders to facilitate reliability. Summary analyses, including descriptive statistics, were used to answer the first research question and two-variable cross-tabulations were used to answer the second research question. All analyses were conducted in an Excel database exported from iSearch.

A total of 103 grants met the inclusion criteria and were funded by 16 different ICOs (Table 1). A majority (57%) of grants were funded through the DIRH program announcements, 87% were reviewed by the SIHH/DIRH study section, and most were R01s (64%) or R21s (22%). Ten grants did not involve interventions and rather were solely focused on developing databases, conducting modeling, developing or validating measures, retrospectively studying policies or grants portfolios, or were in early pre-implementation exploratory phases prior to developing an intervention. Over two-thirds of grants (69%) targeted two or more levels of the social-ecological model; predominantly the individual (66%) and organizational (62%) levels, with the fewest targeting the societal level (6%). The most common intervention settings included healthcare organizations (67%) and the community (18%). Accordingly, common intervention implementers included healthcare providers (63%), lay health personnel (17%), clinic support staff (15%) and clinic administrators (13%). Less common implementers included policymakers (4%), social service workers (5%), public health professionals (7%), and implementation support practitioners like knowledge brokers, technical assistance providers, or practice facilitators (8%). [INSERT Table 1 HERE]

Table 1

Characteristics of included grants (N = 103)
	Number (%)
Funding ICO
NCI	27 (26)
NHLBI	17 (17)
NIMH	16 (15)
NIDA	11 (11)
NICHD	9 (9)
Others	23 (22)
Study section
DIRH/SIHH	90 (87)
Others	13 (13)
RFA PAR #
DIRH PAR (19–274 or 19–275)	59 (57)
Others	44 (43)
Funding mechanism
R01	66 (64)
R21	23 (22)
R37	6 (6)
Others	8 (8)
Fiscal year
2020	48 (47)
2021	55 (53)
Intervention, Yes	93 (90%)
Intervention Level*
Individual	68 (66)
Interpersonal	24 (23)
Organizational	64 (62)
Community	19 (18)
Societal	6 (6)
Intervention Setting*
Community setting or community-based organization	19 (18)
Faith-based organization	4 (4)
Health department	2 (2)
Healthcare	69 (67)
Home	8 (8)
Justice/Legal	2 (2)
Online or mobile phone	5 (5)
School	6 (6)
Workplace	1 (1)
Other	3 (3)
Intervention Implementers*
Clinic administrator/manager	13 (13)
Clinic support staff	15 (15)
Educator or school health personnel	11 (11)
Healthcare provider	65 (63)
Implementation support practitioner/professional	8 (8)
Lay health personnel	18 (17)
Policymaker	4 (4)
Public health practitioner/professional	7 (7)
Social service worker	5 (5)
Others	23 (22)
Health Disparities-Relevant Research, Yes	47 (46)
Disparities population*	Number (% out of n = 47)
Racial or ethnic minorities	22 (47)
Underserved rural	18 (38)
Socioeconomically disadvantaged	22 (47)
Sexual or gender minorities	6 (13)
Other	4 (9)
*Responses not mutually exclusive

Table 2 shows that most studies (87%) included some form of community or partner engagement. Engagement was planned across the research continuum, with formative research (61%) and implementation strategy design (52%) being the most common and theory development (6%) being least common. On average, each study engaged 2.5 different partner types with direct service providers (77%) and patients/survivors (54%) being the most common, and product makers (3%) and purchasers or payers (7%) being the least common. The highest level of engagement ranged Sanders Thompson et al.’s (36) continuum with consultation being the most common (56%) and partnership (3%) and outreach (4%) being the least common. Practice-based research was a common approach (23%), as was CBPR (16%), while one-third of studies did not specify any particular approach to engagement. On average, each study used 2.2 engagement strategies, with the most common being key informant data collection (77%) and project advisory groups (39%). Nearly half (48%) of studies did not describe any indicators for equitable engagement, but for those who did, influence (28%) and mutual benefit (26%) were most clearly articulated. Finally, only 16% of engaged research grants reported formally measuring engagement. [INSERT Table 2 HERE]

Table 2

Characteristics of the engagement in included grants
When is engagement planned?*	Number (% out of N = 103)
In theory development	6 (6)
In intervention design	32 (31)
In implementation strategy design	54 (52)
In formative research	63 (61)
In evaluation and measurement	34 (33)
Only stated but not explained in depth	5 (5)
Other	6 (6)
Not at all	13 (13)
Who will be engaged?*	Number (% out of n = 90)
Direct service providers or practitioners	69 (77)
Partner organizations	29 (32)
Patient/survivors or their families	49 (54)
Policy makers	16 (18)
Product makers/intervention designers	3 (3)
Program/system administrators	33 (37)
Public/General community members	19 (21)
Purchasers or Payers	6 (7)
Other	6 (7)
Highest level of engagement in study
Outreach and Education	4 (4)
Consultation	50 (56)
Cooperation	21 (23)
Collaboration	12 (13)
Partnership	3 (3)
Engagement approach
Practice-based research	22 (23)
Community-based participatory research (CBPR)	14 (16)
Community engaged research (CEnR)	10 (11)
Other	13 (14)
Not specified	31 (33)
Engagement strategies*
Brainstorming/Crowdsourcing Ideas	4 (4)
Capacity-building	20 (22)
Human-centered design	10 (11)
Information dissemination	15 (17)
Key Informant data collection in needs assessment, formative research, or research agenda setting	69 (77)
Group meetings for consensus building or prioritization	20 (22)
Process, system, intervention, or implementation mapping	8 (9)
Project steering committee, advisory group or other decision-making body	35 (39)
Partners as active members of research team	15 (17)
Others	3 (3)
Indicators of equity in engagement*
Decision-Making	14 (16)
Influence	25 (28)
Mutual Benefit	23 (26)
Ownership	0 (0)
Power and control	1 (1)
Resource-sharing	13 (14)
Responsibility	2 (2)
Not described	43 (48)
Is engagement measured? Yes	14 (16)
*Responses not mutually exclusive

In looking at themes that emerged with respect to engagement and key implementation domains, studies that were categorized as D&I or de-implementation all involved engagement at some level, whereas 14% (n = 11) of the grants categorized as implementation-only studies had no engagement (Table 3). Similarly, 12% (n = 11) of studies coded as involving an intervention reported no engagement (data not shown in table). Compared to grants without community or partner engagement, engaged research projects were more likely to be focused on implementation phases of testing strategies (61% vs. 31%), scaling up (20% vs. 8%), and/or sustainability (19% vs. 8%). Grants involving engagement were more likely to use a case study (3% vs. 0%) or observational design (37% vs. 23%) and non-engaged grants were more likely to use modeling (15% vs. 7%); there was no substantial difference in the use of experimental or quasi-experimental designs. All grants set in the community or within a community-based organization involved some level of engagement (data not shown in table). Grants involving community or partner engagement appeared more likely to leverage less-common D&I theories, models, and frameworks or those from other disciplines compared to grants without engagement (60% vs. 23%). Compared to grants without community or partner engagement, engaged research projects were more likely to apply the following implementation strategies: adapt and tailor to context (50% vs. 38%), develop interrelationships (36% vs. 23%), engage consumers (40% vs. 8%), provide interactive assistance (32% vs. 23%), support clinicians/service providers (29% vs. 8%), train and educate (48% vs. 23%), and use evaluative and iterative strategies (52% vs. 23%). Among the grants involving engagement, 50% were relevant to health disparities. [INSERT Table 3 NEAR HERE]

Table 3

Characteristics of implementation domains for all included grants, and by whether or not they were engaged research
	Number (% of total grants N = 103)	Number (% of grants with engagement n = 90)	Number (% of grants without engagement n = 13)
Study type
Dissemination	0 (0)	0 (0)	0 (0)
Implementation	76 (74)	65 (72)	11 (85)
De-implementation	2 (2)	2 (2)	0 (0)
Both D&I	23 (22)	23 (26)	0 (0)
Modeling only	2 (2)	0 (0)	2 (15)
Phase of Implementation*
Pre-implementation	41 (40)	37 (41)	4 (31)
Describe implementation process/factors	68 (66)	60 (67)	8 (62)
Tests implementation strategy(ies)	59 (57)	55 (61)	4 (31)
Scale up	19 (18)	18 (20)	1 (8)
Sustainability	18 (17)	17 (19)	1 (8)
Measurement development	6 (6)	6 (7)	0 (0)
Mechanisms and pathways	18 (17)	17 (19)	1 (8)
Study Design*
Case study	3 (3)	3 (3)	0 (0)
Experimental	64 (62)	57 (63)	7 (54)
Quasi-experimental	7 (7)	6 (7)	1 (8)
Observational	36 (35)	33 (37)	3 (23)
Pre-post	6 (6)	5 (6)	1 (8)
Modeling	8 (8)	6 (7)	2 (15)
Other	8 (8)	8 (9)	0 (0)
Hybrid study
Type 1	23 (22)	22 (24)	1 (8)
Type 2	24 (23)	18 (20)	6 (46)
Type 3	13 (13)	12 (13)	1 (8)
Not hybrid	43 (42)	38 (42)	5 (38)
Theory*
Consolidated Framework for Implementation Research (CFIR)	43 (42)	38 (42)	4 (31)
Reach, Effectiveness-Adoption Implementation, Maintenance (RE-AIM) or Practical, Robust Implementation and Sustainability Model (PRISM)	35 (34)	29 (32)	6 (46)
Exploration, Preparation, Implementation, Sustainment (EPIS)	12 (12)	8 (9)	2 (15)
Others	57 (55)	54 (60)	3 (23)
None	8 (8)	6 (7)	2 (15)
Implementation Strategies*
Adapt and tailor to context	50 (49)	45 (50)	5 (38)
Change infrastructure	15 (15)	13 (14)	2 (15)
Develop partner interrelationships	35 (34)	32 (36)	3 (23)
Engage consumers	37 (36)	36 (40)	1 (8)
Provide interactive assistance	32 (31)	29 (32)	3 (23)
Support clinicians/service providers	27 (26)	26 (29)	1 (8)
Train and educate partners	46 (45)	43 (48)	3 (23)
Use evaluative and iterative strategies	51 (50)	47 (52)	3 (23)
Utilize financial strategies	10 (10)	9 (10)	1 (8)
Other	6 (6)	5 (6)	1 (8)
None	6 (6)	3 (3)	3 (23)
*Responses not mutually exclusive

In looking at themes that emerged with respect to engagement in the subset of grants focused on populations with health disparities, all but two had some level of engagement (96%). Compared to non-disparities studies, disparities-relevant studies were about twice as likely to engage partners at the level of partnership or collaboration (19% vs. 11%) (Fig. 3). About 26% of disparities-focused studies were conducted in the community or at a community-based organization compared to just 5% of non-disparities studies. CBPR was the most common engagement approach described in studies with populations experiencing disparities at 26% compared with only 4% in studies not focused on disparities. [INSERT FIGURE 3 HERE]

This analysis illuminated gaps in the application of engagement in D&I research and opportunities to study engagement processes and outcomes in the context of D&I research (Fig. 4). [INSERT FIGURE 4 HERE]

For our primary research question of to what extent and how are community members and partners engaged in recent NIH-funded D&I research, it was not surprising to see that nearly nine in ten grants reviewed involved engagement in some way, given the increased attention to engaged research in recent years. However, engagement scientists might argue that the more than half of studies involving partners at the level of consultation are not truly meeting the spirit of engagement when partners are simply serving as research participants, key informants, or advisory board members. Level of engagement did not appear to vary by type of partner, though that was difficult to examine since multiple partners were often engaged in each grant. Full partnership may not be warranted or necessary in every research study, but the rigor and relevance of D&I research would likely benefit if investigators were engaging communities and partners at the higher levels of collaboration and partnership more often. This opportunity goes beyond engagement just in formative research and implementation strategy design. To fully appreciate the potential benefits of such engagement, opportunities across the research continuum will likely be necessary. For example, community and partner engagement in theory development could result in theories, models, and frameworks that are more relevant and applicable to the populations of interest. Likewise, engagement in intervention design and evaluation may lead to more effective and sustainable interventions.

It was encouraging to see that multiple diverse groups of partners, on average, were engaged in each study. For intervention implementers, opportunities remain to increase engagement of policymakers, social service workers, public health professionals, and implementation support practitioners (e.g., knowledge brokers, technical assistance providers, practice facilitators). Further, engagement with product makers, purchasers or payers, and health equity experts, could be considered to potentially increase outcomes related to “designing for dissemination” including acceptability, cost-effectiveness, scalability, and sustainability (39, 40).

Whereas an engagement approach was not specified in one in three grants that had engagement activities, it was encouraging to see that nearly one quarter of these grants were employing a practice-based research approach, and more than a quarter were conducting CBPR or following generic community-engaged research approach. Grant applications that did not identify a specific approach were most commonly operating at the consultation level, reinforcing critiques that they are likely not meeting the true spirit of engagement. Opportunities remain for incorporating strategies that involve deeper engagement approaches with both end-user implementers and beneficiaries of innovations such as human or user centered design (41), process mapping (42), systems mapping (43), intervention mapping (44), or implementation mapping (45), and other longitudinal and hands-on strategies. Future research can explore the impact of these increasing levels of engagement and various strategies on relevant implementation outcomes.

Indicators of equitable engagement were challenging to extract and infer from the proposals, despite having access to the full narrative and related budget documents. Where indicators were apparent, partners’ influence on the research and mutual benefit of engagement were most commonly evident. Little to no description of issues of ownership, responsibilities, or power and control over the data, findings, or otherwise was found. It is possible that issues of equitable engagement emerged and were addressed over the course of the project but were not included in the proposal for several reasons: not part of grant evaluation criteria; lack of space; not yet defined with partners, thought to be irrelevant to the science. Regardless, the issues around the influence of power, responsibility, ownership, and control in engaged research and as they relate to dismantling structural drivers of social determinants of health (SDOH) are ripe for future research. Exploring power dynamics in engaged research and proactively co-creating acceptable engagement structures could enhance community participatory research (46). In 2021, Shelton, Adsul, and Oh (47) outlined recommendations to the D&I research field for addressing structural racism that included leveraging engagement as an essential method, assessing and addressing power differentials, and applying multi-level approaches for implementing interventions, policies, and strategies to address structural discrimination and advance health equity. In 2022, Stanton et al. (48) proposed a typology of 3 types of power that appear in implementation efforts and outlined a number of research avenues to investigate how power operates and influences implementation and health equity-related outcomes. Consistent with these recommendations, the findings of this analysis suggest that opportunities remain to explicitly address equitable engagement in D&I research.

D&I research offers promise for more systematically applying engagement and in so doing, providing a lens for advancing the science of engagement. Distinct from participatory research that uses engagement techniques, engagement science examines the methods and outcomes of engagement in order to develop an evidence base for why engagement matters and how to do it well (49–51). Certainly, studying engagement and relating it to D&I strategies or outcomes is not possible without measuring engagement in some way. Given that most grants reviewed did not formally assess engagement, there remains a significant opportunity for D&I scientists to fully embrace engagement approaches and rigorously measure these processes to ideally make an impact on the sciences of D&I and of engagement moving forward.

The second research question strived to uncover any trends in engagement variables based on key study characteristics. There were noted missed opportunities for engaging key community members and partners in a portion of studies focused solely on implementation. Furthermore, there was a notable lack of engagement described in studies that involved delivery of an intervention; future studies could likely benefit from more active attempts to engage the implementers, decision makers, and intervention beneficiaries more substantially and to examine the impact that may have on study design, execution, and outcomes. Although there were fewer grants in total that were both D&I (n = 23), they all involved engagement of some sort, possibly due to the increased challenge of studying and successfully accomplishing both dissemination and implementation within a single time-limited grant. Similarly, grants involving testing strategies, scaling up, or studying sustainability were more likely to involve community or partner engagement, possibly due to the active and complex nature of these phases of implementation. Many implementation efforts require engagement with a decision maker to facilitate the test of change, and studying scale up and sustainability are intuitively enhanced with the involvement of partners responsible for expanding and maintaining program implementation. Similarly, case study and observational study designs appeared to be more common in engaged grants, likely because they required access through partners to collect data. In contrast, grants without engagement were more likely to utilize modeling approaches, which may not obviously require input from partners. Yet, partner engagement in modeling approaches to interventions remains a relatively unexplored area of inquiry that has the potential to make such efforts more responsive and relevant to communities they aim to serve (52). Indeed, some initial efforts to develop a “participatory systems dynamics modeling” approach has shown promise. Logically, the implementation strategies employed in engaged grants tended to be strategies that stemmed from that engagement (e.g., engaging consumers, developing interrelationships, training). Similarly, community-situated studies all involved engagement and there was greater indication of decision making and power and control in studies at higher levels of engagement.

It was encouraging to find that nearly half of grants overall were relevant to disparities in some way and half of the engaged D&I grants were disparities-relevant. Given the potential of D&I research and engagement approaches to address SDOH and advance health equity among underserved populations who experience disparities (22, 53, 54), opportunities remain for growing the amount of disparities-related and equity-focused engaged D&I studies. In particular, there was limited engaged research with some populations experiencing health disparities such as underserved rural, sexual and gender minorities, and others (e.g., disability communities). With respect to engagement themes among disparities-focused grants, nearly all these grants involved engagement at some level, and they engaged partners more deeply at the levels of collaboration or partnership as compared to non-disparities focused grants. This may be in part due to the longitudinal commitment necessitated by equity focused work as well as the noted increasing federal support for engaged research approaches to advance health equity. The disparities-focused grants were more likely to be conducted in a community setting or at a community-based organization and were more likely to involve multi-level interventions as compared to non-disparities grants. These findings perhaps indicate that D&I researchers are beginning to answer the calls to truly partner with communities and devise multi-pronged strategies that address complex problems, build capacity among D&I researchers, and incorporate equity across implementation focus areas (47, 55).

Strengths And Limitations

This portfolio analysis had several strengths, including the use of dual coders and a structured codebook. In addition, as an internal NIH project, coders had access to the full grant proposal narrative including the research summary and budget documents to be able to code more accurately.

Yet, important limitations were present as well. The inclusion criteria were limited to a specific set of program announcements and a single study section; this likely omitted some D&I research funded across NIH through other mechanisms and study sections. However, the included grants do incorporate the majority of D&I research supported by the NIH. Coders reviewed grant proposals, which may or may not be a true reflection of study activities implemented once funded. Some engagement variables coded, such as approach and equity indicators, are elements that are not required to be specified or described in proposals, potentially leading to an overcount of “not specified/described.” Finally, despite training coders, developing a thorough codebook with definitions, and providing support throughout the project, coding is subject to human judgement and manual data entry errors, though this was mitigated by dual coding to agreement.

Based on this portfolio analysis, the D&I research field appears to regularly integrate community and partner engagement approaches and strategies, though opportunities remain to deepen engagement and diversify partners. Further, to meaningfully advance health equity, there are critical opportunities for D&I research that leverages community and partner engagement to address health disparities, especially among understudied populations. Through this increased attention to engagement within D&I research, new opportunities to study engagement-related research questions may be pursued that will likely increase the rigor, relevance, and impact of both engagement science and D&I research.

CBPR Community-Based Participatory Research

COVID-19 Coronavirus-2019

D&I Dissemination and implementation

DIRH Dissemination and Implementation Research in Health

HIV Human immunodeficiency virus

ICO [NIH] Institutes, Centers and Offices

NIH National Institutes of Health

SDOH Social determinants of health

SIHH Science of Implementation in Health and Healthcare

TIDIRC Training in Dissemination and Implementation Research in Cancer

Ethics approval and consent to participate

Not applicable. Ethics approval and informed consent to participate were not required as this manuscript does not report on human subjects research.

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and analyzed during the current study are not publicly available due the confidential nature of grant proposals but a limited dataset is available from the corresponding author on reasonable request.

Competing interests

Other than being affiliated with the National Institutes of Health, whose funded research portfolio is the subject of this analysis, the authors declare that they have no competing interests.

Funding

This study was conducted by NIH staff members, trainees, and contractors as part of their official duties. No additional funding was used to support this research.

Authors' contributions

AV, DBD, APL conceptualized the study’s research questions and methods. AV led project administration and supervision of coders, development of the codebook, data cleaning and analysis, manuscript writing and editing. DB was a major contributor to coding, data cleaning, and analysis. APL and DBD were also major contributors in writing the manuscript and editing. MB contributed to advanced data analysis and manuscript review and editing. All authors read and approved the final manuscript. The views expressed in this article are those of the authors and do not necessarily represent the official position of the National Institutes of Health.

Acknowledgements

The authors would like to thank their colleagues who helped with coding: Brenda Adjei, Fernando Bruno, Lori Ducharme, Kalina Duncan, Andrea Horvath Marques, Marion Koso-Thomas, Keith Mintzer, Elizabeth Neilson, Gila Neta, April Oh, Suzanne Pollard, Laura Prakash, Jenelle Walker, Jackie Wallace, Miya Whitaker, and Tisha Wiley. The authors also appreciate the thoughtful feedback on earlier drafts of this manuscript by David Chambers, Christine Hunter, and Tilda Farhat.

PAR-19-274: Dissemination and Implementation Research in Health (R01 Clinical Trial Optional) [Internet]. [cited 2022 Nov 28]. Available from: https://grants.nih.gov/grants/guide/pa-files/PAR-19-274.html
PAR-19-276: Dissemination and Implementation Research in Health (R03 Clinical Trial Not Allowed) [Internet]. [cited 2022 Nov 28]. Available from: https://grants.nih.gov/grants/guide/pa-files/PAR-19-276.html
PAR-19-275: Dissemination and Implementation Research in Health (R21 Clinical Trial Optional) [Internet]. [cited 2022 Nov 28]. Available from: https://grants.nih.gov/grants/guide/pa-files/PAR-19-275.html
SIHH | NIH Center for Scientific Review [Internet]. [cited 2022 Nov 28]. Available from: https://public.csr.nih.gov/StudySections/DABP/HSS/SIHH
D&I - About | AcademyHealth [Internet]. [cited 2022 Nov 28]. Available from: https://academyhealth.org/page/2022-di-about
Training Institute for Dissemination and Implementation Research in Cancer (TIDIRC) OpenAccess | Division of Cancer Control and Population Sciences (DCCPS) [Internet]. [cited 2022 Nov 28]. Available from: https://cancercontrol.cancer.gov/is/training-education/training-in-cancer/TIDIRC-open-access
Holt CL, Chambers DA. Opportunities and challenges in conducting community-engaged dissemination/implementation research. Transl Behav Med. 2017 Sep;7(3):389–92.
Blachman-Demner D, Wiley T, Chambers D. Fostering integrated approaches to dissemination and implementation and community engaged research. Translational Behavioral Medicine. 2017 Sep;7.
Hursting LM, Chambers DA. Practitioner Engagement in Implementation Science: Initiatives and Opportunities. J Public Health Manag Pract. 2021 Apr 1;27(2):102–4.
Glandon D, Paina L, Alonge O, Peters DH, Bennett S. 10 Best resources for community engagement in implementation research. Health Policy Plan. 2017 Dec 1;32(10):1457–65.
Mensah GA, Curry JS, Engelgau MM, Johnson LE. Stakeholder Engagement in Late-Stage Translation Research and Implementation Science: Perspectives From the National Heart, Lung, and Blood Institute. Glob Heart. 2019 Jun;14(2):191–4.
Hudon C, Chouinard MC, Bisson M, Danish A, Karam M, Girard A, et al. Case Study With a Participatory Approach: Rethinking Pragmatics of Stakeholder Engagement for Implementation Research. Ann Fam Med. 2021 Dec;19(6):540–6.
Ramanadhan S, Davis MM, Armstrong R, Baquero B, Ko LK, Leng JC, et al. Participatory implementation science to increase the impact of evidence-based cancer prevention and control. Cancer Causes Control. 2018 Mar;29(3):363–9.
Stalker KC, Brown ME, Evans CBR, Hibdon J, Telep C. Addressing Crime, Violence, and Other Determinants of Health through Community-Based Participatory Research and Implementation Science. American Journal of Community Psychology. 2020;66(3–4):392–403.
Ramanadhan S, Donaldson ST, Siqueira CE, Rackard-James C, Miller E, Tappin J, et al. Connecting Implementation Science, Community-Engaged Research, and Health Promotion to Address Cancer Inequities in Massachusetts: The UMB/DF-HCC U54 Outreach Core. Am J Health Promot. 2022 May 1;36(4):597–601.
Skolarus LE, Sales AE, Zimmerman MA, Corches CL, Landis-Lewis Z, Robles MC, et al. Stroke Ready: a multi-level program that combines implementation science and community-based participatory research approaches to increase acute stroke treatment: protocol for a stepped wedge trial. Implementation Science. 2019 Mar 7;14(1):24.
Adam MB, Minyenya-Njuguna J, Karuri Kamiru W, Mbugua S, Makobu NW, Donelson AJ. Implementation research and human-centred design: how theory driven human-centred design can sustain trust in complex health systems, support measurement and drive sustained community health volunteer engagement. Health Policy and Planning. 2020 Nov 1;35(Supplement_2):ii150–62.
RADx Programs [Internet]. National Institutes of Health (NIH). 2020 [cited 2022 Nov 28]. Available from: https://www.nih.gov/research-training/medical-research-initiatives/radx/radx-programs
Community Engagement Alliance (CEAL) Against COVID-19 Disparities | Community Engagement Alliance [Internet]. [cited 2022 Nov 28]. Available from: https://covid19community.nih.gov/
Pinto RM, Park S (Ethan), Miles R, Ong PN. Community engagement in dissemination and implementation models: A narrative review. Implementation Research and Practice. 2021 Jan 1;2:2633489520985305.
Shea CM, Young TL, Powell BJ, Rohweder C, Enga ZK, Scott JE, et al. Researcher readiness for participating in community-engaged dissemination and implementation research: a conceptual framework of core competencies. Transl Behav Med. 2017 Sep;7(3):393–404.
Beidas RS, Dorsey S, Lewis CC, Lyon AR, Powell BJ, Purtle J, et al. Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem. Implementation Science. 2022 Aug 13;17(1):55.
Organizing Committee for Assessing Meaningful Community Engagement in Health & Health Care Programs & Policies. Assessing Meaningful Community Engagement: A Conceptual Model to Advance Health Equity through Transformed Systems for Health. NAM Perspectives [Internet]. 2022 Feb 14 [cited 2022 Nov 28]; Available from: https://nam.edu/assessing-meaningful-community-engagement-a-conceptual-model-to-advance-health-equity-through-transformed-systems-for-health/
Haapanen KA, Christens BD. Community-engaged Research Approaches: Multiple Pathways To Health Equity. American Journal of Community Psychology. 2021;67(3–4):331–7.
Payán D, Zawadzki M, Song A. Advancing community-engaged research to promote health equity: considerations to improve the field. Perspect Public Health. 2022 May 1;142(3):139–41.
Field N, Cohen T, Struelens MJ, Palm D, Cookson B, Glynn JR, et al. Strengthening the Reporting of Molecular Epidemiology for Infectious Diseases (STROME-ID): an extension of the STROBE statement. The Lancet Infectious Diseases. 2014 Apr;14(4):341–52.
Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, et al. An Overview of Research and Evaluation Designs for Dissemination and Implementation. Annual Review of Public Health. 2017;38(1):1–22.
Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed Method Designs in Implementation Research. Adm Policy Ment Health. 2011 Jan 1;38(1):44–53.
Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science. 2015 Feb 12;10(1):21.
Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging Research and Practice: Models for Dissemination and Implementation Research. American Journal of Preventive Medicine. 2012 Sep 1;43(3):337–50.
Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science. 2015 Apr 21;10(1):53.
Potter MA, Quill BE, Aglipay GS, Anderson E, Rowitz L, Smith LU, et al. Demonstrating Excellence in Practice-Based Research for Public Health. Public Health Rep. 2006 Jan 1;121(1):1–16.
Wallerstein N, Duran B. Community-Based Participatory Research Contributions to Intervention Research: The Intersection of Science and Practice to Improve Health Equity. Am J Public Health. 2010 Apr;100(Suppl 1):S40–6.
Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, et al. A New Taxonomy for Stakeholder Engagement in Patient-Centered Outcomes Research. J Gen Intern Med. 2012 Aug 1;27(8):985–91.
PCORI’s Stakeholders [Internet]. PCORI’s Stakeholders | PCORI. 2014 [cited 2023 Jan 13]. Available from: https://www.pcori.org/about-us/our-programs/engagement/public-and-patient-engagement/pcoris-stakeholders
Sanders Thompson VL, Ackermann N, Bauer KL, Bowen DJ, Goodman MS. Strategies of community engagement in research: definitions and classifications. Translational Behavioral Medicine. 2021 Feb 1;11(2):441–51.
Key KD, Furr-Holden D, Lewis EY, Cunningham R, Zimmerman MA, Johnson-Lawrence V, et al. The Continuum of Community Engagement in Research: A Roadmap for Understanding and Assessing Progress. Progress in Community Health Partnerships: Research, Education, and Action. 2019;13(4):427–34.
HDPulse - Health Disparities and Minority Health Resources (NIMHD) [Internet]. [cited 2022 Nov 28]. Available from: https://hdpulse.nimhd.nih.gov/index.html
Brownson RC, Jacobs JA, Tabak RG, Hoehner CM, Stamatakis KA. Designing for Dissemination Among Public Health Researchers: Findings From a National Survey in the United States. Am J Public Health. 2013 Sep;103(9):1693–9.
Mora Pinzon MC, Myers S, Renken J, Eggert E, Chewning B, Mahoney JE. Essential elements to “design for dissemination” within a research network—a modified Delphi study of the Community-Academic Aging Research Network (CAARN). Implementation Science Communications. 2021 Feb 12;2(1):18.
Beres LK, Simbeza S, Holmes CB, Mwamba C, Mukamba N, Sharma A, et al. Human-Centered Design Lessons for Implementation Science: Improving the Implementation of a Patient-Centered Care Intervention. J Acquir Immune Defic Syndr. 2019 Dec;82 Suppl 3(3):S230–43.
Kononowech J, Landis-Lewis Z, Carpenter J, Ersek M, Hogikyan R, Levy C, et al. Visual process maps to support implementation efforts: a case example. Implementation Science Communications. 2020 Nov 25;1(1):105.
Kiekens A, Casterlé BD de, Vandamme AM. Qualitative systems mapping for complex public health problems: A practical guide. PLOS ONE. 2022 Feb 25;17(2):e0264463.
Bighetti N. Intervention Mapping [Internet]. Intervention Mapping. [cited 2022 Nov 28]. Available from: https://www.interventionmapping.com/
Fernandez ME, ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Parcel G, et al. Implementation Mapping: Using Intervention Mapping to Develop Implementation Strategies. Frontiers in Public Health [Internet]. 2019 [cited 2022 Nov 28];7. Available from: https://www.frontiersin.org/articles/10.3389/fpubh.2019.00158
Roura M. The Social Ecology of Power in Participatory Health Research. Qual Health Res. 2021 Mar 1;31(4):778–88.
Shelton RC, Adsul P, Oh A. Recommendations for Addressing Structural Racism in Implementation Science: A Call to the Field. Ethnicity & Disease. 2021 May 20;31(Suppl):357–64.
Stanton MC, Ali SB, SUSTAIN Center Team. A typology of power in implementation: Building on the exploration, preparation, implementation, sustainment (EPIS) framework to advance mental health and HIV health equity. Implementation Research and Practice. 2022 Jan 1;3:26334895211064250.
Meissner P, Cottler LB, Eder M “Mickey”, Michener JL. Engagement science: The core of dissemination, implementation, and translational research science. J Clin Transl Sci. 4(3):216–8.
AcademyHealth. Engagement Science: Introducing Inclusive Research Practices & Potential Impacts [Internet]. [cited 2023 Jan 13]. Available from: https://academyhealth.org/blog/2019-01/engagement-science-introducing-inclusive-research-practices-potential-impacts
Science of Engagement PCORI Funding Announcement -- Cycle 2 2022 [Internet]. Science of Engagement PCORI Funding Announcement -- Cycle 2 2022 | PCORI. 2022 [cited 2023 Jan 13]. Available from: https://www.pcori.org/funding-opportunities/announcement/science-engagement-pcori-funding-announcement-cycle-2-2022
Zimmerman L, Lounsbury DW, Rosen CS, Kimerling R, Trafton JA, Lindley SE. Participatory System Dynamics Modeling: Increasing Stakeholder Engagement and Precision to Improve Implementation Planning in Systems. Adm Policy Ment Health. 2016 Nov 1;43(6):834–49.
Baumann AA, Cabassa LJ. Reframing implementation science to address inequities in healthcare delivery. BMC Health Services Research. 2020 Mar 12;20(1):190.
Emmons KM, Chambers DA. Commentary: Policy Implementation Science - An Unexplored Strategy to Address Social Determinants of Health. Ethnicity & Disease. 2021 Jan 21;31(1):133–8.
Adsul P, Chambers D, Brandt HM, Fernandez ME, Ramanadhan S, Torres E, et al. Grounding implementation science in health equity for cancer prevention and control. Implementation Science Communications. 2022 Jun 3;3(1):56.

STROBEchecklistcrosssectionalCEDIPA2023.docx

Download PDF

Journal Publication

published 11 Jul, 2023

Read the published version in Implementation Science Communications →

Editorial decision: Major revision
24 Mar, 2023
Reviewers agreed at journal
03 Feb, 2023
Reviewers invited by journal
03 Feb, 2023
Editor assigned by journal
16 Jan, 2023
Submission checks completed at journal
15 Jan, 2023
Editor invited by journal
15 Jan, 2023
First submitted to journal
13 Jan, 2023

You are reading this latest preprint version

Community and Partner Engagement in Dissemination and Implementation Research at the National Institutes of Health: An Analysis of Recently Funded Studies and Opportunities to Advance the Field

Status:

Journal Publication

Version 1

Abstract

Figures

Contributions To The Literature

Background

Methods

Results

Discussion

Strengths And Limitations

Conclusions

Abbreviations

Declarations

References

Supplementary Files

Status:

Journal Publication

Version 1