Demographic characteristics of surveyed scientists are presented in Table 1. Of the 159 scientist responses to the survey, 159 provided qualitative responses to any of the three open-response items. Specifically, we received 95 responses for questions one and two and 159 responses from question three. The majority of scientists were from university or research settings in the U.S. (69%) or Canada (13%). They were from a mix of clinical (33%) and community settings (68%). The majority were from behavioral health (43%) or public health disciplines (42%); 26% had received formal training in D&I and there was a wide distribution in years since highest academic degree. The project was approved by the Colorado Combined Institutional Review Board (COMIRB), including a waiver of written consent to participate.
What follows are reports of key themes present in the data organized by each of the open-ended question. Additional illustrative quotes are also provided in Tables 2-4.
Question 1: How is your dissemination of research findings to non-research audiences evaluated?
Of the 95 responses to this question three distinct themes emerged: 1a) promotional review; 1b) funding requirements; and 1c) absence of evaluation of dissemination activities.
1a) Promotional review
Many respondents reported that when the dissemination of research findings to non-research audiences was evaluated, this evaluation occurred during an annual review. Researchers expressed a combination of ways in which the review took place. “When going up for promotion or other awards we indicate the impact our work has made on practice/policy. I also share events (i.e., policy changes) with my leadership through email.” It was also expressed that the evaluation of dissemination activities is considered under the “broad umbrella of community service”. Additionally, it was noted that evaluation of dissemination often occurred only “qualitatively” during the performance review conversation, perhaps indicating that these activities garnered less attention than more easily-quantifiable productivity.
1b) Funding requirements
Overall, there was general agreement that many, but not all, grant funders expected the dissemination of research findings. Researchers noted that “certain agencies specifically ask for this [dissemination] in an annual report” and also have “explicit expectations for our research to lead to clinical practice change.” Further, scientists mentioned specific funder requirements regarding dissemination expectations. “There is an expectation that evidence-based information and programs will be taken to the community. Number of community contacts is part of what is measured, as well as support activities of these programs in community.” Scientists reported that particular funding agencies require “completion of milestones, presentation of results at meetings of various kinds” as well as the ‘reporting of how stakeholders (including patients, caregivers, policy makers, etc.) were engaged throughout the research process’ as means of assuring the dissemination of research.”
1c) Lack of acknowledgement of dissemination activities
External evaluation of dissemination activities was often non-existent. A majority of researchers said that it is simply “not evaluated”, as demonstrated by such statements as it “is not an important part of my job.” Other respondents also noted they were either “unsure” or “not certain” if the university requires such an evaluation. “My institution only recognizes peer-reviewed research in scholarly journals; dissemination among practitioners or stakeholders is derivative by an order of magnitude.” There were a modest number of scientists that stated the information was recorded in the “CV section on dissemination” as well as through “publications, citations, media, social media etc.” Table 2 presents additional quotes of researchers’ experience with disseminating findings to non-research audiences.
Question 2: How would you improve the system for credit or recognition for disseminating research to non-research audiences?
Of the 95 responses to this question three distinct themes emerged: 2a) dissemination as a distinct component of research; 2b) requirement of dissemination plan; and 2c) dissemination metrics.
2a) Dissemination as a distinct component of research
Participants described the need for a clear path for promotion that is inclusive of metrics associated with disseminating research, noting to “make it more desirable to reach these audiences (more recognition within formal reports etc - not everything should be based on publications).” Specific suggestions provided by researchers were to “include it [dissemination] in metrics for academic promotion”, “recognition of practice-based reports equal to academic journals”, and “link to tenure process”.
2b) Requirement of dissemination plan
Scientists agreed in general that recognition for dissemination activities would receive more attention if they were a required element of grant proposals, noting to “add a dissemination outcome or plan in grant proposals; or a requirement to report dissemination channels and impact to non-research audiences…” Even more definitively, there were specific requests for “funders require it’ and that “NIH put more weight on it.” Researchers more explicitly recommended to “make it [dissemination activities] a valued component of KT [knowledge translation] plans on grant applications, include a section in manuscripts for other ways to learn about the results of this study, assign higher priority to these activities in Canadian Common CV/salary awards.”
2c) Dissemination metrics
Researchers generally believe that a clear measure to capture dissemination efforts will create a path for recognition within the academic setting. There was a general consensus that the current system does not have metrics in place to evaluate dissemination, noting the need to “develop a measure for how much time and effort a researcher puts into this, what the outcomes are, and make it as important as publishing in peer-reviewed journals.” Table 3 presents additional quotes of researchers’ recommendations on ways to improve the system.
Question 3: What is the one thing you could do that would most enhance your efforts to disseminate your research to non-research audiences?
Of the 159 responses, three primary themes emerged from question 3: 3a) development of skills and allocation of resources for dissemination activities; 3b) utilization of non-traditional (or alternative) dissemination mediums; and 3c) identify and address issues of priority for stakeholders.
3a) Development of skills and allocation of resources for dissemination activities
Scientists felt ill-equipped to effectively disseminate research findings. Specifically, it was noted that lack of financial resources and staffing to support such efforts are strong barriers to dissemination. Scientists stated explicitly, the need to “learn dissemination skills. Have a dedicated staff to assist with effective dissemination strategies. Its time-consuming and not all researchers are good at this part, so it would be good to work with creative staff and partners on this.” Scientists largely agreed that dissemination needs to begin early in the study design process, but is actually accomplished post-hoc (if it is even considered at all). Furthermore, scientists emphasized that dissemination and traditional research activities often require different skills and hiring staff specifically to support dissemination activities would improve impact. To rectify this, it was noted to prospectively prioritize dissemination activities, for example, including in research budgets. Further, the need for resources was continually emphasized.
3b) Utilization of non-traditional (or alternative) dissemination mediums
Scientists conveyed the need to increase their own use of non-traditional methods of reporting research findings. It was suggested to “create a website with regular blogs, social media posts, and press releases” to share research findings to the non-research community. It was also recommended to create “visual abstracts for sharing on social media would likely garner more attention and interest - graphic design and visuals are very helpful communication tools.”
3c) Identify and address issues of priority for stakeholders
Scientists referenced both practical and logistical challenges to meeting with stakeholders and the need to partner with them during all phases of the research process, noted that “non-researchers often don't care about or want to use research, so knowing how to make them interested in it to begin with would be my silver bullet.”
Specifically, the need to “better understand their [stakeholders] priorities” was expressed as well as the need to “to develop a formal dissemination plan as part of the research plan, including stakeholders in this process.” Tale 4 presents additional quotes regarding researchers’ recommendations on ways to improve the system.