Knowledge translation (KT) research investigates the complex factors that define whether and how research-based knowledge finds its way into better healthcare, policy, and practice [1, 2]. KT research frequently entails participatory research (PR) approaches, which involves establishing co-creating relationships between researchers and knowledge users (i.e., those who influence, administer, and/or make use of healthcare systems) [3,4]. PR is defined as:
“An umbrella term to include all partnered research, including community-based participatory research (CBPR), action research, participatory action research, participatory evaluation, community engagement and patient engagement” [5, p. 2].
Successful PR requires establishing connections across multiple sectors, including community, government, and education, to better create and implement products and tools generated using KT approaches [1]. PR products, developed for and with healthcare providers, decision-makers, industry, and patients, include policy briefs, clinical practice guidelines, education and awareness strategies (e.g. promotion campaigns and video communication), and technologies, such as digitized decision support systems and mHealth/eHealth applications [1,6]. While there are many sources of guidance on the development of PR and KT and implementation strategies [7, 8], there are comparatively fewer on assessing their impacts [7, 9, 10].
The effectiveness and impact of PR products are rarely tested, empirically investigated, or systematically evaluated [7, 9, 10]. Evaluating how a PR product influences its users considering different mechanisms of such interaction, and identifying ways to reduce any barrier that may be interfering with the product’ intended influence (co-designed objectives), are necessary steps to understand and enhance the effect of PR products [6, 7, 8, 9, 10]. This type of impact and effectiveness assessment could help to ascertain ways in which PR methods are positively influencing health-related outcomes [7]. This lack of empirical evaluation procedures represents a gap between research and practice, which is constantly manifested in calls for evaluation frameworks and impact metrics connected to PR [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]. Thus, bridging this gap through the evaluation of PR products is necessary because billions of dollars are lost yearly in health research that fails to create a significant impact [7].
Design requirements for generating an inductive evaluation framework of PR products
Evaluations of PR have been criticized for focusing on the short-term effects of their products and mainly using qualitative studies and results; their findings are rarely generalizable or transferable because they concentrate on small samples or highly specialized topics [7]. Therefore, to address these gaps, frameworks for conducting cross-sectional and longitudinal evaluations of PR products should adhere to the design standards for public health evaluations [7, 13, 14, 15].
The standards for public health evaluations are utility, feasibility, propriety, and accuracy [7, 13, 14, 15]. Utility refers to the purpose of the product and how users will benefit from it [7, 13]. Feasibility refers to how measurable is the product within each stage of its development/use and its practicality [7, 13]. Propriety focuses on examining user experience by asking questions about the intended and unintended consequences of the product [13]. Accuracy relates to the success or failure of the product and its design while detecting and tracking such characteristics in a replicable manner across different populations, topics, and sectors [13, 12, 1]. Addressing these standards for generating an inductive evaluation of PR products requires a mixture of formative and summative evaluation methods [16].
Formative evaluation involves assessing process, experiential, or narrative aspects associated with the performance of the product [16]. These assessments are typically conducted using qualitative research methods (e.g., interviews) [16]. The summative evaluation considers the short-term to long-term effectiveness and impact of the product, primarily using quantitative measures [16]. A mixture of formative and summative evaluations embedded within frameworks designed to support productive partnerships to enact change and quality improvements while involving the community [17] may help to meet the utility and feasibility standards for the evaluation of PR products.
The Learning, Evaluation and Planning (LEAP) framework facilitates the mixture of formative and summative evaluations [17]. LEAP describes an iterative procedure for co-creating resources and service responses through participatory learning, evaluation, and planning [17]. The framework enables an assessment of needs, planning, implementation, empowerment, and review of actions, focusing on responding collaboratively to framing questions [17]. These questions could be used to guide the process of merging different evaluation tools to measure and track the impact of PR products.
Additionally, utility, feasibility and propriety standards can be met and tracked using a Body Mapping technique [14], which is an adaptable research tool that can capture baseline data, changes, and longitudinal feedback about a product or service from various types of knowledge users, including people with limited literacy or/and language difficulties [14]. Body Maps typically depict a simplified figure of a human body with open-ended questions about the value, acceptance, utility, and comprehension of the product [14].
The combination of LEAP and Body Mapping can collect the perspective of knowledge users to assess the impact of PR products as well as identify their unintended consequences across time by conducting multiple measurements of the same product’s effectiveness longitudinally [13, 14]. Regarding the accuracy and propriety standards, the Net Promoter Score (NPS) [18], a metric tool from business research, can be used to evaluate products or services across large samples from a consumer-centric perspective [19, 20]. The NPS calculates the number of respondents expressing positive views about a product or service (“Promoters”), minus those with negative views (“Detractors”) while disregarding, within its score calculation, all neutral responses (“Passive”) [19].
Each of the mentioned techniques and metrics seems to bring an inductive but relevant strategy to respond to current evaluation gaps in the measurement of PR products’ effectiveness and impact. Combining and reconfiguring them to create and test an operationalizable and replicable evaluation framework is therefore a necessary and an important contribution to improve health research implementation (knowledge and practice). Consequently, we are examining our inductive approach by co-developing, reconfiguring, redefining, and explaining such a procedure as an evaluation methodology, which is introduced and tested in this manuscript.