Relational aspects of building capacity in economic evaluation in an Australian Primary Health Network using an embedded researcher approach

doi:10.21203/rs.3.rs-1325619/v1

Download PDF

Research Article

Relational aspects of building capacity in economic evaluation in an Australian Primary Health Network using an embedded researcher approach

https://doi.org/10.21203/rs.3.rs-1325619/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background: Health organisations are increasingly implementing ‘embedded researcher’ models to translate research into practice. Against specified aims, this paper examines the impact of an embedded researcher model known as the embedded Economist (eE) Program that was implemented in an Australian Primary Health Network (PHN) located in regional New South Wales, Australia. The site, participants, program aims and design are described. Insights into the relational facilitators, challenges and barriers to the integration of economic evaluation perspectives into the work of the PHN are provided.

Methods: The eE Program consisted of embedding a lead health economist on site, supported by off-site economists, part-time, for three- and three-quarter months to collaborate with PHN staff. Evaluation of the eE at the PHN included qualitative data collection via semi-structured interviews (N= 34), observations (N=8) and a field diary kept by the embedded economists. A thematic analysis was undertaken through the triangulations.

Results: The eE Program successfully met its aims of increasing PHN staff awareness of the value of economic evaluation principles in decision making and their capacity to access and apply these principles. There was also evidence that the program resulted in PHN staff applying economic evaluations when commissioning service providers. Evaluation of the eE identified two key facilitators for achieving these results. First, a highly receptive organisational context characterised by a work ethic, and site processes and procedures that were dedicated to improvement. Second was the development of trusted relationships between the embedded economist and PHN staff that was enabled though: the commitment of the economist to bi-directional learning; facilitating access to economic tools and techniques; personality traits (likeable and enthusiastic); and because the eE provided post-embedding support for PHN projects.

Conclusions: This study provides the first detailed case description of an embedded health economics program. The results demonstrate how the process, context and relational factors of engaging and embedding the support of a health economist works and why. The findings reinforce international evidence in this area and are of practical utility to the future deployment of such programs.

Trial registration: N.A.

Health economics

economic evaluation

program evaluation

embedded researcher

health services research

value-based healthcare

primary care

commissioning

Australia

Based on efficiency and equity, Australia has one of the world’s most highly ranked health systems [1, 2]. This status has been achieved despite spending less on healthcare than many other countries – including the United States [3]. Australia has some excellent systems for deciding what healthcare should be funded, such as medicines that receive a listing on the Pharmaceutical Benefits Scheme (PBS). As economic evaluations are central to new PBS listings, the Australian Government has the information to understand both cost and benefit from a medicine. That is, the decision-making process includes an assessment on ‘value for money’. While this evaluation is common at the national level, economic evaluations are relatively rare at the local level [2]. The local level includes local health districts, hospitals, and primary care and it is where Australia spends most of its health budget. There are a multitude of reasons why economic evaluations are not done at the local level, but key ones are a lack of health economic skills in the health workforce and/or a lack of access to appropriately experienced health economists outside health services [2].

The gap in health economic skills in local health services was an important finding in a national report on local level evaluation [2]. The report was a product of New South Wales Regional Health Partners (NSWRHP), an Australian research translation centre accredited by the National Health and Medical Research Council (NHMRC) and funded by health and research partners, and the Medical Research Future Fund (MRFF).

Insights from this work showed local health services wanted to develop their own internal capacity and capability in evaluation, particularly in health economics [2]. They also wanted to work with experts who would be focused on the priorities of the health service and share their skills with staff, a focus that was often lacking when private sector or academic expertise was engaged [2]. The novel embedded Economist (eE) Program was developed to address this need by embedding a health economist in health services to build capacity in economic evaluation. The embedding was undertaken over approximately three months each in four acute health services and also in a Primary Health Network (PHN). This paper reports the evaluation of the eE in the Hunter New England Central Coast Primary Health Network (HNECCPHN); the initial and unique site.

The paper firstly presents the distinct PHN context and the background that informed the ‘economist-in-residence’ approach. Following sections present methods, findings and impacts of the eE program, before distilling our study results into a more general argument in support of developing local capacity and capability in evaluation.

Australian Primary Health Networks (PHNs) were established on 1^st July 2015 by the Australian Federal government to increase the effectiveness of medical services for patients, particularly those at risk of poor health outcomes, and to improve coordination of care. The PHNs are not for profit organisations which do not directly provide health services. Instead, they use a commissioning model to work with primary health care providers, secondary care providers and hospitals. Commissioning includes a range of activities to assess the needs of the population, plan and prioritise services, purchase those services and monitor the quality of the services being provided. PHN commissioning is intended to move the local health system towards more sustainable models of care by not only procuring new or additional services but also transforming and reorganising existing services.

All PHNs are required to undertake a comprehensive needs assessment to identify at-risk and underserved populations. The needs assessment and service planning documents are reviewed by the Federal government. Once approved, services are commissioned to meet identified gaps. The information gathered by regularly undertaking and revising the region’s needs assessment, along with information gained from contract management and relationships with system partners, feeds into the decisions about what services will be purchased and where they will be located [4].

There is little research overall on Australian PHNs, [4-6] and none on commissioning in practice. Commissioning for value relies on evidence informed planning and evaluation of cost against effectiveness, including valid and feasible measures of quality and/or outcomes [7, 8]. In England, however, where commissioning services have been studied, decisions based on anecdotal evidence are reported and the commissioning process described as pragmatic, involving ‘juggling competing agendas, priorities, power relationships, demands and personal inclinations to build a persuasive, compelling case’, necessitating a change in approach by researchers to one that ensures the provision of localised and useful information [7]. Attempts in England to address this skills gap in commissioning organisations have included workshops on systematic evidence searching and critical appraisal of evidence quality [9]. There has also been advocacy for the use of economic modelling tools [10].

Embedded researchers

Embedded researchers, or researchers-in-residence, are researchers who work as part of an operational team. Their role is not simply to bring new skills and expertise to the team. They are charged with ‘mobilising knowledge and creating new evidence for local use and wider dissemination’[11]. Embedded researchers are required to ‘negotiate their expertise, integrate it with the expertise of their colleagues’ and as, where necessary, compromise to reach ‘shared understanding and solutions’ [11].

This approach has been implemented in areas including education, the law, and social care [12]. Researchers have embedded across a wide variety of clinical contexts in the United Kingdom [13-15]; including public, private and voluntary settings that either commission and/or deliver primary, community and/or acute care [11, 14, 15]. Similar initiatives have been implemented in the United States and Canada [16]. There is a small but growing number of embedded researcher initiatives in Australia, most of which have taken the form of a senior academic with clinical experience collaborating with health service staff on co-producing traditional research outputs [17, 18].

Regardless of jurisdiction or clinical setting, the literature on embedded researchers is rated by Ward and colleagues [15] as lacking in analysis ‘disaggregating the components’ of such initiatives, concentrating instead on ‘overviews of the principles of embedded research’. The result is little information about what how embedded researcher initiatives look in practice [15]. There is agreement on the need for more research into how the models are implemented, how they work, under what conditions and for whom [19-22]. Varallyay et al [23] regard this as ‘…particularly important given the lack of clarity about the core features and conditions of ‘embeddedness’ minimally required to achieve the objective of evidence-informed decision-making for health programme improvement.’

Despite the lack of formal evaluations to date, guidance on designing embedded initiatives is not lacking. The literature identifies and lists a number of key success factors for embedded research [20, 24-26]. The following are listed as process factors: nominated lead(s) within the organisation(s) to help navigate the politics, systems, and explain the ways of doing and being [25]; a designated planning or scoping and introductory period of at least three months in which all stakeholders are involved and agree on project scope [12] and secure adequate funding [21, 25, 27, 28]. It is also suggested that clear guidelines are developed which include defining the role of the researcher (including negotiating access to research participants [20, 25, 27]), the types of activities the researchers will be able to undertake, study timeframes and feedback processes [20]; attendance at team meetings and other formal and informal events (to explain the researcher’s role, and explore and utilise opportunities for co-production; be responsive to site needs and identify champions or leaders [12, 20, 25, 27]) and researcher immersion including being physically located with staff (having a desk, access to IT systems, administrative support, communal areas and meeting rooms) [20, 25].

In terms of contextual factors, the literature suggests it is important for the embedded researchers to respond and adapt to the organisation’s decision-making processes to counteract any program resistance [21, 25, 29]. Organisational receptiveness is considered as contingent on the intervention being tailored to local needs [20, 26]; the organisation being open to new learning and ways of working [20, 25] and staff having the willingness and commitment to drive improvements in new and innovative ways [25].

The literature further identifies high-level relational factors that include building on pre-existing relationships [12, 25, 28] developing good personal relationshipsto facilitate trust, and creating shared goals so researchers are seen as team members [20].

The embedded Economist (eE) Program

Against this background, this article now sets out results from embedding a health economist in the first of six program sites, a regional New South Wales primary healthcare network (PHN). This site was unique – the others being large health services centred on hospitals. The full planned program protocol, including a brief literature review and overview has been reported in an earlier edition of this journal [30]. The conceptual underpinnings are also reported elsewhere in detail [31]. The basis is ‘slow science’ [32] and focus achieving impact in the site of research, with academic outputs such as publication not an objective (but to be supported if desired by the site).

By way of summary, the eE Program had three aims:

To increase health service staff awareness of the value economic evaluation can bring to decision making.
To develop health service staff knowledge and capacity to access and apply economic evaluation principles, methods and tools in decision making through formal training and extended exposure to an embedded economist.
To facilitate health service practice change and the routine application of economic evaluation principles in decision making [30].

The aims of the evaluation are to capture the outcomes and impact of embedding an economist and to evaluate the contextual, procedural and relational aspects that facilitated or acted as a barrier to the program [30]. More detail is provided on the relational findings as they add considerably to the published literature.

Setting

The Hunter New England Central Coast Primary Health Network (HNECCPHN is a new (approximately 6 years old) and medium sized organisation, consisting of a Chief Executive; four senior executives; 11 board members; a clinical council; a community council; and approximately 100 full-time equivalent staff. The HNECCPHN region covers 130,000 square kilometres, as set out in Figure 1, and has a population of 1.2 million people who live in small rural and remote rural villages, in regional towns and in densely populated urban centres. The organization has three main offices in Erina, Newcastle and Tamworth located hundreds of kilometres apart.

Participants

The Chief Executive, all senior executives and managers as well as all staff employees had some form of interaction with the eE.

Site study design

A senior ‘front of house’ Lead Health Economist (Professorial level) was embedded within the HNECCPHN from 15 October 2019 – through to 29 February 2020, although allowing for Christmas and New Year breaks, the elapsed time was approximately three and three quarter months. The economist worked with PHN staff to conduct economic evaluations and advise on evaluation and impact design and the inclusion of economics principles, as appropriate, in day-to-day business. While the Lead Economist conducted all the face-to-face interactions with PHN staff, ‘back of house’ support Economists were engaged from the Hunter Medical Research Institute (HMRI) Health Economics team to provide assistance in conducting the economic evaluations and to provide specialist expertise as needed.

The program was structured into three phases: planning, embedding and post-embedding [30]. A description of each phase and how they were implemented in the HNECCPHN is set out in Table 2.

Evaluation design

The evaluation took place concurrently with the eE program, with the last interviews conducted in mid-July 2020, approximately five months after the embedding phase was complete. Ethics approval was obtained from the Human Research Ethics Committee of the University of New England (approval number H-2018-0005), with written consent provided by all participants. All methods were carried out in accordance with relevant guidelines and regulations.

Potential evaluation participants were all senior executives and staff members participating in the eE program. Complementary data was collected at each phase of the project to capture insights into the context, processes, relational aspects and outcomes of the program [28]. Methodological triangulation was undertaken by cross-checking or comparing and contrasting the different data sources which add value to each other by illuminating different aspects of an issue and potentially explaining unexpected findings. For example, the reflective field diaries kept by the embedded economist provided data on what the embedded economists think and do [25, 28]. Interviews allowed issues identified by the evaluator to be explored in more detail with the economist, site executives and participants. Interviews were semi-structured and followed an interview guide based on the program aims and key themes from the embedded researcher literature. Observation notes enabled the Social Scientist to identify if what people said in interviews and field diaries matched what was done in real-time.

Data Collection and Data Analysis

The data that were collected prior during and after the economist embedded at the PHN are listed in Table 1.

Table 1: Data collected for evaluation

PHASE	DATA COLLECTED
Planning	-Observation notes from 3 interactions between economists and PHN senior executives; -6 interviews (4 with PHN senior executives and 2 with HMRI economists) -11 HNECCPHN corporate documents (including the HNECCPHN Strategic Plan; 6 Activity/Work Plans; Needs Assessment Framework and assessment; Commissioning Guidance and Strategy; The Health Planning Compass; the HNECCPHN website); the economist’s field diary; emails between the site and economist and researchers; team and site meeting agendas and minutes; as well as the Introductory PowerPoint presentation and draft eE program operational plan
Embedding	-Observations from 5 interactions between economists and participating PHN staff; -20 interviews (2 with PHN senior executives; 2 with the Lead Economist; 16 with PHN staff participants); -The economists’ field diary; -Emails between the site and economist and researchers, research team meeting agendas and minutes
Post embedding (after the economist left the site)	-12 interviews (3 with PHN senior executives; 1 with the Lead Economist and 2 with Support Economists; 6 with PHN staff participants who were also interviewed during the embedded phase) -Emails between the site and economists and researchers; as well as research team meeting agendas and minutes

Interviews were recorded with consent and professionally transcribed. Participants were invited to review their de-identified transcripts and request amendments before they are entered into QSR International’s NVivo qualitative data analysis software (Version 12). Documents were provided by the sites. The researchers also produced field diaries and researcher observations. All data were entered into NVivo.

A thematic analysis was conducted on the data by applying a coding framework using predetermined themes. The themes were based on a review of the embedded literature conducted by the researchers, used to design the program (and summarised in the introduction of this paper) as well as the program aims. The coding framework was tested and refined by two of the researchers (DP and CJ) each blind coding six interviews and refining the manual and coding based on a comparison of their coding of these initial interviews. The refined coding manual was then applied to the remaining transcripts via an iterative, reflexive and concurrent process of refining themes [33, 34]. The coded themes were corroborated and legitimated by scrutinising the previous stages to ensure that clustered themes were representative of the initial data analysis and assigned codes.

Table 2 Overview of the embedded program as delivered at the HNECCPHN

PHASE	TIMING	PLANNED ACTIVITIES	PARTICIPANTS	PURPOSE	ACTUAL ACTIVITIES
PLANNING	6 months prior	Project Briefing to all Sites at NSWRHP Translational Committee meeting	NSWRHP Translational Committee Members including PHN Site Lead	To introduce the program and understand the context, priorities and expectations of the NSWRHP partner sites and engage the sites as participants	As per plan
	1 month prior	Formation of Program Advisory Committee (With meeting schedule including two meetings during the embedded phase)	eE Steering Committee made up of external chair, site representatives, external economist, consumer representatives, the Program Lead, Lead Economist, and Program Manager/Social Scientist as well as two Social Science Professors (forming a Social Science Sub-committee)	To strategically advise on the program protocol and risk management and to form a Sub-Committee of Senior Social Scientists to advise on program evaluation	As per plan
		Introductory meeting PHN Senior Exec, Economist and Researchers	Program Lead Lead Economist Support Economist Program Manager/Social Scientist Site Lead Chief Executive 1 Additional Executive	To introduce the program and understand the context, priorities and expectations of the PHN	As per plan Executive were very supportive and had few preconceptions of how the program would evolve
	3 October, 2019	Presentation of Introductory Seminar on health economics and the program, covering: Core concepts of health technologies and health technology lifecycle An economic perspective on some problems in healthcare A summary of the range of possible economic evaluations AND the questions they can answer Followed by discussion on: How could health economics be useful to the PHN? If the PHN had specific projects in mind? At the end of the embedded economist program, what would “success” of this program look like? (2 hours)	PHN Seminar attendees (N=10) 4 Senior Exec 2 Health Analysts 3 Health Planners/Managers 1 Commissioning Manager	To begin capacity building and introduce the program as well as the Site Lead, Economists and Social Scientist to a broader group of staff To prioritise areas the economist might focus on	All elements of the presentation were delivered without modification The discussion questions prompted a brainstorming session on what the economist could work on. The Site Lead wound the seminar up by stating the next steps would be for her to collect ideas and feedback from participants and discus with economist when he embedded
	November 2019	Drafting operational plan The Lead Economist and an additional HMRI economist developed a generic draft operational plan to be tailored to the PHNs’ specific needs during the embedded component. This operational plan was designed to provide clarity over the delivery and evaluation of the embedded component as well as set out expectations of what sites are to provide to enable the embedded Economist program to be successful, and the requirements of the embedded Economist to optimise benefit of the program to health services. The plan contained the following sections: Seminar on health economic evaluations Guidelines for initial meetings with health services Protocol for the Embedded Economist placement Evaluating the Embedded Economist program Health service expectations Timelines for the Embedded Economist program Employment considerations Debrief – post placement	Lead Economist Program Manager/Social Scientist Site Lead	To provide clarity over the delivery and evaluation of the embedded component as well as set out expectations	The Plan was an effective starting point for all parties to discuss how embedding would occur

EMBEDDING	16 October, 2019 – 29 February 2019 (with a one month hiatus for Christmas and New Year)	Lead Economist embeds at PHN Newcastle Office 2-3 days per week in person and via email and telephone 1 day a week. Support economists from HMRI work on projects as directed by the Lead Economist	Lead Economist & 4 support economists	To build capacity in economic evaluation	The Lead and other HMRI economist time equated to 328.7 hours during the embedded phase or approximately 22 hours per week There was a total of 233 engagements between the economists and staff during the embedded phase, with 15.3 average engagements per week, and a median number of 7 engagements per week. (N.B. some staff engaged more than once, accordingly numbers represent episodes of engagement). The two satellite sites were only visited once each, because of program resources. Lack of a full-time face-to-face presence was supplemented at ALL sites, including Newcastle with Skype, Zoom and email PHN staff wanted a combination of advice about incorporating evaluation into ‘business as usual’ as well as training is basics such as: costing a model of care, how to do an impact assessment, what outcomes are important etc. Impact assessment was introduced using the Framework to Assess the Impact from Translational health research (FAIT)[35]
	Engaging 25^th October 2019 4^th November 2019	Presentation by Lead Economist at Annual All Staff Forum focused on how staff can include health economics in their everyday work Gastro-diplomacy event (a morning tea and/or a meet and greet/pitch) 10am in person at PHN Newcastle Offices 10.30am via Skype to Erina office 11am via Skype to Tamworth Office	Lead Economist and ALL PHN staff Lead Economist & Program Manager/Social Scientist, 36 PHN staff across 3 site offices	To improve engagement of staff with the economist	These engagement events were an addition to the operational plan, co-designed Lead and Lead Economist after less than expected engagement in the first two weeks of embedding. Both were successful, resulting in greater engagement post events. An engagement strategy should have been considered in Operational Plan
	Celebrating	Website	Lead Economist Project Lead Site Communications Officer Site Lead Program Manager	To celebrate the achievements of the site and create broader program visibility for future sites, the funder and interested researchers	A website housing program resources including one video case study was developed, see: https://embeddedeconomist.com.au

POST EMBEDDING	4^th March 2020 (Week following eE leaving site)	Exit meeting - 10am-11am via telephone	Lead Economist Site Economist Site Lead	To discuss its achievements and lessons learnt and sustainability of capacity building	As per plan
	February 2020 – February 2021	Virtual contact with project lead to finalise projects (time not collected)	Lead Economist 3 Site participants	To finalise projects	The Site wanted ongoing support and access to a health economist. The Lead Economist received requests for support up to twelve months post placement. This included mentoring for PHN staff to complete their own cost-benefit model. The eE’s field diary reveals the ongoing contact took the form of email and Zoom contact sporadically over the twelve months post-placement, leading to the conclusion that the-e is a need to build in an exit strategy and ‘tailing-off’ phase.

Key outputs

The key outputs include six projects that were undertaken by the HNECCPHN whilst the economist was embedded. Table 3 presents an overview of the projects. A variety of economic evaluation skills and tools were developed and applied via these projects including developing a business case; applying cost modelling / cost consequence tools and processes; developing and implementing a program logic model; and identifying and collecting appropriate outcome measures to support program evaluation and impact assessment.

Table 3 Summary of the embedded Economist HNECCPHN Projects

PROJECT

BACKGROUND

NEED

OBJECTIVE

OUTPUTS TO DATE

Upskilling GP Administrative Staff Impact Assessment

Pilot Project (Medical Practice Assistant Project - MPA)

This project addressed national priorities in the rural health workforce by upskilling GP Administrative staff in a number of non-invasive procedures

A business case and an impact assessment (including a cost consequence component) was needed to examine the relative costs and benefits associated with program impacts, such as the time saved by Registered Nurses on tasks which are now be performed by MPAs who have a lower salary, and changes in work satisfaction across different job roles.

To upskill relevant PHN staff in impact assessment and costing

To determine the best method of evaluation and draft evaluation plan including methods

-Funding secured from NSWRHP to conduct impact assessment on the MPA program

-Training PHN staff in impact assessment

- Presentation of a draft protocol at the HNECCPHN MPA conference June 2021 confirmed

-Program Logic for an impact assessment drafted

-Payback metrics identified which cover the domains of research impacts of the MPA Program

-Cost model development commenced

Central Coast Diabetes Alliance Cost Study

Modifications in diabetes care service delivery were made on the Central Coast. A feasibility study of the new model of care (case conferencing) was designed and implemented. The study had an evaluation arm based on process and outcome metrics but no costing.

How much does it cost to deliver case conferencing (for each stakeholder and overall)?

Is it more expensive than the previous model?

Is it more effective than the previous model?

To upskill relevant PHN staff in costing and cost-consequence evaluations.

To add a cost component to the existing evaluation to determine whether this model of care is feasible and cost efficient.

-Cost consequence analysis conducted

-Business case template developed – tailored to stakeholder needs

-Study outcomes reporting template developed

-Capacity built on comparing cost to consequence & conducting a feasibility study

-Demonstrated net revenue (profit) for one stakeholder, which would not have been identified without cost model

-Support and expertise from eE created a substantial and data driven business case

-Staff Lead has built capacity and utilised skills elsewhere in health services & can now build her own cost models

General Practice Fracture Prevention Cost Study

(Study currently on hold due to cost constraints)

A new GP initiative to improve the identification and management of patients with osteoporosis in general practice and reduce the number of re-fractures in patients with osteoporosis who have experienced previous minimal trauma fractures had been developed

How much does it cost?

-What outcomes need to be measured to evaluate the project?

-To design an appropriate cost / cost consequence model

-To upskill a PHN staff member to conduct the costings and develop and collect appropriate outcome measures to support program evaluation

-Upskilling PHN member to conduct the costing and evaluation

-Co-producing a program logic and working cost model template

-Enhanced connections with external experts in evaluation, economics and statistics

-Enhanced internal collaboration between subject matter experts and Health Planning

Healthy Weight Evaluation Project

The increase in the overweight and obese population is a health and strategic concern, with potential impacts on downstream chronic disease. This project pre-dated the eE. The economic evaluation was able to extend in scope due the presence of the eE

Is the Healthy Weight program cost effective?

Can it be financially viable from a PHN and GP perspective?

How will households engage?

To engage PHN staff in an economic evaluation that would normally have taken place as a standard ‘consultancy’ (i.e. at arms-length from the PHN). i.e. the aim was to better engage PHN staff in the process of the evaluation

To ensure optimal implementation of this model of care

-The cost model for this project utilised the model developed for the Diabetes Alliance cost study set out above

-Economic evaluation, pricing models of the Healthy Weight Program, and design modifications of the Program improved effectiveness and efficiency

Rural Communities Project &

Trusted Advocate Project

-The Rural Communities project is a community engagement initiative to assist primary care in areas affected by drought

-The Trusted Advocate Project is an intervention in Scone, where the PHN provided training to selected people who were often approached by community members seeking advice/referral (including but not limited to advice on mental health issues).

How can these projects be evaluated and what is their impact?

To build staff capacity to conduct impact assessments on both projects

-Detailed and project specific advice about introducing impact assessment to both programs was provided

-Upskilling PHN member to conduct assessments, including but not limited to drafting logic models and assessment tools

-Evaluation and impact assessment currently being conducted by PHN staff member

Building impact assessment into operational decisions

The PHN increasingly needed to demonstrate ‘value for money’ to government funders.

The PHN needs an organisation-wide approach to measure the impact of its programs and build staff capacity in applying the approach

To integrate and build organisational capacity in applying the FAIT Impact Assessment Framework[35]

-Staff training on impact assessment via workshops conducted

-FAIT applied to a major PHN initiative (Medical Practice Assist – publication of results with PHN co-authorship out for review)

The following three sub-sections set out the process, contextual and relational facilitators and barriers encountered.

Process facilitators and barriers

Two process issues facilitated the planning phase: senior and active executive sponsorship within the HNECCPHN and the seniority of the Lead Economist and applied experience of all the Support Economists. Within the PHN, the eE Program was facilitated by the appointment of a Site Lead, a senior executive who ‘concierged’ the program from within the organisation, liaising with the Program Manager and acting as a gateway for staff to access the economists. The Lead Economist and the Support Economists, with more than twenty-years of applied experience working with health services, and their understanding of the regulatory context in which the PHN operated were considered crucial to encourage executive teams to consider new ways of thinking and working.

Three process issues facilitated the embedded phase: the physical location of the economist; administrative support to book formal meeting requests; and identification of existing in-house expertise. Placement of the embedded economist in an open plan office was a major facilitator as the economist was highly visible to PHN staff. This also provided easy and non-formidable access for staff to initiate informal conversations, which led to both more formal engagement in project work as well as one off-advice. A single contact point and process for booking meetings between staff and the economist was necessary. This administrative support was provided by the economist’s organisation (HMRI), alleviating the need for the site to coordinate across organisational diaries. Although not known prior to the eE Program, internal capacity at the PHN in the form of an individual with economics training was identified during the embedded phase. This ‘find’ was a consequence of informal conversations with the Lead Economist, and provided the PHN with the potential of capitalising on this existing human resource during and after the eE program. Some capacity building was tailored around this key individual’s skills and interest, potentially facilitating sustainability of skills beyond the eE Program.

Project challenges in the embedded phase included: depth of embedding (economist time available per week); the need for a communication and visibility strategy; the need for an exit strategy; and the overall length of embedding period.

The economist embedded for 328.7 hours - approximately 22 hours per week. This partially embedded model was implemented due to the available project resources. It meant however that the economist was not fully relieved from their everyday jobs and were constrained by competing interests and demands. Despite this, the economist was available outside of their physically embedded time by telephone and via virtual meetings.

The first two weeks of embedding resulted in less engagement than expected (N=20 staff members). A more detailed communication strategy was then developed consisting of: three morning teas (one-face-to-face and two conducted via Skype with satellite offices) where the Lead Economist explained the program to managers; a ‘pitch’ emailed by the Site Lead to managers asking for project nominations to be ranked in order of organisational priority by executive; and a presentation by the Lead Economist at the annual ‘All Staff Day’ to increase program visibility and foster staff engagement. Given the success of the Lead Economists’ presentation at the ‘All Staff Day’ and with the benefit of hindsight, the Site Lead thought earlier communication, engaging all staff, and clarifying the process for engaging with the economist to ensure a strategic approach to a limited resource, during the planning and embedding phases would have been beneficial. Staff also wanted improved communication throughout the embedded phase, including about what other projects the economist was working on. Overall, embedded research takes time to build visibility and momentum, timeframes may need to be adjusted to accommodate for this and staff need to be made aware that not everyone will have access, particularly when the economist was only intended to be placed at the PHN for a limited period.

Despite a slow start, a crescendo of projects presented themselves toward the end of the embedded period, predominantly as a result of visiting a satellite site for the first time during the last week of embedding, resulting in ongoing remote work. The desirability of developing an exit strategy and moderate expectations about how much work can be done and when the work will end earlier in the program was identified.

The embedded component of the HNECCPHN project was scheduled for three months but actually went over time by approximately three weeks. Three months was perceived by all participants as the shortest possible timeframe, with six months cited as more realistic, as projects presented late into the embedded phase, right up to the last day of embedding. A longer lead time for the economist to immerse within the organisation and identify appropriate projects and priorities would have been beneficial.

These process issues are all interrelated as they represent consequences of limited economist time and eventual high demand for services. For instance, a slightly slow start would not have been identified as an issue with a different schedule or if the engagement was not perceived as very high value.

Contextual facilitators and barriers

A number of organisational characteristics impacted positively on the planning and embedding phases of the HNECCPHN program. Board and senior executive support for the eE Program was pivotal in facilitating engagement of staff. The Chief Executive’s leadership, the not-for-profit status of the PHN and its medium size (approx. 100 FTE) cultivated an enthusiastic and receptive culture where employees would easily see the benefit and impact of the eE program. These factors made it feasible for the eE program to achieve broad organisational reach. An openness to innovation in the PHN also facilitated involvement.

They're really dynamic and really innovative thinkers. eE 1

Relevance of the eE Program to organisational function was another facilitator: Given the focus of the eE Program was on upskilling participants in economic evaluation, its relevance to commissioning was easily understood. More than this, participants believed that enhanced evaluation skills would improve commissioning.Senior executives described improvement currently occurring in the PHN with evaluations being designed to inform future decision-making:

We’re changing a lot of the way primary mental health service is delivered and funded. And so we are doing a pre-emptive evaluation framework at the moment … that will hopefully help us know in four or five years’ time whether the changes that we’ve made are the right ones. PHN PARTICIPANT 3

This executive team member was keen to ‘to get decisions based on reliable information’ and reflected on the feelings of vulnerability created by the current commissioning processes:

And we’ve felt a little bit vulnerable as some of our decisions could be challenged and it would be really great to have some more rigour behind some of the things that we’re doing. PHN PARTICIPANT 3

This executive felt the PHN lacked the information to measure value:

On what basis are we measuring value?…We talk a lot about the quadruple aim – but I just don’t think we have the information that we need necessarily to measure value. PHN PARTICIPANT 3

The need to gain practical skills in measuring the impact or value of their programs on their population and improve on those impacts was a motivator for participating in the embedded Economist program:

We want improvements in quality of life for patients …That's really important to us. SITE LEAD

In line with senior executives, staff recognised the lack of skills and capability, especially in relation to economic evaluation where they were described as ‘basic to non-existent’. PHN PARTICIPANT 1.

There was broad consensus amongst staff of the need for a significant improvement in the quantity and quality of evaluation undertaken:

It’s one of those things we know we should be doing but we’re not really sure how. PHN PARTICIPANT 14

I think we need to develop it. It’s not a strong thing in the PHN really, around evaluation and use of good economic decision making around which programs we’re going to run …. I don’t think that we always are able to identify or articulate … whether what we’re planning on doing is going to be appropriate. PHN PARTICIPANT 5

There was also a general interest amongst staff to know when to dis-invest in programs, so as to optimise impact from their work.

The eE program therefore came along at a time that aligned with the PHNs’ stage of organisational evolution. The organisation was moving from an establishment phase to attempting to commission services and support improvement in a range of services and demonstrate impact and were hoping the eE program would provide tools and training to inform more systematic or rigorous way of making decisions about planning and commissioning.

I think we’ve been in a very rigorous establishment process as the PHN for a few years now, attempting to commission services and support improvement in a range of services [but] we haven’t necessarily had the capacity and capability to demonstrate our impact sufficiently [although] the planning team has developed excellent health outcomes and wellbeing framework. … it’s part of our evolution and maturity. So I think it’s an opportunity to build a bit more skill base across the PHN team around effective evaluation or analysis, particularly obviously with an economic slant. PHN PARTICIPANT 4

Staffs’ limited time impacted on engaging with the economists, with some participants expressing regret at the end of the program that they had not had time to engage more.

Geography impacted negatively on the eE Program at HNECCPHN. Embedding at least partially face-to-face on-site was considered essential to the program by all participants, predominantly because of the opportunities this presented for quick, unplanned interactions and real-time feedback. There was the perception staff in the Newcastle office benefited more from the project than staff in other offices because of the greater face-to-face contact.

Relational facilitators and barriers

The following three sections set out the themes and sub-themes that emerged from data coded under ‘relationality and engagement’. This code collected data that addressed: How and why did engagement occur or not occur? What relational processes, mechanisms, attributes & skills were required for engagement to occur?

The PHN had been utilising some services from the HMRI economists prior to the eE Program, to evaluate some commissioning contracts. According to the Lead Economist, previous relationship building impacted positively on the eE Program at the PHN:

We had established and good relationships … so we are already at first base (eE 1 email 21.08.2020).

The embedded Economists’ attributes

When asked to comment on the necessary attributes for an embedded researcher from the point of view of replicating the program in other sites, the Lead Economist stated a quick thinking, solution-based, confident and facilitative approach was necessary:

The need to think on your feet super quickly. You’ve got … an hour and … your minds got to be going through what potential solutions might be. You’ve got to be confident enough to say, if there’s isn’t a solution, ‘you’re going to have to wait and I’m going to have to come back to you’ … And there’s been times that people have told me their problems and I actually know, that’s not my skill set, ‘So you need to talk to [someone else] …‘if I said something I’d be guessing’. And you need the confidence to be able to say that. eE 1

Senior executives (N=3) and staff participants (N=12) expressed a number of positive attributes held by the economist that they perceived as facilitative of the program. The economists’ intellect and knowledge of the subject matter was foundational (N=3 staff participants):

Just a really smart head... he knows his stuff. PHN PARTICIPANT 5

Successful engagement was perceived to be a result of the economist’s ability to communicate in a way that provided clarity to the specialised subject matter (N=2 senior executives; N=4 staff participants):

[eE] has a really nice way of communicating those things in a very – I wouldn’t say simplistic, but it is quite simple and it’s not onerous for people who don’t have that kind of evaluation perspective, or don’t have that experience or background in thinking about evaluation or thinking about value-based health-care or those sorts of things. PHN PARTICIPANT 11

Additional attributes related to successfully conveying specialised knowledge included the ability to make health economics interesting and relevant to participants’ work (N=2 senior executives; N= 5 staff participants):

He’s been well received …he’s just really interested …in making it relevant for us.

PHN PARTICIPANT 17

Other communication skills were also core facilitators for the program. Participants valued the economists’ receptive, easy-going, open communication style including their enthusiasm and encouragement (N= 2 senior executives; N=4 staff participants):

I guess, it comes back to the response that she received, and the encouragement she received helped her go further. SITE LEAD

As well as their approachability and responsiveness (N=5 staff participants):

He’s been very approachable and has been very open to having discussions with anybody and everybody. PHN PARTICIPANT 9

Well they need to be quite open people, people who are open to new ideas, or open to being approached in different ways. Because, I think [eE] has been approached in a million different ways but he’s never closed the door. PHN PARTICIPANT 5

Well obviously good and positive was [eE’s] willingness and ability to jump into something very quickly in a very short turnaround time and give us some feedback, which was great. And … he also provided feedback over a weekend, which was really, really generous of him. So he’s quite responsive. Yeah, yeah, very responsive. PHN PARTICIPANT 1

More than this, the economist openly displayed genuine interest in participants’ work (N= 8 staff participants):

[eE] is able to demonstrate an actual interest in the work that people are doing PHN PARTICIPANT 3

Leading one participant to conclude:

I love [eE]. I think he’s amazing! PHN Participant PHN PARTICIPANT 18

Ways of working

The theme ‘ways of working’ captured data on what facilitated mutual understanding and knowledge transfer and capacity building for each of the six projects the economist worked on with PHN staff. Two sub-themes emerged: ‘coaching’ and bi-directional engagement.

Coaching

The economists avoided both traditional consultancy and complex academic approaches. Instead, providing tailored support and capacity building to enable the application of relevant tools and approaches to specific problems with the health service. The approach taken was described by one staff participant as ‘coaching’:

It is about that mentoring, coaching approach … [the eE] just takes people through step by step, doesn’t land a whole lot of information on somebody’s desk and expect them to digest it. He sits down with you, with the information in front and goes through it. So not a dump and run, a gentle reading. PHN PARTICIPANT 20

Participants (N=12) in four of the six projects undertaken with the economist commented on this coaching approach. They described it as being democratic, with the economist viewing participants as equal partners:

He can relate to anyone at their level. He doesn’t come across as being superior.

PHN PARTICIPANT 20

Rather than as research subjects:

I think that really has resonated with people so they don’t feel like there was a risk that we could have been guinea pigs, and sort of treated like lab rats in some ways, but that’s definitely not the experience. PHN PARTICIPANT 3

‘Coaching’ was also characterised by the Lead Economist as problem solving rather than imposing a solution:

Understanding that what health services need and what the academic institutions think they need are really two different things. That it can be quite insulting to them to have an academic tell them "There's a better of doing this you know."…The attitude I think that's going to be important is, right, you've got a problem, let's do the best we can to get this problem sorted. Not telling them how to reshape their business model…There's a bit of work in doing that. eE 1

Post embedding the economist’s willingness to continue coaching three staff members, impacted positively on the program in that it enabled further capacity building to occur and ensured the trust built throughout the embedded phase was not broken by abandonment at the end of the program timeframe. During this phase ‘coaching’ was less intense:

There might be a three-month period of where you have the intensity of the intervention, but you need to allow a tail period of where you will still have some contact…I think this is quite okay too because you can’t just abandon them…ultimately in capacity building you’re trying to not do the work for people, you’re passing the skills over for them to do. So as part of that tailing down period more and more of that gets passed over. And all I’m doing now with the PHN staff, I’m only providing feedback on what they’re doing, and providing guidance that you’re on track, or you’re off track, and this is what you should consider. eE 1

The eE’s field diary reveals ongoing sporadic email and Zoom contact about two projects over approximately two- and one-half months and email contact for approximately one-year post-embedding.

Bi-directional learning

Coaching was underpinned by bi-directional knowledge exchange, with the economists’ learning as much as staff participants:

…we're learning as we do this program…I would have to say, for my personal development, has been a river of gold. eE 1

Economists reported learning more about the need for a greater applied approach to economic evaluation at the local level than originally contemplated:

When I went into the PHN it was a bit of a rude shock for me because even though I felt we were very applied, we’d focus on the problems that health services bring to us, but when I actually got into the PHN I had a completely different perspective and it was a new learning for me that we were not as applied as I thought we were…people were asking me issues around evaluation and economic evaluation that were almost like first steps that we tend to ignore…normally we would have just launched straight into, oh, that’s the project, you want an evaluation of that, this is what the evaluation’s going to look like. So yes, it’s applied in that sense that we focused on their problems, but what we were missing previously is that understanding that they needed help with that background work of what are the pathways to determining you actually have a project that even warrants an economic evaluation? eE 1

They also reported learning more about the context of local health service delivery:

It was really interesting to find out more about health care systems at the local level. So, it’s been a good satisfying learning experience for me… My knowledge of health systems and how they might be run at a local level as opposed to at a state or a federal level, or at a hospital level, has expanded. So, I’m a better health economist for having done those jobs. I can certainly say that. eE 3

This way of working was described by an economist as different to their usual way of work in that closer relationships were formed facilitating greater learning for both the economists:

In a normal situation, usually if I do a job for somebody here at HMRI, they won’t learn much about what I do. I mean they’ll learn something. If they’ve never seen a cost analysis before, of course they’ll see it for the first time and I’ll talk about how I did it. But they won’t get a chance to work with me. In this case I actually encouraged [DE-IDENTIFIED]… to make changes in the spreadsheet…we worked together closely… It was more hands on in this case. So, it wasn’t like a normal job. It was very much one where it was set up on the initial understanding of the closer working relationship with a practical aspect. eE3

As well as staff participants:

I want [LEAD and SUPPORT ECONOMISTS] to support me rather than them do all the work and then just hand me a cost model at the end because then I don’t learn anything PHN PARTICIPANT 18

Impact

Interviews with all program participants suggests the program successfully increased staff awareness of the benefits of economic evaluation. Increased awareness took the form of looking at work differently and thinking about evaluation more, from a program or initiatives inception. Both senior executive (N=2) and staff participants (N=10) believed the eE program had developed their capacity to access and apply economic evaluation. It did so by providing additional information, knowledge and tools. In addition, interviews with the Site Lead and staff participants (N=3) suggest there was emerging evidence to demonstrate practice change and the routine application of economic evaluation principles in commissioning. Evidence included increased use of logic models:

We’re starting to use some logic models which – which is helping obviously, and I think there’s some documentation out on that. PHN PARTICIPANT 5

As well as an increased consideration of the need to evaluate commissioning contracts:

I put into my contracts with my new providers, as part of a deliverable – and [the eE] gave me permission in a way to do this… they have to submit an impact evaluation report. [And that wasn’t in there before the eE Program]? No. There was no evaluation report whatsoever PHN PARTICIPANT 8

When I’ve got my managers in a monthly management meeting, those types of questions will come up. Or … I’ll see an email where they’re pitching something to me, and they’ve actually considered those questions. And I can see that they’ve thought about it and they’ve written the answers to those questions in the email to try … and get it over the line. Whereas normally that wouldn’t be their argument. Normally their argument … would be just around relationships or it could be about,’ Well, this is how we’ve always done it,’ or, ‘This is government policy’. Or ‘health has funded us to do this.’ While all those things are still really important, they’re now adding in that other element. ‘Is it really the best way of doing it?’ ‘Could we do an evaluation?’ ‘Could we spend this $20,000 on an evaluation, rather than just rolling it over again this year?” That’s the sort of stuff that I’m getting, which is really great. That’s really great. SITE LEAD

However, senior executive (N=2) and staff participants (N=12) stated they wanted ongoing support if capacity building was to continue:

Ongoing support would be ideal…what we found was we – we were sorry that the time was coming to an end because we saw an ongoing need for the embedded Economist in providing further support and upskilling, and capability development, which I suppose reflects the success of the initiative in its early stages. PHN PARTICIPANT 4

Strengths and limitations

These results must be considered in light of the following limitations: The eE programembedded a particular skill set – health economics – in a particular context – a primary health network located in regional Australia. The evaluation of future local health district (care provider) sites at which the economist will embed within the NSWRHP footprint may produce quite different results. This is a small scale, qualitative case study. Whilst all program participants from this site were approached to participate in the evaluation, participants self-selected which increases the risk of bias. The evaluation was conducted by a Social Scientist who also occupied the dual role of Program Manager. However, to ensure the research remained theoretically and methodologically sound, the embedded Economist program is overseen by the eE Social Science Research Committee.

Overall, our results confirm the commentary available on embedded research. We note a number of similarities between our process, contextual and relational facilitators, challenges and barriers to the high-level ones identified by previous researchers examining the implementation of embedded models in health care [11, 12, 18, 28, 36, 37]. The high-level facilitators and challenges and barriers encountered in this program point to a number of practical lessons that should be considered in planning and implementing future embedded research, as catalogued below and mapped to similar recommendations from existing literature.

Process factors

We confirmed the importance of a number of previously suggested process facilitators including the need to appoint site leads [28, 37]. Whilst our process for communication and engagement did result in securing executive engagement and the identification of a champion in the form of a trained economist, difficulty came in explaining how the economist would work and managing expectations about level of embeddedness and chiefly, how the eE would leave the site. Engagement should have occurred earlier and prior to embedding, in a longer designated planning phase [18, 36, 37]. The development of an operational plan helped with scope, but the volume of work in the available time and exit from the site were difficult to manage – especially considering the longer than expected lead time to co-produce the scope of work. We all had ‘little experience of the roles and therefore it took time and learning from all parties to embed in the role’ [27]. However, we advocate for the need for any planning phase to ensure program flexibility: Once health services identify the problems they wish to address, the program needs to be able to pivot to provide whatever skills are required. This might mean there is a need to find another researcher with a particular expertise, for example, statistical modelling.

Our program funding extended only to a part-time embedded researcher for three months. The time it took to gain momentum, the need for work to be conducted post-embedding and the economists’ struggles with competing work interests suggests more secure funding to allow flexibility in length and depth of embedding would have been preferable [18, 27, 28, 37]. The quanta of funding we had available created tensions; all parties tried extremely hard, but this was not a sustainable model.

Co-located workdays (including having a desk, access to IT systems, administrative support, communal areas and meeting rooms) enabled staff to seek informal support, which was greatly valued by staff in this particular location. Pain and colleagues suggest ‘Feelings produce impacts produce feelings’: emotional dimensions are not side-effects but are active in generating impact [24]. The warm relationships developed were central to program success. Establishing relationships requires a large investment at the front end of a program necessitating a ‘high level of physical presence and face-to-face interactions.’ [12]. Once established these relationship may be maintained virtually, however some on-going face to face contact is necessary [12]. Lack of co-location was challenging for the geographically dispersed PHN sites that did not have face-to-face access, confirming Vindrola-Padros’ (2019) research [12].

Contextual factors

The form, size and culture of the site all contributed positively to the program [18]. The PHN is a medium size organisation (approx. 100 FTE) – so a small number of economists could feasibly effect change – with new conversations about evaluation documented as occurring widely. It is a relatively new organisation (established 2015), open to learning and actively seeking to improve the way it works and develop better processes. The PHN committed to working with the economists to upskill staff in economic evaluation, a skill deemed necessary and lacking by both senior executives and other staff to improve decision making. The economists approached the embedded phase well informed of the organisational context and with a willingness to be responsive and adapt to organisational needs. The PHN itself was open and responsive to the eE Program. Staff were highly motivated to increase their skills in economic evaluation to improve commissioning and ‘make a difference’ [37]. The subject matter was seen as responsive and tailored to organisational needs as PHN funders are placing increasing emphasis on the need to demonstrate value (external context) and much of the co-designed work had a focus on impact assessment, highly relevant for all the work of a commissioning organisation [36, 38].

Relational factors

The literature acknowledges the importance for embedded researchers of social and interpersonal skills as well as technical or topic specific expertise. Desirable interpersonal skills include inquisitiveness, receptiveness and enthusiasm as well as communication skills [21, 25, 27]. Our findings are confirmatory: Embedded researcher skills and attributes are pivotal in establishing the way of working or interactions during the embedded period, which is in turn pivotal in ensuring success.

However, our research goes further and adds to the importance of relational issues in embedded research: The greatest facilitator for this program was the way of working, the ‘coaching’ or ‘slow science’ [21, 32] approach facilitated by the economists. Whilst previous authors have highlighted a variety of ways of working based on degrees of objectivity and levels of embeddedness [17, 21, 39]; the ‘role domains’ of knowledge brokering [14, 40] and ‘mechanisms of collaboration for co-production’ [41] we argue that embedded research will benefit from viewing the work done or interactions through an additional lens of embedded processes as ‘slow science’ [31, 32] – especially where capacity build/education element is needed. In line with our philosophic commitment to slow science [31], these results build on that thinking by unpacking how a process of capacity building can occur during embedding.

Building capability in health economics, or any other expert skill within health or commissioning organisations, calls for a ‘re-invention’ of health service research: moving from the pursuit of grand scientific findings that apply to all services everywhere, to a more nuanced combination of understanding local context and needs, workforce capacity building, design of realistic plans of action, and programmes of investigation targeting priority problems of the health service. Instead of bypassing complex challenges in order to adhere to theoretical-methodological strictures, this health service research re-invention ensures we take local complexity as our primary point of departure. But this only becomes possible when we engage healthcare staff as co-researchers and prioritise practical achievement and future-facing learning over the production of discipline-specific knowledge and method-delimited evidence [31].

The quest for discipline specific knowledge and evidence is generally legitimated with reference to professional researchers’ ability, anchored in methodological rigour, to impersonate a ‘point-of-view from no-where’ [42]. This rigour and positioning preclude or at least limit public negotiation over its theoretical and methodological principles and its practical consequences. It is for this latter reason that such science is described in Stengers’ latest book as ‘fast’ or ‘industrialised’ science [32]. Fast science prioritises its self-defined rules and interests, not just with regard to how such science is conducted, but also with regard to whomever is given a say over its funding, its orientation, its execution, and its real-life impact.

It is against this background that Stengers [32] is motivated to advocate for ‘slow science’ – a science whose priority is to negotiate its orientation, execution, and real-life impact with local actors in the sites where such science is conducted and where its findings are to count. ‘Soft’, in this context, refers to a norm according to which all dimensions of science are submitted for and remain open to public deliberation. Elsewhere we outlined the view that healthcare research in particular warrants adopting a more local, contextualised and practical approach in order to tackle health service complexity with requisite levels of fine-grained sensitivity and local relevance [31] . To do justice to the complexity of contemporary care, slow science in health embeds its researchers in services, or places them ‘in residence’, to shrink the distance between research and practice.

To our knowledge, there are no studies that have explored researcher embedding in a regional Australian commissioning organisation.

This article demonstrates the previously reported evaluation components of the study protocol [30] to be effective in identifying barriers and facilitators to embedded research. Whilst the evaluation findings largely support the guiding principles set out in the literature to date as applicable to a regional commissioning organisation, the detail goes beyond affirming these-high level principles in ways will be of utility to other groups designing and evaluating such programs.

This article has teased apart the relational aspects of engagement of an embedded economist with health service managers across all phases of the project including the planning, embedding and post embedding phases of the program, setting out the facilitators and challenges and barriers to embedded research in this context.

Our evaluation contributes to the empirical evidence on embedded research in three main ways. First, it examines embedded researchers in a previously unexplored context: a regional Australian primary health commissioning network. Secondly, it describes a detailed design/schema for an embedded program and ideas for improvement that can be adapted by others. Thirdly, it adds to the importance of relational issues in embedded research: the greatest facilitator for this program was the way of working, the coaching or ‘slow science’ approach employed by the economists.

Despite its stretched resources, this embedded research project had impressive outcomes. The organisational outputs, in the main will not be reflected by academic outputs. Hopefully, this case study describes is the start of a journey where continued funded engagement will develop increased PHN staff skills and the development of research endeavors that are democratically designed and undertaken and also contribute to the academic literature.

Overall, this project worked because the organisation was highly receptive and the skills offered were a perfect fit for a commissioning; because of the democratic stance taken by the economists (‘also learning’) and their effort to encourage the accessibility of their tools and techniques (‘its not rocket science’) and; finally, due to the warmth and enthusiasm all parties brought to the engagement.

ComPrac: Community of Practice

eE: embedded Economist

HMRI: Hunter Medical Research Institute

HNECCPHN: Hunter New England Central Coast Primary Health Network

IT: Information Technology

NSWRHP: New South Wales Regional Health Partners

PHN: Primary Health Network

Ethics approval and consent to participate

Ethics approval for the embedded component and evaluation for the HNECCPHN site was granted by the University of New England Ethics Committee, approval number: HE19-196. Ethics approval for the education component for the HNECCPHN site was granted by the Hunter New England Research Ethics Committee approval number: 2020/ETH00012. These approvals were ratified and registered with the University of Newcastle Research Integrity Unit number: H-2020-0234.Written informed consent was obtained from all participants.

Consent for publication

Not applicable

Availability of data and materials

The datasets generated and analysed during the current study are not publicly available, due to the small sample size and size of the participating organisation: The researchers did not receive consent from participants to make data publicly available, as participants required anonymity to ensure they spoke freely about their organisation and the program. To access available data sets, please contact Dr Lisa McFayden [email protected]

Competing interests

The authors declare that they have no competing interests.

Funding

This eE Program pilot study is funded by the Australian Government’s Medical Research Future Fund (MRFF) as part of the Rapid Applied Research Translation program. (MRF9100005). The study protocol has been independently peer reviewed by the funding body. The funding entity had no other role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Authors' contributions

All authors have provided input to, reviewed, edited and approved the final version.

Conceptualization: CJ, RI, NG, AS and DP

Funding acquisition: CJ

Methodology: CJ, RI, NG and DP

Project administration: CJ, DP and LM

Writing - original draft: DP

Writing - review & editing: DP, CJ, RI, NG, AS and LM

Acknowledgements

We acknowledge the participation of the Hunter New England Central Coast Primary Health Network staff and Hunter Medical Research Institute embedded economists.

We also acknowledge the efforts and help of the following members of the embedded Economist Steering Committee who provide oversight, advice and direction on the program design and implementation: Mr Peter Johnson (consumer representative), Ms Wendy Keech (Director, Health Translation SA), Mr Michael Di Rienzo (Chief Executive, Hunter New England Local Health District), Dr Antonio Penna, (Executive Director Office for Health and Medical Research) and Professor Rachel Morton (Director, Health Economics - NHMRC Clinical Trials Centre, Faculty of Medicine and Health, The University of Sydney).

1. CSIRO: Future of Health: Shifting Australia’s focus from illness treatment to health and wellbeing management. In. Canberra; 2018.

2. Searles A, Gleeson M, Reeves P, Jorm C, Leeder S, Karnon J, Hiscock H, Skouteris H, Daly M: The Local Level Evaluation of Healthcare in Australia. In. Newcastle, NSW: NSW Regional Health Partners; 2019.

3. Australian Institute of Health and Welfare: Australia's Health 2016. Canberra: Australian Institute of Health and Welfare; 2016.

4. Henderson J, Javanparast S, MacKean T, Freeman T, Baum F, Ziersch A: Commissioning and equity in primary care in Australia: Views from Primary Health Networks. Health & social care in the community 2018, 26(1):80-89.

5. Freeman T, Baum F, Javanparast S, Ziersch A, Mackean T, Windle A: Challenges facing primary health care in federated government systems: Implementation of Primary Health Networks in Australian states and territories. Health policy 2021, 125(4):495-503.

6. Anstey M, Burgess P, Angus L: Realising the potential of health needs assessments. Australian Health Review 2018, 42(4):370-373.

7. Wye L, Brangan E, Cameron A, Gabbay J, Klein JH, Pope C: Evidence based policy making and the ‘art’of commissioning–how English healthcare commissioners access and use information and academic research in ‘real life’decision-making: an empirical qualitative study. BMC health services research 2015, 15(1):1-12.

8. Clarke A, Taylor-Phillips S, Swan J, Gkeredakis E, Mills P, Powell J, Nicolini D, Roginski C, Scarbrough H, Grove A: Evidence-based commissioning in the English NHS: who uses which sources of evidence? A survey 2010/2011. BMJ open 2013, 3(5):e002714.

9. Sabey A: An evaluation of a training intervention to support the use of evidence in healthcare commissioning in England. International journal of evidence-based healthcare 2020, 18(1):58-64.

10. Sanders T, Grove A, Salway S, Hampshaw S, Goyder E: Incorporation of a health economic modelling tool into public health commissioning: evidence use in a politicised context. Social Science & Medicine 2017, 186:122-129.

11. Marshall M, Eyre L, Lalani M, Khan S, Mann S, de Silva D, Shapiro J: Increasing the impact of health services research on service improvement: the researcher-in-residence model. Journal of the Royal Society of Medicine 2016, 109(6):220-225.

12. Vindrola-Padros C, Eyre L, Baxter H, Cramer H, George B, Wye L, Fulop NJ, Utley M, Phillips N, Brindle P: Addressing the challenges of knowledge co-production in quality improvement: learning from the implementation of the researcher-in-residence model. BMJ quality & safety 2019, 28(1):67-73.

13. Marshall M, Eyre L, Lalani M, Khan S, Mann S, de Silva D, Shapiro J: Increasing the impact of health services research on service improvement: the researcher-in-residence model. J R Soc Med 2016, 109(6):220-225.

14. Wye L, Cramer H, Beckett K, Farr M, Le May A, Carey J, Robinson R, Anthwal R, Rooney J, Baxter H: Collective knowledge brokering: the model and impact of an embedded team. Evidence & Policy: A Journal of Research, Debate and Practice 2020, 16(3):429-452.

15. Ward V, Tooman T, Reid B, Davies H, Marshall M: Embedding researchers into organisations: a study of the features of embedded research initiatives. Evidence & Policy: A Journal of Research, Debate and Practice 2021.

16. Vindrola-Padros C: Can We Re-Imagine Research So It Is Timely, Relevant and Responsive? Comment on "Experience of Health Leadership in Partnering with University-Based Researchers in Canada: A Call to 'Re-Imagine' Research". Int J Health Policy Manag 2021, 10(3):172-175.

17. Coates D, Mickan S: The embedded researcher model in Australian healthcare settings: comparison by degree of “embeddedness”. Translational Research 2020, 218:29-42.

18. Churruca K, Ludlow K, Taylor N, Long JC, Best S, Braithwaite J: The time has come: Embedded implementation research for health care improvement. Journal of evaluation in clinical practice 2019, 25(3):373-380.

19. Coates D, Mickan S: Challenges and enablers of the embedded researcher model. Journal of health organization and management 2020.

20. Vindrola-Padros C, Pape T, Utley M, Fulop NJ: The role of embedded research in quality improvement: a narrative review. BMJ Qual Saf 2017, 26(1):70-80.

21. Churruca K, Ludlow K, Taylor N, Long JC, Best S, Braithwaite J: The time has come: Embedded implementation research for health care improvement. J Eval Clin Pract 2019, 25(3):373-380.

22. Marshall M, Pagel C, French C, Utley M, Allwood D, Fulop N, Pope C, Banks V, Goldmann A: Moving improvement research closer to practice: the Researcher-in-Residence model. BMJ Qual Saf 2014, 23(10):801-805.

23. Varallyay NI, Langlois EV, Tran N, Elias V, Reveiz L: Health system decision-makers at the helm of implementation research: development of a framework to evaluate the processes and effectiveness of embedded approaches. Health research policy and systems 2020, 18(1):1-12.

24. Pain R, Askins K, Banks S, Cook T, Crawford G, Crookes L, Darby S, Heslop J, Holden A, Houston M: Mapping Alternative Impact: Alternative approaches to impact from co-produced research. 2016.

25. Cheetham M, Wiseman A, Khazaeli B, Gibson E, Gray P, Van der Graaf P, Rushmer R: Embedded research: a promising way to create evidence-informed impact in public health? J Public Health (Oxf) 2018, 40(suppl_1):i64-i70.

26. Walley J, Khan MA, Witter S, Haque R, Newell J, Wei X: Embedded health service development and research: why and how to do it (a ten-stage guide). Health Res Policy Syst 2018, 16(1):67.

27. Gradinger F, Elston J, Asthana S, Martin S, Byng R: Reflections on the researcher-in-residence model co-producing knowledge for action in an Integrated Care Organisation: a mixed methods case study using an impact survey and field notes. Evidence & Policy: A Journal of Research, Debate and Practice 2019, 15(2):197-215.

28. Wye L, Cramer H, Carey J, Anthwal R, Rooney J, Robinson R, Beckett K, Farr M, le May A, Baxter H: Knowledge brokers or relationship brokers? The role of an embedded knowledge mobilisation team. Evidence & Policy: A Journal of Research, Debate and Practice 2019, 15(2):277-292.

29. Rapport F, Clay‐Williams R, Churruca K, Shih P, Hogden A, Braithwaite J: The struggle of translating science into action: foundational concepts of implementation science. Journal of evaluation in clinical practice 2018, 24(1):117-126.

30. Searles A, Piper D, Jorm C, Reeves P, Gleeson M, Karnon J, Goodwin N, Lawson K, Iedema R, Gray J: Embedding an economist in regional and rural health services to add value and reduce waste by improving local-level decision-making: protocol for the ‘embedded Economist’program and evaluation. BMC health services research 2021, 21(1):1-13.

31. Jorm C, Iedema R, Piper D, Goodwin N, Searles A: “Slow science” for 21st century healthcare: reinventing health service research that serves fast-paced, high-complexity care organisations. Journal of Health Organization and Management 2021.

32. Stengers I: Another science is possible: A manifesto for slow science: John Wiley & Sons; 2018.

33. Yukhymenko MA, Brown SW, Lawless KA, Brodowinska K, Mullin G: Thematic Analysis of Teacher Instructional Practices and Student Responses in Middle School Classrooms with Problem-Based Learning Environment. Global Education Review 2014, 1(3):93-110.

34. Willmott L, White B, Yates P, Mitchell G, Currow DC, Gerber K, Piper D: Nurses’ knowledge of law at the end of life and implications for practice: A qualitative study. Palliative Medicine 2020, 34(4):524-532.

35. Searles A, Doran C, Attia J, Knight D, Wiggers J, Deeming S, Mattes J, Webb B, Hannan S, Ling R et al: An Approach to Measuring and Encouraging Research Translation and Research Impact. Journal of Health Research Policy and Systems 2016, 14(60).

36. Vindrola-Padros C, Pape T, Utley M, Fulop NJ: The role of embedded research in quality improvement: a narrative review. BMJ quality & safety 2017, 26(1):70-80.

37. Cheetham M, Wiseman A, Khazaeli B, Gibson E, Gray P, Van der Graaf P, Rushmer R: Embedded research: a promising way to create evidence-informed impact in public health? Journal of Public Health 2018, 40(suppl_1):i64-i70.

38. Walley J, Khan MA, Witter S, Haque R, Newell J, Wei X: Embedded health service development and research: why and how to do it (a ten-stage guide). Health research policy and systems 2018, 16(1):67.

39. Duggan JR: Critical friendship and critical orphanship: embedded research of an English local authority initiative. Management in Education 2014, 28(1):12-18.

40. Glegg SM, Hoens A: Role domains of knowledge brokering: a model for the health care setting. Journal of Neurologic Physical Therapy 2016, 40(2):115-123.

41. Heaton J, Day J, Britten N: Collaborative research and the co-production of knowledge for practice: an illustrative case study. Implementation Science 2015, 11(1):1-10.

42. Nagel T: The view from nowhere: oxford university press; 1989.

No competing interests reported.

Download PDF

Editorial decision: Major revision
12 Apr, 2022
Reviews received at journal
22 Feb, 2022
Reviewers agreed at journal
22 Feb, 2022
Reviewers invited by journal
21 Feb, 2022
Editor assigned by journal
12 Feb, 2022
Editor invited by journal
11 Feb, 2022
Submission checks completed at journal
11 Feb, 2022
First submitted to journal
03 Feb, 2022

You are reading this latest preprint version

Relational aspects of building capacity in economic evaluation in an Australian Primary Health Network using an embedded researcher approach

Status:

Version 1

Abstract

Figures

Introduction

Background

Methods

Results

Key outputs

Process facilitators and barriers

Contextual facilitators and barriers

Relational facilitators and barriers

The embedded Economists’ attributes

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1