Exploring factors influencing the consistent adoption of a post-stroke upper extremity outcome measure using Normalisation Process Theory

doi:10.21203/rs.3.rs-4840455/v1

Download PDF

Research Article

Exploring factors influencing the consistent adoption of a post-stroke upper extremity outcome measure using Normalisation Process Theory

https://doi.org/10.21203/rs.3.rs-4840455/v1

This work is licensed under a CC BY 4.0 License

You are reading this latest preprint version

Background

Stroke rehabilitation guidelines recommend using outcome measures like the Fugl-Myer Assessment (FMA) and the Action Research Arm Test for post-stroke upper extremity function assessment. Despite these recommendations, challenges persist in their integration into everyday clinical practice, highlighting the crucial need to comprehend and implement outcome measurement aligned with evolving healthcare models.

Objective

Our study aimed to use a theory-based approach to identify the barriers and facilitators to routine use of the FMA for post-stroke upper extremity function among hospital-based occupational therapists.

Methods

Employing a mixed-method sequential exploratory study design rooted in the Normalisation Process Theory (NPT), we gathered quantitative data through a validated survey followed by qualitative insights analysed with directed content analysis from focus group discussions involving occupational therapists from four hospitals.

Results

The online survey (n = 34) revealed barriers predominantly associated with NPT constructs collective action and coherence. Facilitators were primarily linked to the cognitive participation construct. Focus group discussion revealed key barriers including inadequate coaching, competing tasks and priorities, and the perceived limited value of the FMA by therapists. Key facilitators were legitimation of therapists’ role in outcome measurement and an open learning culture.

Conclusions

Through a theory-based approach, we identified distinct challenges and enablers to routine outcome measurement previously not reported in the literature. Our findings provide insights on designing implementation strategies to embed the use of the FMA into routine practice.

Stroke

outcome measure

upper limb

rehabilitation

implementation science

Stroke rehabilitation clinical practice guidelines worldwide recommend the use of outcome measures to evaluate upper extremity function after stroke.^1,2 Outcome measures serve the dual purpose of establishing the status of upper extremity function and evaluating changes in upper extremity function over time.^3,4 The Fugl-Myer Assessment (FMA) and the Action Research Arm Test are two post-stroke outcome measures that are not only widely used in clinical practice but are also recommended outcome measures for research when studying upper extremity sensorimotor recovery after stroke.^5,6 However, despite these recommendations, integrating post-stroke upper limb outcome measures into routine clinical practice remains a significant and under-researched challenge.

Stroke rehabilitation professionals often endorse the use of and compliance to recommended outcome measures in stroke rehabilitation clinical practice guidelines, yet less than half consistently use outcome measures in their routine daily practice.^7,8 This lack of consistent use is not unexpected, considering the persistent resistance of practice settings to successful and sustained implementation of evidence-based practice change.⁹ Barriers to using outcome measures among rehabilitation professionals exist at individual, managerial, and organisational levels, as summarised in a systematic review by Duncan & Murray (2012).¹⁰ Their review identified four major themes, including knowledge, education, and perceived value of outcome measures; support/priority for outcome measurement; practical considerations; and patient considerations.¹⁰ Some barriers identified included the lack of knowledge about psychometric properties of outcome measures, low perceived value, and time constraints for assessment.¹⁰ While insights from the review shed light on the reasons behind the limited integration of outcome measures in clinical practice, there is a pressing need to delve deeper into the barriers and facilitators to routine outcome measurement to address several knowledge gaps.

Knowledge gaps encompass several key areas. Firstly, there is a lack of understanding regarding the barriers and facilitators influencing the routine use of outcome measures in post-stroke rehabilitation, particularly beyond the early implementation stages. Our implementation experiences, marked by limited success in embedding the routine use of the FMA in clinical practice, suggest that distinct barriers and facilitators exist beyond the early stages of outcome measurement implementation. Only one of the 15 studies included in the Duncan & Murray’s systematic review¹⁰ investigated the barriers and facilitators following the implementation of a range of strategies aimed at increasing physiotherapists’ use of outcome measures.¹¹ Secondly, existing literature often lack a solid theoretical foundation when examining barriers and facilitators. Most studies included in Duncan & Murray’s review¹⁰ did not use a theory to examine the barriers and facilitators to the routine use of outcome measures. Using a theory that can explain behavior change in healthcare professionals is important as it provides a theoretical base to^12,13: (1) understand the factors underlying current patterns of behaviors of rehabilitation professionals in the use of post-stroke upper extremity outcome measures, (2) identify appropriate targets for implementation strategies, and (3) explain and predict the mechanisms of change involved with implementation strategies. Lastly, there is a notable lack of published studies examining the systematic implementation of recommended post-stroke upper extremity outcome measures in clinical practice. Extremely limited evidence exists concerning factors influencing the integration and embedding of outcome measures for evidence-informed, patient-centred upper extremity stroke rehabilitation care.

Addressing knowledge gaps in the integration and embedding of outcome measurement within stroke rehabilitation due to two key healthcare trends. Firstly, the increasing adoption of value-based care model in healthcare systems worldwide places a heightened emphasis on outcome measurement, an essential facet of value-based healthcare.¹⁴ Secondly, the assessment and measurement of outcomes were central themes in the key recommendations for advancing post-acute stroke care systems and the proposed criteria for designating rehabilitation readiness for Comprehensive Stroke Centers.¹⁵ Consequently, integrating and embedding outcome measures in routine clinical practice is necessary to align with evolving paradigms within healthcare systems and comprehensive stroke care.

Aim

Seeking to expand upon the findings from Duncan & Murray’s systematic review and address identified knowledge gaps, our study adopted an implementation science perspective and aimed to identify the barriers and facilitators to routine use of a post-stroke upper extremity outcome measure among occupational therapists (OTs) in Singapore using a theory-driven approach.

Normalisation Process Theory

We used the Normalisation Process Theory (NPT) to guide our study as we wanted to explore how the FMA can be embedded in daily work processes such that it is normalised and accepted as part of routine clinical practice beyond early implementation stages.¹⁶ The NPT considers the normalisation of a set of practices as the product of four social mechanisms: cognitive participation, collective action, collective action, and reflexive monitoring. ¹⁶ Each mechanism is influenced by factors that promote or inhibit routine embedding of a set of practices and are also in a dynamic relationship and interact with the wider local practice context of group processes and conventions, organisational structures, and social norms.¹⁷ The four social mechanisms (i.e. constructs) are each divided further into four components, i.e. 16 components in total.¹⁶ Table 1 describes the NPT constructs and components in relation to the practice of using the FMA.

Table 1

The constructs and components of the NPT in relation to the use of the FMA.
Construct	Component	Description
Coherence		Process of understanding the purpose and benefits of using the FMA
	Differentiation	Actions OTs take to understand how using the FMA differs from other outcome measures used in clinical practice
	Communal specification	Actions OTs do collectively (e.g. a group or a team) to understand the aims, objectives, and benefits of implementing and/or using the FMA
	Individual specification	Actions individual OTs take to understand the efforts needed to implement and use the FMA
	Internalisation	The work OTs do to understand the value, benefits, and importance of using the FMA in their practice
Cognitive participation		The relational work to define and organise the efforts needed to build the routine practice of using the FMA
	Initiation	Actions taken at the individual and/or group level to initiate and drive the use of the FMA among OTs
	Enrolment	The work of engaging OTs to support the routine use of the FMA
	Legitimation	The work that OTs do (at individual and/or group level) to justify their involvement in the routine practice of using the FMA
	Activation	The continuing work to sustain the routine practice of using the FMA
Collective action		The operational work needed to enact the routine practice of using the FMA
	Interactional workability	The interactional work to operationalise the routine practice of using the FMA
	Relational integration	The knowledge work to build OTs’ confidence in administering the FMA and to instill responsibility to use the FMA in their routine clinical practice
	Skill set workability	The work of dividing and allocating tasks and resources according to one’s competency in using the FMA
	Contextual integration	The work of integrating the use of the FMA into existing work processes and contexts. It includes allocation of resources or endorsing procedures to support the use of the FMA
Reflexive monitoring		The work of appraising and evaluating the effectiveness of routinely using the FMA
	Systemisation	The work of collecting and analysing data to determine the usefulness of the FMA and the adherence to routine use of the FMA among OTs
	Communal appraisal	Actions taken at a group level to evaluate the value (usefulness, worth) of the FMA and adherence to routine use of the measure
	Individual appraisal	The work by individual OTs to evaluate the value (usefulness, worth) of using the FMA on their clinical practice and patients
	Reconfiguration	Actions taken after evaluating and appraising the effectiveness of routinely using the FMA
FMA: Fugl-Meyer Assessment of the Upper Extremity; OTs: Occupational therapists

Study design

We employed a mixed-method sequential explanatory study design comprising two phases. In Phase 1, we used an online survey to collect quantitative data, followed by focus groups in Phase 2 to explain and elaborate on findings from the online survey.

Setting

We conducted our study at four public hospitals from the same hospital network in Singapore. Two are acute hospitals that provide hyperacute and acute stroke treatment, and OTs work in the acute stroke units and inpatient wards to provide early rehabilitation for patients with stroke. The other two are community hospitals that provide short-term post-acute care and rehabilitation to facilitate recovery and transition back to daily lives and communities. OTs working in community hospitals provide post-acute stroke rehabilitation services. The FMA had been implemented at all four hospitals as a post-stroke upper extremity outcome measure, with the year of implementation ranging between 2013 and 2021. The measure was operationalized as a routine outcome measure for patients with a newly diagnosed stroke referred for occupational therapy services on admission and discharge to the inpatient rehabilitation programs or units.

Phase 1 (Quantitative phase)

Online Survey

The Normalisation Measure Development (NoMAD) survey is a 23-item survey instrument based on the NPT (Supplementary file 1).¹⁸ The survey was organised into three parts: Part A comprised of demographic questions; Part B included three general questions about FMA use and responses were rated on an 11-point visual analogue scale; and Part C contained a set of detailed questions about the use of the FMA to reflect the 16 components of the NPT. Responses for Part C were rated on a 5-point Likert scale.

Participants

All OTs employed at the four public hospitals were invited to participate in the study via email. The primary inclusion criteria were OTs working with persons with stroke or who had previous clinical rotations where they worked with persons with stroke at the time of the survey. OT trainees who had not obtained their full practicing license were excluded.

Data collection

The customised NoMAD survey was administered via an online platform developed by the Government Technology Agency, Singapore. The survey link was included in the email invitation to all OTs, and we also sent a reminder email a week before the end of our survey. The survey was available for 6 weeks from October to December 2021.

Data analysis

We used descriptive statistics to summarise participant characteristics and survey responses. In Part C of the survey where each item corresponded to an NPT component, we examined the proportion of responses to determine whether the NPT component was a barrier or facilitator. We considered an NPT component as a barrier if the corresponding survey item had less than 50% of participants rating strongly agree or agree; otherwise, the NPT component was considered a facilitator. Data analysis was conducted using Statistical Package for Social Sciences (SPSS) version 6.

Phase 2 (Qualitative phase)

Participants

We recruited participants from those who completed the online survey in Phase 1. At the end of the online survey, we included the question “Are you willing to be contacted for a follow-up focus group discussion?” Survey participants who indicated agreement were contacted for informed consent for our focus groups.

Data collection

Focus groups were held virtually and moderated by two of our authors [SXC, JY]. We used a semi-structured interview protocol and formulated guiding questions based on Phase 1 survey results and the NPT theoretical framework. Questions were designed to prompt participants to explain the reasons behind the survey ratings to gain a more in-depth understanding of the barriers and facilitators to routine use of the FMA. To facilitate an honest discussion between focus group participants and the moderator, we ensured that the moderator who led the discussion did not belong to the same hospital as the participants. All focus groups lasted about 1 hour and were audio-recorded and supplemented with field notes.

Data analysis

Audio recordings were transcribed verbatim for two of the three focus groups conducted. Due to technical errors, audio recordings from the third focus group were not viable for transcription and field notes were used for data analysis instead. We used directed content analysis with an initial coding framework based on the NPT. Analysis was undertaken deductively in a systematic approach, where transcripts were reviewed independently by three authors [SCX, JY, and SA] and first broadly coded with the four NPT constructs, followed by detailed coding using the NPT components of each construct. Data could be coded to more than one NPT construct, and data that did not fit within the NPT were assigned stand-alone codes. The three authors met regularly to discuss findings and agree and verify data interpretations to ensure consistency in our data analysis. Codes and themes were mapped and displayed using NVivo version 11.

Consistent with our study design, analysed data from the survey and focus groups were amalgamated and we report study results as a unified whole framed by the NPT.

Participant characteristics

Survey participants

We received responses from a total of 34 participants, which represented 47.2% of the OTs who fit our inclusion criteria. Participants’ characteristics are summarised in Table 2. Most participants (n = 19, 55.9%) had 1–2 years of working experience at their current organisation. All but one participant was involved in administering the FMA; only one participant held a managerial role who oversaw the use of the FMA by OTs in their hospital.

Table 2

Survey participant characteristics (N = 34).
	n (%)
Site of Practice
Sengkang General Hospital	4 (11.8)
Singapore General Hospital	10 (29.4)
Outram Community Hospital	14 (41.2)
Sengkang Community Hospital	6 (17.6)
Years worked at current hospital
< 1 year	3 (8.8)
1–2 years	19 (55.9)
3–5 years	9 (26.5)
6–10 years	3 (8.8)
Main role in relation to the use of the FMA
Involved in administering the FMA	33 (97.1)
Managerial role	1 (2.9)
FMA: Fugl-Meyer Assessment of the Upper Extremity

Focus group participants

Eight survey participants consented to follow-up focus groups, and we conducted a total of three focus groups. Two focus groups had only two participants due to scheduling conflicts.

Participant characteristics. There were participants from all four hospitals (Table 3), and their working experience at the hospital ranged from 2 to 10 years.

Table 3

Focus group participant characteristics (N = 8).
Participant		Practice setting
1.	Aw	Tertiary acute hospital, stroke unit and inpatient wards
2.	Anna	Tertiary acute hospital, stroke unit and inpatient wards
3.	Amber	Tertiary acute hospital, stroke unit and inpatient wards
4.	Emma	Tertiary acute hospital, stroke unit and inpatient wards
5.	Chloe	Community hospital, inpatient wards
6.	Emily	Community hospital, inpatient wards
7.	Diana	Acute hospital, inpatient wards
8.	Mel	Acute hospital, outpatient clinic
Note: All names are pseudonyms.

Participants’ practice setting. The OT’s practice settings were similar for both community and one acute hospital, which all followed a generalist model of care. These OTs worked with patients with various medical conditions in their daily practice, including treating patients with stroke. OTs practicing at the acute tertiary acute hospital followed a specialist model of care and therapists primarily worked with patients with neurological conditions (including stroke) daily.

The OTs’ practice settings were similar between both community hospitals and an acute hospital which followed a generalist model of care. These OTs worked with patients with various medical conditions in their daily practice, and only worked with patients with stroke on an opportunistic basis (i.e., if a patient with a stroke was admitted to the ward they are working at). For OTs practicing at the tertiary acute hospital, their practice setting followed a specialist model of care and therapists primarily worked with patients with neurological conditions (including stroke) daily.

Use of the FMA

Survey responses to the general questions about the use of the FMA are shown in Table 4. When asked about the familiarity of the FMA, the mean rating was 6 (SD = 2). Similarly, the mean rating was 5 (SD = 3) when participants were asked if they felt the use of the FMA was a normal part of their work.

Table 4

Summary of responses to Part B of the survey (N = 34)
Survey Question	Mean	SD	(Min, Max)
When you use the FMA, how familiar does it feel?^*	5.68	2.31	(0, 10)
Do you feel the FMA is currently a normal part of your work?^†	5.09	2.88	(0, 10)
Do you feel the FMA will become a normal part of your work?	6.53	2.02	(2, 10)
FMA: Fugl-Meyer Assessment of the Upper Extremity; Min: minimum; Max: maximum * Anchors of 11-point Likert Scale: 0 – Still feels very new, 10 – Feels completely familiar ^† Anchors of 11-point Likert Scale: 0 – Not at all, 5 – Somewhat, 10 - Completely

Familiarity with the FMA was related to the OTs’ practice setting. For participants whose practice followed a generalist model of care, their opportunity to work with patients with stroke only occurred if such a patient was admitted to the ward they were working at. Their infrequent use of the FMA was therefore the main reason for the lack of familiarity with the measure. In contrast, for participants whose practice setting followed a specialist model of care, the frequent use of the FMA in their daily practice contributed to their familiarity with the measure.

Focus group discussions also revealed different interpretations of ‘normal part of work’. Some participants associated ‘normal’ with their frequency of using the FMA; because they did not administer the FMA frequently in their daily practice, they felt using the FMA was not a normal part of their work. Other participants associated ‘normal’ with the work processes related to the FMA, rather than the frequency of using the FMA. These participants regarded the use of the FMA as a ‘normal’ part of work and usual care practice as the FMA was operationalised as a standard upper extremity outcome measure for patients with stroke in their clinical setting.

Barriers and facilitators to routine use of the FMA framed by the NPT

Coherence

All four NPT components within the Coherence construct were interpreted as facilitators as more than 50% of survey participants agreed to the relevant survey statements (Fig. 1). However, the proportion of participants agreeing with the statements ranged from 52.9–67.6%. OTs highly valued understanding the purpose and benefits of using the FMA beyond a mandated work process, particularly how the use of the measure could directly influence patient care. As explained by Participant 6, Emily:

I guess the intention has to be clear whenever we roll out such assessments… if we want a good buy-in for therapists to use an assessment tool, the intention to why we bring in such a tool has to be very clear. And how does this [the FMA] actually affect the way we conduct rehab? Or how can we use the tool to motivate patients or to give a prognostication of how the upper limb [function] will return?

A key theme arising from the focus groups was the perceived value of the FMA which acted both as a facilitator and a barrier. Both participants from Focus Group 3 shared that the FMA was used to demonstrate objective improvements in the affected upper extremity to both patients and the multi-disciplinary team. Improvements in FMA scores served as a motivation for patients, while the OTs used score improvements to justify the extension of inpatient rehabilitation length of stay. Furthermore, focus group discussions revealed that OTs who were trained to use FMA scores to guide intervention planning highly valued the measure as its use enhanced their immediate clinical reasoning process. By administering the FMA, these OTs were able to use FMA scores to systematically match patients’ upper extremity function to task difficulty and tailor functional task practice accordingly:

“I think things changed for me when I understood the use of the Fugl-Meyer for intervention planning... it was difficult to integrate it in the past, not because I didn’t feel like I had enough training. I think it’s more that I didn’t understand the purpose of it; it was just part of a coordinated care pathway. But once I understood the importance of the Fugl-Meyer and you can directly implement it to treatment immediately, it was easier to integrate [into daily practice] for me.” (Participant 4, Emma)

“I think really understanding the view of the Fugl-Meyer and how it impacts intervention planning, I think it made a big difference... I learned it purely as a [assessment] tool and then many years later, I learned how it is used to help with intervention planning. And I think that was really helpful to change my entire mindset about the purpose or the use of this assessment.” (Participant 1, Aw)

In contrast, OTs who were not aware of how to interpret FMA scores to guide intervention planning had a low perceived value of the measure, regarded the use of the FMA as only a mandated work process to evaluate patient outcomes and effectiveness of rehabilitation services in the hospital:

“I also think that with the scores that we get from the FMA, it may not actually help guide a therapist in thinking of possible interventions for them... It doesn’t come naturally to them because it’s just a score to them. It’s ‘Oh I’ve been told I need to administer a tool, so I’m just going to administer it for the sake of doing it.’” (Participant 6, Emily)

Consequently, insufficient training, specifically the lack of training on score interpretation, was a recurrent theme raised by several focus group participants that were identified as a barrier to routine use of the FMA in clinical practice.

Cognitive Participation

Most survey participants agreed to the statements linked to the NPT components of the cognitive participation construct, and thus, were interpreted as facilitators (Fig. 2). The statements ‘I believe that participating in the use of the FMA is a legitimate part of my role’ and ‘I’m open to working with colleagues in new ways to use the FMA’ in particular, had 76.4% and 79.4% of participants agreeing to them respectively.

Recognising the use of the FMA as a legitimate part of OT’s role in stroke rehabilitation was an important facilitator. Rehabilitation of the upper extremity is one of the major focuses of occupational therapy practice because upper extremity function is closely related to one’s ability to carry out basic and instrumental activities of daily living. OTs work collaboratively with patients to facilitate and improve control and movements in the upper extremity after stroke to maximize their ability to perform daily tasks. Thus, focus group participants emphasized the need for OTs to assess upper extremity function using outcome measures such as the FMA.

An open learning culture was another theme that emerged as a facilitator from the focus groups. Participants shared various self-directed initiatives they took to familiarize themselves with the FMA, beyond the training and educational resources provided in their departments. These initiatives included making time for extra practice administering the FMA, approaching supervisors or senior colleagues for coaching sessions, and seeking help from peers, seniors, and/or supervisors when in doubt about administering or scoring the FMA. Their self-directed initiatives were reciprocated by their colleagues, which facilitated their learning of the FMA.

Key champions to drive the use of the FMA was another theme identified, acting both a facilitator and a barrier. Key champions facilitated implementation by building support for the new practice of using the FMA and obtaining needed resources to operationalise its use. Conversely, the lack of or limited number of champions was deemed a barrier by focus group participants. They expressed the significance of collective influence and persuasiveness from a group of champions to drive and elicit changes:

“It seems like there is only one senior therapist… trying to drive the use of the FMA... But I think there is insufficient people to push this assessment forward and getting others involved because of the proportion…We need more people to actually advocate that this assessment tool is very good. We need to try it; everyone let’s try it.” (Participant 5, Chloe)

Focus group participants further emphasized the importance of building a ‘team culture’ of using the FMA routinely. Doing so will exert greater influence on colleagues to drive the use of the FMA forward, to embed, and integrate it into clinical practice.

Collective Action

Figure 3 shows survey responses to statements related to collective action construct. NPT components, interactional workability and skill set workability, were interpreted as barriers. Only 44.1% agreed to the statement ‘I can easily integrate the FMA into my existing work’ (Interactional workability), and a mere 32.4% agreed to the statement ‘Sufficient training is provided to enable staff to use the FMA’ (Skill set workability). The statement ‘Using the FMA disrupts working relationships’ was negatively worded, and thus, we regarded the NPT component relational integration, as a facilitator despite 79.4% of participants rating disagree and strongly disagree. The remaining NPT component, contextual integration, was also considered a facilitator albeit the proportions of participants agreeing to the statements were only 50–55.9%.

From the focus group discussions, the lack of equipment, unintuitive resources, and multi-step work procedures were three barriers identified that affected the integration of the FMA into existing work. There is no standardised FMA toolkit as most test items involve only upper extremity movements and the minimal equipment required are inexpensive daily items (e.g. ball, pen, and cup). Although such equipment was typically available in therapy gyms/areas, it was inconvenient for OTs to search and gather these items whenever they administered the FMA. The FMA administration guidelines also provided only descriptions of the upper extremity movements, lacking graphic elements to aid comprehension and learning. Lastly, the FMA was not integrated into the electronic health record, and OTs thus found the multi-step procedures to score, tabulate, and transfer FMA scores from paper to electronic records inefficient and did not fit easily into existing workflows.

Focus group participants provided insights into the fourth barrier – insufficient training. They clarified that while there is already training on the administration of the FMA, there was a dearth of coaching on FMA scoring and how to interpret FMA scores (see Coherence). Participants perceived coaching as OTs administering and scoring the FMA on patients while their senior colleague or supervisor observed the therapy sessions. Thereafter, they clarified scoring discrepancies and discussed strategies to improve work processes and efficiency related to FMA use. Focus group participants shared that their positive coaching experiences helped build their confidence in using the FMA and therefore regarded receiving coaching as a facilitator. However, coaching sessions were disrupted or even stopped during the COVID-19 pandemic due to safe distancing measures imposed within the hospitals. As Participant 6, Emily, explained:

One thing that we didn’t do was to actually have coaching with a patient...Because of COVID restrictions, we were not able to cross wards that easily. Also, we only had two senior occupational therapists who were driving the stroke program. It was difficult to do it [coaching] individually with each patient and staff.

This inadvertently led to new or recently rotated staff not receiving coaching on the FMA, which created a perception of a lack of training and a barrier to using the measure in routine practice. In addition, as coaching was the primary method used to teach OTs how to assign FMA scores to the movements observed in the affected upper extremity, some focus group participants also felt that there was a lack of calibration for the scoring of the FMA.

A facilitator recognised only by participants of the first focus group was how the FMA was used to allocate work to therapy assistants. The systematic approach to match patients’ upper extremity function to task difficulty using FMA scores enabled these OTs to prescribe and progress upper extremity training in a similar manner. This reduced practice variation between OTs, allowed consistent delegation of upper extremity training to therapy assistants, and clearer communication between OTs and therapy assistants of the targeted upper extremity movements.

Reflexive Monitoring

Survey participants agreed with most statements linked to NPT components of the reflexive monitoring construct (Fig. 4). At least half of the survey participants agreed to the statements related to communal appraisal (52.9%) and individual appraisal (58.8%) and these NPT components were interpreted as facilitators. There was no mention in any focus group discussions that formal and/or informal group meetings were conducted to evaluate the usefulness or worth of the practice of using the FMA. However, partaking in our survey and focus group discussions unintentionally provided participants an opportunity to reflect on their own experiences with the use of the FMA. There was positive appraisal at the individual level during the focus groups about how the FMA has been useful in clinical practice, as evident in the themes presented earlier in this section (see Coherence and Collective Action).

Most participants disagreed (58.9%, n = 20) or were neutral (14.7%, n = 5) to the statement “I am aware of the reports about the adherence rate of the use of the FMA” (NPT component systemization). Focus group participants clarified there were no formal audits on the consistency of FMA use at their hospitals, and therefore, there were no such reports on the adherence rates of the use of the FMA. They rationalised that such reports might be useful as it can be used as a basis to start conversations with OTs about the challenges and facilitators in using the FMA in routine practice.

In this age of healthcare transformation, integrating outcome measures seamlessly into routine clinical practice is crucial for effectively monitoring patient progress, planning interventions, and demonstrating the value and efficiency of healthcare services. In this context, our study set out to investigate the barriers and facilitators to the routine use of post-stroke upper extremity outcome measures, using the FMA as an exemplar, among OTs in Singapore. Our study yielded novel insights, shedding light on previously unreported facilitators and barriers within rehabilitation literature, extending beyond post-stroke upper extremity rehabilitation and stroke rehabilitation domains. These include the presence of key champions, an open learning culture, lack of audit/appraisal, insufficient training, and practical considerations. We also found factors consistent with existing evidence – perceived value of outcome measurement and legitimation.¹⁰ Notably, a bi-directional relationship exists regarding OTs’ perceived value of the outcome measure, where a higher perceived value corresponds to a greater likelihood of outcome measure utilisation, and conversely, a lower perceived value reduces its likelihood of use. OTs also acknowledge their legitimate role in upper limb rehabilitation and the utilisation of outcome measures.

Our study adopted a theory-led approach, enabling a more comprehensive understanding of the process involved in embedding and integrating the post-stroke UE outcome measures into routine clinical practice. The analysis of quantitative survey data highlighted facilitators encompassing all four NPT constructs, aligning with the theoretical premise that all four mechanisms are necessary for the seamless integration of outcome measurement into routine practice. Additionally, insights gleaned from focus group discussions provided a nuanced perspective on specific NPT components in relation to the barriers and facilitators to routine outcome measurement.

Facilitators and barriers to routine outcome measurement previously not reported

Presence of key champions

Our study revealed the pivotal role of influential advocates, often referred to as champions, in promoting the use of the FMA. This factor aligns with the construct of cognitive participation within the NPT, which encompasses collaborative efforts undertaken to define and organise the efforts needed to build the routine practice of using the FMA. Key champions identified in our study were OTs regarded by their peers and colleagues for their expertise in stroke rehabilitation and proficiency in using the FMA. They were also informal/formal leaders who had a degree of control over the contextual aspects of their practice. This authority enabled them to enact essential changes to accommodate the new practice of FMA and select and adapt implementation strategies as they were well-attuned to the challenges associated with FMA use in their practice setting. The champions’ clinical competence and trustworthiness, together with their strategic leadership positions, therefore empowered them to exert substantial influence to promote and facilitate the adoption of FMA use in clinical practice.

The presence of key champions has not been identified as a facilitator for outcome measurement implementation, however, prior evidence from a systematic review underscores the efficacy of clinical champions in increasing the use of innovations across healthcare settings.¹⁹ The use of champions is a also widely utilised implementation strategy for overcoming barriers and enhancing implementation outcomes in the use of innovations.¹⁹ Champions are individuals who volunteer or are appointed to promote and drive the adoption of an implementation initiative.^19,20 They are committed to advocating for and leading implementation efforts, including addressing resistance at the organisational level, and leveraging their intrinsic motivation for change to inspire others through their influential positions.²¹

Open learning culture

Another facilitator identified was associated with the NPT component enrolment, which involves the processes of engaging OTs to support the routine use of the FMA, was an open learning culture. We found that an open learning culture, which encourages self-directed learning and fosters the sharing of experiences, was integral in promoting the routine use of the FMA. OTs not only gained proficiency in using the FMA through structured learning processes but also through informal self-directed and peer-based learning. The willingness of peers, colleagues, supervisors, and managers to support less-experienced staff when administering the FMA contributed significantly to building confidence and expertise in its use. It is crucial to acknowledge that while continuous individual learning is essential, that alone may not be adequate to influence knowledge performance.²² A collective learning experience is equally indispensable for empowering others to gain confidence and mastery in the use of outcome measures.²³ To foster this collective learning experience, a scoping review highlights the need to cultivate and integrate a conductive climate or culture into organisational practices and structures.²² This integration will transform learning into an interactive and interdependent process that will promote group learning and ultimately facilitate organisational improvement.²⁴

Lack of audit/appraisal

One barrier hindering the routine use of the FMA was the absence of audit and appraisal processes. Audit and feedback, a strategy involving the compilation and dissemination of data regarding specific aspects of clinical practice to encourage practice improvement, is commonly and increasingly employed strategy in various clinical settings.²⁵ The formal audits gauge the effectiveness of FMA implementation activities. However, no such audits had been conducted at most of the hospitals, leaving OTs unaware of the success or failure of their efforts to integrate the FMA into routine practice, as well as the lack of means to monitor the adherence of FMA use. There was also an absence of formal group-level appraisals to assess the value (usefulness and worth) of the FMA and adherence to its routine use. The lack of allocated time to discuss the challenges and successes of using the FMA hindered the identification of any emerging obstacles and adaptations required to sustain the ‘new’ practice over time. Audit and appraisal processes are vital because, according to the theory of alignment, this evaluation process is a precursor to aligning outcome measurement with the practice context.²⁶ Therapists first need to ‘try on’ an outcome measure and evaluate its compatibility with their context and assess its suitability.²⁶ Subsequently, therapists and their departments can make necessary adjustments to implementation activities to align outcome measurement with their practice context, thereby achieving a better fit between the two.²⁶ This proactive, real-time approach not only ensures the ongoing use of outcome measures in routine practice but also serves to ‘normalise’ the ‘new’ practice of outcome measurement.

Insufficient training

Another barrier identified in our study pertained to insufficient training, which was an unexpected finding given that education and training for administering the FMA were already in place across all four hospitals. The training deficiencies identified by OTs were primarily related to the interpretation of FMA scores, which indirectly influenced their perceived value of the FMA. Specifically, OTs assessed the value of this outcome measure at an individual professional level, particularly in terms of how FMA scores could inform immediate clinical decision-making and direct patient care. This aligns with the NPT construct coherence, highlighting the importance of OTs comprehending the purpose and advantages of using the FMA.

Our finding suggests that training needs are dynamic and evolve at various stages of implementation. In the initial phases of implementation, the focus lies on establishing competency, ensuring that therapists can proficiently administer and score the outcome measure. However, as we progress towards sustaining the practice of using outcome measures, the training needs shift. During the sustainment phase, the emphasis should be on elucidating the benefits of utilising the measure, encompassing score interpretation, linking scores to immediate clinical reasoning processes, and aiding in intervention planning.

Practical considerations

OTs brought to light practical challenges that hindered the integration of the FMA into routine practice, primarily revolving around work processes. These challenges encompassed the absence of seamless integration with electronic medical records and the involvement of multi-step work processes. While practical obstacles have been previously reported, previous factors were predominantly related to the outcome measure itself (e.g., the number of test items, complexity of instructions, and cost) and time constraints.¹⁰

In parallel with training needs, our findings suggest the practical considerations required to support the utilisation of outcome measures also undergo changes at different stages of implementation and evolve over time. During the initial stages of implementation where an outcome measure is first introduced into practice, work processes are nascent and relatively unrefined in terms of incorporating the measure into existing processes. However, to sustain the use of the outcome measure, it becomes imperative to ensure that these work processes undergo periodic review and refinement to improve efficiency and integration with existing processes. Additionally, these work processes must remain adaptable to changes in the healthcare system, such as the introduction of electronic medical records or changes to the platform.

Implications for practice

Based on our study findings, we propose the following recommendations when embedding and integrating post-stroke UE outcome measures into routine clinical practice:

Identify key champions: Recognise and engage influential individuals who are informal/formal leaders to drive change at various levels, facilitating the adoption of outcome measures and coordinating implementation efforts.

Cultivate an open learning culture. Foster a culture that supports self-directed learning at all levels and encourages peer-based learning. Maintain in-person coaching for skill acquisition and ensure uninterrupted learning even during challenging situations like the safe-distancing measures during the COVID-19 pandemic.

Conduct regular audits and appraisals. Implement routine audits to assess adherence to outcome measurement and share audit reports with frontline staff. Encourage open discussions to address challenges and successes in routine measure use. Allocate time for periodic reviews of work processes and to identify new barriers and facilitators (if any).

Simplify work processes. Review existing work processes and aim to continuously simplify them. Ensure resources related to outcome measurement are user-friendly to reduce the cognitive load on therapists.

Align the focus of training to implementation stage. After initial implementation stages, training should be focused on improving therapists' understanding of the rationale and advantages of utilising outcome measures, to continuously increase their perceived value of the measure. Customised training could encompass the practical utilisation of measure scores, such as incorporating scores into their clinical reasoning processes, intervention planning, and directing patient care. Also, ensure adaptability in training approaches, allowing for adjustments in response to dynamic healthcare circumstances (e.g. COVID-19 pandemic and associated safety protocols) to ensure uninterrupted training.

Study limitations

Our study was the first of its kind to examine the barriers and facilitators to outcome measurement beyond the initial implementation stages and focused on various inpatient settings within hospitals. Future research should extend this examination to community settings (e.g. day rehabilitation centres) where clinical practices may differ, thereby uncovering new barriers and facilitators. To encourage survey participation and honest responses, we ensured a level of respondent anonymity for our study; only OTs who were keen to participate in follow-up focus group discussions identified themselves. Thus, our focus group participants may already have a vested interest and commitment to outcome measurement, as well as a high perceived value of outcome measures. Maximal variation sampling could have offered a more comprehensive insight into the barriers and facilitators, however, de-anonymising the survey might introduce social desirability bias in the responses. Lastly, while adopting a theory-driven approach offers several key advantages, our qualitative analysis involved a priori coding, potentially imposing predefined categories onto the data that might have differed if solely emerging from the data itself.

While there exists compelling evidence for the use of upper extremity outcome measures in stroke rehabilitation, there is limited success in the translation of this evidence to practice. Embedding and integrating outcome measurement in routine clinical practice continues to be a challenge. Our study demonstrates the use of a theory-driven approach to identify the barriers and facilitators to routine administration of an upper extremity outcome measure among OTs in Singapore. In doing so, we were able to fully explore and understand how individual, system, and environmental influences may affect the uptake, implementation, and full integration of the practice of using the measure beyond initial implementation stages. Findings from our study provide a stronger evidence-based selection and tailoring of implementation strategies, which may improve the adherence rates and achieve greater success in embedding the routine administration of the FMA in clinical practice.

FMA

Fugl–Meyer Assessment

NPT

Normalisation Process Theory

NoMAD

Normalisation Measure Development

OTs

Occupational therapists

Standard deviation

Ethics approval and consent to participate

Ethics approval was obtained from the SingHealth Centralised Institutional Review Board (CIRB Ref: 2021/2559). Prior to commencing the online survey, the webpage provided participants with brief study information and a statement detailing that the completion of the survey indicated their consent to participate in our study. Written informed consent was obtained for all focus group participants.

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Competing interests

The authors declare that there is no conflict of interest.

Funding

This work was supported by the SingHealth Duke-NUS Academic Medicine Research Grant (Special Category: Health Service Research) [Grant number: AM\HRT016/2021].

Authors’ contributions

SXC reviewed the literature and conceptualised the study, involved in data collection and analysis, drafted, and critically revised the manuscript. JY and SAMR contributed to data collection and analysis. CJL and JBT contributed to the data collection. EL and JT contributed to the study conception and data analysis. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Acknowledgements

We extend our heartfelt gratitude to all the participants who generously dedicated their time and shared their valuable insights for this study.

Winstein Carolee J, Joel S, Ross A, et al. Guidelines for adult stroke rehabilitation and recovery. Stroke. 2016;47:e98–169.
Heart and Stroke. 5.1 Management of the Upper Extremity Following Stroke. Canadian Stroke Best Practices, https://www.strokebestpractices.ca/en/recommendations/stroke-rehabilitation/management-of-the-upper-extremity-following-stroke/ (accessed 28 August 2023).
Santisteban L, Térémetz M, Bleton J-P et al. Upper limb outcome measures used in stroke rehabilitation studies: a systematic literature review. PLoS ONE; 11. Epub ahead of print 6 May 2016. 10.1371/journal.pone.0154792
Beaton DE, Bombardier C, Katz JN, et al. A taxonomy for responsiveness. J Clin Epidemiol. 2001;54:1204–17.
Murphy MA, Resteghini C, Feys P, et al. An overview of systematic reviews on upper extremity outcome measures after stroke. BMC Neurol. 2015;15:292.
Kwakkel G, Lannin NA, Borschmann K, et al. Standardized measurement of sensorimotor recovery in stroke trials: Consensus-based core recommendations from the Stroke Recovery and Rehabilitation Roundtable. Int J Stroke Off J Int Stroke Soc. 2017;12:451–61.
Van Peppen RPS, Maissan FJF, Van Genderen FR, et al. Outcome measures in physiotherapy management of patients with stroke: a survey into self-reported use, and barriers to and facilitators for use. Physiother Res Int J Res Clin Phys Ther. 2008;13:255–70.
Burton L-J, Tyson S, McGovern A. Staff perceptions of using outcome measures in stroke rehabilitation. Disabil Rehabil. 2013;35:828–34.
Dogherty EJ, Estabrooks, Carole A. Why do barriers and facilitators matter? In: Richards DA, Hallberg IR, editors. Complex Interventions in Health: An overview of research methods. Routledge; 2015. pp. 273–81.
Duncan EA, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Serv Res. 2012;12:96.
Abrams D, Davidson M, Harrick J, et al. Monitoring the change: current trends in outcome measure usage in physiotherapy. Man Ther. 2006;11:46–53.
Eccles M, Grimshaw J, Walker A, et al. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005;58:107–12.
Portela M, Pronovost P, Woodcock T et al. June. How to study improvement interventions: a brief overview of possible study types. BMJ Qual Saf Published Online First, 10.1136/bmjqs-2014- 003620 (2015, accessed 1 2016).
Noël F. Accelerating the pace of value-based transformation for more resilient and sustainable healthcare. Future Healthc J. 2022;9:226–9.
Duncan PW, Bushnell C, Sissine M, et al. Comprehensive stroke care and outcomes. Stroke. 2021;52:385–93.
May C, Finch T. Implementing, embedding, and integrating practices: an outline of Normalization Process Theory. Sociology. 2009;43:535–54.
Finch TL, Mair FS, O’Donnell C, et al. From theory to ‘measurement’ in complex interventions: methodological lessons from the development of an e-health normalisation instrument. BMC Med Res Methodol. 2012;12:69.
Finch TL, Girling M, May CR, et al. Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT). BMC Med Res Methodol. 2018;18:135.
Santos WJ, Graham ID, Lalonde M, et al. The effectiveness of champions in implementing innovations in health care: a systematic review. Implement Sci Commun. 2022;3:80.
Titler MG, Everett LQ. Translating research into practice. Considerations for critical care investigators. Crit Care Nurs Clin North Am. 2001;13:587–604.
Morena AL, Gaias LM, Larkin C. Understanding the role of clinical champions and their impact on clinician behavior change: The need for causal pathway mechanisms. Front Health Serv. 2022;2:896885.
Verbeek FHO, van Lierop MEA, Meijers JMM, et al. Facilitators for developing an interprofessional learning culture in nursing homes: a scoping review. BMC Health Serv Res. 2023;23:178.
Singer SJ, Benzer JK, Hamdan SU. Improving health care quality and safety: the role of collective learning. J Healthc Leadersh. 2015;7:91–107.
Marsick VJ, Watkins KE. Demonstrating the value of an organization’s learning culture: The dimensions of the Learning Organization Questionnaire. Adv Dev Hum Resour. 2003;5:132–51.
Colquhoun HL, Lamontagne M-E, Duncan EA, et al. A systematic review of interventions to increase the use of standardized outcome measures by rehabilitation professionals. Clin Rehabil. 2017;31:299–309.
Skeat J, Perry A. Exploring the implementation and use of outcome measurement in practice: a qualitative study. Int J Lang Commun Disord. 2008;43:110–25.

No competing interests reported.

Appendix.docx

Download PDF

Editorial decision: Revision requested
05 Aug, 2024
Editor assigned by journal
02 Aug, 2024
Submission checks completed at journal
02 Aug, 2024
First submitted to journal
01 Aug, 2024

You are reading this latest preprint version

Exploring factors influencing the consistent adoption of a post-stroke upper extremity outcome measure using Normalisation Process Theory

Status:

Version 1

Abstract

Background

Objective

Methods

Results

Conclusions

Figures

Background

Aim

Methods

Normalisation Process Theory

Study design

Setting

Phase 1 (Quantitative phase)

Online Survey

Participants

Data collection

Data analysis

Phase 2 (Qualitative phase)

Participants

Data collection

Data analysis

Results

Participant characteristics

Survey participants

Focus group participants

Use of the FMA

Barriers and facilitators to routine use of the FMA framed by the NPT

Coherence

Cognitive Participation

Collective Action

Reflexive Monitoring

Discussion

Facilitators and barriers to routine outcome measurement previously not reported

Presence of key champions

Open learning culture

Lack of audit/appraisal

Insufficient training

Practical considerations

Implications for practice

Study limitations

Conclusions

Abbreviations

Declarations

References

Additional Declarations

Supplementary Files

Status:

Version 1