Developing feedback literacy in physiotherapy students: a pilot study

doi:10.21203/rs.3.rs-1618114/v1

Background: The ability to meaningfully use feedback is an important life skill especially in the healthcare industry. Traditionally, feedback has been viewed as rather one-way information. A new feedback literacy paradigm stresses the importance of student agency and reconceptualising feedback as a sense-making process. However, research on how to improve feedback literacy is still in the development stage.

Methods: This mixed method pilot study explored the effectiveness of a novel feedback literacy workshop for physiotherapy students. Participants’ level of feedback literacy was measured before and after the workshop using a customised questionnaire. Participants also shared in a focus group their perceptions and experiences of feedback before the workshop, as well as anticipated changes after the workshop.

Results: Thematic analyses from the focus group illustrated mixed reactions to the workshop. While there were mismatches of expectations about the workshop, participants anticipated positive change resulting from it. They also demonstrated feedback seeking behaviour after the workshop, a feature that the new paradigm aims to promote. However, owing to the small and possibly biased sample, no statistically significant difference was found between the pre-test and post-test feedback literacy score in the Wilcoxon signed rank test (P>0.05).

Conclusions: There is potential for a curriculum-embedded and structured feedback literacy training to improve feedback literacy in physiotherapy students. The pilot results provide insights on workshop designs and implementation strategies for a future larger scale study.

Feedback

Feedback literacy

Physiotherapy education

Feedback in the healthcare industry

Feedback is not a new concept. Every person encounters some forms of feedback every day, be it at home, in school or at work. Most people define feedback as information or comments they receive on their performance (1). Most feedback givers expect feedback receivers to use the comments and change their actions as a result. For example in school, teachers expect students to interpret the feedback and improve their subsequent submission or performance in a similar assessment (2). At work, employers expect their employees to reflect on appraisal feedback for better job performance in the subsequent year.

Feedback is especially crucial in the healthcare field as the quality of work directly influences clients’ wellbeing. In real world healthcare settings, healthcare professionals often need to give/receive feedback on different aspects of their work on a daily basis to/from their clients and their families on the effectiveness of the healthcare procedure; from their peers in the context of interprofessional practice; and from administrative staff about the effectiveness and efficiency of overall workflow. All these are highly valued in healthcare settings.

Given the high demand for feedback in the healthcare industry, various studies have addressed this important topic in healthcare education programmes. Previous studies explored how feedback practice could be improved by various means. One study shared the experience of nursing students’ use of a tangible tool called the Daily Feedback Tool to better seek feedback from their clinical educators (3). The tool improved communication between them; however it is unclear if students’ understanding of the feedback process developed through the use of this tool.

Sargeant et al (4) reported the use of the R2C2 model (model that builds relationship, explores reactions and content, and coaches for performance change) in fostering coaching and feedback use during medical student residency. The model included four phases: building relationships, exploring reaction to feedback, exploring understanding of feedback content and coaching for performance change. Their pilot study result was positive in that the engagement of residents’ feedback discussion and overall performance were enhanced. This is promising; however the model appears to only occur during residency where some students may not have enough time to build up their understanding of the feedback process.

O’Malley et al (5) explored the effectiveness and efficiency of the 5 minute feedback form (5MFF) used in physiotherapy clinical placement settings in Ireland. The 5MFF provides a feedback structure that requires contribution from both students and clinical educators in order to help standardise feedback practice and improve frequency of feedback between different actors. This cross-sectional survey study showed that the 5MFF enhanced the timing, content and exchange of feedback between physiotherapy students, practice educators and tutors. However, it is unclear if training was provided on how to use the 5MFF prior to its implementation. Similar to other previously mentioned studies, this research appeared to focus on whether a tool helped with the exchange of information, but not on the improvement of understanding of feedback or feedback practice as a whole.

The reconceptualisation of feedback as a sense-making process

Previous literature seems to focus on the traditional paradigm of how and what feedback is given to learners (6). Yet, even if students receive feedback from their teachers and supervisors, they may not know how to interpret it, let alone use it meaningfully to the level teachers or feedback givers expect (2). Why is it so? This is likely related to the dominant and traditional view on feedback as information. The old paradigm of feedback practice focuses primarily on how information is delivered but fails to explore how it should be used (7). Teachers and supervisors are assumed to take the primary role of actively providing feedback, while students are assumed to automatically use it if it is given in an optimal way. Sadly, their uptake of feedback is often unsatisfactory in reality (7).

Recently, it is argued that the process of using feedback information is more important than what and how information is delivered. It appears that feedback without further engagement or uptake is only viewed by students as non-essential information. As long as they are expected to passively receive feedback without the need to actively engage in the process, feedback continues to be information only. This perhaps explains why feedback has been one of the areas of most dissatisfaction in higher education (8, 9) – students simply do not find feedback useful.

As such, a new paradigm that builds on Sutton’s suggestion (10) of conceptualising feedback as a sense-making process is proposed. The new framework (7) suggests four features of student feedback literacy in order to shift the dominant discourse of feedback as information to feedback as a meaningful process. In the new framework, students need to be able to appreciate what feedback is and why it is important (Appreciating Feedback), be able to evaluate the quality of work and feedback (Making Judgements), and be able to manage affect associated with receiving feedback (Managing Affect). Students need to develop these three interrelated attributes before they are able to take practical actions to use or uptake feedback (Taking Actions). The details of these four pillars are described below:

Appreciating Feedback

To appreciate feedback, students need to recognize their active role in the feedback process, and the value of feedback. Instead of only passively receiving feedback, feedback literate students recognise and actively take responsibility to acquire academic knowledge for better feedback uptake as learners. Students also need to appreciate that feedback can have different forms (e.g., text, audio, video) and can come from sources other than teachers.

Making Judgements

Making judgements refers to students’ ability to accurately judge the quality of their own work or others’. Authentic practice is one important way to improve students’ judgement on feedback. For example, when doing assignments, students should produce internal feedback primarily by self-evaluation instead of relying on teachers’ feedback. In addition to internal feedback, peer feedback is also an opportunity to practise judgement. Different students might have different interpretations of the criteria for an assignment, thus producing entirely different pieces of work. By actively participating in the peer feedback process and comparing different pieces of work, students can improve their judgement through exchange of views.

Managing Affect

Managing affect refers to students’ capacity to constructively manage their emotion and reaction, in particular defensiveness, when facing critical feedback. The capacity to manage affect allows students to elicit constructive dialogue with peers and teachers for continuous improvements. Developing resilience to accept both positive and negative feedback is crucial.

Taking Actions

Taking actions refers to students’ initiative to make sense of feedback received and to apply it in their later work. Students need to have agency and responsibility for their own learning; and this student agency can translate into taking actions to apply or make use of feedback. Such actions include actively clarifying confusions with others in the feedback process, refining internal interpretation of feedback, planning on executable strategies for feedback application, and executing them in actuality. This element in the new feedback paradigm is the most critical and differentiates the definition of feedback from the old and new paradigm, and it is what makes the new paradigm student-centred.

Training on healthcare students’ feedback literacy

Since the introduction of this new feedback literacy paradigm, limited research has been conducted to study the effectiveness of feedback training based on it. Winstone, Mathlin and Nash (11) studied the perceived usefulness and effectiveness of the Developing Engagement with Feedback Toolkit (DEFT) they developed and implemented in an undergraduate Psychology programme. The DEFT included feedback resources (e.g. feedback glossary), workshops (e.g. reflection on previous feedback) and portfolios (e.g. feedback synthesis in one place). It was co-created with students and the authors found favourable perceptions about it. There was also preliminary data provided to support the effects of DEFT in improving feedback literacy in psychology students.

Noble et al (12) also conducted a qualitative study to review a feedback literacy programme aiming to enhance healthcare students’ feedback literacy. One hundred and five Australian healthcare students from physiotherapy, pharmacy, social work, medicine, and nursing attended a feedback literacy programme prior to their clinical placements. The programme consisted of an online primer, a real-world workshop, and requirement on continuous feedback logs from students during placements. After placements, 27 students were interviewed to investigate their feedback experience during placements. Qualitative findings revealed that improvement of healthcare students’ feedback literacy through training was possible. Yet, traditional feedback culture and habits, such as relying on feedback from experienced practitioners especially in a complex healthcare environment, place challenges on students’ feedback experience.

O’Connor and McCurtin (13) studied the effects of using an online platform for healthcare students to upload their received feedback and develop action plans on it. They found that the engagement rate of such platform was poor and they then moved on to study a feedback literacy module co-created by various stakeholders. While the experiences with this co-created module were generally positive, there were practical concerns raised by participants regarding the implementation of this module. These practical concerns include workload, knowledge of trainers and where the module within the curriculum should be located to achieve maximal learning outcomes. The findings of this study provide thought-provoking insights on future development of feedback literacy training.

Gaps in the literature and aims of the study

Current research on feedback as a sense-making process centres on conceptualisations and possible theories underpinning them. There are few studies in the healthcare field that investigated how this new concept can be operationalised. Given the crucial role of feedback (as a sense-making process) in the healthcare field, this pilot study aimed to design a novel Feedback Literacy Workshop for physiotherapy students and to test its effectiveness on developing their feedback literacy. The workshop was designed according to the four features in the new feedback paradigm. Training on soft skills such as understanding and using feedback is new, especially in healthcare education including physiotherapy. Thus, results of this pilot study will contribute to appreciating if and how feedback literacy can be improved using workshop-based training. Another contribution is that the findings lay the foundation of the content and delivery method to study future feedback literacy training that extends beyond the physiotherapy discipline to other health professions or other applied disciplines, and perhaps to teachers’ training at large. After all, empirical studies with larger sample sizes are needed to explore the effectiveness of such training.

Study design, sampling and data collection

This pilot mixed methods study employed a single group pre-test/post-test design to study the effectiveness of a novel feedback literacy workshop for physiotherapy students. Effectiveness is defined as improvement in the feedback literacy, as measured by a customised 18-item survey before and after the workshop. In addition, perceptions and experiences of the workshop collected through the focus group serve to provide in-depth information on how it impacted participants’ feedback literacy. The study was funded by the Katie Shu Sui Pui Charitable Trust of the university. Ethical clearance was approved by the Research Ethics Committee of the university (Reference No.: HE-KS2021/01) prior to the start of the project.

The original plan was to recruit 10 participants from the Year-1 cohort of the Bachelor of Science with Honours in Physiotherapy programme to join this study as 10 is deemed an appropriate subject number for a pilot study (14). However, since participation was voluntary, only five participants (all male) joined the study after two rounds of email and personal recruitment. We are aware that this is not the optimal sample size even for a pilot study; however, findings could still provide insights for a future larger scale study. In addition, since this study employed a mixed methods approach to collect both quantitative and qualitative data, triangulation of data was included and this factor enhanced the credibility of results. Informed consent was sought to ensure that participants joined the research voluntarily. Each participant who successfully completed the workshop received a HKD$300 supermarket coupon as a token of appreciation.

Since no validated questionnaires were available at the time of this study, a customised questionnaire with 18 items was developed with reference to Carless and Boud’s student feedback literacy framework (7) and the 14-item survey that Winstone, Mathlin and Nash (11) created for their research project. After creating the questionnaire for this study, the authors invited an expert in this research area to provide opinions on the appropriateness of this customised questionnaire, who deemed the survey items legitimate. The items focus mainly on students’ level of appreciation towards feedback, ability to make judgement on feedback, capacity to manage affect when facing feedback, and proactiveness in taking action according to feedback. The survey items are listed in Table 1. Participants rated their level of agreement of each item, before and after the workshop, on a five-point Likert Scale where 1 means strongly disagree and 5 indicates strongly agree. The maximum score on the questionnaire was 90, with higher scores indicating a higher level of feedback literacy.

The focus group was conducted in Cantonese immediately after participants attended the workshop. It lasted for about 60 minutes and was audio recorded for transcription. The focus group allowed the exploration of in-depth meanings of participants’ experiences in this workshop. Since they were from the same cohort and shared similar experiences (i.e. the workshop, the courses taken in the term), it was deemed appropriate to facilitate an interactive group dynamic and generate fruitful discussion that may otherwise have been impossible (15). The research used cued recall, aided recall and non-sensitive questions to facilitate an open and accurate discussion, and to ensure credibility of the discussion process (16).

Table 1

Student Feedback Literacy Questionnaire with Pre-test/Post-test Mean Scores

Component	Items	Pre-test mean score	Post-test mean score
Appreciating Feedback	I understand the role of feedback in improving work	4.25	4.6
	I appreciate the role of feedback in improving work	4.75	4.4
	I do not need to seek feedback proactively in the feedback process*	4	3.4
	Feedback is as important for student's learning as other forms of learning activity	4.75	4.6
	Feedback should develop learning beyond just the next piece of work	4.75	4.2
	I understand that feedback information comes in different modes and from different individuals	5	4.8
Making Judgement	I do not understand the role of standards and criteria in judging the work of myself*	3.5	2.8
	I self-assess and reflect on my own work	4.5	3.8
	We give feedback to each other with my friends	3.25	4
	I discern feedback information that is constructive versus non-productive	3.5	3.4
Managing Affect	I am open to receiving comments from others without displaying defensiveness	3.75	3.6
	I am proactive in asking for comments	3.5	4.2
	I continue feedback dialogue with people as needed	4	4.4
	I often think of ways to improve based on feedback	3.5	3.8
Taking Action	I analyse and record feedback information in appropriate forms so I can easily retrieve and act on it	2.75	2.2
	I collate feedback information from different sources	3	4
	I respond to feedback information by planning how it might be useful in future work	3.75	4
	I seek out examples as a way to make sense of standards of work	4.75	4.6

*Items 3 and 7 were reversed-coded questions

Intervention – Feedback literacy workshop

All five participants attended a four-hour activity-based workshop designed according to the new paradigm of feedback literacy. The workshop spanned a period of four weeks with each session lasting for one hour. The focuses of the four sessions were aligned with the four components of feedback literacy: appreciating feedback, making judgement, managing affect and taking actions. Table 2 shows the contents and activities of each session. The workshop used an activity-based approach. Activity-based Learning (ABL) is known as a student-centred teaching and learning model, which utilises group activities to facilitate students’ discussion and problem-solving process (17). It stimulates higher-order thinking skills, such as analysis, synthesis and evaluation, through extensive use of discussion and dialogue embedded in activities (18). ABL helps students shape their own learning experience, as learning under the student-centred model happens through students’ active engagement in activities and open communication with teachers and other students. Unlike the traditional one-way teaching, ABL emphasises mutual communication and interaction within the classroom (19). The nature of ABL resonates with the notion of feedback as a two-way process instead of just a one-way telling. Students are encouraged to actively come up with their own perspectives and solutions through participation in activities with the teacher’s guidance. Therefore, this pilot study aspires to apply the ABL principle in the design of the student workshop. Activities and learning materials are utilised in the workshop, and students are prompted to engage in discussion with the small group. Table 2 contains sample activities conducted in the workshop.

Table 2

Content summary of the Workshop

Content	Duration
Appreciating Feedback Introduction to the concept of feedback, feedback literacy and the new paradigm Activity: the Feedback Figure game that demonstrates the effectiveness of the two-way feedback process Foundational elements to an effective feedback process Activity: reflective worksheet – sharing self-evaluation as a feedback giver and a receiver	1-hour session
Making Judgement Types of feedback: appreciative; coaching; evaluative The growth mindset Criteria of helpful feedback Objectivity; specificity; constructiveness Common feedback in physiotherapy settings Communication skills Seek feedback with a specific context Check one’s understanding Activity: Judging and clarifying feedback Activity: Judging the quality of work	1-hour session
Managing Affect Psychoeducation: negativity when facing feedback Possible stress reaction when considering feedback as threats Psychological reactions when facing threats Cognitive distortions that create negative emotions Activity: Cognitive distortions matching game Solution: cognitive-behavioural strategies Strategies for feedback givers and receivers to calm emotions Steps of cognitive reframing Activity: recalling a negative feedback experience Solution: mindfulness Introduction to mindfulness and its possibility to turn an automatic reaction to feedback into mindful response Activity: short mindfulness practice (e.g. breathing exercise, grounding exercise) Quick mindfulness technique: breathing space Mindfulness resources in Hong Kong	1-hour session
Taking Action Tips for forming and executing plans Introduction to feedback logging for recording progress Activity: form an actionable plan with customised templates Psychological barriers of feedback utilisation Enabler of progress: self-acceptance	1-hour session

Including activities for managing affect was perhaps one of the unique features in this workshop because handling emotion in the feedback process is an important yet often overlooked component. This workshop introduced mindfulness as a strategy for managing affect when facing feedback. This session included a mindfulness practice using an app (20). Mindfulness is a state of non-judgmental awareness of the present moment’s experience (21). Mindfulness practices aim to enhance an individual's level of mindfulness and might reduce emotional stress at work, and increase job satisfaction (22).

In recent years, researchers have been investigating the effectiveness of mindfulness-based training and techniques’ on healthcare professionals’ wellbeing. Such interventions were found effective in reducing healthcare professionals’ psychological distress, stress, depression and anxiety. It is also found to be beneficial in enhancing healthcare professionals’ well-being and lessening burnout (23). It is anticipated that mindfulness techniques could be suitable strategies for students to utilise when they want to manage affect stimulated by the feedback process. Therefore, short mindfulness practices were incorporated in the managing affect session. In addition, free online local mindfulness resources were introduced to participants, in the hope that participants would utilise these resources when needs arise.

Data Analysis

The Wilcoxon signed rank test was used to evaluate the differences in survey score pre- and post-workshop because of the small sample size and also because the data were not normally distributed. The significant level was set at 0.05. The Statistical Package for the Social Sciences (SPSS) version 26 (IBM Corp, Armonk, NY) was used to conduct the Wilcoxon-signed rank test and the basic demographics of participants.

Qualitative data from the focus group was used to explore the perceptions and experiences of participants’ usual feedback practice as well as of the workshop. Focus group audio data was transcribed into traditional Chinese and was read by the first and second authors separately for thematic analysis. The first author then cross-checked the themes generated from the two analyses to identify agreements and disagreements in them. Discussions between the two authors were held to come up with the finalised themes for this analysis. Having separate coders is beneficial in that the identified themes are more likely to be accurate in representing the qualitative data than having only one coder. As consensus was reached between different researchers, the identified themes were less prone to individual bias (24).

Both quantitative and qualitative data were triangulated and interpreted by all authors to get a complete picture of participants’ experience of the workshop and whether it was beneficial in improving their feedback literacy.

Five Year-1 male students in the BSc (Hons) Physiotherapy programme joined the workshop. Their ages ranged from 19 to 32. All participants completed the questionnaire and joined the focus group.

Feedback literacy questionnaire result

A Wilcoxon signed rank test was performed to compare the differences between the total mean score on the questionnaire pre- and post-workshop. Results indicated that the post-workshop mean score was not statistically significant different compared to the pre-test mean score (Z=-4.05, p=0.686). The mean scores on the pre- and post-feedback literacy questionnaire were 71.25 (SD=7.50) and 70.8 (SD=8.50), respectively. The pre-test and post-test mean scores of each survey item are listed in Table 1.

Focus group analysis

Four themes were identified from the focus group analysis, two of which related to the participants’ usual perceptions and experiences of feedback and the other two answered the question on the effectiveness of the workshop.

Theme 1 – Mixed feedback experience

Participants described their usual feedback perceptions and experiences as both positive and negative. The positive experience related to the alignment between received feedback and results of self-reflection, the ability to observe reactions to feedback or changes in performance according to feedback.

“After I finished a task, I knew there were something that I did not do well, and the feedback I got matched with what I did not do well.” (S3)

“They did not give me feedback directly but I would observe their momentum and their level of engagement, improvement and joy and I would know the effects of my class.” (S4)

There was also a mention of the negative emotional aspect of feedback, which obviously hindered the motivation to act on it.

“I had to request [the tutor] for the written feedback, but the written feedback talked about something else, and that I did not meet the requirement … I think I should have passed, I felt rejected…” (S4)

“I had received some negative feedback in the past, it was overwhelming and I wanted to fix it. But I continued to receive negative feedback, then it piled up and affected my motivation.” (S2)

It appeared that some participants had not had very good feedback experiences in the past. The negative experiences might have created a barrier for them to meaningfully act on feedback and this is one of the major components in the new feedback paradigm. The quotes illustrated that participants were somewhat aware of the influences of negative feedback but they were left with no resources to change the situation. The word ‘rejected’ in S4’s quote is a strong word that signifies a rather negative feeling and emotion associated with the feedback process.

Theme 2 – Feedback depends on “relationship”

Participants mentioned multiple times that they felt the relationship between the feedback giver and receiver needs to be instrumental for feedback to be useful. The characteristics that make a feedback giver more effective, according to the participants, include being professional, experienced, competent, direct, convincing and context-specific. Perhaps the most influential attribute mentioned by the participants was trust and rapport.

“Even if the feedback is good, if I don’t have a trusted relationship with him [the feedback giver], I will not accept that feedback.” (S5)

“Maybe he [feedback giver] did not see things [when giving feedback] from my perspectives, and I also did not view from his perspectives, then that created a big misunderstanding.” (S5)

These quotes suggested that a rapport between the feedback giver and receiver is essential to spearhead the use of feedback. The rapport depends on personal and professional attributes which are not new to the field of feedback research. However, they may also be related to the feedback receiver’s ability to manage their affect if a trustworthy and respectful relationship exists.

Theme 3 – Expectation mismatch

Participants found that the workshop activities, such as reflection on past experiences, practices of mindfulness and discussions on authentic feedback scenarios provided valuable experiences for them to learn how to apply the new feedback process. However, they equally mentioned a number of expectation mismatches of the workshop. They expected having expert guest lectures to deliver part of the workshop, lecture notes for revision, more techniques on giving feedback effectively instead of receiving feedback, and more activities in the workshop. The most significant expectation mismatch was the lack of organised take-home tasks:

“After each session, it would be great to have an informal assignment for using different strategies to give feedback. Then you [the facilitator] can ask us in the next session on how the person [feedback receiver] responded.” (S3)

“[We] must do these assignments outside, not during the workshop, to note down the person’s feelings, to discuss afterwards [so that we] can observe [our improvement].” (S1)

These quotes illustrated that participants value authentic practice outside of the workshop and that it was a major part missing in this training. In fact, this authentic practice was originally planned in the workshop in the form of seeking assignment feedback from course instructors. However, the workshop sessions could not be planned to coincide with the assignment return dates, so this activity did not happen. In hindsight, the activity could be modified to seeking or giving feedback from/to peers.

In addition, participants suggested an alternative workshop mode, which provided insights on effective feedback training from students’ perspective.

“ [The training] needs an appropriate timing, to put it in the practicum, maybe before the practicum, or to run the workshop for a bit longer then there may be more development [for us].” (S5)

This quote signifies that embedding feedback literacy training into the curriculum, in particular before clinical placement, would be a more effective way for students to learn and perhaps actualize the feedback process.

Theme 4 – Changes after the workshop: from perceptions to actions

Despite the expectation mismatches of the workshop, participants anticipated some changes in their feedback practice. They said they would be more critical and contemplate feedback information from different perspectives:

“[I] will think more about the feedback [comment], if it’s a good one, or if it’s clear enough.” (S1)

“[I] will be more critical [about the feedback], I won’t just think about one aspect but I will think about both the positive and negative parts about the feedback.” (S3)

“After this workshop, when I receive feedback I will pause and think for a second, and see if I will consider it as positive or negative.” (S5)

These quotes indicate the anticipated differences in how participants will process feedback after the workshop as compared to before. Arguably, these are only anticipated changes of the participants and the design of this study does not include a follow-up phase on their behavioural change. However, at the end of the focus group discussion, one participant asked the facilitator for feedback on their workshop participation:

“For this workshop, is there any feedback you [facilitator] could give us [participants]? Is there anything we could do better in this workshop?” (S5)

This was a self-initiated and unexpected action from the participant that happened after the workshop. This action may serve to illustrate the increased awareness of the importance of seeking feedback, and that the participant turned this knowledge into action - one of the ultimate goals of the new feedback paradigm.

This mixed method pilot study investigated the effectiveness of a novel feedback literacy workshop on physiotherapy students’ feedback literacy. The study also explored participants’ perceptions and experiences with feedback practice prior to the workshop, and perceived changes in their feedback practice after the workshop. Survey results showed that participants’ level of feedback literacy did not differ significantly before and after the workshop. Qualitative findings from the focus group discussion suggested a mixed reaction about the effectiveness of the feedback literacy workshop. This mixed reaction partially stemmed from expectation mismatches. Nevertheless, participants discussed both enabling and constraining factors in their feedback processes, as well as anticipated positive changes in their feedback practice as a result of this workshop. It is worthwhile to note that some actual behavioural changes of participants were observed during the focus group and after the workshop. Despite the mixed results of this pilot study, findings preliminarily support the embedding of feedback literacy training into the physiotherapy education curriculum and shed light on the future development of structured feedback literacy training.

There are multiple reasons underpinning the mixed results on the effectiveness of the workshop. Regarding the quantitative measure of the feedback literacy score, this study failed to illustrate a statistically significant score improvement, which is different from the result of the DEFT study by Winstone, Mathlin and Nash (11). The biggest drawback of our study is the unexpectedly small sample size, while the DEFT study included 77 psychology students in this specific part of their study. Due to the pilot nature of our study, as previously mentioned, we originally expected to recruit 10 participants; however only five participants eventually joined the workshop despite two rounds of recruitment. Nevertheless, this pilot data can be used to estimate the required sample size for future studies.

Although the DEFT study also included a voluntary sample of participants, it used course credit in exchange for study participation. The voluntary participation appeared to happen during class time. In contrast, this present study recruited students to participate in a stand-alone workshop unrelated to any part of the curriculum. This may explain the small sample size as students may not have seen any incentive to participate.

The voluntary participation evidently created a sample bias of this study. Coincidently, all participants were students with previous tertiary education experience who somewhat understood the importance of feedback at baseline, as illustrated by their relatively high pre-test feedback literacy score. They had experienced feedback processes from previous education and work experiences. This relatively high feedback literacy level at baseline may translate to little room for improvement with the workshop. Participants’ own life experience might have already largely shaped the way they engage in feedback processes prior to joining the workshop.

In fact, this observation is described by Nieminen et al (25) as the ecological perspective of the theoretical framework of student agency of feedback literacy. The framework describes four perspectives that influence students’ active role in seeking, obtaining, creating and acting on feedback. The ecological perspective states that an individual’s past experiences, potential trajectories, and judgement on contextual appropriateness affect how one acts on feedback. Carless and Boud (7) also stressed the importance of student agency and responsibility in the taking action element of the new feedback paradigm. As all participants in the present study had some concrete experiences of engaging with feedback in previous work and education, this workshop may not have served the higher-level needs of this group of participants in improving their feedback literacy score as well as in meeting their expectations of this workshop.

The execution of the feedback literacy workshop was constrained by pragmatic factors, for example, the inability to schedule the workshop time around the assessment time of a course happening in the same semester. The original idea was to match the time of the workshop together with assignment submission time of that course so as to allow participants to seek assignment feedback with the skills they learnt in the workshop. However, with participants’ busy school and work schedule, it was not possible to conduct the workshop around the assessment time. Therefore, no real-life feedback practice was available. This pragmatic constraint concurs with O’Connor and McCurtin’s findings of where feedback literacy training should be located within the curriculum to benefit the learning outcomes of students (13). Future training schedule should consider the curriculum as a whole to maximise the benefits of it.

Although this study tried to utilise the strategy as in the DEFT study (11) by asking participants to bring samples of their previous works for peer feedback, our participants did not find this activity particularly helpful. These pragmatic constraints evidently contributed to some of the expectation mismatches as stated in Theme 3 in the findings. There could have been confidence issues with regards to sharing their assignment feedback and results with others. In addition, this type of constraint is also an example described in the socio-material perspective of the theoretical framework of student agency of feedback literacy (24). The socio-material perspective stresses the importance of the interaction between human factors (e.g. teacher, classmates, family, and the student themselves), non-human factors (e.g. learning platforms, digital hardware and software, physical settings of classroom, rubrics and feedback forms) and the relationship between the two. Simply put, the socio-material perspective concerns whether students are equipped by various means, be it equipment or training or opportunity, to practise feedback. The lack of authentic practice in the design of this workshop failed to provide an opportunity for participants to authentically experience what they had learnt in the workshop and led to disappointment and suboptimal effectiveness of the workshop.

Yet, despite the expectation mismatch and insignificant change in feedback literacy score with the workshop, participants projected some positive changes regarding their feedback process as illustrated in Theme 4 of the findings. This result is comparable to the three previously-mentioned studies on training on feedback literacy (11, 12, 13) where there were perceived usefulness and benefits of structured feedback literacy training. Beyond projection, participants in this study actually demonstrated feedback-seeking behaviour; one participant actively asked the workshop facilitator for feedback on their workshop behaviours, and later on another participant asked how to give feedback to a lecturer on her teaching. These behavioural changes are encouraging as actions are more impactful than words alone.

We attributed these changes to the rapport between the workshop facilitator and participants. The participants mentioned a number of times in the focus group that trustworthiness, credibility and competence of the teacher encourage their practices of feedback. Rapport is another element stated in the ecological perspective of the theoretical framework of student agency of feedback literacy, in particular, the judgement on contextual appropriateness in terms of the relational aspect (25). The participants might have subconsciously evaluated their relationship with the workshop facilitator throughout the workshop. Rapport might also have been built up during the interactive activities. These judgments promoted a trustworthy relationship which in turn supported their feedback-seeking behaviours.

Study Limitation

The small sample size evidently limited the statistical analysis of this study. We tried to overcome this weakness by including both quantitative and qualitative data for result triangulation. Still, the sample size of five participants is likely insufficient for both quantitative and qualitative analyses. The results should therefore be treated with caution and as pilot findings to inform any future larger scale study.

At the time when the study was conducted, no validated scale for measuring feedback literacy existed. The first author had to create a customised questionnaire based on examples from previous studies to measure students’ feedback literacy. The questionnaire was not validated but expert opinion was sought to improve it prior to its use. For any future study, a validated student feedback literacy scale should be used.

This pilot study provided valuable insights on feedback literacy training specifically for physiotherapy students. Factors such as rapport and authentic practice appeared to facilitate students’ feedback uptake. These factors incidentally align with two aspects in the student agency framework for feedback literacy, indicating the importance of embedding these features in future student feedback literacy training. The training can be conducted in the curriculum, before or during clinical placement, so as to maximise opportunities for students to practice feedback uptake in an authentic physiotherapy practice environment. Future research with a larger sample size can be conducted to study the impact of curriculum-embedded, student agency-focused feedback literacy training.

5MFF: the five-minute feedback form

ABL: Activity-based Learning

DEFT: Developing Engagement with Feedback Toolkit

R2C2 Model: A model that builds relationship, explores reactions and content, and coaches for performance change

Ethics approval and consent to participate

Ethics approval was obtained from the Research Ethics Committee of Hong Kong Metropolitan University (Reference number HE-KS2021/01). Informed consent was obtained from all participants included in the study. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no potential conflict of interest and competing interest.

Funding

The study was funded by the Katie Shu Sui Pui Charitable Trust – Research and Publication Fund (Applied Research) (Project number KS 2021/01). The funding allowed for the appointment of a research assistant to provide support in developing the workshop, transcribe the focus group discussion, and support statistical and content analyses of the data.

Authors' contributions

DC conducted the focus groups and analysed/interpreted all qualitative and quantitative data. SC contributed to analysing and interpreting the qualitative data. WT contributed to analysing and interpreting the quantitative data. All authors contributed to the explanations of the overall findings. DC and SC were major contributors in writing the manuscript while all authors read and approved the final manuscript.

Acknowledgements

The authors would like to acknowledge the participants in this research.

Ajjawi R, Regehr G. When I say ... feedback. Med Educ. 2019;53(7):652-4.
Sadler DR. Beyond feedback: developing student capability in complex appraisal. Assess Eval High Educ. 2010;35(5):535-50.
Allen L, Molloy E. The influence of a preceptor-student 'Daily Feedback Tool' on clinical feedback practices in nursing education: A qualitative study. Nurse Educ Today. 2017;49:57-62.
Sargeant J, Mann K, Manos S, Epstein I, Warren A, Shearer C, et al. R2C2 in action: Testing an evidence-based model to facilitate feedback and coaching in residency. J Grad Med Educ. 2017;9(2):165-70.
O’Malley E, Scanlon A-M, Alpine L, McMahon S. Enabling the feedback process in work-based learning: An evaluation of the 5 minute feedback form. Assess Eval High Educ. 2020;46:1-15.
Bing-You R, Hayes V, Varaklis K, Trowbridge R, Kemp H, McKelvy D. Feedback for learners in medical education: What is known? A scoping review. Acad Med. 2017;92(9):1346-54.
Carless D, Boud D. The development of student feedback literacy: Enabling uptake of feedback. Assess Eval High Educ. 2018;43(8):1315-25.
Higher Education Funding Council for England. National student survey results 2016. 2016 [Available from: https://webarchive.nationalarchives.gov.uk/ukgwa/20170712122907/http://www.hefce.ac.uk/lt/nss/results/2016/.
Social Research Centre. 2016 Student experience survey: National report. Melbourne: Social Research Centre; 2017.
Sutton P. Conceptualizing feedback literacy: knowing, being, and acting. Innov Educ Teach Int. 2012; 49:31-40.
Winstone NE, Mathlin G, Nash RA. Building feedback literacy: Students’ perceptions of the Developing Engagement With Feedback Toolkit. Front Educ. 2019; 4.
Noble C, Billett S, Armit L, Collier L, Hilder J, Sly C, et al. “It’s yours to take”: Generating learner feedback literacy in the workplace. Adv Health Sci Educ. 2020;25(1):55-74.
O’Connor A, McCurtin A. A feedback journey: Employing a constructivist approach to the development of feedback literacy among health professional learners. BMC Med Educ. 2021;21:486.
Hertzog MA. Considerations in determining sample size for pilot studies. Res Nurs Health. 2008;31(2):180-91.
Grix J. The foundations of research. 3rd ed: London Red Globe Press; 2019.
Bernard HR. Social research methods: Qualitative and quantitative approaches. 2nd ed. Los Angeles, [Calif.]; London: SAGE Publications, Inc; 2013.
Leu E, Price-Rom A. Quality of education and teacher learning: A review of the literature. American Institutes for Research under the EQUIP1 LWA Academy for Educational Development; 2007.
Bloom BS, D. EM, J. FE, H. HW, R. KD. Taxonomy of educational objectives: The classification of educational goals. Handbook 1. London: Longman; 1956.
Ginsburg M. Improving educational quality through active-learning pedagogies: A comparison of five case studies. Educ Res. 2010;1:62-74.
Newlife.330 [Mobile application software]. (Version 2.7.0) Hong Kong: New Life Psychiatric Rehabilitation Association. (2021).
Brown KW, Ryan RM. The benefits of being present: Mindfulness and its role in psychological well-being. J Pers Soc Psychol. 2003;84(4):822-48.
Hülsheger UR, Alberts HJ, Feinholdt A, Lang JW. Benefits of mindfulness at work: The role of mindfulness in emotion regulation, emotional exhaustion, and job satisfaction. J Appl Psychol. 2013;98(2):310-25.
Spinelli C, Wisener M, Khoury B. Mindfulness training for healthcare professionals and trainees: A meta-analysis of randomized controlled trials. J Psychosom Res. 2019;120:29-38.
Harding T, Whitehead D. Nursing and midwifery research: Methods and appraisal for evidence-based practice. 5th ed. Schneider Z, Whitehead D, LoBiondo-Wood G, Haber J, editors. Mosby, Marrickville, Sydney: Elsevier; 2016.
Nieminen JH, Tai J, Boud D, Henderson M. Student agency in feedback: Beyond the individual. Assess Eval High Educ. 2022;47(1):95-108.

No competing interests reported.

Developing feedback literacy in physiotherapy students: a pilot study

Status:

Version 1

Abstract

Background

Methods

Results

Discussion

Conclusion

Abbreviations

Declarations

References

Additional Declarations

Status:

Version 1