The ReHapticKnob
The ReHapticKnob (11, 12) is an end-effector device for sensorimotor rehabilitation of the hand and forearm after stroke (Fig. 1). In previous clinical trials, therapy assisted by the ReHapticKnob and supervised by a therapist was shown to be equivalent (i.e., non-inferior) to carefully dose-matched conventional therapy (13).
A set of seven therapy exercises implemented on this device are based on the neurocognitive therapy concept, which focuses on the integration of motor, sensory, and cognitive functions when performing a task (12, 14). The exercises focus on the passive or active training of grasping or forearm pronosupination and target subjects with different levels of impairments. Furthermore, each exercise follows an assessment-driven concept, meaning that the initial difficulty level is tailored to the results of specific assessments performed with the ReHapticKnob before the start of the training (for more details see (15)).
To address the challenges raised by unsupervised use, a major focus was placed on improving the usability of the robot, including pilot evaluations with stroke patients and therapists (12). For example, the graphical user interface was redesigned to be more intuitive and pleasant. Furthermore, clinically-inspired algorithms based on the action and decision process usually performed by therapists in a supervised session were implemented, with the objective of automatically monitoring, controlling, and adapting the content of therapy sessions (16, 17). For instance, these algorithms automatically adapt the difficulty of an exercise or add a new, more challenging, exercise based on the performance, and provide feedback to guide users through the therapy sessions. Generally, they further decrease the number of actions that need to be learned to interact with the device, thereby increasing usability, and avoiding the need for a therapist to monitor and adjust therapy content over an extended period of time.
Study protocol
This pilot study was approved by the Swiss notified body regulating the use of medical devices (Swissmedic 102681300) and the cantonal ethics commission of Ticino (CE TI 3577). A detailed description of the study protocol is provided in (18).
In short, a sample of 13 participants was chosen, which was considered large enough for a feasibility study while also taking into account a similar group size and drop-out rate (around 20%) compared to our previous studies with the ReHapticKnob (13). Participants were recruited from the stroke inpatients of the Clinica Hildebrand Centro di riabilitazione Brissago. Inclusion criteria were age between 18 and 90 years old, inclusion within 6 weeks from stroke onset, pre-stroke modified Rankin score (19) ≤ 1, National Institutes of Health Stroke Scale (NIHSS, (20)) ≥ 1 in at least one of the items regarding motor or sensory function and ataxia, and signed informed consent form. Exclusion criteria were moderate to severe aphasia (Goodglass-Kaplan’s scale (21) < 3), moderate to severe cognitive deficits (levels of cognitive functioning-revised (LCF-R, (22)) < 8), functional impairment of the upper limb due to other pathologies, severe pain in the affected arm (visual analogue scale for pain (VASp) ≥ 5), other pathologies possibly interfering with the study, pacemakers and other active implants, and modified Ashworth Scale (23) > 2 for one or more of the following muscles: shoulder adductors, forearm pronator and supinator, flexors and extensors of elbow, wrist, and fingers.
In order to teach participants to confidently use the device in an unsupervised manner and reduce the risk for adverse events, we specifically designed a systematic protocol for the progressive transition from supervised to unsupervised use of the device (18).
For each participant, the study protocol lasted four weeks. The first week consisted of 5 sessions of 45 minutes of supervised therapy, where a therapist was present to explain how to use the device and perform the different exercises. The second week consisted of 5 sessions of 45 minutes of minimally supervised therapy, i.e., participants tried to perform the therapy session independently, but a therapist was still in the room. Despite being present, in this phase the therapist remained in the background and intervened solely upon participant’s request or when necessary (e.g., for safety reasons). At the end of the minimally supervised week, the therapist evaluated the participant’s readiness to continue to the unsupervised phase as well as independence with respect to mobility (e.g., ability to independently access the device) with a custom-made checklist. If participants reached all the goals on the checklist, they proceeded to two weeks of fully unsupervised training. In the unsupervised phase, the device was kept turned on in a freely accessible room in the clinic, and participants could train during a 45-minute timeslot indicated on their daily schedule (business days only), as well as in their free time, evenings, and weekends (the latter not indicated on their schedule). Although for business days a timeslot was booked on their daily schedule to avoid interfering with the conventional therapy plan and to ensure device availability, it is important to note that participants were clearly told that therapy with the ReHapticKnob was voluntary, and no recommendations were given on the daily therapy dose to achieve. Access to the robot was also not monitored nor directly encouraged during the unsupervised phase. If, at the end of the minimally supervised week, a participant was deemed not ready to safely train unsupervised, an additional week of minimally supervised therapy was added. At the end of this second minimally supervised week, the checklist was repeated and if the requirements were met, the participant could train unsupervised for the final week. If not, a third week of minimally supervised therapy was performed.
During each phase, all the robot-assisted therapy sessions were an addition to the conventional therapy plan of the participants (usual care). Participants with very limited mobility (i.e., unable to move on their own from their room to the various therapy stations) were accompanied to the ReHapticKnob upon request by clinical staff dedicated to escorting patients to the various rooms, as for any other conventional therapy. These staff were not trained in the use of the ReHapticKnob, so while they could assist patients in positioning themselves in front of the robot, they were not allowed to help them in interacting with it.
At the beginning and at the end of the study, clinical assessments were performed. Questionnaires to evaluate usability and user experience were performed after the first week of minimally supervised therapy (Usability 1) and at the end of the study (Usability 2), to evaluate the change in perceived usability due to therapists’ absence during the robot-assisted therapy sessions. User experience was further evaluated at the end of each therapy session with the ReHapticKnob by automatically presenting the question “How was your therapy session today?”, to which subjects could answer with a 5-point Visual Analogue Scale represented by different emoticons (VAS – Smiles).
Primary outcome measures
Feasibility of the proposed protocol was measured as the number of subjects who could proceed to the unsupervised phase, safety of use (i.e., number of adverse events and device deficiencies), and attendance during the unsupervised phase. Attendance is here defined as the percentage of days where the participant trained with the ReHapticKnob at least once (detected by the login in the therapy account) out of the total number of offered days for unsupervised therapy (i.e., 14 or 7).
A further outcome was the dose of unsupervised robot-assisted therapy, measured as therapy duration in minutes per day and total minutes over two complete weeks of unsupervised therapy, number of task repetitions, and percentage increase in physical therapy time due to the robot-assisted therapy with respect to the conventional physical therapy time (i.e., upper limb and lower limb physio- and occupational therapy) during the unsupervised phase. The latter metric reflects the increase in therapy dose that could be achieved with minimal use of the clinical resources.
An additional primary outcome was the change in usability and user experience between Usability 1 and Usability 2. Performed usability questionnaires included the System Usability Scale (SUS, (24)), the raw Task Load Index (TLX, (25)), and the Post-Study System Usability Questionnaire (PSSUQ, (26)). User experience was characterized with the net promoter score (NPS, (27)) and the customer satisfaction (CSAT) score. In this case, the NPS reflected the probability (on a scale from 0 to 10) with which a participant would recommend therapy with the ReHapticKnob to another patient. Participants are divided into promoters (score 9 or 10), passively satisfied (score 7 or 8), and detractors (score < 7). The final score is given by the subtraction of the percentage number of detractors from the percentage of promoters, with higher values corresponding to a higher ratio of promoter to detractors. For the CSAT, participants had to rate their level of satisfaction with the therapy with the ReHapticKnob on a 5-point scale ranging from “very unsatisfied” (i.e., 1) to “very satisfied”.
The difference between VAS – Smiles ratings given in the three phases was used to further investigate how user experience changed depending on therapist’s presence. The results of the custom-made checklist were also used to identify aspects of the device possibly requiring improvement.
Secondary outcome measures
The difference in the content of the therapy sessions between the three phases (supervised, minimally supervised, and unsupervised) was evaluated to investigate whether, in the absence of the therapist, participants really engaged in the exercises of their personalised therapy plan and did not, for example, simply start the exercises without performing them. Thus, for all three phases we expected no significant difference in therapy content. The metrics used to assess therapy content were intensity (i.e., number of task repetitions per minute), task performance (i.e., correct responses out of the total number of repetitions), and ratio of effective therapy time (i.e., net therapy time without breaks) to total duration of a therapy session.
An additional secondary outcome was functional recovery, calculated as the difference between final and baseline scores for the clinical assessments. The assessments performed at both time points were the Fugl-Meyer Assessment of Upper Extremities (FMA-UE, (28)), ABILHAND (29), Box and Block test (BBT, (30)), Motor Evaluation Scale for Upper Extremities in Stroke Patients (MESUPES, (31)), and modified Ashworth Scale (mAS, (23)).
Furthermore, parameters possibly influencing unsupervised therapy dose or attendance were investigated. These parameters included age, baseline clinical assessments scores, and dose of conventional therapy in minutes during the unsupervised phase. The impact of cognitive deficits and of the level of independence with respect to mobility, measured with the Barthel Index and custom questions, on the ability to proceed to the unsupervised phase was also investigated.
Data analysis
Descriptive statistics (mean and range (min-max) or boxplots) were computed for study population, attendance and therapy dose in the unsupervised phase, user experience, platform usability, assessments scores, and functional recovery. Therapy dose as total minutes over the unsupervised phase was calculated only for subjects who achieved two complete weeks of unsupervised therapy.
The paired samples Wilcoxon test was used to compare the data collected during Usability 1 and Usability 2. Subjects who did not perform unsupervised therapy and subjects who did not complete both usability sessions were excluded from this analysis.
The Friedman test was performed to compare therapy content (i.e., mean intensity, performance, and effective therapy time for each subject) and mean VAS – Smiles ratings between the three phases. The Wilcoxon signed rank test with Bonferroni correction was used for post hoc analysis. Subjects who did not reach the unsupervised phase were excluded from the analysis.
Linear fixed-effects models were computed to investigate parameters possibly influencing the achieved total dose of unsupervised therapy and attendance in the unsupervised phase. The parameters included were age, dose of interdisciplinary conventional therapy in minutes (as an estimate of fatigue), and baseline scores for the clinical assessments (i.e., ABILHAND, BBT, FMA-UE, MESUPES). Given the limited dataset, we had to restrict the number of independent variables of the model. Therefore, we selected the parameters that are known soon after admission to the clinic and that in the future could potentially be early predictors of which patients are good candidates for unsupervised therapy. Subject who did not perform two complete weeks of unsupervised therapy were excluded from this analysis. Significance level was set to 0.05.