Participants
Jakob Nielsen’s theory (19) is widely accepted regarding the number of users enough to evaluate a system with the aim of detecting most of the usability problems that can affect a product. According to Nielsen between three and five users could identify the 85% of the most relevant usability problems. In this case, it was decided to recruit 12 patients in different stages of post-stroke upper limb recovery evolution due to the heterogeneity of the study population, in order to test as many system features as possible.
Patients with a stroke were recruited at the University Hospital Reina Sofía of Cordoba, Spain. Participants were divided in three different groups depending on their stage: subacute between 2–6 months of evolution, chronic of short evolution between 6–12 months and long-term chronic (more than 12 months). Four patients were recruited from each stage. The inclusion criteria to participate in the study were: subjects over 18 with upper limb hemiparesis after stroke, unilateral paresis and cognitive ability to understand, accept and actively participate in the usability study. The patients who presented bilateral motor deficit, severe spasticity, psychiatric illness, and/or cognitive impairment were excluded.
Having Wi-Fi at home, as well as a table of 110 × 68 cm to have enough space for MERLIN system installation was also considered a requirement to participate in the study. All the subjects were duly informed about the study and all of them gave written consent before the first session.
Study design
This interventional study is an open label trial with a single group and a longitudinal design. Each patient used the MERLIN system for 3 weeks: one week of training at the IMIBIC (Maimonides Biomedical Research Institute in Cordoba, Instituto Maimónides de Investigación Biomédica de Córdoba, in Spanish) facilities with a physiotherapist supervision, and 1 week at patient's home with the physiotherapist supervision and 1 week at patient’s home by their own with physiotherapist remote support and supervision to organize the rehabilitation sessions.
Arm and hand functions were evaluated the first day before starting the training as baseline, and the last day of participation. System usability and participants’ motivation was evaluated the last day of their participation using different validated scales as it will be explained later in this manuscript.
The MERLIN robotic-assisted telerehabilitation system
The MERLIN robotic telerehabilitation system has been developed to bring the neurorehabilitation to the post-stroke patients homes with the aim of providing daily, intensive, motivating and patient tailored rehabilitation, with therapists’ indirect supervision (20). The system is composed by the ArmAssist (AA) cost-effective robotic system based on serious games developed by TECNALIA, and the Antari Home Care platform (21) to supervise, organize and customize the patients’ daily training remotely, which has been developed by GMV (22). The AA system is a modular solution which includes an affordable and portable robotic device for an upper limb complete rehabilitation, and a software platform based on serious games for the patients engagement and training assessment (18). The AA robotic device can measure the patient’s self-directed active movements which are performed on a mat put on a regular table thanks that the robotic system includes different sensors (see Fig. 1) The key movements that can be measured are shoulder horizontal abduction-adduction, flexion-extension in the elbow (vertical force), prono-supination movements of the wrist and hand opening and closing (grasping) (23). This version of the system is addressed to patients that can actively do the movements and thus is more appropriate for the patients that are in mildly or moderate motor impairment according to Fugl-Meyer scale.
The AA system facilitates recovery of upper limb motor control and function through, interactive gaming goal-oriented, and functional tasks assisted, that require variable cognitive engagement designed to motivate the user to train longer and more effectively. It includes games for both training and assessment (24). 7 training games are available, which were co-designed with patients and physiotherapists (25). The games can be configured to train shoulder abduction/adduction, wrist prono-supination, hand open/close, or elbow flexion-extension and combinations of those movements. The game demands to the user doing the exercise beyond the threshold of the patient’s range motion. This threshold is previously set using the assessment games, and can be modified when needed, i.e. when motor improvement is detected by the physiotherapist. Different levels can be also configured for each patient depending on the motor and cognitive capabilities. The games have been designed suitable for the target group taking into account the possible cognitive or vision problems, for example (26). Additionally to those designed games, the system offers the option to play online games available in Internet using the robot device as a regular mouse. This option is recommended for patients with good movement control and cognitive capabilities.
In the MERLIN system, the patients could access to the daily therapy previously organized by the physiotherapist, as well as to a summary of the results obtained during the therapy (see Fig. 3 Right). A communication tool with the therapist, similar to mailing, was also added.
The AA system has been previously tested in clinical setting by therapists and patients with positive results on acceptance (11) and effectiveness with the improvements in the motor function of the patients after the use of it (24). Previous studies also demonstrated that the therapy using AA system is enjoyable and motivating because engages patients (27). In this study, the system has been adapted for a home use. With this aim, the software was prepared to work on a tablet, a package was designed to transport the system, and the mat was adapted (see Fig. 2).
As it was previously explained, the Antari HomeCare platform has been also integrated in the MERLIN system. This telecare platform which is prepared for managing patients’ treatments and doing an online follow-up, was adapted for customizing the rehabilitation therapies remotely. Using this online platform, the therapist customized each patient therapy selecting the games to be used and the movement to train, the number of days to be repeated, the time to play each game. Also, the evolution of the patient and the score obtained in the different games or tasks proposed for each day, as well as duration and frequency of training could be checked by the therapist using this online system (see Fig. 3 Left). The communication tool similar to mail is also accessible via Antari HomeCare system.
Intervention sessions
Rehabilitation therapy included 11 sessions using MERLIN system to be done in 3 weeks. The first week was used as training to teach users and caregivers how to use the system correctly as well as getting used to the rehabilitation system, robot movements and protocol times. Training sessions were organized every day with a duration of one hour per day at IMIBIC facilities (see Fig. 4 Left). Special emphasis was addressed in the correct positioning of the arm and shoulder for a proper rehabilitation. In addition, each participant received a user manual copy which also included a telephone number to contact in case of any technical or clinical issue. The physiotherapist installed the system at the participant’s home in the beginning of the second week, the chair height was adjusted and explained to the patient the correct position for the back and shoulder for doing the training and having the arm in a comfortable position during the rehabilitation when supervision was not available (see Fig. 4 Right). 3 sessions of 30 minutes were held on alternative days. During this week the patient did the assigned therapy for each day with the physiotherapist supervision. Third week have same structure but participants trained using the system completely autonomously at home. The physiotherapist used the tele-care platform daily, to follow up remotely the participants’ progress and system use as well as organizing next session.
The training movements and games used for that purpose were selected by the physiotherapist, who decided the therapy intensity level and movements to train according to the patient's evolution or cognitive conditions. The therapist organized the rehabilitation sessions beforehand using the tele-care platform prepared with that aim. Prior starting the therapy, patients were requested to do a calibration process to set up the threshold according to their range of motion. This allows that participants were challenged to exercise at their maximum capacity. Then, during the system use, the range of motion for each game and patient was controlled by the system itself.
Assessment
Study data were collected and managed using REDCap (28) electronic tool hosted at FIBICO (Foundation for Biomedical Research in Cordoba; Fundación para la Investigación Biomédica de Cordoba in Spanish) (29). REDCap (Research Electronic Data Capture) is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources (30, 31). REDCap is HIPAA (Health Insurance Portability and Accountability Act) (32) and 21 CFR Part 11 (33) compliant, which means that it complies a minimum security for data in clinical investigations. However, no personal data were recorded on REDCap to comply with the European General Data Protection Regulation (GDPR) (34) as participants were European citizens.
Primary outcomes measurements. Usability and acceptance data.
Feasibility of use of the system and motivation were evaluated by patients using semi-structured interviews and different usability questionnaires with Likert scales during the clinical trials which had a duration of 3 weeks of intervention. Used validated scales were: System Usability Scale (SUS) (35), Adapted Intrinsic Motivation Inventory (IMI adapted) (36), Quebec User Evaluation of Satisfaction with assistive Technology (QUEST) (37), and AA Usability Assessment Questionnaire (18).
SUS scale is used to evaluate the usability of the system. Scores are ranging to 0–100%, where a high score means better usability with a threshold of 68%. The IMI is a multidimensional questionnaire which measures interest/enjoyment, perceived competence, pressure/tension, value/usefulness, felt pressure and tension, and perceived choice that patients experience during the performance of an intervention with the device. This scale consists of 20 item questionnaire with options from 1 ‘strongly agree’ to 7 ‘strongly disagree’. The purpose of QUEST is to evaluate the patient’s satisfaction with the device and with the services they experienced. It consists of 12 questions: 8 related to the device and 4 related to services, which must be rated in a Likert from 1 ‘Not satisfied at all’ to 5 ‘Very satisfied’, AA Usability Assessment Questionnaire consists of 17 items survey. It was specifically designed for the AA device which is used in MERLIN system. The questions are rated by patients and therapists from 1 ‘strongly agree’ to 7 ‘strongly disagree’ to evaluate the satisfaction with the system and the therapy. It also includes 3 open-ended questions about the participant’s subjective opinion such as the aspects that she like most, the identified negative aspects, and any proposals for improve the system.
In addition two short questions were added to ask to participants about their willingness-to-pay for the MERLIN system as therapy.
Secondary outcome measurements. Clinical information.
With the aim of quantifying general arm function and any effect of the system on it, clinical standardized scales were used, before patients start with the therapy using MERLIN and after finishing the clinical trial. Upper Limb Fugl Meyer Scale (Fugl-Meyer) (38) and Modified Ashworth Scale (MAS) (39) were used to evaluate the clinical condition of the patients before their enrolment in the study to confirm their participation according to the inclusion criteria. Same scales were repeated at the end of the therapy using MERLIN to confirm the system safety and did not cause negative effects on the patient such as arm function reduction. Fugl-Meyer and MAS could be used also to measure the effectiveness of the system although small improvements were expected due to the short duration of the intervention achieve clinical evidence.
The Fugl-Meyer is an index to assess the sensorimotor impairment in individuals. MAS measures muscle tone during passive soft tissue stretching by rotating a joint and estimating the resistance, and it is used as a simple measure of spasticity.
Statistical analysis
Statistical outcomes were analysed using IBM SPSS Statistics© (40) software for Windows© Operating System. Descriptive summary statistics (mean with standard deviation, SD) was used to process the quantitative data provided by the Likert scales items in SUS, QUEST and IMI Adapted. The qualitative data obtained in the open-ended questions were analysed using thematic analysis.
To the Clinical assessment, a one-tailed, paired t-test, with a significance level of p < .05, was used to compare pre- and post-intervention Fugl Meyer and MAS outcome measures.