A comprehensive analysis of a novel real-time adaptive assist-as-needed controller for robot-assisted rehabilitation for stroke patients

Background: Rehabilitation therapy plays an essential role in assisting stroke patients in regaining function. For this reason, many studies have been conducted to optimize rehabilitation interventions to improve effectiveness and efficiency. In this context, robotic devices for rehabilitation and assistance can be effective. Several studies have demonstrated that using a robot as a therapy tool can significantly reduce motor impairment. However, the slacking behavior, in which the patient lets the robot guide their movements even when they are capable of doing so by themselves, has been identified as a major barrier to reaching the full potential of robot-assist rehabilitation. This paper developed a novel electromyographical-based adaptive assist-as-needed (aAAN) controller aiming to avoid this slacking behavior. Methods: Five stroke patients were recruited to test the controller. Motor impairment status was documented with the Fugl-Meyer (FMA) assessment. In this experiment, horizontal arms tasks were conducted with the robot off and on to assess the subject’s performance in both scenarios. Velocity, time, and position were quantified as performance parameters during the training. Arm and shoulder EMG and electroencephalography (EEG) were used to assess the performance of the controller. Results: The cross-sectional results showed strong second-order relationships between Fugl-Meyer score and outcome measures, where performance metrics (path length and accuracy) were sensitive to change in participants with lower functional status. In comparison, speed and electrophysiological metrics (EMG and EEG) were more sensitive to change in participants with higher functional status. EEG signal amplitude increased when the robot suggested that the robot was inducing a challenge during the training tasks. Conclusion: The preliminary results were very promising; slacking was avoided for all participants during training with the aAAN controller.


Introduction
Stroke is a leading cause of adult long-term serious disability and the second cause of mortality worldwide (1). According to the Public Health Agency of Canada and the Heart and Stroke Foundation, approximately 741,800 adults (365,000 men and 376,800 women) aged over 20 years old live with the effects of a stroke (2012/13). Stroke damages the brain, and 80% of people who suffered a stroke have hemiparesis, which is an impaired motor control on the affected side (2). Most survivors of hemispheric stroke present limited movement of the contralesional upper extremity.
Robots for assessment, treatment, and assistance of people with impaired motor control have shown some promise in improving motor recovery post-stroke. Although not readily available in clinics, the use of this technology in the rehabilitation field is widely thought to be potentially beneficial (3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13).
Most of the advanced work in the robot-assisted therapies field has been done in recent decades. Several studies have shown how the strategy of control used in robot-assisted rehabilitation can significantly and positively impact the rehabilitation process's effectiveness and efficiency (3,4,6,(8)(9)(10)(11)(12). However, these studies indicate that most of the strategies used in robotassisted therapies have drawbacks such as being susceptible to a "slacking" behavior where the subjects let the robot drive the movements without performing sufficient physical effort to benefit from the training. Moreover, an overall analysis of the subject's progress, including muscle and brain activity, is crucial to understand motor learning and neuroplasticity process in stroke patients.
Prior studies also do not utilize an online adaptation of the control system based on physiological measurements that can appropriately scale assistance or resistance according to user performance and progression. To date, no study has done a comprehensive analysis of cortical activity when applied to a robotic-assisted training program for improving motor recovery in poststroke patients (3,9,11,12).
The present study aimed to develop and validate a novel adaptive AAN (aAAN) algorithm for upper-extremity robot-assisted therapy for stroke. This new algorithm's control system integrates sEMG and a haptic system to provide a patient-cooperative non-slacking roboticassisted therapy to facilitate and accelerate upper extremity motor function recovery. Changes in performance metrics, muscle, and brain activity between the baseline and training sessions were examined for each subject. Relationships of these responses to participants' functional status using the Fugl-Meyer (FMA) clinical assessment were explored to quantify the robot training system's sensitivity. This enabled us to validate how outcomes measures would be expected to change with improvement in functional status. Specifically, we address the following research questions: Does overall mean EMG entropy amplitude increase when training with the aAAN algorithm?; Does overall mean EEG power density increase when training with the aAAN algorithm?; Are performance metrics that are assessed during training sensitive to functional status via the FMA score?; Is mean EMG entropy amplitude assessed during training sensitive to functional status via the FMA score?; Is mean EEG power density assessed during training sensitive to functional status via the FMA score?

Methods
To develop the robot-assisted training with an adaptive assist-as-needed system, the following essential requirements were prescribed. First, the robot shall provide enough force to allow the user to complete the task and motivate them but do so minimally to avoid the slacking effect. Second, the system shall adapt to the user's needs by, for example, reducing the assistance and increasing the challenge as the user's performance improves. Third, the training should be non-monotonic in order to be more engaging. These elements are expected to encourage voluntary control during training, known to promote motor plasticity in the brain (19,20). Finally, multiple sensors should be used for measuring movement performance and muscle and brain activity during the training to capture outcomes relevant to motor recovery.

Instrumentation
The system was implemented using a PHANTOM 3.0 device from SensAble technologies, with open-source software that allows the user to interact with it in a broad range of applications.
This device is mostly used for rehabilitation, the development of games, and entertainment (15). It can provide low-level control, support the visualization of 3D objects, and can generate force effects. The PHANTOM device is a high-precision instrument connected to the computer through a serial connection and provides a large workspace and high forces with high fidelity and low friction. Information such as velocity, force, and position can be provided from this device.
This study's EMG system was the Trigno™ Wireless EMG (Delsys, Inc., Natick, Massachusetts). This system has 16 active wireless sensors, with an EMG signal range of ±5V and a significant 909V/V ±5% gain. It also has an active analog bandpass filter between 10-500Hz, 160dB/Dec. The electrode placement and target group muscle that was used in this control system were the biceps (BI), triceps (TI), pectorallis major (PM), anterior deltoid (AD), middle deltoid (MD), and posterior deltoid (PD). The electrode placement followed the SENIAM guidelines.
The EEG, from Artinis company (Artinis Medical Systems, Einsteinweg, The Netherlands) has 128 channels EEG for full head coverage. Although the EEG signal is a spatially imprecise recording technique due to volume conduction (activation several centimeters away is recorded by electrodes), it is a very useful technique as it records at a very high sample rate of up to several tens of Kilohertz.

Control algorithm
The information acquired in real-time from the PHANTOM and EMG systems was used to control the robot-assisted system. Prior studies have utilized muscle activity or user's performance and velocity data to develop strategies of control with different levels of assistance.
To our best knowledge, no studies have validated this approach or integrated the capability of challenging the users to keep them engaged. Past research shows this is a requirement to promote motor plasticity and a faster recovery (5-7, 10,11,17).
For the novel patient-cooperative non-slacking system, the system's initial trigger is regulated based on the EMG signal. The EMG control part of the system was similar to the approach described in Zhang et al. (27), where the sample entropy was computed for the EMG signal in order to detect the muscle activity onset. The entropy of an EMG signal represents the complexity and randomness of the signal. This measurement takes advantage of this dynamic system's nonlinear properties to distinguish voluntary surface EMG signals from spurious background spikes. The calculation of the entropy of the EMG signal starts by embedding the scalar time series { 1 , 2 , … , , } in a delay m-dimensional space as the following: −1 , = 1, 2, … , − + 1 Then, the sample entropy is calculated as the following: where B(r) is the probability of two sequences matching for m points, which is calculated by counting the average number of vector pairs where the distance is lower than the tolerance r and where A(r) is an embedding dimension of m+1. Once this is done, the EMG trigger threshold is determined based on the signal resulting from this process.
The EMG signal of three muscles was constantly analyzed. The velocity was also constantly acquired as soon as the EMG trigger threshold first trigged the system. The robot only assisted the patient if the system has been triggered and the muscle is maintained activated, but the insufficient motion is detected to achieve the target. Figure 1 illustrates how this method works. The system adapts to the subject's progress based on their sEMG signal, the velocity, and force data he or she can generate throughout the training sessions. The EMG threshold value gradually increases, and some resistance force is gradually added to the system once the driving muscle can generate stronger EMG signals. The performance of the subject improves. The initial threshold (buffer) was calculated based on the EMG collected during the baseline (robot-off) phase. The threshold was calculated as 20% of the maximum EMG entropy amplitude. This threshold was adapted for each repetition during the training (robot-on) phase. This strategy was used to encourage the user's effort throughout the entire rehabilitation process. This way, the device does not provide assistance if not needed and provides only as much as needed.
At the beginning of the session, baseline data were collected, and velocity and performance parameters were collected. The performance parameters were also calculated for each move, letter A to letter B being one move, B to C another move, and so on, similar to the process proposed by Krebs et al. (28). The first performance parameter is the average deviation from the trajectory measured as follows: where ( , ) is the target position and ( , ) is the handle position, and N the number of samples.
Therefore, 1 represents the average of the distance between the PHANTOM handle and the target along the trajectory. If a subject takes longer to start moving and "wobbles" around the shortest linear distance between the handle and the target, 1 increases. On the other hand, a quick and linear movement from the handle position to the target yields a lower value of 1 . Figure 2 shows an example of the measurements along a trajectory with the subject moving the handle from letter A to letter B. The second performance measure is the root-mean-square deviation along with the normal distance to the target axis measured as following: where is the normal distance to the target axis as defined above. The coordinates ( , ) represent the position of the letter of origin, ( , ) is the coordinates for the target letter, and ( , ) is the handle position. Therefore, 2 represents the summation of the distance between the PHANTOM handle and the line from the previous target to the next target. In this equation, the reference is the smallest distance between the two targets. Hence, if the patient has high dexterity and coordination, 2 is small; otherwise, as shown in Figure 3, the 2 increases. Figure 4.3 represents the subject moving the handle from letter A to letter B.

Participants
Five chronic post-stroke patients were enrolled in this study. This sample size is consistent with many previous studies where robot-assisted therapy's performance was evaluated (21-25). Before the experiment, each subject was informed of the study requirements and was asked to provide written informed consent. Both sexes were included in this study. The stroke must have occurred at least six months ago. Participants had to be able to actively move their shoulder, elbow, and wrist, as determined by an occupational physiotherapist. Excluded were those who had: history of severe neurological injuries other than (spinal cord injury or stroke); severe concurrent medical diseases: infections, circulatory, heart or lung, pressure sores; problematic spasticity (e.g.,   The table was a transparent plexiglass acrylic sheet on top of a monitor mounted to a metal structure. The monitor showed the letters, as shown in Figure 5. The patient was asked to hold the robotic arm's end effector and move until it touches the letters. They were able to feel when they reached the letter through a haptic force (touch effect) programmed in the robot. One letter was shown at a time. Prior to data collection, MAS and FMA scores were acquired. The patients were also asked to inform the activities they were involved in the previous weeks that involved any physical activity, using the ARM-A questionnaire.
The next step was to set up the EEG and EMG electrodes. The subject was asked to wear a cap where the EEG electrodes were placed, as shown in Figure 6. The EEG electrodes required gel between the sensor and scalp to reduce the impedance. This procedure usually took around 15 minutes. The EMG sensors were placed directly on the participant's skin with a sensor adhesive interface. Nine EEG electrodes and six EMG electrodes were placed on the subject.  The letter positions can be seen in Figure 8. After that, letters appeared in a sequence which the subject had to reach. For example, letter A shows first, then letter F, so the subject simulates a reaching forward task with the elbow half extended. Next, letter B appears, then letter H simulating lateral reaching with elbow half extended, and so on. The participant is given 5 seconds to reach the target without the assistance of the robot.
During this baseline test, if they are unable to get to the target within 5 seconds, the target disappears, and the next target shows up on the screen.
The baseline phase was used to collect information such as ROM, velocity, force, and electromyography signal (EMG). No assistance was provided from the robot during the baseline phase.
For the following training protocols, the patient had to repeat the same movements as the baseline phase. However, this time, the letters appeared on the screen randomly. The letters appear 30 times for training tests one, two, and three, and 60 times for training tests four and five, with an interval between them of 5 seconds. The subjects had 60 seconds to rest between the tests. In this part of the training, the robot-arm was assisting the participants when needed; if the patient could not reach the target, the robot-assisted, the patient in getting to the target. The robot was continuously evaluating the participant's EMG signal throughout the entire trajectory to achieve the target and only assists if the EMG signal is still detectable and there is no movement. The robot also applied some resistance to challenge the participant if both the velocity and the EMG were higher than the EMG and velocity threshold. The EMG signal threshold necessary for the robot to assist/resist increased based on the participant's improvement.

Data analysis
This study collected EEG, sEMG, and performance data in order to analyze the effectiveness of the novel AAN algorithm. The sEMG signal was used as an input for the control system and was also used to assess the training effects in conjunction with the performance parameters and EEG data. Internal validity of the system's responses was evaluated by using curve fitting methods to assess the relationship between the participant's FMA scale and performance metrics (average velocity, parameter 1 and 2 ), EMG entropy and EEG power density. Details of the analyses follow.
The performance parameters were first normalized by dividing the mean P1 and mean P2 by the assistance-resistance force (positive/negative force) provided by the robot during training.
To calculate the mean EMG entropy, we created an index based on the MAS summation, which gave us a normalized spasticity score from 1 to 10 and used to normalize the overall mean EMG entropy for every participant. This variable was called Mean Entropy, which represents the spasticity normalized mean entropy.
The EEG signal was first band-pass filtered from 1 to 30 Hz and visible artefacts were removed using an independent component analysis procedure to calculate the EEG spectra.
Following this, the data were processed using a common average reference, and the data were divided into rest and active. The active and rest data were then divided into epochs of 2 seconds using a Hamming-window, and the power spectral density was calculated using the fast Fourier transform (FFT). The power spectral density was then averaged for each condition.
To answer research questions 1 and 2, EMG and EEG measurements were compared between the robot off (baseline) and on (training) conditions. To answer research questions 3 through 5, curve fitting was used to quantify the relationship between functional status (via the FMA score) and training outcomes measures: performance metrics, EMG, and EEG. Polynomials and rationales were used to find the simplest equation that best fit the experimental data. These analyses will provide internal validity whereby performance parameters, muscle and brain signals, are expected to vary with the user's functional status, thereby revealing the system's sensitivity.

Results
To demonstrate the movement task and its relative difficulty for the participants to perform, Figure 9 shows the hand horizontal trajectory for two participants: the highest functioning (FMA=65) and lowest functioning (FMA=36) during a task that required reaching a letter target and returning to the start letter. Figure 9: a-Trajectory Y-X from the subject with FMA score of 65; b-Trajectory Y-X from the subject with FMA score of 36.
As can be seen in Figure 9, the higher functioning participant moved smoothly between targets with only moderate difficulty returning to the start point. In contrast, the lower functioning participant had difficulty reaching the target and often did not return to the origin before having to move to the next target.
This demonstrates that the population sample was appropriate for the study. Below we present the results of testing the algorithm's effect on the main outcomes measures and their relationships with (sensitivity to) functional status.

Effect of the aAAN algorithm
Research question 1 tested if EMG increases when using the aAAN algorithm in training compared to baseline (with robot off). The EMG entropy for each of the six muscles for the high and low functioning participants can be seen in Figure 10.  Research question 2 was to test if EEG activity increases when using the aAAN algorithm during training relative to baseline. Brain activity maps using the EEG data can be seen in Figure   12 for baseline (robot off) and training (robot on). Although there was no statistically significant change in EEG spectral density between baseline and training (t=1.18, df=4, p=0.31), the difference trended toward increased EEG activity.

Relationships to Functional Status
Research question 3 evaluated the relationship between functional status (FMA score) and normalized performance metrics P1 and P2. Based on the equation used to calculate the performance parameters P1 and P2, lower values mean better dexterity and coordination. However, when dividing this value by the assistance-resistance force, a higher P1norm and P2norm values represent better performance. This happens because the assistance-resistance force goes from -6 to +6, where a negative force represents robot resistance, and a positive represents assistance.
The results showed that performance parameter mean P1 could be predicted from the clinical FMA score by following the formula from general model rational with coefficients (with 95% confidence bounds) and 2 = 0.9955: The fitted curve for the parameter FMA and mean P1 is shown in Figure 13. The shaded region shows P1norm is sensitive to functional status in the lower FMA score region. The fitted curve for the parameter FMA and mean P2 is shown in Figure 14. The shaded region shows P2norm is also sensitive to functional status in the lower FMA score region. The fitted curve for the parameter FMA and average velocity is shown in Figure 15.
Conversely, the shaded region shows velocity is sensitive to functional status in the higher FMA score region. The fitted curve for normalized EMG entropy versus FMA is shown in Figure 16.
Likewise, the shaded region shows EMG Mean Entropy is sensitive to functional status in the higher FMA score region The fitted curve for the parameter FMA and EEG activity is shown in Figure 17. As above, the shaded region shows EEG spectral density is sensitive to functional status in the higher FMA score region. The shaded region shows EEG spectral density is sensitive to functional status in the higher FMA score region.

Discussion
This paper describes the development and validation of a new EMG-based adaptive AAN algorithm that adapts to user requirements in real-time based on performance and EMG entropy, providing assistance and resistance as needed to improve robot-assisted rehabilitation for the poststroke population. To date, no study has done a comprehensive analysis to validate the sensitivity of an aAAN algorithm and method for delivering upper-extremity rehab training to patients with chronic hemiplegia post-stroke.
The algorithm developed in this study used an online adaptive control system based on physiological measures, specifically intending, to avoid slacking behavior and improve engagement during training. The robot was used to assist and challenge the patient during treatment, keeping them engaged throughout the entire training. To better understand the potential of this new method to impact people with stroke, we first compared EMG and EEG between the robot off and on states to test whether training with the aAAN algorithm increases peripheral and central activity, respectively, and; we evaluated how sensitive the performance and electrophysiological outcome measures were to the user's functional status.
Overall, the behavior of the system and how it challenged participants based on their functional status (as shown in Figure 4.6) was consistent with results presented by Krebs et al. (28) and Benitez (4). Although our sample size was small, based on these findings, we are confident the post-stroke sample we recruited represented a sufficiently broad range to evaluate the behavior of the new aAAN algorithm.
Results of research questions 1 and 2 showed us that we achieved the aim of avoiding slacking by providing a challenge at all times, which can be seen in Figure 4.7 where the muscle activity increased during training (robot on) when compared to the baseline (robot off) even for the subject with a low score, showing that the slacking behavior is not happening even when the robot is assisting the subject as expected. This contradicts the results presented by Dipietro et al. (29), where the EMG signal was also used to trigger the robot, and their results showed that when the robot was on, the EMG amplitude reduced for all subjects. The authors explain this decrease in EMG as a validation that their robot was assisting the subject in completing the task, but it also indicates that slacking might occur. In contrast, our results showed that slacking never occurred (see Figure 4.8) when the robot was assisting, which was accomplished by the algorithm adding sufficient challenge when performance was good.
Another relevant finding was that higher brain activity was observed during training for some subjects (Figure 4.9), which is expected when more effort is required from the participant (13,14,17). Although there was only a small change for some subjects, resulting in a statistically insignificant difference, the trend was increasing. This represents that the participant had to do more work when the robot was on, which may infer that the robot's use increases the engagement needed from the subject, potentially improving neuroplasticity in the long term.
Research questions 3 through 5 showed there was a strong relationship between functional status and measurement outcomes, but the relationship was not a simple straight-line linear relationship; rather, it was always a higher-order equation that best fit the data, as seen in Performance parameters P1norm (Fig. 4.10) and P2norm (Fig. 4.11) were more sensitive to change in participants with lower FMA scores, suggesting that these parameters may be more important to monitor subjects with more severe limitations at an earlier stage of motor recovery.
Conversely, the results for velocity (Fig. 4.12), mean EMG entropy (Fig. 4.13), and EEG spectral density ( Fig. 4.14) showed that these outcomes were more sensitive to change in participants with higher functional status.
Based on these findings, it may be hypothesized that participants with lower functional status may be expected to improve more in their performance measures before showing significant changes in muscle and brain activity during a longitudinal study of repeated training sessions. On the other hand, participants with higher functional status would have little performance improvement (already performing quite well). Still, they would show higher muscle and brain activity changes due to the increased challenge introduced by the aAAN algorithm for these patients. Future studies will be needed to verify if the novel control system can accelerate motor recovery over time.

Conclusion
This paper presented a new EMG-based adaptive AAN algorithm that was tested in five stroke subjects across a range of FMA scores from low UE sensory-motor function to high UE sensory-motor function during one session. A comprehensive analysis was performed to understand the algorithm's effectiveness regarding the brain, muscle activity, and performance improvement. Unlike other adaptive AAN algorithms, this new algorithm adapts in real time to the participant's needs based on their performance and EMG entropy, providing assistance and resistance as needed. The preliminary results were very promising; slacking was avoided for all participants during training with the aAAN controller. The system's sensitivity made sense from a motor learning perspective that the clinician can exploit to direct and monitor rehabilitation progress. Future studies with a larger sample size will be necessary to confirm the effectiveness of the new algorithm. Moreover, a complete treatment should be accomplished with weekly training sessions for a long period to understand the effects of the use of this new algorithm on a long-term basis.