Effects of task-specic virtual reality training using a Leap Motion Controller on hemiparetic upper extremity in patients with stroke

Background: Task-specic training has been proven to be effective in promoting recovery of the hemiparetic upper extremities after a stroke. This study was to develop a new and innovative task-specic VR (TS-VR) program using a Leap Motion Controller VR device and the Unity3D program for distal hand function training, and to investigate whether a two-week program of TS-VR training would promote recovery of the hemiparetic upper extremity in patients with chronic stroke. Methods: We designed the TS-VR program based on seven general hand function tasks used in the activities of daily living that require upper limb movement, such as pushing open a door and pouring water. Then, we examined the content validity of the TS-VR according to the views of an expert panel and through eld testing on patients with stroke. The nal version of the TS-VR was tested on 20 patients suffering from chronic stroke with upper extremity hemiparesis over 2 weeks, 5 sessions per week, 30 minutes per session. Outcomes were assessed using the Fugl-Meyer Assessment-Upper Extremity score (FMA-UE), the Wolf Motor Function Test (WMFT), and the Motor Activity Log (MAL). Data were collected at the rst session (week 0), last session (week 2), and follow-up session (week 5). Patients’ arm impairments were stratied into lower- and higher-functioning groups according to the Functional Test for the Hemiplegic Upper Extremity (FTHUE). Results: Signicant improvements in upper extremity functions were found after TS-VR training in FMA-UE Total score (χ2=34.219, p=0.000), FMA-UL subscore (χ2=31.2, p=0.000), FMA-Hand subscore (χ2=22.6, p=0.000), and WFMT score (χ2=27.8, p=0.000) among the three time occasions, but no signicant effect on grip strength was found. Moreover, the higher-functioning group (levels 5-7 in FTHUE) beneted more from the TS-VR, as indicated in outcome measures of FMA-UL, FMA-Hand, FMA-UE Total, and WMFT respectively, as well as amount of use score in MAL, but this was not the case for those in the lower-functioning group (levels 1-4 in FTHUE). Conclusions: Our ndings suggest that our new TS-VR training system was useful for upper extremity recovery in patients with chronic stroke. It has potential to be applied in clinical settings in future. Post-hoc analysis using Multiple Wilcoxon signed ranks tests with Bonferroni correction (p ≦ 0.05/3) revealed that there were signicant differences (p=0.000) in (1, 2) of FMA-UE Total score, FMA-UL subscore, FMA-Hand subscore, and WFMT respectively. There were also signicant differences (p=0.000) in (1, 3) of FMA-Total, FMA-UL, FMA-Hand, and WFMT. However, there were no signicant differences in (2, 3) of FMA-UE Total score (p=0.231), FMA-UL subscore (p=0.536), FMA-Hand subscore (p=0.076), or WFMT (p=0.405). The results imply that TS-VR promotes improvements in affected hand functional performance of individuals with stroke after 10 sessions of treatment. There was still a sustaining positive impact on the functioning of individuals’ affected hand after the three-week follow up when compared to their baseline performance, and patients also maintained their performance post-training across time.


Introduction
Upper extremity hemiparesis is one of the major concerns for patients who have suffered a stroke following their discharge from hospital. Over 60% of chronic stroke patients have motor dysfunction in their upper extremities, with only 5% demonstrating complete functional recovery [1]. Among them, only 5-20% of acute stroke patients regained full function, while 33-60% continued to have limited function after 6 months [2].
According to the current international clinical guidelines for stroke management, intensive and frequent practice of task-speci c training in the activities of daily living (ADL) has been proven to be effective in promoting recovery of the upper extremities following a stroke [3]. Many research ndings have indicated that the best way to relearn a given functional task in ADL is to train speci cally for that task through repetitive and consistent practice [4]. Task-speci c training has the bene ts of promoting cortical reorganization and associated functional improvements, hence, the rehabilitation outcomes are more successful when the tasks are meaningful to the person [5]. Previous studies have found that taskspeci c upper limb training can lead to signi cant improvements in motor function and daily use of the hemiparetic limb among stroke patients [6,7]. There is also an increasing trend for task-speci c training to move away from traditional approaches that uses objects in ADL towards the use of novel technology so as to meet the evolving needs of stroke patient care [8].
Virtual reality is nowadays widely used as part of rehabilitation because of its ability to bridge the gap between laboratory training and the real-life tasks involved in daily living. Virtual reality (VR) is a technology that allows people to view, navigate, and interact with a simulated three-dimensional world in real time. VR comes with different kinds of interactivity according to different immersion levels. Two of its advantages are that it poses no threat to or physical limitations upon patients in the simulated environment, and it can easily be modi ed to change levels of di culty, which may not be possible in the real world [9]. It also has advantages over normal computer-based rehabilitation programs. It can address real-time aspects of information processing, enhance dynamic interaction with the virtual environment, and promote patients' intention and willingness to do training [10]. Moreover, a VR environment allows real-time data to be collected and analyzed, so that therapists can adjust and monitor patients' progress in order to increase the e ciency and effectiveness of rehabilitation [11]. There is mounting evidence showing that VR technology provides therapeutic bene ts in the improving of the motor function of the upper extremities during stroke rehabilitation [12]. However, most existing VR-based rehabilitations focus on training of the proximal upper extremity, and there is limited research on its effectiveness on the distal upper extremity [13]. For this reason, it would be useful to develop a taskspeci c VR training program for a hemiplegic hand and examine its effectiveness on patients following a stroke.
The VR system comprises ve classical elements. These are the VR engine; input and output devices; the software and database; the user; and the task. Unity3D is a cross platform game engine used to develop 3D games and applications for computer, consoles, websites, and mobile devices. The input and output devices include traditional Head-Mounted Display (HMD), VR gloves, etc. Recently, HoloLens was developed, providing holographic experiences for the user and allowing interaction with digital content dynamically in real time [14]. These devices, however, are usually expensive and complicated. On the other hand, a lot of VR training focuses on collecting patients' cognitive performance data rather than their physical performance and motion data. The Leap Motion Controller (LMC), shown in Figure 1, is a modern, low-cost, contact-free, and portable input device used for real-time gesture and position tracking of hands [15]. The accuracy of ngertip position tracking is very precise (approximately 0.01 millimeter) [16,17]. Given these bene ts, it was judged that the LMC could best serve as a cost-effective input device for motor recovery training focused on the distal upper extremity.
Recently, Fernández-González et al. [18] designed a video game-based therapy that used the LMC to provide upper limb rehabilitation for patients with Parkinson's disease. The LMC was used to capture the movement of the patient's forearms and hands. Wang et al. [19] explored the effects of a Leap Motionbased virtual reality system on subacute stroke. The trainings were designed based on the pinching, grasping, and individuating motor skills of ngers. Home-based training is particularly important for patients after hospital discharge, allowing them to practice within a familiar environment, as well as better enabling them to recall their memories during daily practices. In this study, we sought to substantiate an innovative treatment that involves promoting the use of the hemiparetic upper extremity in chronic stroke patients by developing a VR task-speci c training program with three stages. In stage 1, we aimed to design a new and innovative task-speci c VR program (TS-VR) using Unity3D and LMC as input devices. The program consists of seven general hand function tasks involved in ADL. In stage 2, we examined the content validity of the TS-VR according to the views of an expert panel and through eld testing on patients with stroke. In stage 3, we aimed to recruit patients with stroke to participate in our two-week TS-VR training program. Our purpose was to investigate its clinical utility. In particular, we wanted to determine whether the lower functioning or the higher functioning group of clients with paretic upper extremity functional impairment would bene t more from the TS-VR program. The ndings of this study could further contribute to the effective care of those with stroke by helping to extend the application of TS-VR to home-based training, as well as to rehabilitation in hospitals and community settings.

Participants
Twenty patients with stroke were recruited from within a community self-help group for stroke, using convenience sampling. The inclusion criteria were: (1) stroke with unilateral hemispherical involvement; (2) aged 18 or above; (3) stroke onset of six months or more before the study; (4) able to understand verbal instructions and follow one-step commands; (5) no excessive spasticity, de ned as a score of 2 or higher on the Modi ed Ashworth Scale [20]; (6) no complaints of excessive pain or swelling over the paretic upper extremity; (7) no prior participation in any experimental or drug studies within one month; and (8) able to understand the meaning of the study and give informed consent to participate. The choice of a post-stroke period of six months or above as an inclusion criterion for patients with chronic stroke re ected the probability that any spontaneous recovery would have slowed down by the time of the study, thus enabling more brain reorganization in response to the therapeutic intervention under study. Before commencement of the study, written and informed consent was obtained from all patients. The study was carried out in accordance with the principles of the Declaration of Helsinki. Ethical approval was sought and obtained from the Human Ethics Committee of the Hong Kong Polytechnic University (Ref. no.: HSEARS20180503002). In order to investigate which level of severity of upper extremity impairment would bene t more from the treatment, patients were further strati ed into two groups according to whether their hemiparetic upper extremity functional impairment was severe to moderate or mild: Group 1, the lower functioning group, included eight patients with severe to moderate impairments to a hemiparetic upper extremity (i.e., FTHUE level 1 -4) who were able to move with bilateral arms, to demonstrate a mass exion pattern in the shoulder/elbow, then an active extension control of the shoulder/elbow; Group 2, the higher functioning group, included 12 patients with moderate to mild hemiparetic upper extremity impairments (i.e., FTHUE level ≧ 5) who were able to demonstrate the beginnings of an ability to combine components of strong mass exion and strong mass extension patterns, and who were able to perform some release of the hand, as well as isolated control of forearm and hand with fair strength.
Task-speci c virtual reality training program (TS-VR) In stage 1, we developed the TS-VR using Unity3D (version 2017.3.1f1) and the LMC (version 3. Unity3D is a cross-platform game engine that has been widely adopted by various industries for the development of VR programs, augmented reality (AR) games, computer simulations, and other experiences. Figure 3 shows the work ow of the development process for the TS-VR program. The 3D virtual environment and models are rst created using SolidWorks. The models are then exported to FBX format and imported to the gaming engine. The imported models are usually gray in color, and the Unity engine supports and provides a physically based rendering material system to create realistic textures and color in various virtual scenes. Custom textures like wood and bricks can be added to the 3D models. The engine also provides a user-friendly interface for the adjusting of material values and parameters, such as "Metallic" and "Smoothness". This enables the creation of a natural virtual environment under different lighting in real time.
The LMC is used to capture the 3D position of the hand, including the wrist and ngers. The LMC has a one-meter hemispherical view area with a 150 degrees eld of view, and it can track users' nger movements at a rate of over 300 frames per second, such that the screen display changes continually along with the user's every movement. The LMC uses structured infrared light emitters to create a 3D depth map with two cameras. The cameras capture data in the form of grayscale stereo images and process this data by extracting tracking information such as the position of wrist and ngers. The tracking algorithms then interpret the 3D data and infer the positions of the hand [23]. Using the LMC, the feature points in the wrist and ngers are extracted to interpolate the motion of the hand.
The LMC works using the Cartesian coordinate system in millimeters (mm). At the beginning, the origin is located at the middle of the device. These coordinates are converted to screen coordinates depending on the screen size or window size of the application, calculated by Equation 1 [24]: , where Vx and Vy are the xand y-coordinates with respect to (w.r.t.) the screen coordinates; Ux and Uy are the width and height of the window; and Ox and Oy are the xand y-coordinates w.r.t. the LMC coordinates.
To train the Hemiplegic Upper Extremity functions of the patients, interactions between different virtual models in the TS-VR training program are required. For instance, when patients push a door or press the password pad, the door will open and the password will display on the screen. In this case, a collider de ned by common primitives in the space of the virtual objects is used to determine the interaction with the patients. A collider is a bounding box (Figure 3) surrounding the virtual models. An interaction is triggered when the distance between the models and the nger is smaller than the distance threshold. Figure 1b shows the pseudo code of the pressing password pad task. In this task, the password is randomly generated. When the virtual hand model triggers the collider, the corresponding password number will be recorded and compared with the generated password. The task ends when the password is correct; otherwise, this task will restart. Similarly, other tasks, such as holding a handbag or placing an object also need to trigger the collider, and then a check is done to determine whether the task has been completed.

Rehabilitation and assessment
In stage 2, we sought qualitative comments for the TS-VR from an expert panel, composed of 10 occupational therapists, each with at least three years of experience in stroke rehabilitation. The panel interviewed 10 clients with stroke of various impairment levels recruited by convenience sampling in the community. After the interview, object component, program structure, and demonstration videos of the program were modi ed according to the comments of the panel, but the VR tasks remained similar in terms of content and di culty.
In stage 3, we investigated whether a two-week program using TS-VR training would promote recovery of the paretic hand in patients with chronic stroke, using a single group longitudinal design study. Twenty patients were invited to participate in the TS-VR at the university laboratory. There were a total of 10 sessions, each 30 minutes in length, held ve days per week over two consecutive weeks.
The equipment was set up as shown in Figure 1. The LMC was installed on the table located in front of the monitor. The patients were seated in front of the computer and provided with enough space for them to move the upper extremity in order to complete the TS-VR tasks. The tasks were displayed on the computer monitor in front of the patients. The study was carried out under the one-to-one supervision of an occupational therapy student who was one of the investigators and had undergone training in using the TS-VR. Patients were instructed to sit on the chair, or else on their own wheelchair. The LMC was placed on another chair, in front of the participant and at an appropriate distance to ensure best sensitivity. In the rst session, demonstrations were given by the investigator for each level of the TS-VR training tasks to ensure that patients understood how to perform the task correctly. In the following training sessions, the training regimens were tailored to the needs of the patients by adjusting the modes of training (i.e., the progressive and repetitive mode, respectively). For progressive mode, patients were instructed to perform the TS-VR tasks from level 1 to the highest level they could attain. For repetitive mode, patients were instructed to perform a speci c task level repetitively for 5-10 minutes. Both progressive mode and repetitive mode were used in each 30 minutes training session, depending on patients' needs, performance, and fatigue level. Patients with lower functioning (FTHUE of 1-4) were allowed to perform the TS-VR training task level 1 or 2 with the assistance of the non-affected hand.
At the beginning, during, and after each training session, patients were allowed to take a rest and do stretching exercises to relieve any tightness, tiredness, or discomfort of their paretic upper extremity. During the training, verbal and tactile cues were given by the investigator to correct patients' abnormal posture and associated movement when they were performing the TS-VR tasks. Visual and audio cues which were built-in to the TS-VR also provided sensory feedbacks to the patients. The time and error records after each completion were also recorded in order to provide immediate feedbacks to the patients.

Outcome measurements
Outcome measurements were conducted at baseline (pre-treatment, rst session, week 0), post-treatment (last session, week 2), and follow-up (week 5) at the university laboratory. The primary outcomes were two laboratory-based assessments for arm functions, including the Fugl-Meyer Assessment-Upper Extremity score (FMA-UE) [25,26] and the Wolf Motor Function Test (WMFT) [27]. The FMA-UE [28] is a well-known impairment test which evaluates patients with stroke through selective movements. The upper limb subscore consists of 32 items, each of which represents movement components, scored on a three-point scale (0, 1, 2). A higher score denotes a better recovery. The WMFT was chosen because the score range of this test is relatively greater and therefore more suitable for capturing both higher and lower levels of arm functioning in stroke patients [29]. The secondary outcome was a self-reported questionnaire, the Motor Activity Log (MAL) (including the Amount of Use scale (AOU) and the Quality of Movement scale (QOM)), indicating how often and how well patients used their affected arm in ADL [30]. It was administered for the actual arm use of the affected hand in daily life. It was carried out by asking patients about the use of their hemiparetic upper limb in 30 daily living activities over one week. Both subscales in the MAL -the AOU and the QOM -were shown to the patients while going through each item over the course of a semi-structured interview.

Statistical analysis
Data analysis was performed using SPSS. Baselines, including demographic data and pre-treatment scores for the outcome measures, were calculated according to descriptive statistics. Nonparametric tests were used because the data collected was either categorical or because sample size was small. The Friedman Test was used to evaluate any signi cant difference in the hand functions and arm use of patients before and immediately after the training and at within-group follow-up sessions. The Wilcoxon Signed-Rank Test was used for post hoc analysis, in which the measurements taken for each variable on the three occasions (i.e., before and immediately after the training, and at follow-up) were compared to determine where any signi cant differences lay. The level of signi cance was set at p ≦ 0.05 for the Friedman Test after Bonferroni adjustment (0.05/n; n = number of tests). The Friedman Test and the Wilcoxon Signed-Rank Test were later used for separate analyses when the patients were strati ed into lower-functioning and higher-functioning groups according to the severity of their hemiparetic arm functional impairment.

Results
Patients' demographic characteristics are listed in Table 1. Tables 2 shows the results of primary and secondary outcome measures at the baseline, post-test, and three-week follow-up evaluations for the whole sample and the higher or lower function groups.

Effects on Hand Functions
Twenty patients with stroke were recruited. There were signi cant differences among the three times of evaluation -pre-test (1), post-test (2), and three-week follow-up (3)    Our study found that the higher-functioning group bene ted more from the TS-VR, as indicated in outcome measures of FMA-UL, FMA-Hand, FMA-UE Total, and WMFT respectively, as well as AOU score in MAL. Compared with the higher-functioning group, patients in the lower-functioning group only participated in certain lower levels with limited use of distal joints and muscles with lower speed, whereas patients in the higher-functioning group could perform a greater variety of movements, with the involvement of greater use of different joints and muscles in upper extremity from proximal to distal, especially hand and isolated nger movement. Moreover, the nature of our TS-VR training is durationxed (30 minutes) but not repetitive or interval-xed. Patients in the higher-functioning group were able to perform higher repetitions because of their faster rate of performance. This result highlights the signi cance of the design and nature of TS-VR training in achieving differential bene ts to stroke patients with various levels of arm severity. Further research on training duration, repetition, and frequency of the TS-VR program would bring out signi cance differences in the bene ts for higher-and lower-functioning groups.
The signi cant increase in MAL-QOM score after TS-VR training in the higher-functioning group but not in the lower-functioning group could possibly be explained by their different functional levels. Following completion of the TS-VR program, the overall upper extremity function increased in both groups. However, in the lower-functioning group, the improvement might only be seen in the proximal joint movement (i.e., increased shoulder exion, external rotation, and elbow extension). The improvements were insu cient for hand functions in the MAL questionnaire, like holding a cup, opening a drawer, or eating with a spoon., Hence, the overall quality of movement in these tasks remained similar after the TS-VR program. The grip strength of both groups showed no signi cant improvement after the course of TS-VR training. This is understandable as the TS-VR training did not include any strengthening element in any of its levels but only movement control and range of motion.
One of the reasons for the signi cant improvements after TS-VR training might be attributed to training with high frequency and appropriate intensity. Our treatment program consisted of a total of 10 sessions, each lasting 30 minutes, held ve days per week over two consecutive weeks. Previous research has revealed that the intensity of task-speci c trainings need not necessarily be high in order to achieve optimal effects. Training sessions that are as short as 15 minutes have resulted in lasting changes in cortical representation [5]. Our ndings are consistent with those from numerous previous studies [5][6][7], which have suggested that less intense (i.e., 30-45 minutes) task-speci c training regimens can lead to signi cant improvements in the use and functions of the affected limb.
We conducted our training on a one-to-one basis with supervised therapy. During each supervised session, the facilitator selected activities that were suitable to the functional level of the patients and determined the repetitions of activities to be performed. Moreover, the facilitator provided speci c and ongoing verbal and tactile cues on participants' performance. This extrinsic and instant feedback on movement quality can effectively correct patients' unwanted or compensatory movement patterns, which is also a key principle in skill acquisition [31]. In addition, the 3D and graphically 'enriched' environments with multimodal feedback (visual, auditory, tactile) in VR settings also provided sensory input to the patients, which made them more motivated to attend the training than they would have been to attend conventional training, which is less playful and interactive.
It might also be possible to explain improvements in MAL as resulting from one of the unique features of our VR training program, that is, it's adoption of task-speci city in a progressive manner. In order to be 'task speci c', each action in the training tasks must specify an object to reach for. The tasks are simulating conditions in real life in order to enhance carry-over into daily life activities (Cunningham et al., 2016). Our TS-VR program is composed of seven functional tasks according to seven ordered levels of increasing complexity. The complexity of the tasks is related to the Brunnstrom theory of development of the hemiparetic upper limb in recovery, with reference to the respective key actions corresponding to levels 1-7 in the FTHUE [21,22,30,31]. Our study, to our knowledge, is the rst one incorporating taskspeci city with increasing levels of complexity into VR training for the hemiparetic upper extremity. The results may be useful as supporting evidence for integrating task-speci city and virtual reality into programs for the treatment of upper limb motor impairments during stroke rehabilitation.
In our study, we attempted to explore the effectiveness of TS-VR rehabilitation that focused on the distal upper extremity up to the forearm, using the LMC as an input device. The advantage of the LMC is that it is cheap and easily set up, requiring only a desktop computer or laptop, the LMC, and our TS-VR training program. There are also existing studies focused more on proximal upper limb rehabilitation [13]. Kinect, which is a widely commercially available motion sensing device, recognizes the body's joint positions, especially for proximal joints like shoulder and elbow. However, VR systems like Kinect do not bene t the distal joints, which are important in hand function training such as grasp and release. To address this limitation, we used the LMC in our TS-VR program, as this motion sensor is able to capture the motion of wrist and of each of the nger joints, enabling it to accurately measure ne motor movements [15]. To further develop our TS-VR program in the future, we suggest combining the use of Kinect and the LMC, as there are currently no VR systems involving both proximal and distal upper extremity movements at the same time [32]. Further research to explore the effectiveness of the combined use of devices that can measure both gross motor and ne motor abilities would be useful is recommended as a direction for future investigations.

Conclusions
This study showed that our newly designed TS-VR training system was useful and led to signi cant improvements in the upper extremity functions of patients with chronic hemiparetic stroke after a 10session training program. TS-VR is user-friendly and simulates daily hand function tasks, with levels of di culty customized for different patients, thus motivating them to perform similar tasks in real life. When undertaken with high frequency and at appropriate intensity, it is hoped that this low cost and innovative training program will provide signi cant bene ts to people with stroke in the future.  Example of pseudo code of the pressing password task. Pseudo code of the pressing password task in Figure 2(d) -The program starts by generating a 4-digit random password. All game objects in the virtual scenes will be searched. If the game object is a password pad and the collider is triggered, then the corresponding digits of the input number will be saved. Finally, the input number will be checked to see if it matches with the random password. The task will end if the password is matched; otherwise, the task will restart.