A Deep Neural Network for Gait Classification Based on Inertial Sensors in Post-Stroke Patients


 Background: Stroke survivors usually experience partial disability, due to abnormal gaits, which vary widely and require tailored rehabilitation programs. However, most gait classifications are based mainly on clinical assessments, which can be influenced by the therapist’s experience. Inertial measurement units (IMUs) are devices that combine accelerometers and gyroscopes to detect movement. IMUs have been successfully used for assessing gait characteristics. Here, we aimed to develop a Deep Neural Network (DNN) model that incorporated information from a motion capture system and multi-labeling IMUs information. This DNN was developed to recognize individual gait patterns in patients affected by stroke to facilitate the design of suitable rehabilitation strategies and promote functional recovery.Methods: We recruited ten patients, aged 20–75 years, with a first-ever, unilateral, ischemic stroke, which caused mild to moderate leg paresis 4 weeks after stroke and ten neurologically normal healthy controls. We applied a motion capture system integrated with multi-label IMUs to acquire the gait information. The motion capture system measured gait information by detecting movement of LED markers attached to each participant. In addition, the IMUs were attached to each participant’s lower limbs to measure kinematic data. These measurements were then applied to the development of a DNN model that could recognize gait characteristics in patients after a stroke and in normal controls.Results: The DNN model achieved an average accuracy of 98.28% in differentiating the stroke gait from the normal gait. Among patients with stroke, the DNN model had an average accuracy of 96.86% in classifying the gait abnormality as either a drop-foot gait or a circumduction gait. We also applied a publicly available dataset, the Physical Activity Monitoring Data Set, which contained IMU information from another independent set of participants to validate our DNN model. We found an average accuracy of 98.60%.Conclusions: We developed a DNN model based on integrated information from a motion capture system and multi-label IMU inputs. This model might assist clinicians and therapists in identifying abnormal gaits more accurately and in applying suitable training programs within the “golden time window” of rehabilitation, after the onset of stroke.


Introduction
Stroke is a common medical emergency with a high mortality rate; it has ranked second as the leading cause of death in the last 15 years [1]. Patients who survive strokes commonly experience partial disability and inconveniences in their daily lives. Therefore, post-stroke patients usually require health care services and long-term rehabilitation. In the USA, the annual costs associated with managing stroke are about 34 billion US dollars [2]. On average, each stroke patient costs about 60,000 US dollars per year, and 30% of those costs are expended on rehabilitation and medical care [3].
Stroke survivors often have abnormal gaits due to neurological sequelae. These abnormalities include longer swing phases and reduced stance phases on the paretic side of the body. Abnormal post-stroke gaits vary widely and require personalized rehabilitation with therapists. The two most prevalent gait abnormalities observed post-stroke are known as the drop-foot [4,Error! Reference source not found.] and the circumduction gait [5,7]. The drop-foot gait develops when weakness or paralysis in the leg limits the ability to raise the front part of the foot. In the swing phase, the foot plate is unable to perform dorsi-flexion movements, and consequently, the toes are dragged when walking. Several rehabilitation strategies are applied for patients with drop-foot [Error! Reference source not found.]: first, a prosthesis, like braces 5 or splints, is applied to help the patient hold the foot in the normal position; second, physical therapy is applied to strengthen the leg muscles and help the patient maintain adequate range of motion in the knee and ankle. The circumduction gait is also known as a hemiplegic gait and it is caused by weakness in the muscles, including the knee flexors, hip flexors, and dorsiflexors. The rehabilitation programs for patients with a circumduction gait usually require two therapists at the same time: one therapist needs to manually adjust the patient's pelvic motion and weight shifting, and the other therapist concomitantly assists the patient in stepping and controlling the lower limb during the stance and swing phases [8].
Inertial measurement units (IMUs), which include combinations of accelerometers and gyroscopes, can be used to measure the quality and quantity of physical activity in both healthy and pathological populations [10]. They have been successfully used for assessing gait characteristics (i.e., gait spatio-temporal parameters and gait variability). A few previous studies have attempted to design a computer program that could recognize abnormal gaits, based on IMU data, but classification systems for characterizing individual gait patterns are limited. One previous study found significant differences in the durations of gait phases between 10 healthy children and 10 children with hemiplegia, based on IMU data [11].
Due to the variability in gait problems encountered in survivors of stroke, it is 6 important to determine each individual gait abnormality early after a stroke to ensure the timely design of an appropriate training strategy. Therapies applied within a 'golden time window of rehabilitation' improve the functional outcome. However, most current gait training programs are based mainly on clinical assessments, which might be influenced by the therapist's experience.
Many studies have attempted to identify post-stroke walking patterns by taking objective measures in patients [12,13]. However, to date, no studies have been conducted on the classification of stroke gaits. Therefore, the present study aimed to develop a deep learning-based model based on integrated information from a motion capture system and multiple IMUs to determine accurate individual gait patterns in patients that survived a stroke. This information will be useful for designing suitable rehabilitation strategies and improving functional recovery.

Participants
We recruited ten patients aged 20-75 years with a first-ever, unilateral, ischemic stroke that exhibited mild to moderate leg paresis 4 weeks after the stroke. Inclusion criteria were: age> 20 years; stable medical and neurological conditions; Brunnstrom Stages 3-5 [Error! Reference source not found.], assessed on the lower extremity; 7 Functional Ambulation Category [14] stages 3-5; a Mini-Mental State Examination [16] score >24; sufficient cognitive ability to follow the instructions and report any discomfort; the ability to walk 10 m indoors with or without aid devices; the ability to stand up on their own, with a handrail and aids; and physical condition sufficient to complete 3 min of supported walking. Patients were excluded when they had other orthopedic or neurological disorders or cardiac conditions that could be affected by a physical load.
We also recruited ten healthy participants without neurological disorders as the normal reference group. All participants provided signed informed consent. This study was approved by the institutional ethics board committee of the National Taiwan University Hospital.

Data Collection and Processing
We collected gait information from each participant to develop a Deep Neural Network (DNN) model [17] that could recognize individual stroke gaits. A motion capture system [17] and IMUs [19] were applied to participants to acquire gait characteristics. The motion capture system detected LED markers that were attached to test subjects. Gait information was measured at a sampling rate of 100 Hz and a resolution of 0.015 mm, as the subject walked a distance of 1.2 m. We attached the 8 LED markers to the subject's lower limbs, according to the Helen Hayes Marker Set [20]: four on the waist, two on the thighs, two on the knees, two on the calves, two on the ankles, two on the toes, and two on the heels (Figure 1a). The IMUs consisted of a 3-axis accelerometer, a 3-axis gyroscope, and a 3-axis magnetometer. The IMU devices detected kinematic data at a maximum sampling rate of 128 Hz. We attached IMUs to the subjects' calves, to record gait information (Figure 1b). We recorded 3-axis accelerations and 3-axis angular velocities for both legs as the subject walked.
All participants were required to complete two 10-min treadmill walking tests, where they walked at their most comfortable pace. Between the two tests, participants rested for 10 to 15 min to ensure that the second test would not be affected by fatigue.
To develop the DNN model, we recorded the angular velocity of the calf on the sagittal plane [21] (i.e., the y-axis in Figure 1b Gait data collected from patients with strokes are referred to as the "stroke gait" and data from healthy controls are referred to as the "normal gait". Figure 2 shows a patient with abnormal trembles and vibrations in the paretic leg during walking. This characteristic was used in the model to distinguish stoke gaits from normal gaits. The stroke gaits were further classified as a stroke gait with a drop-foot (SGwDF) and a stroke gait with circumduction (SGwC), based on clinical diagnoses from therapists.
Because patients showed variability in abnormal gaits, we created four designations to build a multi-label classification model (Table 1). For example, patient P2 exhibited both a drop-foot and circumduction in the right leg, but patient P1 exhibited only a drop-foot in the left leg. Conversely, patient P10 exhibited neither an obvious circumduction nor a drop-foot, although his gait had been affected by the stroke. We applied the gait data shown in Table 1 to establish a gait dataset with 22,650 gait cycles, including 12,553 stroke gait cycles and 10,097 normal gait cycles (Appendix A).

Model Architecture
We used the normalized gait data to build a multi-output gait recognition model.
The model architecture included a detection part and a classification part (Figure 3). 10 The detection part first judged whether the input gait was a normal gait or a stroke gait. It contained an input layer, six hidden layers, and the detection output. Each fully connected layer included 100 neurons. The detection output included two neurons, which labeled the gait as a normal gait [1,0]  To develop the DNN model, we included three neuron-based functions: the activation function, the loss of function, and the optimizer.
where z was the neuron input and ReLU(z) was the neuron output. This function 11 could effectively overcome the vanishing gradient problem [24]; i.e., the neural network would not continue training, when the gradient value was small.
Moreover, the computing load was reduced, because this function judged whether the input was greater than 0. When the input was ≤0, then ReLU(z)=0, and this neuron was directly deleted. Thus, the total number of neurons was reduced, and rapid convergence ensued.
On the other hand, to activate the output layers, we selected the following sigmoid function [25]: where z was the neuron input and (z) was the neuron output. This sigmoid function converted a scalar number to a binary number [0, 1] (Figure 4b). When (z) was above a threshold of 0.5, it was considered to belong to the 1. Because the probability of each neuron was independent, the sigmoid function was mostly used to activate the output layer for multi-label classifications.
2. The Loss Function: This function was applied to evaluate how well the algorithms interpreted the given data. It evaluated the loss (error) of the model and updated the weights to reduce the loss on the next evaluation. We applied the following cross-entropy equation [26] as the loss function: 3. The Optimizer: We selected Adam [28] as the optimizer of the DNN model. Adam is an adaptive learning rate optimization algorithm designed specifically for training DNNs. It combines the advantages of Adagrad [29] and RMSprop [30] by calculating the gradients and updating the weights [27].

Model Training and Validation
We applied the k-fold cross-validation test [31] to evaluate model performance.
We set k=5 by dividing all classes of gait data into five parts (Fold 1, Fold 2, …, Fold 5), and we arranged them randomly for training and validating. Each training run used four of the five folds as a training dataset, and the remaining fold was used for validation. Figure 5 shows the training and validating flow chart. The 5-fold cross-validation was repeated five times. In the training process, 500 samples were selected for each model training run (batch size=500) to update the weights. This 13 training process was repeated 60 times (Epochs = 60). The phenomena of overfitting and excessive time in the training process were avoided by adding a Dropout [32], with a dropout rate of 0.2, to each fully connected layer in the classification part of the model. Consequently, each neuron had a 20% probability of being deleted.
We applied the Confusion matrix [33] for quantitative fine-tuning and correcting the model ( Table 2). Based on the data in Table 2 (12) where Accuracy was the most intuitive indicator, but it might be invalid in some cases [33]. In this study, we applied Accuracy and the F1-score to demonstrate the quality of the developed DNN model. Furthermore, to validate our developed DNN model, we applied data from a public dataset, known as the Physical Activity Monitoring Data Set (PAMAP2) [34], which stored information from IMUs recorded in an independent group of participants. These data were available from the University of California Irvine machine learning repository [35].

Results
The basic clinical characteristics of the 10 patients with strokes and the 10 healthy controls are shown in Table 3. We applied the gait data from individual participants to establish a gait dataset with 22,650 gait cycles, including 12,553 stroke gait cycles and 10,097 normal gait cycles (Appendix A). We applied a 5-fold cross-validation test  (Table 4). Figure 2 shows that the stoke gaits could be easily distinguished from the normal gaits, because they included abnormal trembles, and they were less Among the data coded as stroke gaits, we also differentiated between drop-foot and circumduction gaits with the classification part of the DNN model. The confusion matrix of the classification layer is shown in Table 5. There, we listed the independent output neurons to observe the errors that occurred in classifying the drop-foot and circumduction gaits. The results showed that the model could effectively distinguish between drop-foot gaits and circumduction gaits. The false positive and false negative rates on the off-diagonal terms in the classification layer (Table 5) were slightly higher than those in the detection layer (Table 4), for differentiating between the normal and stroke gaits.
The validation of the DNN model for detecting and classifying stroke gaits are shown in Table 6. The detection layer achieved an average of accuracy of 98.28% and an F1-score of 0.9846 in differentiating between normal gaits and stroke gaits. The 16 classification layer achieved an accuracy of 96.86% and an F1-score of 0.9716 in differentiating between the SGwDF and the SGwC abnormal stroke gaits. That is, the proposed DNN model could effectively detect stroke gaits and classify the two common gait abnormalities.
With this established DNN model, we further applied the public PAMAP2 dataset, which included nine healthy subjects (1 female and 8 males) that wore IMU devices.
Those subjects performed 12 different activity tests, including standing, sitting, and walking. We applied the angular velocity of the calf in the PAMAP2 walking activities as input data for the five DNN models. The testing results are shown in Table 7. The average accuracy was 98.60% and the average F1-score was 0.9929.
These results suggested that the DNN model we developed could effectively differentiate between stroke gaits and normal gaits, based on the angular velocity data recorded with the IMUs attached to the calves of subjects in the public database.

Discussion
This study developed a DNN model that could detect stroke gaits and classify two common abnormal gaits: the drop-foot gait and the circumduction gait, based on the between normal gaits and pathological gaits, including stroke and choreatic gaits.
They applied the maximum class-specific likelihood evaluation and found an overall accuracy of 66.7% [37]. They improved the accuracy to 73.3% by performing further analyses with Hidden Markov Models that included time and frequency domains [37].
In comparison, our model provided higher accuracy (>98%) in detecting stroke gait and in differentiating between drop-foot and circumduction gait patterns (>96%). The major difference between our study and the Mannini study [37] might be explained by the objectivity of the DNN model. The Mannini study [37] partially relied on human knowledge when applying the supporting vector computation, which limited the performance of gait classification. In contrast, we applied multi-layer algorithms in the DNN model to process the large amount of gait information input from IMUs.
Indeed, the main advantage of the DNN model, compared to its predecessors, was that it could automatically detect important features without human supervision. The DNN was also computationally efficient, due to its convolution and pooling operations and its use of parameter sharing [38] However, our model showed lower accuracy for stroke gait classification than for abnormal gait detection (96% vs. 98%). One possible explanation for this discrepancy could be that the number of stroke gaits in the dataset was not sufficiently large enough to analyze all the inter-and intra-subject gait variabilities that characterized the stroke 19 group.

Consent for publication
Not applicable

Availability of data and materials
The datasets used and analyzed during the current study are available from the corresponding author upon reasonable request.

Competing interests
All authors declare no competing interests.