A comparison of machine learning models’ accuracy in predicting lower-limb joints’ kinematics, kinetics, and muscle forces from wearable sensors

doi:10.21203/rs.3.rs-2083365/v1

Download PDF

Article

A comparison of machine learning models’ accuracy in predicting lower-limb joints’ kinematics, kinetics, and muscle forces from wearable sensors

https://doi.org/10.21203/rs.3.rs-2083365/v1

This work is licensed under a CC BY 4.0 License

Journal Publication

published 28 Mar, 2023

Read the published version in Scientific Reports →

You are reading this latest preprint version

Gait analysis outside the laboratory has been possible by recent advancements in wearable sensors like inertial measurement units (IMUs) and Electromypgraphy (EMG) sensors. The aim of this study was to compare performance of four different non-linear regression machine learning (ML) models to estimate lower-limb joints’ kinematics, kinetics, and muscle forces using IMUs and EMGs’ data. Seventeen healthy volunteers (9F, 28 ± 5 yrs) were asked to walk over-ground for a minimum of 16 trials. For each trial, marker trajectories and three force-plates data were recorded to calculate pelvis, hip, knee, and ankle kinematics and kinetics, and muscle forces (the targets) as well as 7 IMUs and 16 EMGs. The most important features from sensors’ data were extracted using Tsfresh python package and fed into 4 ML models; Artificial Neural Network (ANN), Random Forest (RF), Support Vector Machine (SVM) and Multivariate Adaptive Regression Spline (MARS) for targets’ prediction. The RF model outperformed the other ML models by providing lower prediction errors in all intended targets. This study suggested that a combination of wearable sensors’ data with an RF model is a promising tool to overcome limitations of traditional optical motion capture for 3D gait analysis.

Three-dimension gait analysis (3DGA) provides quantitative information on the locomotion system and lower-limb functionality level in patients with movement disorders. 3DGA has been used in various applications, such as providing valuable information about disease progression, developing patient-specific treatment plans, and monitoring changes after rehabilitation and surgeries. Conventional methods for 3DGA rely on a time-consuming procedure that requires traditional optical motion capture (OMC) systems and force plates in a controlled environment. It has been shown that spatial and temporal constraints in the laboratory may affect human gait [1, 2].

With the emergence of lightweight and inexpensive wearable sensors, collecting human gait data outside the clinic has been made feasible. Inertial Measurement Unit (IMU) and Electromyography (EMG) are two types of wearable sensors that are becoming widely used in 3DGA. IMUs are made of a single electronics module combining three accelerometers and three gyroscopes which respectively collect linear acceleration and angular velocity in 3 dimensions [3]. EMG electrodes are placed on the person’s skin at the muscle of interest belly location and indirectly measure the electrical signals transmitted by motor neurons that cause muscles to contract [4, 5]. Although wearable sensors are very promising in motion analysis, barriers exist to their widespread clinical adaptation. First, the integration of acceleration data to determine the IMUs' position and orientation causes numerical drift errors over time [6]. Second, scaling EMG signal to the patient’s maximum voluntary muscle contraction to determine muscle activation is not always feasible in patients with impaired muscle forces, such as stroke patients or children with Cerebral Palsy [7, 8].

To overcome challenges associated with wearable sensors' data processing limitations, regression-based machine learning (ML) techniques can be used. ML models can establish a direct relationship between wearable sensors' data and intended targets; such as joint kinematics, joint kinetics, and muscle forces in this study. Training an ML model would enable us to predict targets for either 1) a specific patient at different time point/session (intra-subject model) or 2) for additional unseen patients (inter-subject model).

Artificial Neural Network (ANN), Random Forest (RF), and Support Vector Machines (SVM) are powerful ML algorithms that can be used for regression even when non-linearity exists inside the targets. In recent years, a few research groups have implemented these algorithms to estimate gait time series from wearable sensors. Some have looked at joint kinematics [9–19], others at joint kinetics [12, 20–22] but also gait parameters such as stride length, velocity, and toe clearance [23–27]. To date, ANN is the most used ML model to predict joint kinematics and kinetics from IMUs [9, 10, 16, 18, 20, 22, 24, 25, 28, 29], which achieved correlation coefficients higher than 0.86 in previous studies. To the best of our knowledge, there is only one research [30] in which ANN is implemented to predict muscle activations from EMG data to estimate joint kinetics in a forward dynamics model. Unfortunately, they did not determine muscle forces based on the predicted muscle activations. Bolam et al. [31] developed an RF model to predict maximum knee flexion angle and provided a reliable workflow to remotely monitor post-operative progress in knee arthroplasty patients. The superior performance of the RF model compared to other ML algorithms is shown in another study for the estimation of hip, knee, and ankle joint angles in the sagittal plane but did not investigate joint kinetics nor muscle forces [17]. SVM models have been used mainly to predict gait parameters (stride length and width, stride time, and foot clearance) [23, 26, 27] rather than prediction of joint kinematics and kinetics. Multivariate Adaptive Regression Splines (MARS) is another powerful ML method that is an extension of linear models and automatically models non-linearities and interactions between variables. It seems that MARS has not been investigated to predict joint kinematics, kinetics and muscle forces during 3D gait analysis yet.

Although the performance of different ML models has been investigated in some studies [10–31], there is a lack of consensus on which ML algorithm is the most suited for predicting joint kinematics, kinetics, and muscle forces. Most studies focused on specific joint angles or joint moments of the lower limbs in one plane. Furthermore, only few studies used automatic feature extraction and selection to train each ML model [31–33].

Therefore, the aim of this study was two-fold: (1) to extract features automatically and determine the most important features for the estimation of each target and (2) to compare the performance of four non-linear regression ML models (ANN, RF, SVM and MARS) to estimate pelvis, hip, knee, and ankle joint angles, moments, and muscle forces in both intra-subject and inter-subject examinations. To this end, we employed a python package called Tsfresh [34] to extract features from EMGs and IMUs data and developed each ML model by using the most important features. Finally, each ML model’s performance was evaluated based on its’ prediction accuracy.

2.1. Data collection

Seventeen healthy adult volunteers (9F, 28 ± 5 yrs, 1.70 ± 0.08 m, 66 ± 10 kg) were assessed in one session with at least one static, one squat, one squat jump, one heel raise, and sixteen over-ground walking trials. Each participant signed an informed consent form prior to collecting data in accordance with the World Medical Association Declaration of Helsinki (1964 last updated in 2013) and was approved by the University of Auckland (New Zealand) human participant ethics committee (reference number 019911). In each trial, marker trajectories from a 12-camera optical motion capture system (Vicon Motion Systems Ltd., UK), ground reaction forces from three gound embedded force plates (Bertec, Columbus, Ohio), EMG (Mini-Wave, Italy), and IMUs (Vicon IMeasureU Ltd., NZ) were recorded. Twenty-seven reflective markers were placed on participants, as shown in Fig. 1, to determine the three-dimensional position and orientation of each body segment. Sixteen EMG surface electrodes were used to record lower limb muscles' activity on both legs (Gluteus maximus, Rectus femoris, Vastus lateralis, Biceps femoris, Semimembranosus, Medial gastrocnemius, Soleus, and Tibialis anterior). Three dimensional acceleration and angular velocity were recorded from 7 IMUs attached to each segment of the lower limbs (one on the pelvis, one on each foot, shank, and thigh). With the exception of marker trajectories data, which was captured at a sampling frequency of 200 Hz, all data were recorded at 1 kHz. For each participant, maximum voluntary contraction (MVC) of lower limb muscles were also collected. During the MVC collections, the lab operator held the participant’s leg in a fixed position and asked them to move their leg to activate the group of muscles of interest.

2.2. Data post-processing

All captured data was synchronized, and marker trajectories were reconstructed through Vicon Nexus software (Version 2.12). The MOtoNMS [35] (Matlab Motion data elaboration toolbox for neuromusculoskeletal applications) was used to filter 3D marker positions and force plate data (Butterworth 4th order, 10Hz low pass filter) and rotate them according to the OpenSim coordinate system (X is perpendicular to the frontal plane pointing forward, Y is perpendicular to the transverse plane pointing upward, and Z is perpendicular to the sagittal plane pointing to the right). MOtoNMS was also used to process the EMG recording to determine muscles’ activations; 1) a zero-lag band-pass filter (4th order Butterworth 30–300 Hz), 2) full-wave rectification, and 3) a low pass filter (4th order Butterworth 4–10 Hz). Finally, the signals were normalized to the maximum value of EMG recorded during the MVC trials to scale underlying muscles' excitations as a number between 0 and 1.

An OpenSim model (gait2392) [36] was scaled using the MAP-client workflow [37] based on marker data using Principal Component Analysis to build a personalized musculoskeletal model for each participant. Pelvis, hips, knees, and ankles kinematics and kinetics were computed using the OpenSim inverse kinematics (IK) and inverse dynamics (ID) tools, respectively (version 3.3). The Calibrated EMG-Informed Neuromusculoskeletal Modelling (CEINMS) [38] toolbox was used to estimate muscle forces. To calibrate musculotendon units (MTUs), we used three walking, one heel raising, and one squat trial in the CEINMS calibration step to adjust musculotendon parameters like tendon slack length, optimal fiber length, and strength coefficient. The objective function for calibration was defined by minimizing the differences between the joint moments estimated by the EMG-driven model and those derived from inverse dynamics during multiple calibration trials. Once calibration was completed, all MTUs activation and forces were predicted using the EMG-assisted approach (hybrid mode) of CEINMS. Finally, IMU, EMG, joint angles, joint moments, and muscle forces data were down sampled to 100 Hz to decrease the computational cost of feature extraction and ML models construction. Joint moments were normalized to each participants’ body weight.

2.3. Models’ development

The development process of each ML model to predict targets (joint angles, joint moments, and muscle forces) from wearable sensors’ data is explained below. To increase the predictive power and facilitate the ML process, we identified all features from raw IMU (acceleration and angular velocity in three directions) and EMG data by using an open-source python package called Tsfresh (Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests) [34]. In order to prepare the IMU and EMG data for Tsfresh, we put them into sequences of consecutive and overlapping windows, where a window is shifted across the data points to create smaller segments of time series signals per target value. In this study, the window size was one second over future data, as recommended by Banos et al. [39].

2.3.1. Feature selection

Tsfresh extracted 788 features for each IMU and EMG data component from an initial 47000 features (6 channels of data for each of 7 IMUs and 16 channels of EMG data). In consideration of computational efficiency, it was necessary to reduce the features list to the top features associated with each target. First, for each target, a primary selection step took place to retain features with non-zero variance. Then the Tsfresh feature selector was used to remove non-significant features in predicting target values. The selected features were then used for training a random forest regressor. The rank of each feature was determined based on Gini Importance [40] for predicting each target by the regressor. Finally, the top ten features related to each target were selected. All top features were put together to build a super feature set, including 500 features (10 for each of the 50 targets). The final feature set was further reduced by removing repeated features to avoid redundancy, and finally, 441 features were used as input to our ML models.

2.3.2. Non-linear regression ML models

The most important features extracted by Tsfresh were used as inputs to our four non-linear regression machine learning (ML) models (Artificial Neural Networks (ANN), Random Forest (RF), support vector machine (SVM) and multivariate adaptive regression spline (MARS)) to predict targets (joint kinematics, joint kinetics, and muscle forces) in over-ground walking. All constructed models were multi-output models, which predicted all targets simultaneously. Scikit-learn (a python library for machine learning) was used to set up the ANN, RF and SVM models, while the MARS model was built by using the py-earth python library. We used the 'GridSearchCV' function of the Scikit-learn library to tune the models' hyperparameters to obtain the best results.

Artificial Neural Networks (ANN) are powerful non-linear regression models that simulate the workings of biological neural networks. The ANN used in this study was a multi-layer (input layer, five hidden layers, and output layer) backpropagation neural network. The activation function in the hidden layers was 'ReLU', which is composed of two linear pieces to account for non-linearities. The 'Adam' solver, a stochastic gradient-based optimizer, was used for weight optimization. As ANN would employ all available variables it would not perform well. Therefore we developed separate ANN models for each target using only the top ten features as input variables.

Random Forest (RF) is a flexible and easy to use machine learning model for regression. RF builds forest (ensemble of decision trees) trained with the bootstrap aggregating (bagging) method and outputs the average of prediction of individual decision trees [41]. The RF model's hyperparameters in this study were set to 500 trees, 21 randomly selected variables in each tree and a maximum depth of 15 for each tree.

Another powerful supervised learning model for non-linear regression is Support Vector Machine (SVM) [42]. In this model, a threshold (𝜀) is set by the user to control the maximum allowable error for the regression setting. When there is non-linearity in the dataset, a kernel function is used to map the input feature vectors to a higher dimensional feature space. In this study, we set $\epsilon =$ 0.2, cost parameter $C=$100, and Radial Basis Function (RBF) as the kernel function.

The last model developed in this study was a multivariate adaptive regression spline (MARS) which is well suited for high-dimensional problems. MARS is an extension of linear models by modelling non-linearities in target values. This model aggregates a set of simple linear functions' results to perform well in predicting any kind of target vector. MARS algorithm automatically discovers the number and type of basis functions to use. We set the number of input variables considered by each piecewise linear function (max_degree) to one.

2.4. Performance evaluation

Each model was trained on 11 gait cycles (stance phase captured from the middle force plate of each trial) for each participant and tested on additional five gait cycles from the same participant to examine the intra-subject models’ performance. Then we built inter-subject models by using a leave-one-out (LOO) cross-validation to investigate the generalizability of each model. The LOO cross validation is similar to a k-fold cross validation with k = 17. To compare the performance of ML models in the testing dataset, the Root Mean Square Error (RMSE) (Eq. (1)), Mean Absolute Error (MAE) (Eq. (2)), and coefficient of determination (${R}^{2}$) (Eq. (3)) between the computed and predicted targets were calculated for each gait cycle and each participant for both intra and inter-subject examinations.

$$RMSE=\sqrt{\frac{\sum _{i=1}^{N}{\left(ytru{e}_{i}-ypre{d}_{i}\right)}^{2}}{N}} \left(1\right)$$

$$MAE= \frac{\sum _{i=1}^{N}\left|ytru{e}_{i}-ypre{d}_{i}\right|}{N} \left(2\right)$$

$${R}^{2}=1-\frac{\sum _{i=1}^{N}{\left(ytru{e}_{i}-ypre{d}_{i}\right)}^{2}}{\sum _{i=1}^{N}{\left(ytru{e}_{i}-\stackrel{-}{ytrue}\right)}^{2}} \left(3\right)$$

The most important features selected to predict each target are available in Supplementary Table 1. The top features were extracted from the IMU signals for kinematics and kinetics prediction. In contrast, most of the top features were extracted from EMG signals for muscle forces prediction. The following results represent the performance of each Machine Learning (ML) model, ANN (pink), RF (blue), SVM (yellow), and MARS (green), for the prediction of intended targets (joint kinematics, joint kinetics, and muscle forces).

3.1. Joint kinematics

The RF model provided the lower RMSE in all joints and planes of motion compared to other models for both intra and inter-subject examinations (Fig. 2). The highest RMSE was related to MARS predictions in most joint angles. The lowest and highest joint kinematics RMSE in RF model were related to pelvic obliquity (RMSE = 1.44 for intra-subject examination and 2.95 degrees for inter-subject examinations) and ankle inversion/evrsion (RMSE = 4.46 for intra-subject examination and 8.86 degrees for inter-subject examinations) angles, respectively. The same trend can be seen by investigating other evaluation metrics such as the mean absolute error (MAE) and the coefficient of determination (${R}^{2}$) presented in Supplementary Table 2.

Examples of the hip, knee, and ankle sagittal plane ROM during a gait cycle for the best participants (based on RF results) for the intra and inter-subject examinations are displayed in Fig. 3. Worst participants results and other joint kinematics are shown in Supplementary Figs. 1 and 2 for one gait cycle. As can be seen, the RF model (dashed blue line) provided the best predictions by following OpenSim inverse kinematics output (solid grey line) better than other models.

3.2. Joint kinetics

The RF model consistently provided lower RMSE than other ML models in all joints moments predictions, analogously to the kinematics results (Fig. 4). The MARS model produced the maximum RMSE in most joints for both intra and inter-subject examinations. The range of RSME for joint kinetics prediction was between 0.028 (hip rotation moment) and 0.214 Nm/kg (pelvic tilt moment) in intra-subject examination. The minimum and maximum joint kinetics RMSE were 0.048 (hip rotation moment) and 0.271 Nm/kg (pelvic tilt moment) in inter-subject examination. MAE and ${R}^{2}$ values are presented in Supplementary Table 3 for joint kinetics predictions by all models.

The best participant predictions of all ML models for one gait cycle for ankle, knee, and hip moments in the sagittal plane are displayed in Fig. 5. Worst participant predictions and other joints kinetics are presented in Supplementary Figs. 3 and 4 for a gait cycle. The RF model (dashed blue line) is providing better predictions of the OpenSim inverse dynamics outputs (grey solid line) compared to the other models.

3.3. Muscle forces

To predict muscle forces, the RF model displayed the lowest RMSE between CEINMS outputs and prediction output from ML models, while the highest RMSEs came from the MARS and the ANN models (Fig. 6). The maximum average RMSE value occurred when predicting the vastus lateralis forces for the intra (208N in RF model) and inter-subject (502N in RF model) examination. The average RMSE for the semimembranosus muscle force prediction was the lowest among all muscles for all ML models in both intra (24N in RF model) and inter-subject (35N in RF model) examinations. MAE and ${R}^{2}$ values between models’ predictions and CEINMS output are presented in Supplementary Table 4 for muscle forces predictions.

Muscle forces predictions by all models can be seen in Fig. 7 for the soleus (ankle plantar flexor muscle), semitendinosus (knee flexor muscle), and rectus femoris (hip flexor muscle) across one gait cycle for the best participants in each examination. Figures demonstrating worst participants results and other muscle forces are available in Supplementary Figs. 5 and 6. The best match between the ML models’ prediction and CEINMS output came from the RF model for both intra-subject and inter-subject examinations.

This study aimed to compare performance of machine learning (ML) models for the prediction of important lower-limb gait time series (joint kinematics, joint kinetics, and muscle forces) from wearable sensors’ data with the aid of automatic feature extraction. The first objective was to extract features automatically and determine the most important features for the estimation of each target. To extract all possible features from raw EMG and IMU data, we used a python package called Tsfresh [34]. Tsfresh’s ability to extract a high number of features and determine their significance makes it more suitable than manual feature extraction methods. Furthermore, the most important features that are essential for predicting a particular target might be neglected when extracted manually [43]. As a result, the top features to predict joint kinematics and kinetics were extracted from the IMU data, and the top features for most of the muscle forces were extracted from EMG data. These results were predictable, as the joint angles are closely related to the angular velocity (gyroscope data), joint moments are associated with linear acceleration (accelerometers data) and angular velocity, and EMG data are correlated with muscle activation and therefore, muscle forces.

The second objective of this study was to compare the performance of four non-linear regression ML models (ANN, SVM, RF, and MARS) to estimate pelvis, hip, knee, and ankle joint angles, moments, and muscle forces in both intra-subject and inter-subject examinations. Each model performance were compared based on their resulted RMSE, MAE and R² against the OpenSim output (used here as ground truth). We found that the RF model gave the best performance in predicting joint kinematics, joint kinetics, and muscle forces for the intra and inter-subject examinations. Based on the figures representing the models’ predictions for one gait cycle, RF model provided smoother outputs, in addition to having less prediction errors compared to other models. The RF algorithm is less prone to overfitting than other models [41], which might explain its higher performance. Moreover, RF is a tree-based model and naturally ranks features by how well they improve the model’s performance and only uses the most important features to build trees. Regardless of the ML model’s type, the level of prediction accuracy decreased when the training dataset did not include any of the testing subjects' trials. This can be partially explained by the fact that individuals' joint motion characteristics are distinct [44].

To avoid limitations of OMC systems like the need for expensive equipment in a controlled environment and time-consuming data processing, previous studies have developed different algortithms to estimate joint kinematics from IMUs [44–55]. One of these algorithms was using filtering approaches to cope with IMU sensor noise and integration drift [44, 52–55]. While these algorithms succeeded to reproduce a similar joint angle waveform, the offset between IMU results and OMC systems is considered relatively high compared to our results. The RMSE ranging from 5 to 10.14 degrees in the hip joint angle sagittal plane was previously reported [45, 46, 48–51, 53, 55], while the present study achieved an RMSE of 2.26 and 5.94 degrees for intra and inter-subject examinations, respectively. Our model produced lower RMSEs in knee joint flexion/extension (2.89 and 7.07 degrees for intra and inter subject examinations) too with reported RMSE between 4.1 to 11.22 degrees in other studies [45, 46, 48–51, 55]. The accuracy of our model for ankle joint dorsi/plantarflexion angle prediction (3.61 and 7.72 for intra and inter-subject examinations) was comparable to previous studies with an RMSE of 1.9 to 9.75 degrees [45, 46, 48, 50, 55]. Other research groups achieved higher accuracy by combining wearable sensors’ data with machine learning techniques for joint kinematics prediction [11–17, 19]. The better performance of this approach (IMUs + ML model) provided lower estimation errors in previous studies, especially in the intra-subject examinations with an RMSE ranging from 1.72 to 3.58 degrees in hip flexion/extension [11, 14, 15], from 2.21 to 3.96 in knee flexion/extension [11, 12, 14, 15] and from 1.81 to 3.58 degrees in ankle dorsi/plantarflexion angle [11, 12, 14, 15]. The performance of our RF model in intra-subject examination was in the range of these studies with an average RMSE of 2.26 degrees at the hip, 2.89 for knee, and 3.61 for ankle angles in the sagittal plane. The prediction error for inter-subject examination in previous studies was higher with hip flexion/extension angle RMSE from 4.6 to 8.85 degrees [14, 16] (5.94 degrees in the present study), knee flexion/extension angle RMSE of 4.5 to 9.28 [14, 16, 19] (7.07 degrees in the present study), and ankle dorsi/plantar flexion angle RMSE of 4.74 to 6.03 [14, 16] (7.72 degrees in the present study). Ren et al. [17] developed five different ML models to predict hip, knee, and ankle joint angles in the sagittal plane and achieved MAE = 4.6, 7.38, and 4.74 degrees, respectively, by using the RF model. They illustrated that the RF model outperforms other ML models (SVR, ANN, multiple linear regression (MLR), and eXtreme gradient boosting (XGboost)) for joint kinematics prediction. The RF model in the current study carried out lower error than their model in hip and knee angles prediction by having MAE = 4.52, and 5.27 degrees, respectively, while we had higher errors than Ren et al. [17] in predicting ankle dorsi/plantar flexion angle (MAE of 5.73 degrees) for inter-subject examination. While previous studies concentrated on joint range of motion in the sagittal plane, our study additionally included the pelvis in all plane, hip int/ext rotation and abd/add and ankle inv/eversion.

Fewer studies were conducted to investigate the performance of different ML models for joint kinetics estimation [12, 20–22] compared to joint kinematics. All previous studies would focus on a specific lower-limb joint kinetics (e.g. knee and ankle moments in sagittal plane [12], knee adduction/abduction moment [20], medial and lateral knee contact forces [21], knee flexion/extension and adduction/abduction moments [22]), while the present study investigated the prediction accuracy for the pelvis (in three planes of motion), hip (in three planes of motion), knee (in sagittal plane) and ankle (in sagittal and frontal planes) joint moments during gait. The RF model presented in our study achieved RMSE of 0.079 Nm/kg for ankle moment prediction in intra-subject examination which is more accurate compared to previous studies with an RMSE of 0.119 Nm/kg [12]. For the knee flexion/extension moment, we had lower accuracy in intra-subject examination compared to other studies (RMSE of 0.102 versus RMSE ranging from 0.042 to 0.068 Nm/kg [12, 20]). However, our model outperformed another study in inter-subject examination (RMSE of 0.18 versus 0.27 Nm/kg) [22].

To the best of our knowledge, there is no other study using wearable sensors’ data to estimate muscle forces. Ardestani et al. [56] used an ANN to estimate muscle activations from EMG signals. They used muscle activations in a forward dynamic model to estimate lower-limb joint moments, but unfortunately, they didn’t report any prediction error for muscle activation or forces. In another study [57], a Gaussian Mixture Regressor (GMR) was employed to estimate muscle kinematics (fiber elongations and moment arms) and EMG data from IMU. They reported NRMSE lower than 30% of muscle activation for all muscles. In the present study, the highest and lowest muscle forces prediction errors were associated with semimembranosus (RMSE of 24 and 35N in intra and inter subject examinations, respectively) and vastus lateralis muscle (RMSE of 208 and 502 N in intra and inter subject examinations, respectively). Higher offset between actual and predicted values for muscle forces prediction compared to joint kinematics and kinetics can be seen in the figures depicting actual and predicted values. The lower accuracy in estimation of muscle forces compared to other targets (joint kinematics and kinetics) was predictable. Data showed different muscle recruitment during walking between individuals and even trials, leading to less consistency in muscle forces across the population. Overall, compared to previous research, we predicted more targets at the same time with a multi-output RF model and achieved prediction errors within the range of what is reported in the literature.

Despite the number of participants (17 total), our RF model resulted to low prediction errors (comparable to the literature) in joints kinematics and kinetics estimation. We will investigate if increasing the number of participants to include a variety of gait profile and self selected speed provides more accurate estimations, specifically for muscle forces prediction. One limitation of the current study is that we used a multi-output RF model to predict many targets at the same time. Although a multi-output models helps us to improve management of the high number of targets, it may result in lower prediction accuracy by feeding many unrelated features to the model for some targets. The differences between performance of a multi-output and a single output model for the prediction of specific gait time series should be explored in a future study.

The personalised musculoskeletal model for each participant were built using the gait 2392 opensim model which lacks the degrees of freedom (dof) on the knee adduction/abduction, knee rotation, and ankle int/ext rotation which does not allow us to study other plane of motion at the knee and ankle. In this study, OpenSim and CEINMS outputs were employed for ML models’ training and considered as the reference for determining the performance of ML models. To produce realistic body motions during gait, movement of each individual joint is constrained in OpenSim inverse kinematics tool. However, this method can not guarantee an optimal solution to inverse kinematics and can lead to inaccurate estimation of body pose. The inaccuracies in inverse kinematics can be transmitted and possibly amplified in later analyses like inverse kinetics and muscle forces estimation [58]. Furthermore, the neural mapping used in CEINMS is a simplification of how muscles are recruited, which may result in inaccurate muscle force predictions [38]. Also, the quality of EMGs can greatly affect CEINMS’ results, as it’s an EMG-informed solution for muscle force estimation.

Although the findings of this study are very promising to benefit the community, more research is required to investigate the optimum number of IMUs needed to achieve these results. Looking back at the top features, it appears that some IMU data are not needed for 3D gait analysis. The optimal number and combination of IMUs can eliminate the need for 7 sensors reducing data processing time and sensor cost. Reducing the number of sensors on the subjects’ bodies will also facilitate workflow implementation in the real world.

This study showed that a combination of wearable sensors and ML techniques is an accurate and promising approach for improving traditional methods of gait time-series prediction. We also demonstrated that the higher performance of RF model compared to other ML models makes it more accurate for predicting lower limb’s joint kinematics, kinetics and muscle forces especially in the intra-subject prediction. Successful implementation of an intra-subject model enables us to remotely monitor changes in patients’ gait outside the clinic. While by having a precise inter-subject model, gait analysis will be possible where optical motion capture system is not available.

Acknowledgement

We would like to thanks the participants in contributing to this study. We also want to acknowledge the Health Research Council NZ and the Science for Technological Innovation NZ for funding this study.

Author Contribution

T.D. and J.C. conceived and designed the study. S.M.M developed and analysed ML models, and prepared original draft. T.D. collected participants’ data. J.C. and T.D. supervised the study and performed funding acquision. S.M.M. and J.C. wrote and revised manuscript. All authors reviewed and approved the manuscript prior to submission.

8. Data Availability Statement

The raw data used in this study to build machine learning models are available on the open source platform SimTK.org (https://simtk.org/projects/ml_sensors).

Additional Information

Authors declare no conflict of interest.

Renggli, D., et al. Wearable inertial measurement units for assessing gait in real-world environments. J. Frontiers in physiology. 11, 90. https://doi.org/10.3389/fphys.2020.00090 (2020).
Takayanagi, N., et al., Relationship between daily and in-laboratory gait speed among healthy community-dwelling older adults. J. Scientific reports. 9, 1-6. https://doi.org/10.1038/s41598-019-39695-0 (2019).
Castillo, P., Lozano, R., and Dzul, A.E. Sensors, modems and microcontrollers for UAVs. J. Modelling and Control of Mini-Flying Machines. 185-232 (Springer, 2005).
Basmajian, J.V., Muscles alive. Their functions revealed by electromyography. J. Academic Medicine. 37, 802 (1962).
Sartori, M., Lloyd, D.G., Besier, T., Fernandez, J., and Farina, D. Electromyography-driven modeling for simulating subject-specific movement at the neuromusculoskeletal level. J. Surface Electromyography: Physiology, Engineering, and Applications, eds R. Merletti and D. Farina (Hoboken, NJ: John Wiley & Sons, Inc.). 247-272. https://doi.org/10.1002/9781119082934.ch09 (2016).
Iosa, M., Picerno, P., Paolucci, S., and Morone, G. Wearable inertial sensors for human movement analysis. J. Expert review of medical devices. 13, 641-659. https://doi.org/10.1080/17434440.2016.1198694 (2016).
De Luca, C.J. The use of surface electromyography in biomechanics. J. Journal of applied biomechanics. 13, 135-163. https://doi.org/10.1123/jab.13.2.135 (1997).
Sartori, M., Farina, D., and Lloyd, D.G. Hybrid neuromusculoskeletal modeling to best track joint moments using a balance between muscle excitations derived from electromyograms and optimization. J. Journal of biomechanics. 47, 3613-3621. https://doi.org/10.1016/j.jbiomech.2014.10.009 (2014).
Argent, R., Drummond, S., Remus, A., O’Reilly, M., and Caulfield, B. Evaluating the use of machine learning in the assessment of joint angle using a single inertial sensor. J. Journal of Rehabilitation and Assistive Technologies Engineering. 6. https://doi.org/10.1177/2055668319868544 (2019).
Błażkiewicz, M. and Wit, A. Artificial neural network simulation of lower limb joint angles in normal and impaired human gait. J. Acta of bioengineering and biomechanics. 20. https://doi.org/10.5277/ABB-01129-2018-02 (2018).
Chen, J., Zhang, X., Cheng, Y., and Xi, N. Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks. J. Biomedical Signal Processing and Control. 40, 335-342. https://doi.org/10.1016/j.bspc.2017.10.002 (2018).
Dey, S., Yoshida, T., Ernst, M., Schmalz, T., and Schilling, A.F. A Random Forest Approach for Continuous Prediction of Joint Angles and Moments During Walking: An Implication for Controlling Active Knee-Ankle Prostheses/Orthoses. IEEE International Conference on Cyborg and Bionic Systems (CBS) (2019).
Farmer, S., Silver-Thorn, B., Voglewede, P., and Beardsley, S.A. Within-socket myoelectric prediction of continuous ankle kinematics for control of a powered transtibial prosthesis. J. Journal of neural engineering. 11, https://doi.org//10.1088/1741-2560/11/5/056027 (2014).
Findlow, A., Goulermas, J., Nester, C., Howard, D., and Kenney, L. Predicting lower limb joint kinematics using wearable motion sensors. J. Gait & posture. 28, 120-126. https://doi.org/10.1016/j.gaitpost.2007.11.001 (2008).
Goulermas, J., Howard, D., Nester, C., Jones, R., and Ren, L. Regression techniques for the prediction of lower limb kinematics. J. (2005).
Luu, T.P., Low, K., Qu, X., Lim, H., and Hoon, K. An individual-specific gait pattern prediction model based on generalized regression neural networks. J. Gait & posture. 39, 443-448. https://doi.org/10.1016/j.gaitpost.2013.08.028 (2014).
Ren, S., et al. Personalized gait trajectory generation based on anthropometric features using random forest. J. Journal of Ambient Intelligence and Humanized Computing. 1-12. https://doi.org/10.1007/s12652-019-01390-3 (2019).
Sivakumar, S., Gopalai, A.A., Lim, K.H. and Gouwanda, D., Artificial neural network based ankle joint angle estimation using instrumented foot insoles. J. Biomedical Signal Processing and Control. 54, https://doi.org/10.1016/j.bspc.2019.101614 (2019).
Wouda, F.J., Giuberti, M., Bellusci, G., and Veltink, P.H. Estimation of full-body poses using only five inertial sensors: an eager or lazy learning approach? J. Sensors. 16, 2138. https://doi.org/10.3390/s16122138 (2016).
Aljaaf, A.J., Hussain, A.J., Fergus, P., Przybyla, A., and Barton, G.J. Evaluation of machine learning methods to predict knee loading from the movement of body segments. International Joint Conference on Neural Networks (IJCNN) (2016).
Giarmatzis, G., Zacharaki, E.I., and Moustakas, K. Real-time prediction of joint forces by motion capture and machine learning. J. Sensors. 20, 6933. https://doi.org/10.3390/s20236933 (2020).
Stetter, B.J., Krafft, F.C., Ringhof, S., Stein, T., and Sell, S., A machine learning and wearable sensor based approach to estimate external knee flexion and adduction moments during various locomotion tasks. J. Frontiers in bioengineering and biotechnology. 8. https://doi.org/10.3389/fbioe.2020.00009 (2020).
Lai, D.T., Shilton, A., Charry, E., Begg, R., and Palaniswami, M. A machine learning approach to k-step look-ahead prediction of gait variables from acceleration data. Annual International Conference of the IEEE Engineering in Medicine and Biology Society (2009).
Luu, T.P., Lim, H.B., Hoon, K.H., Qu, X., and Low, K. Subject-specific gait parameters prediction for robotic gait rehabilitation via generalized regression neural network. IEEE International Conference on Robotics and Biomimetics (2011).
Sandhu, K. and Kamboj, V.K., Role of Artificial Neural Network for Prediction of Gait Parameters and Patterns, AI Techniques for Reliability Prediction for Electronic Components. 124-135 (IGI Global, 2020).
Santhiranayagam, B.K., Lai, D., Shilton, A., Begg, R., and Palaniswami, M. Regression models for estimating gait parameters using inertial sensors. Seventh International Conference on Intelligent Sensors, Sensor Networks and Information Processing (2011).
Zhang, H., Guo, Y., and Zanotto, D. Accurate ambulatory gait analysis in walking and running using machine learning models. J. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 28, 191-202. https://doi.org/10.1109/TNSRE.2019.2958679 (2019).
Ferreira, J.P., Vieira, A., Ferreira, P., Crisostomo, M., and Coimbra, A.P. Human knee joint walking pattern generation using computational intelligence techniques. J. Neural Computing and Applications. 30, 1701-1713. https://doi.org/10.1007/s00521-018-3458-5 (2018).
Mundt, M., et al. Prediction of lower limb joint angles and moments during gait using artificial neural networks. J. Medical & biological engineering & computing. 58, 211-225. https://doi.org/10.1007/s11517-019-02061-3 (2020).
Wang, L. and Buchanan, T.S. Prediction of joint moments using a neural network model of muscle activations from EMG signals. J. IEEE transactions on neural systems and rehabilitation engineering. 10, 30-37. https://doi.org/10.1109/TNSRE.2002.1021584 (2002).
Bolam, S.M., et al., Remote Patient Monitoring with Wearable Sensors Following Knee Arthroplasty. J. Sensors. 21, 5143. https://doi.org/10.3390/s21155143 (2021).
Dindorf, C., Teufl, W., Taetz, B., Bleser, G., and Fröhlich, M. Interpretability of input representations for gait classification in patients after total hip arthroplasty. J. Sensors. 20, 4385. https://doi.org/10.3390/s20164385 (2020).
Gholami, M., Ejupi, A., Rezaei, A., Ferrone, A., and Menon, C. Estimation of knee joint angle using a fabric-based strain sensor and machine learning: A preliminary investigation. 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob). IEEE (2018).
Christ, M., Braun, N., Neuffer, J., and Kempa-Liehr, A.W. Time series feature extraction on basis of scalable hypothesis tests (tsfresh–a python package). J. Neurocomputing. 307, 72-77. https://doi.org/10.1016/j.neucom.2018.03.067 (2018).
Mantoan, A., et al. MOtoNMS: A MATLAB toolbox to process motion data for neuromusculoskeletal modeling and simulation. J. Source code for biology and medicine. 10, 1-14. https://doi.org/10.1186/s13029-015-0044-4 (2015).
Delp, S.L., et al. OpenSim: open-source software to create and analyze dynamic simulations of movement. J. IEEE transactions on biomedical engineering. 54, 1940-1950. https://doi.org/10.1109/TBME.2007.901024 (2007).
Zhang, J., et al. The MAP client: user-friendly musculoskeletal modelling workflows. International Symposium on Biomedical Simulation. Springer (2014).
Pizzolato, C., et al. CEINMS: A toolbox to investigate the influence of different neural control solutions on the prediction of muscle excitation and joint moments during dynamic motor tasks. J. Journal of biomechanics. 48, 3929-3936. https://doi.org/10.1016/j.jbiomech.2015.09.021 (2015).
Banos, O., Galvez, J.-M., Damas, M., Pomares, H., and Rojas, I. Window size impact in human activity recognition. J. Sensors. 14, 6474-6499. https://doi.org/10.3390/s140406474 (2014).
Nembrini, S., König, I.R., and Wright, M.N. The revival of the Gini importance? J. Bioinformatics. 34, 3711-3718. https://doi.org/10.1093/bioinformatics/bty373 (2018).
Breiman, L. Random forests. J. Machine learning. 45, 5-32. https://doi.org/10.1023/a:1010933404324 (2001).
Drucker, H., Burges, C.J., Kaufman, L., Smola, A., and Vapnik, V. Support vector regression machines. J. Advances in neural information processing systems. 9, 155-161 (1997).
Dehzangi, O., Taherisadr, M., and ChangalVala, R., IMU-based gait recognition using convolutional neural networks and multi-sensor fusion. J. Sensors. 17, 2735. https://doi.org/10.3390/s17122735 (2017).
Horst, F., Lapuschkin, S., Samek, W., Müller, K.-R., and Schöllhorn, W.I. Explaining the unique nature of individual gait patterns with deep learning. J. Scientific reports. 9, 1-13. https://doi.org/10.1038/s41598-019-38748-8 (2019).
Dorschky, E., Nitschke, M., Seifer, A.-K., van den Bogert, A.J., and Eskofier, B.M. Estimation of gait kinematics and kinetics from inertial sensor data using optimal control of musculoskeletal models. J. Journal of biomechanics. 95, 109278. https://doi.org/10.1016/j.jbiomech.2019.07.022 (2019).
Karatsidis, A., et al. Predicting kinetics using musculoskeletal modeling and inertial motion capture. J. arXiv preprint arXiv. https://doi.org/10.48550/arXiv.1801.01668 (2018).
Moon, K.S., Lee, S.Q., Ozturk, Y., Gaidhani, A., and Cox, J.A., Identification of Gait Motion Patterns Using Wearable Inertial Sensor Network. J. Sensors. 19, 5024. https://doi.org/10.3390/s19225024 (2019).
Nüesch, C., Roos, E., Pagenstert, G., and Mündermann, A. Measuring joint kinematics of treadmill walking and running: Comparison between an inertial sensor based system and a camera-based system. J. Journal of biomechanics. 57, 32-38 (2017).
Ohtaki, Y., Sagawa, K., and Inooka, H. A method for gait analysis in a daily living environment by body-mounted instruments. J. JSME International Journal Series C Mechanical Systems, Machine Elements and Manufacturing. 44, 1125-1132. https://doi.org/10.1016/j.jbiomech.2017.03.015 (2001).
Tadano, S., Takeda, R., and Miyagawa, H. Three dimensional gait analysis using wearable acceleration and gyro sensors based on quaternion calculations. J. Sensors. 13, 9321-9343 (2013).
Takeda, R., Tadano, S., Natorigawa, A., Todoh, M., and Yoshinari, S. Gait posture estimation using wearable acceleration and gyro sensors. J. Journal of biomechanics. 42, 2486-2494. https://doi.org/10.3390/s130709321 (2009).
Cikajlo, I., Matjačić, Z., and Bajd, T. Efficient FES triggering applying Kalman filter during sensory supported treadmill walking. J. Journal of medical engineering & technology. 32, 133-144. https://doi.org/10.1080/03091900601029627 (2008).
Dong, L., Wu, J., and Bao, X. A Hybrid HMM/Kalman filter for tracking hip angle in gait cycle. J. IEICE TRANSACTIONS on Information and Systems. 89, 2319-2323. https://doi.org/10.1093/ietisy/e89-d.7.2319 (2006).
Sabatini, A.M. Quaternion-based extended Kalman filter for determining orientation by inertial and magnetic sensing. J. IEEE transactions on Biomedical Engineering. 53, 1346-1356. https://doi.org/10.1109/TBME.2006.875664 (2006).
Agostini, V., Gastaldi, L., Rosso, V., Knaflitz, M., and Tadano, S. A wearable magneto-inertial system for gait analysis (H-Gait): Validation on normal weight and overweight/obese young healthy adults. J. Sensors. 17, 2406. https://doi.org/10.3390/s17102406 (2017).
Ardestani, M.M., et al. Human lower extremity joint moment prediction: A wavelet neural network approach. J. Expert Systems with Applications. 41, 4422-4433. https://doi.org/10.1016/j.eswa.2013.11.003 (2014).
Cimolato, A., et al. Hybrid Machine Learning-Neuromusculoskeletal Modeling for Control of Lower Limb Prosthetics. 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob) (2020).
Smale, K.B., et al. Effect of implementing magnetic resonance imaging for patient-specific OpenSim models on lower-body kinematics and knee ligament lengths. J. Journal of biomechanics. 83, 9-15. https://doi.org/10.1016/j.jbiomech.2018.11.016 (2019).

No competing interests reported.

SupplementaryMaterials.pdf

Download PDF

Journal Publication

published 28 Mar, 2023

Read the published version in Scientific Reports →

Editorial decision: Major revision
09 Jan, 2023
Reviews received at journal
06 Jan, 2023
Reviewers agreed at journal
13 Dec, 2022
Reviews received at journal
20 Nov, 2022
Reviewers agreed at journal
15 Nov, 2022
Reviewers agreed at journal
06 Nov, 2022
Reviewers invited by journal
04 Nov, 2022
Editor assigned by journal
04 Nov, 2022
Editor invited by journal
23 Sep, 2022
Submission checks completed at journal
23 Sep, 2022
First submitted to journal
20 Sep, 2022

You are reading this latest preprint version

A comparison of machine learning models’ accuracy in predicting lower-limb joints’ kinematics, kinetics, and muscle forces from wearable sensors

Status:

Journal Publication

Version 1

Abstract

Figures

1. Introduction

2. Methodology

2.1. Data collection

2.2. Data post-processing

2.3. Models’ development

2.3.1. Feature selection

2.3.2. Non-linear regression ML models

2.4. Performance evaluation

3. Results

3.1. Joint kinematics

3.2. Joint kinetics

3.3. Muscle forces

4. Discussion

5. Conclusion

Declarations

References

Additional Declarations

Supplementary Files

Status:

Journal Publication

Version 1