A Deep Neural Network-based Nonlinear Dynamics Model for the Prediction of Lateral-longitudinal Vehicle Dynamics

The vehicle multibody model can be used for accurate coupling dynamics, but it has higher computational complexity. Numerical stability during integration is also very challenging, especially in complicated driving situations. This issue can be substantially alleviated by using a data-driven nonlinear dynamics model owing to its computational speed and robust generalization. In this work, we propose a deep neural network (DNN)-based modeling approach for predicting lateral-longitudinal vehicle dynamics. Dynamic simulations of vehicle systems are performed based on a semirecursive multibody formulation for data acquisition. The data are then used for training and testing the DNN model. The DNN inputs are the torque applied on wheels and the initial vehicle speed that imitates a double lane change maneuver with acceleration and deceleration. The DNN outputs are the longitudinal driving distance, lateral driving distance, ﬁnal longitudinal velocities, ﬁnal lateral velocities, and yaw angle. The dynamic responses obtained from the DNN model are compared with the multibody results. Furthermore, the accuracy of the DNN model is investigated in terms of error functions. The DNN model is ﬁnally veriﬁed via the results of a commercial software package. The results show that the DNN vehicle dynamics model predicts accurate dynamic responses in real time. The DNN model can be used for real-time simulation and preview control in autonomous vehicles.


Introduction
Neural networks can be understood as a set of algorithms, which are loosely modeled after the human brain recognizes patterns. They can interpret many kinds of data using a type of machine perception, clustering and labeling raw input. Neural networks are currently being applied to solve many complex engineering problems, and the demand is increasing with time. A wide range of applications starting from dynamics modeling to intelligent control is being handled by neural networks [4,27,26,13].
Due to the advantages of neural networks, they have been widely used in laterallongitudinal vehicle dynamics and control in recent years. Devineau et al. explored the capability of deep neural networks (DNNs) to capture key characteristics of vehicle dynamics and their ability to perform coupled lateral and longitudinal control of a vehicle [3]. Kumarawadu et al. presented a neural network adaptive model-based combined lateral and longitudinal vehicle control algorithm for highway applications. A nonlinear three-degree-of-freedom dynamic model of a passenger wagon was developed to simulate vehicle motion and for controller design [11]. Tork et al. performed simultaneous control of the lateral and longitudinal dynamics in the nonlinear system of an autonomous vehicle. The numerical results showed maximum tracking errors of 0.02 m/s for longitudinal velocity and 0.04 m for lateral path following [25]. Taghavifar et al. proposed a probabilistic estimation approach by hybridization of optimal robust control and damped least-square backpropagation neural networks. The approach can predict and control vehicle dynamics to improve vehicle handling and stability [23]. Ji et al. presented a novel lateral motion control method that consists of a robust steering controller and an adaptive neural network approximator. The simulation and experimental results suggest that the strategy can robustly track the reference path and maintain vehicle yaw stability at or near the physical limits of tire friction [8]. Melzi et al. developed a layered neural network approach that uses input dynamic parameters acquired from ordinary sensors to estimate the sideslip angle. The results show good consistency between the estimated and measured sideslip angles in different dynamic cases [14]. Kim et al. proposed a sideslip angle estimation scheme combining DNN and nonlinear Kalman filters. The algorithm was verified through both simulation and experimental results [9]. Acosta et al. combined a model predictive controller and feedforward neural networks to achieve autonomous drifting of a vehicle along a wide range of road radii [1].
The literature survey indicates that neural networks can be efficiently used in lateral-longitudinal vehicle dynamics and control. In most studies, lateral and longitudinal control problems have been analyzed in a decoupled way. However, there are strong couplings between the two dynamics [2,15]. As a consequence, the simultaneous inclusion of lateral and longitudinal control becomes unavoidable for improving vehicle performance in a large operating range. However, the major difficulty of neural network-based methods is the number of driving situations required to build a representative training dataset. The direct estimation of vehicle states is sometimes unavailable because numerous road tests of high-performance vehicles are needed. Vehicle dynamics simulations based on commercial software packages are also time consuming since real-time simulations are still very challenging in low-cost hardware [19].
To partially address these issues, data-driven modeling approaches based on realtime data acquisition and deep neural networks provide an efficient solution. In this work, a deep neural network-based model for predicting lateral-longitudinal vehicle dynamics is presented. An efficient semirecursive multibody formulation, which performs faster than real-time simulations for data acquisition, is used to capture the key vehicle characteristics. The originality of this work lies in two contributions. The first contribution consists of the use of an efficient semirecursive multibody dynamics formulation to acquire many training and testing data. The vehicle multibody model can accurately consider and model the nonlinearities of complicated vehicle systems. The lateral-longitudinal coupled dynamics can therefore be acquired, which is much more accurate than the decoupled or simplified vehicle models. The second contribution consists of a DNN modeling approach for predicting lateral-longitudinal vehicle dynamics. Different initial velocities and applied torques over a large range are used to imitate all kinds of on-road driving situations. Furthermore, the DNN model is tested and verified by the testing set and the results obtained from a commercial software package.
The rest of the manuscript is organized as follows. In Section 2, an efficient semirecursive multibody formulation is introduced for data acquisition in real time. In Section 3, the DNN model for lateral-longitudinal dynamics of a vehicle is developed. The DNN training procedure is also provided. In Section 4, the effectiveness of the DNN model in different driving situations are investigated. The error functions are introduced to evaluate the model accuracy. In Section 5, the proposed DNN nonlinear dynamics model is verified in the framework of a commercial software package. Finally, in Section 6, we conclude our work.

Vehicle multibody formulation
In this work, we use an efficient semirecursive multibody method to establish a vehicle dynamics model. For open-loop multibody systems, the motion equations are formulated in terms of relative coordinates based on the Lagrange equations and the first velocity transformation, which can be described as [7,21]: where R d denotes the first velocity transformation matrix, which can be used to express the Cartesian velocities and accelerations in terms of the relative velocities and accelerations. T denotes the path matrix representing the recursive system tree topology.M andQ denote the generalized mass matrix and external forces, respectively.P denotes the velocity-dependent inertial forces.ż andz contain the relative velocities and accelerations, respectively. For closed-loop multibody systems, a series of mechanical joints or slender rods need to be cut or removed to take advantage of the recursive system tree topology. The corresponding loop-closure constraint equations should be considered based on Eq. (1). By introducing the second velocity transformation, a subset of relative coordinates, which are called independent relative coordinates, can be abstracted to describe the relative coordinates. Therefore, the motion equations of the closed-loop multibody system can be achieved in terms of independent relative coordinates. They can be expressed as [16]: where R z denotes the second velocity transformation matrix, which can be used to express the relative velocities and accelerations in terms of independent relative velocities and accelerations. M Σ and Q Σ denote the accumulated generalized mass matrix and external forces, respectively.ż i andz i contain the independent relative velocities and accelerations, respectively. This formulation is the double-step semirecursive multibody formulation, which was proposed by Javier Garcia de Jalon and his coworkers [20,6]. Although the expression of Eq. (2) is more complicated than other multibody formulations, it leads to the smallest set of independent relative accelerationsz i , which results in higher computational efficiency. Furthermore, the ordinary differential form of Eq. (2) enables the stable simulation of closed-loop multibody systems by using various numerical integrators. Low-order integration methods are very suitable for efficient simulations. However, it is important to note that the solution accuracy may be unable to satisfy the requirements as the numerical error accumulates over time. The Adams-Bashforth-Moulton and 4th-order Runge-Kutta methods are typical high-order integration methods that are often used in real-time simulations with longer simulation times [18,17].
The training and testing data of lateral and longitudinal dynamics can be obtained in real time based on the dedicated multibody dynamics model. Thus, we collect historical data and build a vehicle lateral-longitudinal dynamics model by using a DNN. The capacity of the DNN modeling approach to capture the key vehicle characteristics is investigated based on the testing data and verified based on the results obtained from a commercial software package. The data-driven vehicle dynamics model is developed offline via DNN and used online for complex vehicle preview control. The computational burden of the data-driven vehicle model is much smaller than that of the vehicle multibody model. In addition, the data-driven model also improves the dynamics modeling robustness and vehicle preview control.

DNN theory for vehicle dynamics modeling
Neural networks are computational technologies that resemble the human nervous system and involve the use of machine learning and efficient algorithms. A DNN is a kind of neural network with more than three structural layers but without feedback and no interconnection in each layer. It can learn and store several relationships between the input neurons and the output neurons and does not need to reveal the basic mathematical equations in advance. A general DNN is shown in Fig. 1 [22].
In the input layer, it is necessary to fix the number and structure of neurons. Input neurons should be highly reliant on the corresponding physical quantities and must be  very sensitive to the expected results. Therefore, the significant input parameters should be related to the output parameters. In the hidden layers, if a DNN with more hidden layers and hidden layers with more neurons are used, the network should have more accurate predictive performance. However, it may lead to low computational efficiency and apparent local convergence. Technically, there is no advisable method for resolving the specific network structure in each layer. Thus, it is critical to continuously test the influence of neural network prediction results with different numbers of neurons. The different network structures should be trained to select the best network. DNNs are widely used in many fields due to their advantages. They essentially realize a mapping function from the input layer to the output layer. Mathematical theory proved that DNNs can approach any nonlinear continuous function with high precision. Furthermore, the DNN model is robust and fault tolerant. If local or partial neurons of the DNN are destroyed, the overall training result will not be greatly affected.
The DNN training process is mainly composed of forward propagation of the input signal and backpropagation of errors. In forward propagation, the input matrix and the weight matrices and bias matrices are described as: W n = w n 1 , w n 2 , w n 3 , · · · w n m (4) where m denotes the number of samples in the training set, n denotes the total number of hidden layers and input layer, Z 1 represents the input matrix of the model, W n represents the weight matrix of the n-th layer, and B n represents the bias matrix of the n-th layer. The forward propagation steps can be expressed as: where Z i represents the input of the i-th layer, A i represents the output of the i-th layer, and f i denotes the i-th activation function. Note that A n+1 denotes the results of the DNN model. There are many types of activation functions that can help the DNN model fit the linear and nonlinear functions and to classify them. It is important to select an appropriate loss function to measure the differences between the sample results and the DNN results before backpropagation. By means of backpropagation, the weight matrices and bias matrices can be updated to minimize the loss function. In general, the accuracy of the DNN model increases as the value of the loss function decreases. The mean square error (MSE) is the most widely used loss function for prediction. Note that when fitting the nonlinear function, it is easy to cause overfitting. To avoid this issue, regularization is normally used to enhance the generalization performance of the DNN model. The common generalization methods include L1 regularization, L2 regularization, and dropout regularization. In this work, L2 regularization is used to compute the loss function in terms of MSE to avoid overfitting. The MSE, L2 regularization and the loss function can be expressed as [12]: where e denotes the network's cost, Y denotes the value of samples, and λ denotes the regularization coefficient. Subsequently, the adaptive moment estimation optimization (Adam) algorithm is introduced in the backpropagation process [10]. It is one of the most efficient algorithms for accelerating convergence in deep learning and can be regarded as a combination of the adaptive gradient (AdaGrad) algorithm and root mean square prop (RMSProp) algorithm [5,24]. The Adam algorithm is more appropriate for backpropagation than other algorithms because it has a higher computational efficiency in DNN training and does not require much memory. Furthermore, the Adam algorithm is good at handling sparse gradients and nonstationary targets. Hyperparameters of the Adam algorithm have intuitive explanations and require few adjustments. The key parameters of AdaGrad and RMSProp can be expressed as: where dW n and dB n denote the partial differential of the weight matrices and the bias matrices with respect to the loss function. v dwn , v dbn , s dwn , and s dbn represent the AdaGrad and RMSProp parameters in each training iteration for weight matrices and bias matrices, respectively. Note that the values of these parameters are 0 at the first training iteration. The values of hyperparameters β 1 and β 2 , in general, are 0.9 and 0.999. Since the moving exponential weighted average results in relatively large errors to the initial value at the beginning of the iteration, the AdaGrad and RMSProp parameters obtained above need to be corrected as: where these parameters are corrected in each training iteration. The weight matrices and bias matrices, in turn, are updated according to the Adam algorithm: where ε denotes a smoothing parameter, which equals 10E-08 in this work. α represents the learning rate, which needs to be minutely adjusted in the deep learning process.

Lateral-longitudinal vehicle dynamics modeling
In this work, we use the semirecursive multibody formulation presented in the previous section to model the vehicle dynamics to obtain a lateral-longitudinal dynamics dataset. This vehicle system consists of McPherson suspensions in the front, multilink suspensions in the rear, and Pacejka tire models. The vehicle structure diagram is described in Fig. 2. Additional vehicle parameters are described in Table 1.  We randomly divide these 500 samples into a training set and testing set with 450 and 50 samples in the training set and testing set, respectively. The training set is used for training the DNN model, and the testing set is used to evaluate the trained neural network. Furthermore, the training data were processed to eliminate the inconvenience caused by the magnitude of values. It can avoid very small weight matrices that may cause stability problems during numerical integration. In this work, we choose Z-score standardization for data processing, and the algorithm is mathematically expressed as: where x denotes the value of the samples, µ denotes the mean of the samples, and σ denotes the standard deviation of the samples. x * represents the standardization value. The mean value of the processed data is 0, and the standard deviation is 1.
The calculation results of the outputs should be reversed standardized later for DNN training.
There are two approaches to updating the optimization algorithm parameters in deep learning. The first is batch gradient descent, which calculates all the dataset samples. Thus, the number of calculations is large, and the calculation speed is slow. The other approach is stochastic gradient descent, which calculates the loss function of each data point and finds the updated parameters. The calculation speed of this approach is faster, but the convergence performance is not very good. To overcome the drawbacks of these two approaches, minibatch gradient descent is used. This approach divides the data into several batches and updates the parameters according to the batches. Thus, the data in a batch jointly determines the direction of this gradient, and it will not vanish easily when falling and will reduce the randomness. In this work, the number of calculations is small; thus, the batch size is set to 32.
After data training, we finally built the DNN model for predicting lateral-longitudinal vehicle dynamics. The DNN modeling procedure is shown in Fig. 4. The inputs of The ReLU function can effectively avoid the gradual disappearance and explosion of the gradient. The concise calculation formula can reduce the calculation cost. These merits ensure that ReLU is one of the most far-ranging activation functions in deep learning. After the neural network training is completed, the testing set needs to be used for DNN model verification. The MSE of the dataset also needs to be calculated. The trained DNN is qualified and can be used for the prediction if the MSE is very small. The procedure of DNN training for vehicle lateral-longitudinal dynamics modeling is described in Fig. 5 Input learning rate α, regularization coefficient λ Fig. 5 The procedure of DNN training.

DNN model results
To evaluate the accuracy of the trained DNN model, we compare the predicted results of the DNN model and the simulated results of the multibody model. Note that the vehicle initial longitudinal speeds of the DNN model and multibody model both range from 15 m/s to 45 m/s, and the front-wheel driving torque ranges from -500 Nm to 500 Nm. Fig. 6 shows the differences between the DNN model results and the multibody model results. The comparative responses are longitudinal driving distance, lateral driving distance, final longitudinal velocities, final lateral velocities, and yaw angle. Note that for better visualization, only part of the predicted results is included in the figures. Fig. 7 to 11 show the absolute percentage error between the DNN model results and the multibody model results for these five vehicle responses, respectively. Different initial speeds and driving torques in a wide range are used to imitate various driving situations.
According to the results comparison and absolute percentage error analysis, the following conclusions can be drawn.      The DNN model can be trained based on a large quantity of lateral and longitudinal data offline and used for the prediction of lateral-longitudinal dynamics later. Furthermore, because the DNN model completely omits the tedious process of numerical integration, it is especially suitable for the preview control of autonomous vehicles.

Model accuracy and efficiency
In this section, we evaluate the accuracy of the proposed DNN model quantifiably by error functions. The computational efficiency of the DNN model is also investigated. The accuracy evaluation metrics comprise the maximum absolute error (ME), the mean absolute error (MAE), the mean absolute percentage error (MAPE), the root mean square error (RMSE) and R 2 . The ME denotes the maximum absolute difference between the simulated results and the predicted results. The MAE denotes the average numerical value of the differences between the ground truth and predicted values. The MAPE denotes the homologous percentage differences. The RMSE denotes the root mean square of the differences between the simulated results and the predicted results. R 2 denotes the coefficient of determination, a statistical indicator used to reflect the regression model to illustrate the reliability of the dependent variable change. The expressions of these five error functions are written as: where, y i represents the i-th value of the simulated results,ŷ i represents the i-th value of the predicted results,ȳ i represents the i-th mean value of the predicted results, n represents the number of groups, and n equals 7701. These five evaluation metrics are used to quantify the accuracy of the DNN model. The smaller the first four metrics are, the more accurate the DNN model is. However, for R 2 , if its value is closer to 1, the DNN model fits well.
The longitudinal driving distance, lateral driving distance, final longitudinal speed, final lateral speed and yaw angle predicted by the proposed DNN model are analyzed based on the evaluation metrics. The evaluation results are described in Table 2. It is easy to observe in Table 2 that the ME, MAE, MAPE, and RMSE of the lateral-longitudinal responses are relatively small, and R 2 is very close to 1. This indicates that the DNN model has very high accuracy in predicting lateral-longitudinal dynamics. In addition, the longitudinal prediction is slightly more accurate than the lateral prediction.
To investigate the computational efficiency of the proposed DNN model, we compare the CPU time and MAPE of deep neural networks with different numbers of hidden layers. The results are described in Tables 3 and 4. It is important to note that the CPU time is the average time of ten random driving situations, and MAPE is measured based on 7,701 sets of data.  It is obvious in Tables 3 and 4 that the CPU time for lateral-longitudinal dynamics prediction is approximately 1 ms with 2, 3, and 4 hidden layers. The computational efficiency remains the same in these cases. However, in the case of 2 or 3 hidden layers, the MAPE is relatively large, and the predicted results are less accurate than the proposed DNN model.

Model verification
We use the commercial software package CarSim to further verify the effectiveness of the proposed DNN model. It is important to note that dynamic simulations with the same vehicle parameters are performed in the CarSim framework. Due to limited space, we only plot the comparative results of longitudinal distances, lateral distances and longitudinal velocities under the conditions of 300 Nm and -300 Nm driving torques. Figs. 12 to 14 depict the result comparison when the 300 Nm and -300 Nm driving torques are imposed respectively. The detailed view in each figure describes the maximum difference between the DNN result and the commercial software result. 15     As presented in Figs. 12 to 14, the predicted results via deep neural networks coincide with the results of the CarSim software package. The results also are consistent with our earlier multibody model results. The driving situation imitates that the vehicle accelerates and decelerates during a double lane change maneuver with a wide range of initial speeds. Finally, the effectiveness and robustness of the proposed DNN model for vehicle lateral-longitudinal dynamics are verified. It can be used for the preview control of autonomous vehicles.

Conclusion
In this work, a DNN-based approach for vehicle lateral-longitudinal dynamics was proposed. A semirecursive multibody dynamics formulation is used for vehicle modeling and data acquisition. An algorithm for obtaining lateral-longitudinal dynamics data and a DNN modeling procedure were developed. It is significant that massive parallel computations can be performed by using GPUs that are especially suitable for large volumes of data.
To verify the proposed DNN model, the testing data were randomly selected for accuracy evaluation. Error functions were introduced to quantifiably assess the model accuracy. The computational efficiency and solution accuracy were investigated for neural networks with three different structures. Finally, results from a commercial software package were obtained to further verify the effectiveness of the proposed DNN model. The results showed that the DNN model can predict the vehicle lateral-longitudinal dynamics accurately and efficiently. The DNN model used for modeling lateral, longitudinal and vertical coupling dynamics will be investigated in the future. Overall, a DNN-based approach was presented to predict the vehicle's lateral-longitudinal dynamics in real time, which is suitable for the preview control of autonomous vehicles.