A key challenge in the data-driven identification method of nonlinear dynamical systems is developing methods robust to measurement error. Many existing methods either depend on de-noising prior to learning or have a poor identification performance. This paper proposes a novel joint state and parameter estimation method for nonlinear dynamical systems through alternating optimization. The new method can effectively remove or at least attenuate the effect of noise via alternatingly updating the state variables and parameters in nonlinear dynamical systems. The proposed identification framework mainly contains four steps. Firstly, initial estimations of state variables (displacement and velocity) are derived by using the Reproducing Kernel Hilbert Space (RKHS)-based de-noising method. Secondly, using the initial estimation of state variables, the nonlinear structure and parameters are identified by using the sparse regression and Duhamel’s integral, which can correctly select the contributing nonlinear component and identify the parameters. By using the Duhamel’s integral, the second derivative of the displacement signal which is sensitive to noise is no longer required. Thirdly, to further improve the identification accuracy, the state variables are updated based on the identified structure and parameters using the numerical integration. Finally, estimation of the nonlinear structure and parameters in step 2 and update of state variables in step 3 are alternatingly optimized until convergence. The numerical simulations and a dynamic experiment validate the effectiveness of the new identification method developed in this paper.